[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:19:34.468037 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:39533
I20260709 12:19:34.469990 12469 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:19:34.471277 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:19:34.492293 12478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:34.492293 12476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:34.494907 12469 server_base.cc:1061] running on GCE node
W20260709 12:19:34.492466 12475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:35.741689 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:19:35.741966 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:19:35.742264 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599575742213 us; error 0 us; skew 500 ppm
I20260709 12:19:35.743256 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:19:35.749950 12469 webserver.cc:533] Webserver started at http://127.12.45.126:32797/ using document root <none> and password file <none>
I20260709 12:19:35.751046 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:19:35.751309 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:19:35.751852 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:19:35.758912 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "66d4bd663e8b498f889dde288f0bef35"
format_stamp: "Formatted at 2026-07-09 12:19:35 on dist-test-slave-90bm"
I20260709 12:19:35.768193 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.000s	sys 0.008s
I20260709 12:19:35.774140 12484 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:35.775712 12469 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:19:35.776160 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "66d4bd663e8b498f889dde288f0bef35"
format_stamp: "Formatted at 2026-07-09 12:19:35 on dist-test-slave-90bm"
I20260709 12:19:35.776573 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:19:35.858798 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:19:35.860747 12469 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:19:35.861542 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:19:35.954059 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:39533
I20260709 12:19:35.954162 12535 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:39533 every 8 connection(s)
I20260709 12:19:35.964049 12536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:35.994447 12536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35: Bootstrap starting.
I20260709 12:19:36.002363 12536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:36.004908 12536 log.cc:826] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35: Log is configured to *not* fsync() on all Append() calls
I20260709 12:19:36.010720 12536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35: No bootstrap required, opened a new log
I20260709 12:19:36.034237 12536 raft_consensus.cc:359] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d4bd663e8b498f889dde288f0bef35" member_type: VOTER }
I20260709 12:19:36.034965 12536 raft_consensus.cc:385] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:36.035179 12536 raft_consensus.cc:740] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d4bd663e8b498f889dde288f0bef35, State: Initialized, Role: FOLLOWER
I20260709 12:19:36.036193 12536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d4bd663e8b498f889dde288f0bef35" member_type: VOTER }
I20260709 12:19:36.036737 12536 raft_consensus.cc:399] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:19:36.037003 12536 raft_consensus.cc:493] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:19:36.037326 12536 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:36.042725 12536 raft_consensus.cc:515] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d4bd663e8b498f889dde288f0bef35" member_type: VOTER }
I20260709 12:19:36.043653 12536 leader_election.cc:304] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66d4bd663e8b498f889dde288f0bef35; no voters: 
I20260709 12:19:36.045650 12536 leader_election.cc:290] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:19:36.046063 12539 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:36.049225 12539 raft_consensus.cc:697] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 1 LEADER]: Becoming Leader. State: Replica: 66d4bd663e8b498f889dde288f0bef35, State: Running, Role: LEADER
I20260709 12:19:36.050359 12539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d4bd663e8b498f889dde288f0bef35" member_type: VOTER }
I20260709 12:19:36.051638 12536 sys_catalog.cc:565] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:19:36.063941 12540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66d4bd663e8b498f889dde288f0bef35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d4bd663e8b498f889dde288f0bef35" member_type: VOTER } }
I20260709 12:19:36.064172 12541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66d4bd663e8b498f889dde288f0bef35. Latest consensus state: current_term: 1 leader_uuid: "66d4bd663e8b498f889dde288f0bef35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66d4bd663e8b498f889dde288f0bef35" member_type: VOTER } }
I20260709 12:19:36.064872 12540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [sys.catalog]: This master's current role is: LEADER
I20260709 12:19:36.064898 12541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [sys.catalog]: This master's current role is: LEADER
I20260709 12:19:36.068260 12548 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:19:36.083619 12548 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:19:36.086395 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:19:36.104291 12548 catalog_manager.cc:1383] Generated new cluster ID: 430352b60240461ab94bbc6e79eaff9b
I20260709 12:19:36.104671 12548 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:19:36.159111 12548 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:19:36.161633 12548 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:19:36.183763 12548 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35: Generated new TSK 0
I20260709 12:19:36.185043 12548 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:19:36.218760 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:19:36.229743 12558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:36.231422 12559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:36.236683 12561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:36.237262 12469 server_base.cc:1061] running on GCE node
I20260709 12:19:36.238175 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:19:36.238487 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:19:36.238835 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599576238803 us; error 0 us; skew 500 ppm
I20260709 12:19:36.239828 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:19:36.245878 12469 webserver.cc:533] Webserver started at http://127.12.45.65:35607/ using document root <none> and password file <none>
I20260709 12:19:36.246661 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:19:36.246919 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:19:36.247262 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:19:36.248555 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "a37ecd8998584be68aafbd4ef66c3d84"
format_stamp: "Formatted at 2026-07-09 12:19:36 on dist-test-slave-90bm"
I20260709 12:19:36.255470 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:19:36.261485 12566 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.262569 12469 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:19:36.262908 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "a37ecd8998584be68aafbd4ef66c3d84"
format_stamp: "Formatted at 2026-07-09 12:19:36 on dist-test-slave-90bm"
I20260709 12:19:36.263242 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:19:36.283408 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:19:36.284970 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:19:36.287869 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:19:36.293205 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:19:36.293452 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.293795 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:19:36.293959 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.422636 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:41905
I20260709 12:19:36.422705 12628 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:41905 every 8 connection(s)
I20260709 12:19:36.431607 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:19:36.446143 12633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:36.450487 12634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:36.463418 12636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:36.466190 12469 server_base.cc:1061] running on GCE node
I20260709 12:19:36.467738 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:19:36.468194 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:19:36.468690 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599576468639 us; error 0 us; skew 500 ppm
I20260709 12:19:36.469605 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:19:36.473016 12629 heartbeater.cc:344] Connected to a master server at 127.12.45.126:39533
I20260709 12:19:36.473587 12629 heartbeater.cc:461] Registering TS with master...
I20260709 12:19:36.474977 12629 heartbeater.cc:507] Master 127.12.45.126:39533 requested a full tablet report, sending...
I20260709 12:19:36.478950 12469 webserver.cc:533] Webserver started at http://127.12.45.66:46153/ using document root <none> and password file <none>
I20260709 12:19:36.479830 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:19:36.479725 12501 ts_manager.cc:194] Registered new tserver with Master: a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905)
I20260709 12:19:36.480149 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:19:36.480552 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:19:36.482450 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "801602d913e44df6a7c0fb76bc53aa30"
format_stamp: "Formatted at 2026-07-09 12:19:36 on dist-test-slave-90bm"
I20260709 12:19:36.483408 12501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40656
I20260709 12:19:36.492741 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:19:36.498898 12641 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.500449 12469 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:19:36.500914 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "801602d913e44df6a7c0fb76bc53aa30"
format_stamp: "Formatted at 2026-07-09 12:19:36 on dist-test-slave-90bm"
I20260709 12:19:36.502147 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:19:36.540232 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:19:36.541796 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:19:36.544903 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:19:36.548856 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:19:36.549134 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.549398 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:19:36.549588 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.674947 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:37321
I20260709 12:19:36.675088 12703 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:37321 every 8 connection(s)
I20260709 12:19:36.682435 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:19:36.690539 12707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:19:36.694782 12708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:36.698720 12704 heartbeater.cc:344] Connected to a master server at 127.12.45.126:39533
I20260709 12:19:36.699337 12704 heartbeater.cc:461] Registering TS with master...
I20260709 12:19:36.700603 12704 heartbeater.cc:507] Master 127.12.45.126:39533 requested a full tablet report, sending...
I20260709 12:19:36.703965 12501 ts_manager.cc:194] Registered new tserver with Master: 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:36.707267 12501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40672
I20260709 12:19:36.708493 12469 server_base.cc:1061] running on GCE node
W20260709 12:19:36.711292 12710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:19:36.712450 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:19:36.712735 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:19:36.712946 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599576712906 us; error 0 us; skew 500 ppm
I20260709 12:19:36.713611 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:19:36.716800 12469 webserver.cc:533] Webserver started at http://127.12.45.67:40741/ using document root <none> and password file <none>
I20260709 12:19:36.717458 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:19:36.717685 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:19:36.718014 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:19:36.719308 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
format_stamp: "Formatted at 2026-07-09 12:19:36 on dist-test-slave-90bm"
I20260709 12:19:36.725319 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:19:36.729526 12715 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.730984 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:19:36.731395 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
format_stamp: "Formatted at 2026-07-09 12:19:36 on dist-test-slave-90bm"
I20260709 12:19:36.731793 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:19:36.756904 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:19:36.758491 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:19:36.760488 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:19:36.764076 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:19:36.764307 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.764621 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:19:36.764796 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:19:36.882925 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:41687
I20260709 12:19:36.883560 12777 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:41687 every 8 connection(s)
I20260709 12:19:36.903391 12778 heartbeater.cc:344] Connected to a master server at 127.12.45.126:39533
I20260709 12:19:36.903894 12778 heartbeater.cc:461] Registering TS with master...
I20260709 12:19:36.905210 12778 heartbeater.cc:507] Master 127.12.45.126:39533 requested a full tablet report, sending...
I20260709 12:19:36.908150 12501 ts_manager.cc:194] Registered new tserver with Master: 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:36.908341 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019334787s
I20260709 12:19:36.910161 12501 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40688
I20260709 12:19:36.969527 12501 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40692:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:19:36.973970 12501 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 12:19:37.076174 12549 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:19:37.108700 12669 tablet_service.cc:1511] Processing CreateTablet for tablet 55d5f1e66c7745ce9c59ea8361d55e74 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:19:37.117921 12594 tablet_service.cc:1511] Processing CreateTablet for tablet 55d5f1e66c7745ce9c59ea8361d55e74 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:19:37.118621 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d5f1e66c7745ce9c59ea8361d55e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.119652 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d5f1e66c7745ce9c59ea8361d55e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.121110 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 744469240da84c15832aa2f583308505 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:19:37.111209 12668 tablet_service.cc:1511] Processing CreateTablet for tablet a1f3b5d9156947e49d05b26ae1586797 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:19:37.122649 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744469240da84c15832aa2f583308505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.123297 12668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1f3b5d9156947e49d05b26ae1586797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.123867 12665 tablet_service.cc:1511] Processing CreateTablet for tablet c6bf05f9b8be40cd9264cf530db7b1a6 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:19:37.113313 12667 tablet_service.cc:1511] Processing CreateTablet for tablet 9a64ffb6456946b7b33340731974841e (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:19:37.125443 12665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bf05f9b8be40cd9264cf530db7b1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.126425 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a64ffb6456946b7b33340731974841e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.127215 12590 tablet_service.cc:1511] Processing CreateTablet for tablet c6bf05f9b8be40cd9264cf530db7b1a6 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:19:37.127707 12589 tablet_service.cc:1511] Processing CreateTablet for tablet 4a9cac7524ae4180879338590f67b2c7 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:19:37.115806 12666 tablet_service.cc:1511] Processing CreateTablet for tablet 744469240da84c15832aa2f583308505 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:19:37.128794 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bf05f9b8be40cd9264cf530db7b1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.133590 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a9cac7524ae4180879338590f67b2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.134195 12666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744469240da84c15832aa2f583308505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.118065 12593 tablet_service.cc:1511] Processing CreateTablet for tablet a1f3b5d9156947e49d05b26ae1586797 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:19:37.119659 12592 tablet_service.cc:1511] Processing CreateTablet for tablet 9a64ffb6456946b7b33340731974841e (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:19:37.144017 12741 tablet_service.cc:1511] Processing CreateTablet for tablet 9a64ffb6456946b7b33340731974841e (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:19:37.138983 12740 tablet_service.cc:1511] Processing CreateTablet for tablet 744469240da84c15832aa2f583308505 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:19:37.146062 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1f3b5d9156947e49d05b26ae1586797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.164276 12741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a64ffb6456946b7b33340731974841e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.167790 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a64ffb6456946b7b33340731974841e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.169198 12739 tablet_service.cc:1511] Processing CreateTablet for tablet c6bf05f9b8be40cd9264cf530db7b1a6 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:19:37.176574 12740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744469240da84c15832aa2f583308505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.148993 12742 tablet_service.cc:1511] Processing CreateTablet for tablet a1f3b5d9156947e49d05b26ae1586797 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:19:37.156689 12738 tablet_service.cc:1511] Processing CreateTablet for tablet 4a9cac7524ae4180879338590f67b2c7 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:19:37.188908 12664 tablet_service.cc:1511] Processing CreateTablet for tablet 4a9cac7524ae4180879338590f67b2c7 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:19:37.193216 12739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6bf05f9b8be40cd9264cf530db7b1a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.194968 12588 tablet_service.cc:1511] Processing CreateTablet for tablet e201e7a803ea453d956f01684a55efde (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:19:37.196821 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e201e7a803ea453d956f01684a55efde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.199347 12738 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a9cac7524ae4180879338590f67b2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.173799 12743 tablet_service.cc:1511] Processing CreateTablet for tablet 55d5f1e66c7745ce9c59ea8361d55e74 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:19:37.217093 12664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a9cac7524ae4180879338590f67b2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.219854 12742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1f3b5d9156947e49d05b26ae1586797. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.230495 12663 tablet_service.cc:1511] Processing CreateTablet for tablet e201e7a803ea453d956f01684a55efde (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:19:37.232635 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e201e7a803ea453d956f01684a55efde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.235882 12743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d5f1e66c7745ce9c59ea8361d55e74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.264477 12587 tablet_service.cc:1511] Processing CreateTablet for tablet 512500cfa01b4614972925e2e54da037 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:19:37.267104 12587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512500cfa01b4614972925e2e54da037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.273243 12737 tablet_service.cc:1511] Processing CreateTablet for tablet e201e7a803ea453d956f01684a55efde (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:19:37.278789 12737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e201e7a803ea453d956f01684a55efde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.273375 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 512500cfa01b4614972925e2e54da037 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:19:37.302289 12736 tablet_service.cc:1511] Processing CreateTablet for tablet 512500cfa01b4614972925e2e54da037 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:19:37.313210 12736 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512500cfa01b4614972925e2e54da037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.307921 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512500cfa01b4614972925e2e54da037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.345245 12586 tablet_service.cc:1511] Processing CreateTablet for tablet 87698a3aa83b4b7f8ce0189a2a7b4807 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:19:37.349990 12586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87698a3aa83b4b7f8ce0189a2a7b4807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.385215 12661 tablet_service.cc:1511] Processing CreateTablet for tablet 87698a3aa83b4b7f8ce0189a2a7b4807 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:19:37.382102 12796 tablet_bootstrap.cc:492] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:37.396555 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87698a3aa83b4b7f8ce0189a2a7b4807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.393018 12735 tablet_service.cc:1511] Processing CreateTablet for tablet 87698a3aa83b4b7f8ce0189a2a7b4807 (DEFAULT_TABLE table=test-workload [id=1a58fe3436fb4e37b97b3e79f310b361]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:19:37.405948 12735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87698a3aa83b4b7f8ce0189a2a7b4807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:19:37.405984 12797 tablet_bootstrap.cc:492] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:37.467723 12798 tablet_bootstrap.cc:492] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:37.482985 12798 tablet_bootstrap.cc:654] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.491817 12629 heartbeater.cc:499] Master 127.12.45.126:39533 was elected leader, sending a full tablet report...
I20260709 12:19:37.503863 12797 tablet_bootstrap.cc:654] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.524132 12796 tablet_bootstrap.cc:654] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.532095 12798 tablet_bootstrap.cc:492] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:37.532727 12798 ts_tablet_manager.cc:1403] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.000s
I20260709 12:19:37.546147 12798 raft_consensus.cc:359] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.547096 12798 raft_consensus.cc:385] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.547586 12798 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.549820 12796 tablet_bootstrap.cc:492] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:37.550539 12796 ts_tablet_manager.cc:1403] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.193s	user 0.019s	sys 0.008s
I20260709 12:19:37.553665 12796 raft_consensus.cc:359] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.553907 12798 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.554514 12796 raft_consensus.cc:385] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.554847 12796 raft_consensus.cc:740] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.555684 12797 tablet_bootstrap.cc:492] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:37.556252 12797 ts_tablet_manager.cc:1403] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.168s	user 0.006s	sys 0.014s
I20260709 12:19:37.555847 12796 consensus_queue.cc:260] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.559839 12797 raft_consensus.cc:359] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.560709 12797 raft_consensus.cc:385] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.561054 12797 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.562027 12797 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.567508 12796 ts_tablet_manager.cc:1434] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.009s
I20260709 12:19:37.568856 12796 tablet_bootstrap.cc:492] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:37.569159 12797 ts_tablet_manager.cc:1434] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:19:37.570894 12778 heartbeater.cc:499] Master 127.12.45.126:39533 was elected leader, sending a full tablet report...
I20260709 12:19:37.571427 12797 tablet_bootstrap.cc:492] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:37.570382 12704 heartbeater.cc:499] Master 127.12.45.126:39533 was elected leader, sending a full tablet report...
I20260709 12:19:37.579284 12797 tablet_bootstrap.cc:654] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.588449 12796 tablet_bootstrap.cc:654] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.590430 12798 ts_tablet_manager.cc:1434] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.057s	user 0.012s	sys 0.010s
I20260709 12:19:37.592782 12798 tablet_bootstrap.cc:492] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:37.600018 12797 tablet_bootstrap.cc:492] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:37.600685 12797 ts_tablet_manager.cc:1403] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.012s
I20260709 12:19:37.604079 12797 raft_consensus.cc:359] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.605062 12797 raft_consensus.cc:385] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.605419 12797 raft_consensus.cc:740] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.606283 12797 consensus_queue.cc:260] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.609727 12798 tablet_bootstrap.cc:654] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.610054 12797 ts_tablet_manager.cc:1434] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:19:37.611351 12797 tablet_bootstrap.cc:492] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:37.615373 12796 tablet_bootstrap.cc:492] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:37.615967 12796 ts_tablet_manager.cc:1403] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.013s
I20260709 12:19:37.619395 12797 tablet_bootstrap.cc:654] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.619349 12796 raft_consensus.cc:359] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.620344 12796 raft_consensus.cc:385] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.620695 12796 raft_consensus.cc:740] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.621814 12796 consensus_queue.cc:260] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.628409 12796 ts_tablet_manager.cc:1434] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:19:37.629774 12796 tablet_bootstrap.cc:492] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:37.643251 12796 tablet_bootstrap.cc:654] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.644380 12797 tablet_bootstrap.cc:492] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
W20260709 12:19:37.645291 12779 tablet.cc:2406] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:19:37.646453 12798 tablet_bootstrap.cc:492] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:37.647116 12798 ts_tablet_manager.cc:1403] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.006s
I20260709 12:19:37.645256 12797 ts_tablet_manager.cc:1403] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.007s
I20260709 12:19:37.651772 12798 raft_consensus.cc:359] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.652959 12798 raft_consensus.cc:385] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.653391 12798 raft_consensus.cc:740] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.653942 12797 raft_consensus.cc:359] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:37.654811 12797 raft_consensus.cc:385] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.655224 12797 raft_consensus.cc:740] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.656059 12797 consensus_queue.cc:260] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:37.658699 12797 ts_tablet_manager.cc:1434] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:19:37.659850 12797 tablet_bootstrap.cc:492] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:37.661273 12796 tablet_bootstrap.cc:492] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:37.654843 12798 consensus_queue.cc:260] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.661823 12796 ts_tablet_manager.cc:1403] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.006s
I20260709 12:19:37.663545 12798 ts_tablet_manager.cc:1434] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.005s
I20260709 12:19:37.664776 12798 tablet_bootstrap.cc:492] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:37.673262 12797 tablet_bootstrap.cc:654] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.676964 12804 raft_consensus.cc:493] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.665227 12796 raft_consensus.cc:359] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.678791 12796 raft_consensus.cc:385] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.678593 12804 raft_consensus.cc:515] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.679108 12796 raft_consensus.cc:740] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.679941 12796 consensus_queue.cc:260] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.682677 12802 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.683226 12802 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.688484 12796 ts_tablet_manager.cc:1434] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.006s
I20260709 12:19:37.690356 12796 tablet_bootstrap.cc:492] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:37.708971 12804 leader_election.cc:290] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:37.709556 12802 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:37.722290 12804 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.725116 12804 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.727799 12804 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:37.726960 12798 tablet_bootstrap.cc:654] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.752903 12796 tablet_bootstrap.cc:654] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.776758 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
W20260709 12:19:37.779124 12719 leader_election.cc:343] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:37.781875 12796 tablet_bootstrap.cc:492] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:37.788805 12797 tablet_bootstrap.cc:492] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:37.789336 12798 tablet_bootstrap.cc:492] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:37.789397 12797 ts_tablet_manager.cc:1403] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.017s
I20260709 12:19:37.789928 12798 ts_tablet_manager.cc:1403] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.125s	user 0.008s	sys 0.007s
I20260709 12:19:37.792261 12796 ts_tablet_manager.cc:1403] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.006s
I20260709 12:19:37.792953 12797 raft_consensus.cc:359] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.796552 12796 raft_consensus.cc:359] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.797456 12796 raft_consensus.cc:385] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.797760 12796 raft_consensus.cc:740] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.798735 12796 consensus_queue.cc:260] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.801514 12796 ts_tablet_manager.cc:1434] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:19:37.802488 12798 raft_consensus.cc:359] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.803157 12796 tablet_bootstrap.cc:492] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:37.803289 12798 raft_consensus.cc:385] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.803632 12798 raft_consensus.cc:740] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.807623 12753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:37.807862 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:37.808825 12604 raft_consensus.cc:2468] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
I20260709 12:19:37.810808 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:37.810083 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:37.814153 12752 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
W20260709 12:19:37.816556 12643 leader_election.cc:343] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:37.817533 12645 leader_election.cc:304] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:37.817713 12643 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30; no voters: 
I20260709 12:19:37.818610 12804 raft_consensus.cc:2804] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:37.819008 12804 raft_consensus.cc:493] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.794056 12797 raft_consensus.cc:385] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.819416 12804 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.819451 12797 raft_consensus.cc:740] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.805011 12798 consensus_queue.cc:260] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
W20260709 12:19:37.827916 12645 leader_election.cc:343] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:37.828683 12804 raft_consensus.cc:515] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.831161 12804 leader_election.cc:290] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:37.831750 12804 raft_consensus.cc:2804] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:37.832100 12804 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.832477 12804 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.835613 12796 tablet_bootstrap.cc:654] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.836452 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:37.837131 12604 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.840134 12804 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.842504 12804 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:37.844165 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
W20260709 12:19:37.846195 12645 leader_election.cc:343] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:37.847846 12796 tablet_bootstrap.cc:492] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:37.848372 12796 ts_tablet_manager.cc:1403] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.004s
I20260709 12:19:37.849493 12604 raft_consensus.cc:2468] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:37.850983 12645 leader_election.cc:304] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:37.851785 12804 raft_consensus.cc:2804] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:37.852243 12804 raft_consensus.cc:697] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:37.853349 12798 ts_tablet_manager.cc:1434] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.063s	user 0.005s	sys 0.002s
I20260709 12:19:37.853132 12804 consensus_queue.cc:237] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.859056 12796 raft_consensus.cc:359] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.860000 12796 raft_consensus.cc:385] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.860317 12796 raft_consensus.cc:740] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.862170 12796 consensus_queue.cc:260] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.863353 12798 tablet_bootstrap.cc:492] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:37.865989 12797 consensus_queue.cc:260] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.875257 12796 ts_tablet_manager.cc:1434] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.003s
I20260709 12:19:37.876503 12796 tablet_bootstrap.cc:492] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
W20260709 12:19:37.884328 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:41687, user_credentials={real_user=slave}} blocked reactor thread for 66951.9us
I20260709 12:19:37.884356 12797 ts_tablet_manager.cc:1434] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.095s	user 0.012s	sys 0.000s
I20260709 12:19:37.886555 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:37.887425 12797 tablet_bootstrap.cc:492] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:37.888587 12796 tablet_bootstrap.cc:654] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.889251 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:37.889822 12752 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.888103 12498 catalog_manager.cc:5697] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:37.893857 12798 tablet_bootstrap.cc:654] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
W20260709 12:19:37.898483 12643 leader_election.cc:343] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:37.898854 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:37.901866 12679 raft_consensus.cc:2393] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5963ee1f8d1a437aa85e7371df8ac3bd in current term 1: Already voted for candidate 801602d913e44df6a7c0fb76bc53aa30 in this term.
I20260709 12:19:37.902633 12752 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:37.907697 12716 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd; no voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:37.909826 12802 raft_consensus.cc:2749] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:19:37.912827 12802 raft_consensus.cc:493] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:37.913296 12643 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30; no voters: a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:37.913998 12802 raft_consensus.cc:515] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.918068 12824 raft_consensus.cc:2804] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:37.918936 12824 raft_consensus.cc:697] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:37.920102 12798 tablet_bootstrap.cc:492] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:37.920364 12802 leader_election.cc:290] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:37.920935 12798 ts_tablet_manager.cc:1403] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.058s	user 0.010s	sys 0.009s
I20260709 12:19:37.919902 12824 consensus_queue.cc:237] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.921479 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:37.922575 12796 tablet_bootstrap.cc:492] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:37.923576 12796 ts_tablet_manager.cc:1403] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.014s
I20260709 12:19:37.928196 12604 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 0.
I20260709 12:19:37.929700 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:37.930851 12679 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 0.
I20260709 12:19:37.931008 12719 leader_election.cc:304] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:37.932520 12797 tablet_bootstrap.cc:654] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:37.933182 12802 raft_consensus.cc:2804] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:37.933862 12802 raft_consensus.cc:493] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:37.934726 12802 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.937156 12798 raft_consensus.cc:359] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.937778 12798 raft_consensus.cc:385] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.938062 12798 raft_consensus.cc:740] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.942095 12796 raft_consensus.cc:359] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:37.943244 12796 raft_consensus.cc:385] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:37.941062 12798 consensus_queue.cc:260] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.943656 12796 raft_consensus.cc:740] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:37.947777 12796 consensus_queue.cc:260] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:37.962898 12802 raft_consensus.cc:515] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:37.970501 12796 ts_tablet_manager.cc:1434] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.046s	user 0.011s	sys 0.005s
I20260709 12:19:37.975248 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:37.973806 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:37.976135 12604 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.976240 12679 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:37.976625 12796 tablet_bootstrap.cc:492] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:37.977551 12802 leader_election.cc:290] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:37.998395 12798 ts_tablet_manager.cc:1434] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.003s
I20260709 12:19:38.000032 12798 tablet_bootstrap.cc:492] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:38.003269 12604 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 1.
I20260709 12:19:38.008890 12824 raft_consensus.cc:493] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.010968 12796 tablet_bootstrap.cc:654] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.012260 12719 leader_election.cc:304] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.012113 12824 raft_consensus.cc:515] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:38.015998 12803 raft_consensus.cc:493] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.014763 12802 raft_consensus.cc:2804] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.016611 12803 raft_consensus.cc:515] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.019096 12802 raft_consensus.cc:697] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:38.027431 12803 leader_election.cc:290] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.028964 12802 consensus_queue.cc:237] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.041608 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.039237 12803 raft_consensus.cc:493] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.034376 12679 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 1.
W20260709 12:19:38.043383 12643 leader_election.cc:343] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.042789 12803 raft_consensus.cc:515] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.039793 12824 leader_election.cc:290] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687), a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905)
I20260709 12:19:38.052099 12803 leader_election.cc:290] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.052976 12803 raft_consensus.cc:493] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.054030 12797 tablet_bootstrap.cc:492] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:38.056980 12797 ts_tablet_manager.cc:1403] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.170s	user 0.019s	sys 0.030s
I20260709 12:19:38.053666 12803 raft_consensus.cc:515] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.065017 12798 tablet_bootstrap.cc:654] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.071657 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.074673 12803 leader_election.cc:290] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.075171 12802 raft_consensus.cc:493] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.077195 12604 raft_consensus.cc:2468] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
I20260709 12:19:38.077920 12831 raft_consensus.cc:493] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.079226 12802 raft_consensus.cc:515] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.082139 12645 leader_election.cc:304] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:38.084172 12824 raft_consensus.cc:2804] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:38.084816 12824 raft_consensus.cc:493] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.085276 12802 leader_election.cc:290] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:38.085372 12824 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.084844 12797 raft_consensus.cc:359] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.086367 12797 raft_consensus.cc:385] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.086997 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
W20260709 12:19:38.088829 12549 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:19:38.081593 12831 raft_consensus.cc:515] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.086885 12797 raft_consensus.cc:740] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.094702 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.095557 12604 raft_consensus.cc:2468] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 0.
I20260709 12:19:38.094817 12797 consensus_queue.cc:260] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.096762 12679 raft_consensus.cc:2468] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 0.
I20260709 12:19:38.099804 12719 leader_election.cc:304] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.103358 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:38.105151 12802 raft_consensus.cc:2804] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:38.106063 12802 raft_consensus.cc:493] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.106508 12802 raft_consensus.cc:3060] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:19:38.107021 12716 leader_election.cc:343] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.108964 12501 catalog_manager.cc:5697] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:38.110018 12796 tablet_bootstrap.cc:492] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:38.110504 12797 ts_tablet_manager.cc:1434] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.003s
I20260709 12:19:38.103945 12831 leader_election.cc:290] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:38.111788 12797 tablet_bootstrap.cc:492] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:38.110569 12796 ts_tablet_manager.cc:1403] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.134s	user 0.017s	sys 0.010s
I20260709 12:19:38.114843 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.115593 12604 raft_consensus.cc:2468] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 0.
I20260709 12:19:38.116971 12719 leader_election.cc:304] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:38.119477 12796 raft_consensus.cc:359] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.120342 12796 raft_consensus.cc:385] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.120867 12796 raft_consensus.cc:740] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.122726 12831 raft_consensus.cc:2804] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:38.126205 12831 raft_consensus.cc:493] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.126770 12831 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.128252 12824 raft_consensus.cc:515] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:38.129405 12802 raft_consensus.cc:515] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.131899 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:38.132817 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:38.133502 12604 raft_consensus.cc:3060] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:19:38.133811 12643 leader_election.cc:343] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.121853 12796 consensus_queue.cc:260] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.137133 12499 catalog_manager.cc:5697] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 1 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:38.139302 12824 leader_election.cc:290] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Requested vote from peers 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687), a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905)
I20260709 12:19:38.142580 12796 ts_tablet_manager.cc:1434] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:19:38.145442 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:38.146195 12603 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.147624 12604 raft_consensus.cc:2468] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 1.
I20260709 12:19:38.149940 12719 leader_election.cc:304] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.151362 12802 leader_election.cc:290] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:38.161357 12796 tablet_bootstrap.cc:492] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:38.161427 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:38.162828 12678 raft_consensus.cc:3060] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.162375 12802 raft_consensus.cc:2804] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.165269 12802 raft_consensus.cc:697] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:38.177597 12798 tablet_bootstrap.cc:492] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:38.178229 12798 ts_tablet_manager.cc:1403] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.179s	user 0.021s	sys 0.020s
I20260709 12:19:38.179306 12603 raft_consensus.cc:2468] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:38.182404 12645 leader_election.cc:304] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:38.183320 12824 raft_consensus.cc:2804] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.183657 12678 raft_consensus.cc:2468] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 1.
I20260709 12:19:38.183897 12824 raft_consensus.cc:697] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:38.186602 12824 consensus_queue.cc:237] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:38.192392 12797 tablet_bootstrap.cc:654] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.195389 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:38.195546 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.198104 12753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.198935 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:38.198311 12802 consensus_queue.cc:237] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
W20260709 12:19:38.206593 12567 leader_election.cc:343] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.193244 12831 raft_consensus.cc:515] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.202051 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
W20260709 12:19:38.207448 12567 leader_election.cc:343] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.209450 12753 raft_consensus.cc:2393] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a37ecd8998584be68aafbd4ef66c3d84 in current term 1: Already voted for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in this term.
I20260709 12:19:38.202950 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:38.213526 12798 raft_consensus.cc:359] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.214359 12798 raft_consensus.cc:385] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.218992 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:38.215036 12798 raft_consensus.cc:740] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.219370 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:38.221099 12798 consensus_queue.cc:260] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.221941 12603 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:19:38.223120 12716 leader_election.cc:343] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.215341 12678 raft_consensus.cc:2393] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a37ecd8998584be68aafbd4ef66c3d84 in current term 1: Already voted for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in this term.
I20260709 12:19:38.217658 12831 leader_election.cc:290] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:38.225723 12567 leader_election.cc:304] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:38.227809 12803 raft_consensus.cc:2749] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:19:38.229087 12568 leader_election.cc:343] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.229588 12568 leader_election.cc:304] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:38.230825 12803 raft_consensus.cc:2749] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:19:38.235584 12568 leader_election.cc:304] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:38.237867 12798 ts_tablet_manager.cc:1434] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.004s
I20260709 12:19:38.239462 12798 tablet_bootstrap.cc:492] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:38.239080 12499 catalog_manager.cc:5697] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:38.243516 12797 tablet_bootstrap.cc:492] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:38.244264 12797 ts_tablet_manager.cc:1403] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.133s	user 0.006s	sys 0.010s
I20260709 12:19:38.245091 12603 raft_consensus.cc:2468] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 1.
I20260709 12:19:38.245844 12803 raft_consensus.cc:2749] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:19:38.249011 12796 tablet_bootstrap.cc:654] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.250874 12719 leader_election.cc:304] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:38.252369 12831 raft_consensus.cc:2804] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.253055 12831 raft_consensus.cc:697] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:38.257395 12797 raft_consensus.cc:359] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.257982 12831 consensus_queue.cc:237] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.264107 12499 catalog_manager.cc:5697] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 1 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:38.268957 12803 raft_consensus.cc:493] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.269557 12803 raft_consensus.cc:515] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.269548 12797 raft_consensus.cc:385] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.270572 12797 raft_consensus.cc:740] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.271873 12803 leader_election.cc:290] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.273814 12753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.271495 12797 consensus_queue.cc:260] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.264483 12798 tablet_bootstrap.cc:654] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.276809 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
W20260709 12:19:38.276980 12568 leader_election.cc:343] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:19:38.278841 12567 leader_election.cc:343] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:19:38.279127 12797 ts_tablet_manager.cc:1434] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.008s
I20260709 12:19:38.279327 12567 leader_election.cc:304] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a37ecd8998584be68aafbd4ef66c3d84; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:38.280229 12797 tablet_bootstrap.cc:492] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:38.281414 12803 raft_consensus.cc:2749] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:19:38.287992 12798 tablet_bootstrap.cc:492] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:38.288558 12798 ts_tablet_manager.cc:1403] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.004s
I20260709 12:19:38.293414 12824 consensus_queue.cc:1048] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:19:38.294277 12798 raft_consensus.cc:359] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.295001 12798 raft_consensus.cc:385] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.295310 12798 raft_consensus.cc:740] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.302186 12798 consensus_queue.cc:260] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.304527 12501 catalog_manager.cc:5697] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 1 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:38.306334 12798 ts_tablet_manager.cc:1434] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:19:38.307436 12798 tablet_bootstrap.cc:492] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:38.309708 12796 tablet_bootstrap.cc:492] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:38.310334 12796 ts_tablet_manager.cc:1403] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.013s
I20260709 12:19:38.314033 12796 raft_consensus.cc:359] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.315143 12797 tablet_bootstrap.cc:654] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.314795 12796 raft_consensus.cc:385] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.316817 12796 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.317536 12796 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.326301 12798 tablet_bootstrap.cc:654] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.330082 12796 ts_tablet_manager.cc:1434] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:19:38.331187 12796 tablet_bootstrap.cc:492] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: Bootstrap starting.
I20260709 12:19:38.339893 12753 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.343963 12796 tablet_bootstrap.cc:654] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.351608 12804 consensus_queue.cc:1048] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:38.352380 12603 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.362247 12804 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:38.377043 12796 tablet_bootstrap.cc:492] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: No bootstrap required, opened a new log
I20260709 12:19:38.377660 12796 ts_tablet_manager.cc:1403] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.005s
I20260709 12:19:38.387442 12796 raft_consensus.cc:359] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.388273 12796 raft_consensus.cc:385] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.388538 12796 raft_consensus.cc:740] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.389813 12796 consensus_queue.cc:260] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.402782 12796 ts_tablet_manager.cc:1434] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.001s
I20260709 12:19:38.402204 12804 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:19:38.409993 12797 tablet_bootstrap.cc:492] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:38.410643 12797 ts_tablet_manager.cc:1403] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.131s	user 0.010s	sys 0.023s
I20260709 12:19:38.413049 12798 tablet_bootstrap.cc:492] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:38.418310 12797 raft_consensus.cc:359] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.419312 12797 raft_consensus.cc:385] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.419664 12797 raft_consensus.cc:740] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.420719 12797 consensus_queue.cc:260] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.424948 12797 ts_tablet_manager.cc:1434] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:19:38.427634 12798 ts_tablet_manager.cc:1403] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.120s	user 0.030s	sys 0.019s
I20260709 12:19:38.436415 12797 tablet_bootstrap.cc:492] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:38.448884 12831 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:38.457863 12803 raft_consensus.cc:493] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.458487 12803 raft_consensus.cc:515] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.468550 12803 leader_election.cc:290] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.471822 12839 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:19:38.472710 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:38.473546 12797 tablet_bootstrap.cc:654] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.473446 12678 raft_consensus.cc:2468] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 0.
I20260709 12:19:38.457228 12798 raft_consensus.cc:359] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:38.475422 12567 leader_election.cc:304] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.475699 12753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.476859 12798 raft_consensus.cc:385] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.478740 12836 raft_consensus.cc:2804] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:38.485816 12836 raft_consensus.cc:493] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.485922 12821 raft_consensus.cc:493] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.486578 12836 raft_consensus.cc:3060] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.481508 12753 raft_consensus.cc:2468] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 0.
I20260709 12:19:38.487037 12821 raft_consensus.cc:515] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.492748 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.493711 12603 raft_consensus.cc:2393] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 801602d913e44df6a7c0fb76bc53aa30 in current term 1: Already voted for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in this term.
I20260709 12:19:38.488056 12798 raft_consensus.cc:740] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.493450 12821 leader_election.cc:290] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.496816 12798 consensus_queue.cc:260] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:38.501796 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.503304 12798 ts_tablet_manager.cc:1434] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.069s	user 0.010s	sys 0.001s
I20260709 12:19:38.504439 12798 tablet_bootstrap.cc:492] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:38.504679 12643 leader_election.cc:304] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30; no voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:38.507681 12856 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.528442 12797 tablet_bootstrap.cc:492] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:38.529031 12797 ts_tablet_manager.cc:1403] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.093s	user 0.027s	sys 0.019s
I20260709 12:19:38.531687 12836 raft_consensus.cc:515] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.533901 12836 leader_election.cc:290] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.534969 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:38.539301 12677 raft_consensus.cc:3060] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.539942 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:38.540556 12751 raft_consensus.cc:3060] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.536648 12856 raft_consensus.cc:2749] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:19:38.538628 12798 tablet_bootstrap.cc:654] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.538337 12797 raft_consensus.cc:359] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.543817 12797 raft_consensus.cc:385] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.544127 12797 raft_consensus.cc:740] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.544847 12797 consensus_queue.cc:260] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.546370 12677 raft_consensus.cc:2468] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 1.
I20260709 12:19:38.547217 12797 ts_tablet_manager.cc:1434] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:19:38.547663 12567 leader_election.cc:304] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.548156 12797 tablet_bootstrap.cc:492] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: Bootstrap starting.
I20260709 12:19:38.548945 12836 raft_consensus.cc:2804] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.549736 12836 raft_consensus.cc:697] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:38.550741 12836 consensus_queue.cc:237] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.558501 12797 tablet_bootstrap.cc:654] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.579708 12839 raft_consensus.cc:493] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.580550 12839 raft_consensus.cc:515] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:38.584650 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801602d913e44df6a7c0fb76bc53aa30" is_pre_election: true
I20260709 12:19:38.584650 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.586617 12603 raft_consensus.cc:2393] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5963ee1f8d1a437aa85e7371df8ac3bd in current term 1: Already voted for candidate 801602d913e44df6a7c0fb76bc53aa30 in this term.
I20260709 12:19:38.588581 12716 leader_election.cc:304] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd; no voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:38.584875 12839 leader_election.cc:290] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905)
I20260709 12:19:38.589416 12798 tablet_bootstrap.cc:492] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:38.589511 12831 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.589888 12798 ts_tablet_manager.cc:1403] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.086s	user 0.010s	sys 0.029s
I20260709 12:19:38.592892 12798 raft_consensus.cc:359] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.593583 12798 raft_consensus.cc:385] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.593840 12798 raft_consensus.cc:740] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.594677 12798 consensus_queue.cc:260] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.596692 12798 ts_tablet_manager.cc:1434] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:19:38.597803 12798 tablet_bootstrap.cc:492] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: Bootstrap starting.
I20260709 12:19:38.599026 12751 raft_consensus.cc:2468] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 1.
I20260709 12:19:38.600469 12831 raft_consensus.cc:2749] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:19:38.605312 12798 tablet_bootstrap.cc:654] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: Neither blocks nor log segments found. Creating new log.
I20260709 12:19:38.612383 12797 tablet_bootstrap.cc:492] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: No bootstrap required, opened a new log
I20260709 12:19:38.612972 12797 ts_tablet_manager.cc:1403] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.020s
I20260709 12:19:38.617234 12797 raft_consensus.cc:359] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.617036 12499 catalog_manager.cc:5697] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 0 to 1, leader changed from <none> to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:38.618396 12797 raft_consensus.cc:385] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.618810 12797 raft_consensus.cc:740] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.619714 12797 consensus_queue.cc:260] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.627609 12797 ts_tablet_manager.cc:1434] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.005s
I20260709 12:19:38.639101 12798 tablet_bootstrap.cc:492] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: No bootstrap required, opened a new log
I20260709 12:19:38.639617 12798 ts_tablet_manager.cc:1403] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.020s
I20260709 12:19:38.641970 12798 raft_consensus.cc:359] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.642657 12798 raft_consensus.cc:385] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:19:38.642894 12798 raft_consensus.cc:740] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Initialized, Role: FOLLOWER
I20260709 12:19:38.644075 12798 consensus_queue.cc:260] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.645860 12798 ts_tablet_manager.cc:1434] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:19:38.666332 12831 consensus_queue.cc:1048] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:38.683282 12831 consensus_queue.cc:1048] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:19:38.707383 12856 consensus_queue.cc:1048] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:19:38.716291 12831 consensus_queue.cc:1048] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:38.720017 12856 consensus_queue.cc:1048] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:19:38.747409 12802 consensus_queue.cc:1048] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:38.816918 12856 raft_consensus.cc:493] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.817539 12856 raft_consensus.cc:515] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.819464 12856 leader_election.cc:290] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.820545 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.820672 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.821177 12603 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
I20260709 12:19:38.821358 12751 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
I20260709 12:19:38.822461 12645 leader_election.cc:304] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.823344 12856 raft_consensus.cc:2804] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:38.823727 12856 raft_consensus.cc:493] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.824057 12856 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.831686 12856 raft_consensus.cc:515] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.834008 12856 leader_election.cc:290] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.835357 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:38.835357 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:38.836288 12751 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.836283 12603 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.843698 12603 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:38.843708 12751 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:38.845103 12645 leader_election.cc:304] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.845880 12856 raft_consensus.cc:2804] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.846452 12856 raft_consensus.cc:697] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:38.847240 12856 consensus_queue.cc:237] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.856490 12500 catalog_manager.cc:5697] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:38.933861 12856 raft_consensus.cc:493] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:19:38.934653 12856 raft_consensus.cc:515] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.936925 12856 leader_election.cc:290] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.938004 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84" is_pre_election: true
I20260709 12:19:38.938126 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" is_pre_election: true
I20260709 12:19:38.938725 12603 raft_consensus.cc:2468] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
I20260709 12:19:38.938874 12751 raft_consensus.cc:2468] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 0.
I20260709 12:19:38.940294 12645 leader_election.cc:304] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.941032 12856 raft_consensus.cc:2804] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:19:38.941399 12856 raft_consensus.cc:493] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:19:38.941763 12856 raft_consensus.cc:3060] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.949646 12856 raft_consensus.cc:515] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.951572 12856 leader_election.cc:290] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:38.952553 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:38.953351 12603 raft_consensus.cc:3060] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.953352 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:38.954092 12751 raft_consensus.cc:3060] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:19:38.961597 12751 raft_consensus.cc:2468] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:38.961597 12603 raft_consensus.cc:2468] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 1.
I20260709 12:19:38.963245 12645 leader_election.cc:304] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:38.964015 12856 raft_consensus.cc:2804] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:19:38.965247 12856 raft_consensus.cc:697] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:38.966653 12856 consensus_queue.cc:237] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:38.975134 12500 catalog_manager.cc:5697] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.008608 12836 consensus_queue.cc:1048] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:39.025638 12803 consensus_queue.cc:1048] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:39.353461 12677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:55622
I20260709 12:19:39.354195 12677 raft_consensus.cc:606] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:39.363186 12856 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:39.364358 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:55622
I20260709 12:19:39.365068 12678 raft_consensus.cc:606] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:39.374570 12677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a64ffb6456946b7b33340731974841e"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:55622
I20260709 12:19:39.375620 12677 raft_consensus.cc:606] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:39.381512 12856 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:19:39.391021 12855 raft_consensus.cc:993] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.391601 12855 raft_consensus.cc:1081] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.396317 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a64ffb6456946b7b33340731974841e"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33384
I20260709 12:19:39.397022 12604 raft_consensus.cc:493] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.397439 12604 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.406422 12604 raft_consensus.cc:515] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.410104 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:39.410238 12753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "744469240da84c15832aa2f583308505"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:41934
I20260709 12:19:39.410920 12677 raft_consensus.cc:3055] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.411392 12677 raft_consensus.cc:740] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:39.410975 12753 raft_consensus.cc:606] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:39.412289 12677 consensus_queue.cc:260] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.413754 12677 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.420595 12753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:41934
I20260709 12:19:39.421392 12753 raft_consensus.cc:606] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:39.421496 12677 raft_consensus.cc:2468] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.423768 12604 leader_election.cc:290] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:39.424353 12753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:39.425004 12753 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.425962 12567 leader_election.cc:304] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.426936 12874 raft_consensus.cc:2804] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.427378 12874 raft_consensus.cc:697] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:39.428198 12874 consensus_queue.cc:237] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.429881 12751 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:41934
I20260709 12:19:39.430788 12751 raft_consensus.cc:606] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:39.440685 12753 raft_consensus.cc:2468] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.441165 12855 consensus_queue.cc:1048] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:39.465236 12821 consensus_queue.cc:1048] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:19:39.471530 12603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
mode: GRACEFUL
new_leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:33390
I20260709 12:19:39.472371 12603 raft_consensus.cc:606] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 LEADER]: Received request to transfer leadership to TS 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:39.471645 12500 catalog_manager.cc:5697] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 1 to 2, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.515143 12803 raft_consensus.cc:993] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: : Instructing follower 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:39.515980 12874 raft_consensus.cc:1081] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 LEADER]: Signalling peer 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:39.518070 12751 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:41928
I20260709 12:19:39.518798 12751 raft_consensus.cc:493] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.519215 12751 raft_consensus.cc:3060] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.525456 12751 raft_consensus.cc:515] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.527516 12751 leader_election.cc:290] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 2 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:39.528708 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:39.529112 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87698a3aa83b4b7f8ce0189a2a7b4807" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:39.529309 12604 raft_consensus.cc:3055] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.529778 12604 raft_consensus.cc:740] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:39.529743 12677 raft_consensus.cc:3060] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.530776 12604 consensus_queue.cc:260] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.531800 12604 raft_consensus.cc:3060] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.537698 12604 raft_consensus.cc:2468] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 2.
I20260709 12:19:39.539081 12719 leader_election.cc:304] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.539266 12677 raft_consensus.cc:2468] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 2.
I20260709 12:19:39.540169 12802 raft_consensus.cc:2804] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.540778 12802 raft_consensus.cc:697] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:39.541765 12802 consensus_queue.cc:237] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.549185 12855 raft_consensus.cc:993] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.549703 12855 raft_consensus.cc:1081] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.552096 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33384
I20260709 12:19:39.551501 12500 catalog_manager.cc:5697] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 1 to 2, leader changed from a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65) to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 2 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:39.552891 12604 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.553431 12604 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.559836 12604 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.564078 12604 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:39.564199 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:39.563413 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:39.564834 12751 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.565296 12677 raft_consensus.cc:3055] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.566349 12677 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:39.567600 12677 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.568985 12677 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.573158 12751 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.574426 12568 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.575165 12874 raft_consensus.cc:2804] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.575603 12874 raft_consensus.cc:697] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:39.576514 12874 consensus_queue.cc:237] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.577117 12677 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.586325 12500 catalog_manager.cc:5697] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 1 to 2, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.589352 12839 raft_consensus.cc:993] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.590080 12839 raft_consensus.cc:1081] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.592221 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "744469240da84c15832aa2f583308505"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33370
I20260709 12:19:39.592896 12604 raft_consensus.cc:493] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.593321 12604 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.601900 12604 raft_consensus.cc:515] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.604351 12604 leader_election.cc:290] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:39.605551 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:39.605563 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:39.606366 12677 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.606366 12751 raft_consensus.cc:3055] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.607051 12751 raft_consensus.cc:740] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:39.608043 12751 consensus_queue.cc:260] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.609494 12751 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.615132 12677 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.616636 12567 leader_election.cc:304] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.617600 12874 raft_consensus.cc:2804] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.617700 12751 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.618232 12874 raft_consensus.cc:697] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:39.619738 12874 consensus_queue.cc:237] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.629632 12500 catalog_manager.cc:5697] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 1 to 2, leader changed from 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.681231 12802 raft_consensus.cc:993] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:39.681784 12802 raft_consensus.cc:1081] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:39.683722 12677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55608
I20260709 12:19:39.684309 12677 raft_consensus.cc:493] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.684664 12677 raft_consensus.cc:3060] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.691800 12677 raft_consensus.cc:515] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.694016 12677 leader_election.cc:290] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 2 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:39.695112 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:39.695300 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a9cac7524ae4180879338590f67b2c7" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:39.695689 12604 raft_consensus.cc:3060] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.696020 12751 raft_consensus.cc:3055] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.696406 12751 raft_consensus.cc:740] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:39.697163 12751 consensus_queue.cc:260] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.698148 12751 raft_consensus.cc:3060] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.702719 12604 raft_consensus.cc:2468] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 2.
I20260709 12:19:39.703891 12751 raft_consensus.cc:2468] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 2.
I20260709 12:19:39.704000 12645 leader_election.cc:304] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.704879 12821 raft_consensus.cc:2804] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.705384 12821 raft_consensus.cc:697] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:39.706302 12821 consensus_queue.cc:237] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.714082 12500 catalog_manager.cc:5697] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 1 to 2, leader changed from 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.780879 12839 raft_consensus.cc:993] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:39.781417 12839 raft_consensus.cc:1081] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:39.783156 12677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55608
I20260709 12:19:39.783739 12677 raft_consensus.cc:493] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.784170 12677 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.789649 12677 raft_consensus.cc:515] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.791769 12677 leader_election.cc:290] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 2 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:39.792732 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:39.792930 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:39.793293 12604 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.793566 12751 raft_consensus.cc:3055] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.793898 12751 raft_consensus.cc:740] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:39.794991 12751 consensus_queue.cc:260] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.796412 12751 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.799348 12604 raft_consensus.cc:2468] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 2.
I20260709 12:19:39.800514 12645 leader_election.cc:304] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.801527 12855 raft_consensus.cc:2804] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.802022 12855 raft_consensus.cc:697] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:39.802837 12855 consensus_queue.cc:237] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:39.803923 12751 raft_consensus.cc:2468] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 2.
I20260709 12:19:39.812412 12500 catalog_manager.cc:5697] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 1 to 2, leader changed from 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:39.816634 12677 raft_consensus.cc:1275] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:39.821369 12856 raft_consensus.cc:993] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.821676 12874 consensus_queue.cc:1048] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:39.822428 12856 raft_consensus.cc:1081] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:39.824693 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33384
I20260709 12:19:39.825572 12604 raft_consensus.cc:493] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:39.825927 12604 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.832147 12751 raft_consensus.cc:1275] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:39.835369 12875 consensus_queue.cc:1048] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:39.837662 12604 raft_consensus.cc:515] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:39.843161 12604 leader_election.cc:290] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:39.843974 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:39.844638 12677 raft_consensus.cc:3055] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:39.844995 12677 raft_consensus.cc:740] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:39.845875 12677 consensus_queue.cc:260] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:39.847769 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:39.848728 12751 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.848767 12677 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:39.855443 12677 raft_consensus.cc:2468] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.856742 12751 raft_consensus.cc:2468] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 2.
I20260709 12:19:39.856882 12567 leader_election.cc:304] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:39.858156 12875 raft_consensus.cc:2804] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:39.858698 12875 raft_consensus.cc:697] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:39.859529 12875 consensus_queue.cc:237] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:39.867188 12501 catalog_manager.cc:5697] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 1 to 2, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:39.929059 12604 raft_consensus.cc:1275] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:39.930687 12839 consensus_queue.cc:1048] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:39.939438 12677 raft_consensus.cc:1275] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:39.941275 12839 consensus_queue.cc:1048] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:39.965045 12677 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:39.966781 12875 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:39.977131 12751 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:39.978808 12803 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.083120 12751 raft_consensus.cc:1275] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.084870 12803 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.095233 12677 raft_consensus.cc:1275] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.096947 12803 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.175171 12604 raft_consensus.cc:1275] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.176695 12855 consensus_queue.cc:1048] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.186365 12751 raft_consensus.cc:1275] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.188406 12821 consensus_queue.cc:1048] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.285569 12677 raft_consensus.cc:1275] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.287009 12803 consensus_queue.cc:1048] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.299839 12751 raft_consensus.cc:1275] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.299839 12753 raft_consensus.cc:1275] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.301883 12604 raft_consensus.cc:1275] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:40.302467 12804 consensus_queue.cc:1048] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:40.302089 12875 consensus_queue.cc:1048] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:19:40.303879 12821 consensus_queue.cc:1048] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:19:42.512307 12469 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:19:42.513208 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:42.537896 12677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "512500cfa01b4614972925e2e54da037"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:55624
I20260709 12:19:42.538664 12677 raft_consensus.cc:606] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Received request to transfer leadership to TS 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:42.551098 12604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
mode: GRACEFUL
new_leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:33402
I20260709 12:19:42.551658 12604 raft_consensus.cc:606] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Received request to transfer leadership to TS 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:42.553018 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:19:42.553709 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:42.910012 12908 raft_consensus.cc:993] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: : Instructing follower 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:42.910540 12908 raft_consensus.cc:1081] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Signalling peer 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:42.912366 12753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:41928
I20260709 12:19:42.912868 12753 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:42.913141 12753 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:42.918541 12753 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:42.920423 12753 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 3 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:42.921476 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:42.921666 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:42.922001 12604 raft_consensus.cc:3055] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:19:42.922405 12604 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:42.922418 12677 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:42.923183 12604 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:42.924085 12604 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:42.928443 12604 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 3.
I20260709 12:19:42.928638 12677 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 3.
I20260709 12:19:42.929751 12719 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:42.930744 12906 raft_consensus.cc:2804] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:19:42.932019 12906 raft_consensus.cc:697] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:42.932904 12906 consensus_queue.cc:237] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:42.940768 12499 catalog_manager.cc:5697] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 2 to 3, leader changed from a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65) to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 3 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:43.010592 12855 raft_consensus.cc:993] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:43.011122 12855 raft_consensus.cc:1081] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Signalling peer 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:43.012876 12753 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "512500cfa01b4614972925e2e54da037"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:41916
I20260709 12:19:43.013549 12753 raft_consensus.cc:493] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:43.013897 12753 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:43.019575 12753 raft_consensus.cc:515] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:43.021597 12753 leader_election.cc:290] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 2 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:43.022559 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:43.022890 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:43.023173 12604 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:43.023613 12677 raft_consensus.cc:3055] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:43.023962 12677 raft_consensus.cc:740] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:43.024685 12677 consensus_queue.cc:260] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:43.025619 12677 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:43.028453 12604 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 2.
I20260709 12:19:43.029959 12719 leader_election.cc:304] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:43.030979 12906 raft_consensus.cc:2804] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:43.031044 12677 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 2.
I20260709 12:19:43.031584 12906 raft_consensus.cc:697] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:43.032444 12906 consensus_queue.cc:237] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:43.040120 12499 catalog_manager.cc:5697] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 1 to 2, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 2 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:43.469463 12677 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:43.470934 12906 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:43.483225 12603 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:43.485312 12912 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:43.535821 12677 raft_consensus.cc:1275] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:43.537055 12922 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:43.549258 12604 raft_consensus.cc:1275] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:43.550921 12927 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:43.554941 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:43.562788 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:43.563295 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:44.566200 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:44.577368 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:44.577876 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:45.579136 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:45.585760 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:45.586153 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:46.587194 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:46.593534 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:46.594024 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:47.595053 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:47.600232 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:47.600821 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:48.602025 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:48.608569 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:48.609002 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:49.610035 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:49.615428 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:49.616011 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:50.617082 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:50.623757 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:50.624261 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:51.625222 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:51.630228 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:51.630777 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:52.661633 12469 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:19:52.667306 12604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:33390
I20260709 12:19:52.668051 12604 raft_consensus.cc:606] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:52.673364 12753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:41934
I20260709 12:19:52.673840 12753 raft_consensus.cc:606] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:52.678980 12604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a64ffb6456946b7b33340731974841e"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:33390
I20260709 12:19:52.679548 12604 raft_consensus.cc:606] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:52.684715 12604 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "744469240da84c15832aa2f583308505"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:33390
I20260709 12:19:52.685168 12604 raft_consensus.cc:606] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:52.698794 12753 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "512500cfa01b4614972925e2e54da037"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
mode: GRACEFUL
new_leader_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:41934
I20260709 12:19:52.699452 12753 raft_consensus.cc:606] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Received request to transfer leadership to TS 801602d913e44df6a7c0fb76bc53aa30
I20260709 12:19:52.724408 12941 raft_consensus.cc:993] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:52.724951 12941 raft_consensus.cc:1081] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:52.726794 12677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55608
I20260709 12:19:52.727340 12677 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:52.727766 12677 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:52.733119 12677 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:52.735589 12677 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 4 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:52.736775 12753 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:52.736678 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:52.737469 12753 raft_consensus.cc:3055] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:19:52.737551 12604 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:52.737910 12753 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:52.738765 12753 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:52.740077 12753 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:52.744817 12604 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 4.
I20260709 12:19:52.746496 12645 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:52.747362 12753 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 4.
I20260709 12:19:52.747524 12935 raft_consensus.cc:2804] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:19:52.749011 12935 raft_consensus.cc:697] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:52.750623 12935 consensus_queue.cc:237] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:52.758128 12500 catalog_manager.cc:5697] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 3 to 4, leader changed from 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 4 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:53.013661 12940 raft_consensus.cc:993] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.014142 12940 raft_consensus.cc:1081] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.015620 12677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "512500cfa01b4614972925e2e54da037"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55608
I20260709 12:19:53.016224 12677 raft_consensus.cc:493] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:53.016556 12677 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.021646 12677 raft_consensus.cc:515] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.023219 12677 leader_election.cc:290] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:53.024093 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:53.024346 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:53.024747 12604 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.024783 12751 raft_consensus.cc:3055] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:19:53.025247 12751 raft_consensus.cc:740] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:53.025978 12751 consensus_queue.cc:260] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.027125 12751 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.031140 12604 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.032409 12645 leader_election.cc:304] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:53.032981 12751 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.033363 12935 raft_consensus.cc:2804] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:19:53.033855 12935 raft_consensus.cc:697] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:53.034715 12935 consensus_queue.cc:237] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.040016 12949 raft_consensus.cc:993] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.040596 12938 raft_consensus.cc:1081] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.042232 12677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "744469240da84c15832aa2f583308505"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55612
I20260709 12:19:53.042796 12677 raft_consensus.cc:493] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:53.043308 12677 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.042757 12500 catalog_manager.cc:5697] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 2 to 3, leader changed from 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:53.049755 12677 raft_consensus.cc:515] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.052438 12677 leader_election.cc:290] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:53.052594 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:53.052851 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:53.053186 12751 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.053720 12604 raft_consensus.cc:3055] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:19:53.053988 12604 raft_consensus.cc:740] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:53.054738 12604 consensus_queue.cc:260] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.055961 12604 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.058148 12751 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.059314 12643 leader_election.cc:304] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30; no voters: 
I20260709 12:19:53.060086 12935 raft_consensus.cc:2804] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:19:53.060618 12935 raft_consensus.cc:697] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:53.061319 12604 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.061432 12935 consensus_queue.cc:237] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.069702 12500 catalog_manager.cc:5697] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 2 to 3, leader changed from a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:53.096915 12949 raft_consensus.cc:993] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.097433 12938 raft_consensus.cc:1081] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.099084 12677 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55612
I20260709 12:19:53.099531 12677 raft_consensus.cc:493] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:53.099771 12677 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.105746 12677 raft_consensus.cc:515] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:53.105976 12949 raft_consensus.cc:993] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: : Instructing follower 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.106653 12938 raft_consensus.cc:1081] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Signalling peer 801602d913e44df6a7c0fb76bc53aa30 to start an election
I20260709 12:19:53.107717 12677 leader_election.cc:290] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Requested vote from peers 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687), a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905)
I20260709 12:19:53.108551 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:53.108794 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d5f1e66c7745ce9c59ea8361d55e74" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:53.108546 12678 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a64ffb6456946b7b33340731974841e"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
 from {username='slave'} at 127.0.0.1:55612
I20260709 12:19:53.109511 12751 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.109637 12604 raft_consensus.cc:3055] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:19:53.109756 12678 raft_consensus.cc:493] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:53.110085 12604 raft_consensus.cc:740] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:53.110301 12678 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.111104 12604 consensus_queue.cc:260] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:53.112023 12604 raft_consensus.cc:3060] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.116547 12751 raft_consensus.cc:2468] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.117316 12678 raft_consensus.cc:515] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.117779 12643 leader_election.cc:304] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, 801602d913e44df6a7c0fb76bc53aa30; no voters: 
I20260709 12:19:53.117869 12604 raft_consensus.cc:2468] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.118660 12935 raft_consensus.cc:2804] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:19:53.119210 12935 raft_consensus.cc:697] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:53.119583 12678 leader_election.cc:290] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:53.120132 12935 consensus_queue.cc:237] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } }
I20260709 12:19:53.121027 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:53.121083 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a64ffb6456946b7b33340731974841e" candidate_uuid: "801602d913e44df6a7c0fb76bc53aa30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:53.121675 12751 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.121835 12604 raft_consensus.cc:3055] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:19:53.122323 12604 raft_consensus.cc:740] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:53.123087 12604 consensus_queue.cc:260] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.123993 12604 raft_consensus.cc:3060] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:53.128101 12604 raft_consensus.cc:2468] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.129456 12645 leader_election.cc:304] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801602d913e44df6a7c0fb76bc53aa30, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:53.130419 12935 raft_consensus.cc:2804] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:19:53.130928 12935 raft_consensus.cc:697] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Becoming Leader. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:53.131964 12751 raft_consensus.cc:2468] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801602d913e44df6a7c0fb76bc53aa30 in term 3.
I20260709 12:19:53.131852 12935 consensus_queue.cc:237] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:53.133813 12500 catalog_manager.cc:5697] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 2 to 3, leader changed from a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:53.147689 12500 catalog_manager.cc:5697] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 reported cstate change: term changed from 2 to 3, leader changed from a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65) to 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "801602d913e44df6a7c0fb76bc53aa30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:53.160758 12604 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:53.162375 12935 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:19:53.177206 12751 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:53.178431 12935 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:19:53.440475 12751 raft_consensus.cc:1275] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.441860 12935 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:53.453047 12604 raft_consensus.cc:1275] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.455489 12942 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:19:53.458554 12604 raft_consensus.cc:1275] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.461045 12942 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:19:53.471325 12751 raft_consensus.cc:1275] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.473775 12942 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:53.551705 12751 raft_consensus.cc:1275] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.553011 12962 consensus_queue.cc:1048] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:53.563086 12604 raft_consensus.cc:1275] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.564869 12962 consensus_queue.cc:1048] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:19:53.634326 12751 raft_consensus.cc:1275] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.635547 12962 consensus_queue.cc:1048] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:53.645926 12604 raft_consensus.cc:1275] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Refusing update from remote peer 801602d913e44df6a7c0fb76bc53aa30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:53.647537 12962 consensus_queue.cc:1048] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:55.736652 12469 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:19:55.737623 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:55.745040 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "512500cfa01b4614972925e2e54da037"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:55624
I20260709 12:19:55.745759 12678 raft_consensus.cc:606] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:55.748976 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "744469240da84c15832aa2f583308505"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:55624
I20260709 12:19:55.749450 12678 raft_consensus.cc:606] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Received request to transfer leadership to TS 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:55.752364 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:55624
I20260709 12:19:55.752933 12678 raft_consensus.cc:606] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:55.755782 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:55624
I20260709 12:19:55.756266 12678 raft_consensus.cc:606] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Received request to transfer leadership to TS a37ecd8998584be68aafbd4ef66c3d84
I20260709 12:19:55.759163 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e201e7a803ea453d956f01684a55efde"
dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
mode: GRACEFUL
new_leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:55624
I20260709 12:19:55.759750 12678 raft_consensus.cc:606] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Received request to transfer leadership to TS 5963ee1f8d1a437aa85e7371df8ac3bd
I20260709 12:19:55.761063 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:19:55.761583 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:55.825635 12962 raft_consensus.cc:993] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:55.826112 12962 raft_consensus.cc:1081] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:55.827780 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "512500cfa01b4614972925e2e54da037"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33384
I20260709 12:19:55.828280 12604 raft_consensus.cc:493] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:55.828563 12604 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:55.833806 12604 raft_consensus.cc:515] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:55.836863 12604 leader_election.cc:290] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 4 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:55.837841 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:55.837795 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512500cfa01b4614972925e2e54da037" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:55.838734 12678 raft_consensus.cc:3055] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:19:55.838897 12751 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:55.839152 12678 raft_consensus.cc:740] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:55.840323 12678 consensus_queue.cc:260] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:55.841991 12678 raft_consensus.cc:3060] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:55.843967 12751 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 4.
I20260709 12:19:55.845324 12568 leader_election.cc:304] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:55.847088 12970 raft_consensus.cc:2804] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:19:55.848213 12970 raft_consensus.cc:697] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:55.849414 12678 raft_consensus.cc:2468] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 4.
I20260709 12:19:55.849390 12970 consensus_queue.cc:237] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:55.857998 12499 catalog_manager.cc:5697] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 3 to 4, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 4 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:56.036792 12969 raft_consensus.cc:993] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:56.037312 12969 raft_consensus.cc:1081] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:56.039067 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33384
I20260709 12:19:56.039634 12604 raft_consensus.cc:493] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:56.040089 12604 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:19:56.045054 12604 raft_consensus.cc:515] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.046844 12604 leader_election.cc:290] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 5 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:56.047772 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:56.047956 12751 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1f3b5d9156947e49d05b26ae1586797" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:56.048414 12678 raft_consensus.cc:3055] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:19:56.048554 12751 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:19:56.048768 12678 raft_consensus.cc:740] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:56.049638 12678 consensus_queue.cc:260] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.050711 12678 raft_consensus.cc:3060] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:19:56.053771 12751 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 5.
I20260709 12:19:56.054890 12568 leader_election.cc:304] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:56.055608 12970 raft_consensus.cc:2804] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:19:56.056159 12970 raft_consensus.cc:697] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 5 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:56.056269 12678 raft_consensus.cc:2468] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 5.
I20260709 12:19:56.057229 12970 consensus_queue.cc:237] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.066592 12499 catalog_manager.cc:5697] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 4 to 5, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 5 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:56.095165 12962 raft_consensus.cc:993] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:56.095752 12978 raft_consensus.cc:1081] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Signalling peer 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:56.097524 12752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e201e7a803ea453d956f01684a55efde"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:41916
I20260709 12:19:56.098073 12752 raft_consensus.cc:493] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:56.098507 12752 raft_consensus.cc:3060] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:56.104176 12752 raft_consensus.cc:515] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.106690 12752 leader_election.cc:290] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 2 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:56.107676 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:56.107890 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e201e7a803ea453d956f01684a55efde" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:56.108304 12604 raft_consensus.cc:3060] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:56.108604 12678 raft_consensus.cc:3055] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:19:56.108943 12678 raft_consensus.cc:740] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:56.109603 12678 consensus_queue.cc:260] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.110687 12678 raft_consensus.cc:3060] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:19:56.113256 12604 raft_consensus.cc:2468] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 2.
I20260709 12:19:56.114722 12719 leader_election.cc:304] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:56.115729 12940 raft_consensus.cc:2804] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:19:56.116132 12678 raft_consensus.cc:2468] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 2.
I20260709 12:19:56.117542 12940 raft_consensus.cc:697] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:56.118510 12940 consensus_queue.cc:237] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.126538 12499 catalog_manager.cc:5697] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 1 to 2, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 2 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:56.148422 12962 raft_consensus.cc:993] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:56.149021 12978 raft_consensus.cc:1081] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Signalling peer 5963ee1f8d1a437aa85e7371df8ac3bd to start an election
I20260709 12:19:56.150574 12752 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "744469240da84c15832aa2f583308505"
dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
 from {username='slave'} at 127.0.0.1:41916
I20260709 12:19:56.151093 12752 raft_consensus.cc:493] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:56.151414 12752 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:56.157733 12752 raft_consensus.cc:515] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.159529 12752 leader_election.cc:290] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 4 election: Requested vote from peers a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905), 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321)
I20260709 12:19:56.160506 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
I20260709 12:19:56.160544 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744469240da84c15832aa2f583308505" candidate_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:56.161288 12604 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:56.161343 12678 raft_consensus.cc:3055] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:19:56.161798 12678 raft_consensus.cc:740] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:56.162623 12678 consensus_queue.cc:260] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.163846 12678 raft_consensus.cc:3060] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:19:56.168124 12604 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 4.
I20260709 12:19:56.169528 12719 leader_election.cc:304] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:56.170620 12678 raft_consensus.cc:2468] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5963ee1f8d1a437aa85e7371df8ac3bd in term 4.
I20260709 12:19:56.170683 12940 raft_consensus.cc:2804] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:19:56.171290 12940 raft_consensus.cc:697] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 LEADER]: Becoming Leader. State: Replica: 5963ee1f8d1a437aa85e7371df8ac3bd, State: Running, Role: LEADER
I20260709 12:19:56.172367 12940 consensus_queue.cc:237] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.180297 12499 catalog_manager.cc:5697] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd reported cstate change: term changed from 3 to 4, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67). New cstate: current_term: 4 leader_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: HEALTHY } } }
I20260709 12:19:56.223851 12962 raft_consensus.cc:993] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: : Instructing follower a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:56.224313 12962 raft_consensus.cc:1081] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Signalling peer a37ecd8998584be68aafbd4ef66c3d84 to start an election
I20260709 12:19:56.225934 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6"
dest_uuid: "a37ecd8998584be68aafbd4ef66c3d84"
 from {username='slave'} at 127.0.0.1:33384
I20260709 12:19:56.226450 12604 raft_consensus.cc:493] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:19:56.226722 12604 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:56.231017 12604 raft_consensus.cc:515] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.233111 12604 leader_election.cc:290] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 3 election: Requested vote from peers 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321), 5963ee1f8d1a437aa85e7371df8ac3bd (127.12.45.67:41687)
I20260709 12:19:56.234163 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "801602d913e44df6a7c0fb76bc53aa30"
I20260709 12:19:56.234714 12752 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6bf05f9b8be40cd9264cf530db7b1a6" candidate_uuid: "a37ecd8998584be68aafbd4ef66c3d84" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd"
I20260709 12:19:56.235095 12678 raft_consensus.cc:3055] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:19:56.235581 12752 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:56.235581 12678 raft_consensus.cc:740] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 801602d913e44df6a7c0fb76bc53aa30, State: Running, Role: LEADER
I20260709 12:19:56.236781 12678 consensus_queue.cc:260] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.238201 12678 raft_consensus.cc:3060] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:19:56.243649 12752 raft_consensus.cc:2468] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 3.
I20260709 12:19:56.245254 12568 leader_election.cc:304] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5963ee1f8d1a437aa85e7371df8ac3bd, a37ecd8998584be68aafbd4ef66c3d84; no voters: 
I20260709 12:19:56.246028 12970 raft_consensus.cc:2804] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:19:56.245713 12678 raft_consensus.cc:2468] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a37ecd8998584be68aafbd4ef66c3d84 in term 3.
I20260709 12:19:56.246556 12970 raft_consensus.cc:697] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 LEADER]: Becoming Leader. State: Replica: a37ecd8998584be68aafbd4ef66c3d84, State: Running, Role: LEADER
I20260709 12:19:56.247773 12970 consensus_queue.cc:237] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } }
I20260709 12:19:56.254776 12499 catalog_manager.cc:5697] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 reported cstate change: term changed from 2 to 3, leader changed from 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66) to a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "a37ecd8998584be68aafbd4ef66c3d84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 } health_report { overall_health: UNKNOWN } } }
I20260709 12:19:56.257885 12678 raft_consensus.cc:1275] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:56.259429 12970 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:56.271358 12752 raft_consensus.cc:1275] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:56.273783 12970 consensus_queue.cc:1048] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:19:56.486343 12752 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 5 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:19:56.487743 12970 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:19:56.498461 12678 raft_consensus.cc:1275] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 5 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:19:56.500290 12970 consensus_queue.cc:1048] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:19:56.516090 12604 raft_consensus.cc:1275] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:56.517478 12940 consensus_queue.cc:1048] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:56.527971 12678 raft_consensus.cc:1275] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:19:56.529516 12940 consensus_queue.cc:1048] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:19:56.667753 12752 raft_consensus.cc:1275] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:56.669114 12971 consensus_queue.cc:1048] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5963ee1f8d1a437aa85e7371df8ac3bd" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 41687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:19:56.679276 12678 raft_consensus.cc:1275] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Refusing update from remote peer a37ecd8998584be68aafbd4ef66c3d84: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:19:56.681370 12989 consensus_queue.cc:1048] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:19:56.693130 12604 raft_consensus.cc:1275] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:56.694319 12940 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37ecd8998584be68aafbd4ef66c3d84" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 41905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:56.707327 12678 raft_consensus.cc:1275] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Refusing update from remote peer 5963ee1f8d1a437aa85e7371df8ac3bd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:19:56.708690 12992 consensus_queue.cc:1048] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "801602d913e44df6a7c0fb76bc53aa30" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:19:56.762907 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:56.768060 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:56.768477 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:57.769920 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:57.775637 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:57.776191 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:58.777343 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:58.782140 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:58.782589 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:19:59.783588 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:19:59.789422 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:19:59.789990 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:00.791249 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:00.796867 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:00.797369 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:01.798596 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:01.804123 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:01.804498 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:02.805709 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:02.811088 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:02.811616 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:03.812834 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:03.820391 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:03.821090 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:04.822664 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:04.828755 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:04.829275 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:05.865281 12469 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:20:05.866225 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
I20260709 12:20:05.900722 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:05.901895 12469 tablet_replica.cc:333] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.902935 12469 raft_consensus.cc:2243] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:05.903765 12469 raft_consensus.cc:2272] T 87698a3aa83b4b7f8ce0189a2a7b4807 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.906996 12469 tablet_replica.cc:333] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.907621 12469 raft_consensus.cc:2243] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:20:05.908550 12469 raft_consensus.cc:2272] T a1f3b5d9156947e49d05b26ae1586797 P a37ecd8998584be68aafbd4ef66c3d84 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.910447 12469 tablet_replica.cc:333] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.911039 12469 raft_consensus.cc:2243] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:05.911948 12469 raft_consensus.cc:2272] T 512500cfa01b4614972925e2e54da037 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.913820 12469 tablet_replica.cc:333] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.914523 12469 raft_consensus.cc:2243] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:05.915336 12469 raft_consensus.cc:2272] T 744469240da84c15832aa2f583308505 P a37ecd8998584be68aafbd4ef66c3d84 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.917718 12469 tablet_replica.cc:333] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.918385 12469 raft_consensus.cc:2243] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:05.919059 12469 raft_consensus.cc:2272] T 55d5f1e66c7745ce9c59ea8361d55e74 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.921162 12469 tablet_replica.cc:333] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.921728 12469 raft_consensus.cc:2243] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:05.922197 12469 raft_consensus.cc:2272] T 4a9cac7524ae4180879338590f67b2c7 P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.924427 12469 tablet_replica.cc:333] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.925014 12469 raft_consensus.cc:2243] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:05.925491 12469 raft_consensus.cc:2272] T 9a64ffb6456946b7b33340731974841e P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.927307 12469 tablet_replica.cc:333] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.927834 12469 raft_consensus.cc:2243] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:05.928339 12469 raft_consensus.cc:2272] T e201e7a803ea453d956f01684a55efde P a37ecd8998584be68aafbd4ef66c3d84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:05.930152 12469 tablet_replica.cc:333] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84: stopping tablet replica
I20260709 12:20:05.930706 12469 raft_consensus.cc:2243] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:05.931603 12469 raft_consensus.cc:2272] T c6bf05f9b8be40cd9264cf530db7b1a6 P a37ecd8998584be68aafbd4ef66c3d84 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:06.055305 12719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:41905: connect: Connection refused (error 111)
I20260709 12:20:06.058194 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
W20260709 12:20:06.060662 12645 consensus_peers.cc:597] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 -> Peer a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905): Couldn't send request to peer a37ecd8998584be68aafbd4ef66c3d84. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:41905: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:06.061026 12719 consensus_peers.cc:597] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd -> Peer a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905): Couldn't send request to peer a37ecd8998584be68aafbd4ef66c3d84. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:41905: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:06.082355 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
W20260709 12:20:06.095598 12716 consensus_peers.cc:597] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd -> Peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Couldn't send request to peer 801602d913e44df6a7c0fb76bc53aa30. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:06.105743 12716 consensus_peers.cc:597] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd -> Peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Couldn't send request to peer 801602d913e44df6a7c0fb76bc53aa30. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:06.122432 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:06.123294 12469 tablet_replica.cc:333] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.124001 12469 raft_consensus.cc:2243] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.124552 12469 raft_consensus.cc:2272] T e201e7a803ea453d956f01684a55efde P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.127413 12469 tablet_replica.cc:333] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.128132 12469 raft_consensus.cc:2243] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.128679 12469 raft_consensus.cc:2272] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:06.128902 12716 consensus_peers.cc:597] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd -> Peer 801602d913e44df6a7c0fb76bc53aa30 (127.12.45.66:37321): Couldn't send request to peer 801602d913e44df6a7c0fb76bc53aa30. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:06.130805 12469 tablet_replica.cc:333] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.131393 12469 raft_consensus.cc:2243] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.131843 12469 raft_consensus.cc:2272] T c6bf05f9b8be40cd9264cf530db7b1a6 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.133658 12469 tablet_replica.cc:333] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.134328 12469 raft_consensus.cc:2243] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.135284 12469 raft_consensus.cc:2272] T 9a64ffb6456946b7b33340731974841e P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.137708 12469 tablet_replica.cc:333] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.138626 12469 raft_consensus.cc:2243] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.139571 12469 raft_consensus.cc:2272] T 55d5f1e66c7745ce9c59ea8361d55e74 P 801602d913e44df6a7c0fb76bc53aa30 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.142082 12469 tablet_replica.cc:333] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.142755 12469 raft_consensus.cc:2243] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.143625 12469 raft_consensus.cc:2272] T 4a9cac7524ae4180879338590f67b2c7 P 801602d913e44df6a7c0fb76bc53aa30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.145576 12469 tablet_replica.cc:333] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.146099 12469 raft_consensus.cc:2243] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.146586 12469 raft_consensus.cc:2272] T 744469240da84c15832aa2f583308505 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.148403 12469 tablet_replica.cc:333] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.148910 12469 raft_consensus.cc:2243] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.149396 12469 raft_consensus.cc:2272] T 512500cfa01b4614972925e2e54da037 P 801602d913e44df6a7c0fb76bc53aa30 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.151611 12469 tablet_replica.cc:333] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30: stopping tablet replica
I20260709 12:20:06.152211 12469 raft_consensus.cc:2243] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.152673 12469 raft_consensus.cc:2272] T a1f3b5d9156947e49d05b26ae1586797 P 801602d913e44df6a7c0fb76bc53aa30 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:20:06.168802 12719 consensus_peers.cc:597] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd -> Peer a37ecd8998584be68aafbd4ef66c3d84 (127.12.45.65:41905): Couldn't send request to peer a37ecd8998584be68aafbd4ef66c3d84. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:41905: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:06.186793 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:20:06.208343 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
I20260709 12:20:06.241508 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:06.242636 12469 tablet_replica.cc:333] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.243619 12469 raft_consensus.cc:2243] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.244606 12469 raft_consensus.cc:2272] T e201e7a803ea453d956f01684a55efde P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.246798 12469 tablet_replica.cc:333] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.247381 12469 raft_consensus.cc:2243] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.248281 12469 raft_consensus.cc:2272] T 87698a3aa83b4b7f8ce0189a2a7b4807 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.250530 12469 tablet_replica.cc:333] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.251392 12469 raft_consensus.cc:2243] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.252183 12469 raft_consensus.cc:2272] T c6bf05f9b8be40cd9264cf530db7b1a6 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.254838 12469 tablet_replica.cc:333] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.255487 12469 raft_consensus.cc:2243] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.256008 12469 raft_consensus.cc:2272] T 55d5f1e66c7745ce9c59ea8361d55e74 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.257932 12469 tablet_replica.cc:333] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.258575 12469 raft_consensus.cc:2243] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.259032 12469 raft_consensus.cc:2272] T 9a64ffb6456946b7b33340731974841e P 5963ee1f8d1a437aa85e7371df8ac3bd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.260733 12469 tablet_replica.cc:333] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.261268 12469 raft_consensus.cc:2243] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.261705 12469 raft_consensus.cc:2272] T 4a9cac7524ae4180879338590f67b2c7 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.263540 12469 tablet_replica.cc:333] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.264068 12469 raft_consensus.cc:2243] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.264894 12469 raft_consensus.cc:2272] T 744469240da84c15832aa2f583308505 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.266835 12469 tablet_replica.cc:333] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.267359 12469 raft_consensus.cc:2243] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.267791 12469 raft_consensus.cc:2272] T 512500cfa01b4614972925e2e54da037 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.269493 12469 tablet_replica.cc:333] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd: stopping tablet replica
I20260709 12:20:06.270092 12469 raft_consensus.cc:2243] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:06.270608 12469 raft_consensus.cc:2272] T a1f3b5d9156947e49d05b26ae1586797 P 5963ee1f8d1a437aa85e7371df8ac3bd [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.303871 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:20:06.327332 12469 master.cc:562] Master@127.12.45.126:39533 shutting down...
I20260709 12:20:06.425554 12556 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 12:20:06.558048 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:06.558846 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:06.559195 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P 66d4bd663e8b498f889dde288f0bef35: stopping tablet replica
I20260709 12:20:06.580302 12469 master.cc:584] Master@127.12.45.126:39533 shutdown complete.
5963ee1f8d1a437aa85e7371df8ac3bd  1
801602d913e44df6a7c0fb76bc53aa30  4
a37ecd8998584be68aafbd4ef66c3d84  4
5963ee1f8d1a437aa85e7371df8ac3bd  3
801602d913e44df6a7c0fb76bc53aa30  3
a37ecd8998584be68aafbd4ef66c3d84  3
5963ee1f8d1a437aa85e7371df8ac3bd  1
801602d913e44df6a7c0fb76bc53aa30  8
a37ecd8998584be68aafbd4ef66c3d84  0
5963ee1f8d1a437aa85e7371df8ac3bd  3
801602d913e44df6a7c0fb76bc53aa30  3
a37ecd8998584be68aafbd4ef66c3d84  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (32169 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:20:06.633895 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:33649
I20260709 12:20:06.635317 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:06.641788 13010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:06.643105 13011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:06.643575 13013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:06.643999 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:06.645102 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:06.645299 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:06.645408 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599606645397 us; error 0 us; skew 500 ppm
I20260709 12:20:06.645890 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:06.649075 12469 webserver.cc:533] Webserver started at http://127.12.45.126:39027/ using document root <none> and password file <none>
I20260709 12:20:06.649612 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:06.649797 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:06.650041 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:06.651391 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "cdc4b94fd5c84a07ba5248ac42e09786"
format_stamp: "Formatted at 2026-07-09 12:20:06 on dist-test-slave-90bm"
I20260709 12:20:06.656850 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.002s
I20260709 12:20:06.661345 13018 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:06.662683 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 12:20:06.662990 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "cdc4b94fd5c84a07ba5248ac42e09786"
format_stamp: "Formatted at 2026-07-09 12:20:06 on dist-test-slave-90bm"
I20260709 12:20:06.663282 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:06.677603 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:06.679095 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:06.727427 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:33649
I20260709 12:20:06.727586 13069 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:33649 every 8 connection(s)
I20260709 12:20:06.731889 13070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:06.743438 13070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786: Bootstrap starting.
I20260709 12:20:06.749130 13070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:06.754102 13070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786: No bootstrap required, opened a new log
I20260709 12:20:06.756633 13070 raft_consensus.cc:359] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" member_type: VOTER }
I20260709 12:20:06.757115 13070 raft_consensus.cc:385] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:06.757381 13070 raft_consensus.cc:740] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc4b94fd5c84a07ba5248ac42e09786, State: Initialized, Role: FOLLOWER
I20260709 12:20:06.758097 13070 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" member_type: VOTER }
I20260709 12:20:06.758694 13070 raft_consensus.cc:399] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:06.758908 13070 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:06.759194 13070 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:06.763685 13070 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" member_type: VOTER }
I20260709 12:20:06.764307 13070 leader_election.cc:304] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdc4b94fd5c84a07ba5248ac42e09786; no voters: 
I20260709 12:20:06.765906 13070 leader_election.cc:290] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:06.766422 13073 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:06.768255 13073 raft_consensus.cc:697] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 1 LEADER]: Becoming Leader. State: Replica: cdc4b94fd5c84a07ba5248ac42e09786, State: Running, Role: LEADER
I20260709 12:20:06.769124 13073 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" member_type: VOTER }
I20260709 12:20:06.769980 13070 sys_catalog.cc:565] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:06.775331 13075 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdc4b94fd5c84a07ba5248ac42e09786. Latest consensus state: current_term: 1 leader_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" member_type: VOTER } }
I20260709 12:20:06.775372 13074 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdc4b94fd5c84a07ba5248ac42e09786" member_type: VOTER } }
I20260709 12:20:06.776746 13075 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:06.778174 13074 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:06.780573 13080 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:06.790627 13080 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:06.791903 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:06.801837 13080 catalog_manager.cc:1383] Generated new cluster ID: 481fb1fb08064059b46d7c3e6baf5b7a
I20260709 12:20:06.802222 13080 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:06.837502 13080 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:06.838964 13080 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:06.856411 13080 catalog_manager.cc:6070] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786: Generated new TSK 0
I20260709 12:20:06.857172 13080 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:06.923658 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:06.930800 13091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:06.931970 13092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:06.934388 13094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:06.934584 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:06.935951 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:06.936153 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:06.936299 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599606936287 us; error 0 us; skew 500 ppm
I20260709 12:20:06.936812 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:06.939545 12469 webserver.cc:533] Webserver started at http://127.12.45.65:44419/ using document root <none> and password file <none>
I20260709 12:20:06.940042 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:06.940270 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:06.940511 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:06.941673 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "20c2882af1cb46d6b9b683e1619db3c6"
format_stamp: "Formatted at 2026-07-09 12:20:06 on dist-test-slave-90bm"
I20260709 12:20:06.946642 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:20:06.950451 13099 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:06.951490 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:20:06.951893 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "20c2882af1cb46d6b9b683e1619db3c6"
format_stamp: "Formatted at 2026-07-09 12:20:06 on dist-test-slave-90bm"
I20260709 12:20:06.952188 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:06.977447 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:06.978940 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:06.980679 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:06.983585 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:06.983841 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:06.984126 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:06.984318 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:07.035319 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:45355
I20260709 12:20:07.035521 13161 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:45355 every 8 connection(s)
I20260709 12:20:07.041061 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:07.049227 13166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:07.051309 13167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:07.052591 13169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:07.053752 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:07.055835 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:07.056073 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:07.056380 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599607056356 us; error 0 us; skew 500 ppm
I20260709 12:20:07.057081 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:07.060693 12469 webserver.cc:533] Webserver started at http://127.12.45.66:41783/ using document root <none> and password file <none>
I20260709 12:20:07.061480 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:07.061767 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:07.062107 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:07.063311 13162 heartbeater.cc:344] Connected to a master server at 127.12.45.126:33649
I20260709 12:20:07.063571 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
format_stamp: "Formatted at 2026-07-09 12:20:07 on dist-test-slave-90bm"
I20260709 12:20:07.063776 13162 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:07.064782 13162 heartbeater.cc:507] Master 127.12.45.126:33649 requested a full tablet report, sending...
I20260709 12:20:07.067437 13035 ts_manager.cc:194] Registered new tserver with Master: 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355)
I20260709 12:20:07.069720 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:20:07.069875 13035 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56034
I20260709 12:20:07.074506 13174 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:07.075692 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.003s
I20260709 12:20:07.076109 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
format_stamp: "Formatted at 2026-07-09 12:20:07 on dist-test-slave-90bm"
I20260709 12:20:07.076514 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:07.106400 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:07.108204 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:07.110364 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:07.113809 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:07.114038 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:07.114231 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:07.114456 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:20:07.163992 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:35085
I20260709 12:20:07.164146 13236 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:35085 every 8 connection(s)
I20260709 12:20:07.169857 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:07.179524 13240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:07.180580 13241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:07.185490 13237 heartbeater.cc:344] Connected to a master server at 127.12.45.126:33649
W20260709 12:20:07.185391 13243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:07.186048 13237 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:07.186048 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:07.187372 13237 heartbeater.cc:507] Master 127.12.45.126:33649 requested a full tablet report, sending...
I20260709 12:20:07.187438 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:07.187856 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:07.188081 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599607188065 us; error 0 us; skew 500 ppm
I20260709 12:20:07.188802 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:07.190120 13035 ts_manager.cc:194] Registered new tserver with Master: 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:07.191854 12469 webserver.cc:533] Webserver started at http://127.12.45.67:35413/ using document root <none> and password file <none>
I20260709 12:20:07.192005 13035 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56044
I20260709 12:20:07.192535 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:07.192788 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:07.193092 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:07.194751 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "019deb94fe994e468a6407fa87791634"
format_stamp: "Formatted at 2026-07-09 12:20:07 on dist-test-slave-90bm"
I20260709 12:20:07.202529 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:20:07.206892 13248 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:07.208099 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:20:07.208540 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "019deb94fe994e468a6407fa87791634"
format_stamp: "Formatted at 2026-07-09 12:20:07 on dist-test-slave-90bm"
I20260709 12:20:07.208886 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:07.229684 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:07.231554 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:07.233132 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:07.235532 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:07.236069 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20260709 12:20:07.236419 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:07.236624 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:07.288486 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:42297
I20260709 12:20:07.288589 13310 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:42297 every 8 connection(s)
I20260709 12:20:07.305373 13311 heartbeater.cc:344] Connected to a master server at 127.12.45.126:33649
I20260709 12:20:07.305794 13311 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:07.306674 13311 heartbeater.cc:507] Master 127.12.45.126:33649 requested a full tablet report, sending...
I20260709 12:20:07.309140 13035 ts_manager.cc:194] Registered new tserver with Master: 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:07.309834 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017660205s
I20260709 12:20:07.311054 13035 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56056
I20260709 12:20:07.340720 13035 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56070:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:07.343905 13035 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:07.423228 13276 tablet_service.cc:1511] Processing CreateTablet for tablet 912ab74cddcf49ab870e1d92f7dc43c6 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:07.424984 13276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ab74cddcf49ab870e1d92f7dc43c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.428211 13275 tablet_service.cc:1511] Processing CreateTablet for tablet e590b63ed3d84c04aa4413b90cfa83bb (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:07.429776 13275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e590b63ed3d84c04aa4413b90cfa83bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.434913 13273 tablet_service.cc:1511] Processing CreateTablet for tablet 6e3ef3c2f65044ca89fcec134898704d (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:07.435459 13202 tablet_service.cc:1511] Processing CreateTablet for tablet 912ab74cddcf49ab870e1d92f7dc43c6 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:07.434924 13274 tablet_service.cc:1511] Processing CreateTablet for tablet c6b2ca002693479e8153b1d576421784 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:07.436473 13273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e3ef3c2f65044ca89fcec134898704d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.436893 13202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ab74cddcf49ab870e1d92f7dc43c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.437251 13274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b2ca002693479e8153b1d576421784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.439864 13201 tablet_service.cc:1511] Processing CreateTablet for tablet e590b63ed3d84c04aa4413b90cfa83bb (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:07.441632 13201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e590b63ed3d84c04aa4413b90cfa83bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.443694 13200 tablet_service.cc:1511] Processing CreateTablet for tablet c6b2ca002693479e8153b1d576421784 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:07.444418 13272 tablet_service.cc:1511] Processing CreateTablet for tablet 331728bfdb514c50b772b2186b20a7a4 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:07.445189 13200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b2ca002693479e8153b1d576421784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.445820 13272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 331728bfdb514c50b772b2186b20a7a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.446817 13123 tablet_service.cc:1511] Processing CreateTablet for tablet 331728bfdb514c50b772b2186b20a7a4 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:07.447249 13122 tablet_service.cc:1511] Processing CreateTablet for tablet f4198ae3f50142469c2ea4e969a89d61 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:07.447556 13126 tablet_service.cc:1511] Processing CreateTablet for tablet e590b63ed3d84c04aa4413b90cfa83bb (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:20:07.448248 13123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 331728bfdb514c50b772b2186b20a7a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.449550 13125 tablet_service.cc:1511] Processing CreateTablet for tablet c6b2ca002693479e8153b1d576421784 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:20:07.450209 13122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4198ae3f50142469c2ea4e969a89d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.451977 13199 tablet_service.cc:1511] Processing CreateTablet for tablet 6e3ef3c2f65044ca89fcec134898704d (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:07.452136 13124 tablet_service.cc:1511] Processing CreateTablet for tablet 6e3ef3c2f65044ca89fcec134898704d (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:20:07.454684 13198 tablet_service.cc:1511] Processing CreateTablet for tablet 331728bfdb514c50b772b2186b20a7a4 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:20:07.455119 13127 tablet_service.cc:1511] Processing CreateTablet for tablet 912ab74cddcf49ab870e1d92f7dc43c6 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:20:07.456754 13125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b2ca002693479e8153b1d576421784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.458626 13198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 331728bfdb514c50b772b2186b20a7a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.463294 13124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e3ef3c2f65044ca89fcec134898704d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.463114 13199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e3ef3c2f65044ca89fcec134898704d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.474664 13126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e590b63ed3d84c04aa4413b90cfa83bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.478663 13127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ab74cddcf49ab870e1d92f7dc43c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.494234 13197 tablet_service.cc:1511] Processing CreateTablet for tablet f4198ae3f50142469c2ea4e969a89d61 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:07.496263 13197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4198ae3f50142469c2ea4e969a89d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.540423 13275 tablet_service.cc:1511] Processing CreateTablet for tablet f4198ae3f50142469c2ea4e969a89d61 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:20:07.541925 13275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4198ae3f50142469c2ea4e969a89d61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.547842 13125 tablet_service.cc:1511] Processing CreateTablet for tablet d827b78a43c2417484eb0f0db56a2f0a (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:07.551173 13328 tablet_bootstrap.cc:492] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:07.552225 13125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d827b78a43c2417484eb0f0db56a2f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.552946 13196 tablet_service.cc:1511] Processing CreateTablet for tablet d827b78a43c2417484eb0f0db56a2f0a (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:07.554800 13196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d827b78a43c2417484eb0f0db56a2f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.556649 13273 tablet_service.cc:1511] Processing CreateTablet for tablet d827b78a43c2417484eb0f0db56a2f0a (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:20:07.557984 13273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d827b78a43c2417484eb0f0db56a2f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.578523 13328 tablet_bootstrap.cc:654] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.578665 13122 tablet_service.cc:1511] Processing CreateTablet for tablet 306360dadd3244c9968d3eeb1c836413 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:07.580142 13122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306360dadd3244c9968d3eeb1c836413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.601269 13274 tablet_service.cc:1511] Processing CreateTablet for tablet 306360dadd3244c9968d3eeb1c836413 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:07.603019 13274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306360dadd3244c9968d3eeb1c836413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.608863 13197 tablet_service.cc:1511] Processing CreateTablet for tablet 306360dadd3244c9968d3eeb1c836413 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:20:07.609391 13328 tablet_bootstrap.cc:492] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:07.609979 13328 ts_tablet_manager.cc:1403] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.059s	user 0.000s	sys 0.017s
I20260709 12:20:07.613269 13328 raft_consensus.cc:359] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:07.614028 13328 raft_consensus.cc:385] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.614373 13197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306360dadd3244c9968d3eeb1c836413. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.614418 13328 raft_consensus.cc:740] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.615672 13328 consensus_queue.cc:260] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:07.625279 13196 tablet_service.cc:1511] Processing CreateTablet for tablet 512a7557f7a94dab8d8e500facf13712 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:07.631932 13328 ts_tablet_manager.cc:1434] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.002s
I20260709 12:20:07.632939 13311 heartbeater.cc:499] Master 127.12.45.126:33649 was elected leader, sending a full tablet report...
I20260709 12:20:07.633744 13329 tablet_bootstrap.cc:492] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.634399 13125 tablet_service.cc:1511] Processing CreateTablet for tablet 512a7557f7a94dab8d8e500facf13712 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:07.635898 13125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512a7557f7a94dab8d8e500facf13712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.638625 13328 tablet_bootstrap.cc:492] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:07.642163 13329 tablet_bootstrap.cc:654] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.649737 13328 tablet_bootstrap.cc:654] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.651968 13330 tablet_bootstrap.cc:492] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:07.663239 13196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512a7557f7a94dab8d8e500facf13712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.674178 13275 tablet_service.cc:1511] Processing CreateTablet for tablet 512a7557f7a94dab8d8e500facf13712 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:20:07.675931 13275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 512a7557f7a94dab8d8e500facf13712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.676216 13330 tablet_bootstrap.cc:654] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.681077 13329 tablet_bootstrap.cc:492] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.681670 13329 ts_tablet_manager.cc:1403] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.024s
I20260709 12:20:07.692016 13200 tablet_service.cc:1511] Processing CreateTablet for tablet 80b396c16c6d4f91a80832c50df6e865 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:07.693560 13200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b396c16c6d4f91a80832c50df6e865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.694595 13273 tablet_service.cc:1511] Processing CreateTablet for tablet 80b396c16c6d4f91a80832c50df6e865 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:07.695907 13273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b396c16c6d4f91a80832c50df6e865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.704195 13329 raft_consensus.cc:359] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.705708 13329 raft_consensus.cc:385] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.706044 13329 raft_consensus.cc:740] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.706887 13329 consensus_queue.cc:260] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.719795 13328 tablet_bootstrap.cc:492] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:07.720575 13328 ts_tablet_manager.cc:1403] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.082s	user 0.023s	sys 0.003s
I20260709 12:20:07.724496 13328 raft_consensus.cc:359] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.725481 13328 raft_consensus.cc:385] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.725811 13328 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.729423 13162 heartbeater.cc:499] Master 127.12.45.126:33649 was elected leader, sending a full tablet report...
I20260709 12:20:07.729653 13329 ts_tablet_manager.cc:1434] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.048s	user 0.016s	sys 0.004s
I20260709 12:20:07.730970 13125 tablet_service.cc:1511] Processing CreateTablet for tablet 80b396c16c6d4f91a80832c50df6e865 (DEFAULT_TABLE table=test-workload [id=6ed5681d47794e22af5fea5caa3c64f3]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:20:07.728410 13328 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.732710 13125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80b396c16c6d4f91a80832c50df6e865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:07.735571 13328 ts_tablet_manager.cc:1434] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.006s
I20260709 12:20:07.736789 13328 tablet_bootstrap.cc:492] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:07.731079 13330 tablet_bootstrap.cc:492] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:07.737725 13330 ts_tablet_manager.cc:1403] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.086s	user 0.008s	sys 0.027s
I20260709 12:20:07.741223 13330 raft_consensus.cc:359] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:07.742090 13330 raft_consensus.cc:385] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.742493 13330 raft_consensus.cc:740] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.743537 13330 consensus_queue.cc:260] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:07.730993 13329 tablet_bootstrap.cc:492] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.749212 13328 tablet_bootstrap.cc:654] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.754709 13237 heartbeater.cc:499] Master 127.12.45.126:33649 was elected leader, sending a full tablet report...
I20260709 12:20:07.755831 13329 tablet_bootstrap.cc:654] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.760560 13336 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.761137 13336 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.767594 13336 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:07.759680 13330 ts_tablet_manager.cc:1434] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:20:07.777580 13330 tablet_bootstrap.cc:492] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:07.789356 13329 tablet_bootstrap.cc:492] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.790171 13329 ts_tablet_manager.cc:1403] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.012s
W20260709 12:20:07.790565 13163 tablet.cc:2406] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260709 12:20:07.792989 13083 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:07.793561 13329 raft_consensus.cc:359] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.794548 13329 raft_consensus.cc:385] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.796336 13329 raft_consensus.cc:740] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.797300 13329 consensus_queue.cc:260] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.798748 13328 tablet_bootstrap.cc:492] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:07.799347 13328 ts_tablet_manager.cc:1403] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.063s	user 0.024s	sys 0.008s
I20260709 12:20:07.800035 13329 ts_tablet_manager.cc:1434] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:20:07.801126 13329 tablet_bootstrap.cc:492] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.804685 13330 tablet_bootstrap.cc:654] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.808960 13329 tablet_bootstrap.cc:654] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.812777 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
W20260709 12:20:07.815196 13102 leader_election.cc:343] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:07.806203 13328 raft_consensus.cc:359] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.816175 13212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:07.816857 13328 raft_consensus.cc:385] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.817304 13328 raft_consensus.cc:740] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
W20260709 12:20:07.818068 13103 leader_election.cc:343] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:07.818733 13103 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6; no voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:07.818608 13328 consensus_queue.cc:260] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.820499 13336 raft_consensus.cc:2749] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:07.823133 13329 tablet_bootstrap.cc:492] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.823750 13329 ts_tablet_manager.cc:1403] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 12:20:07.824163 13328 ts_tablet_manager.cc:1434] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 12:20:07.825439 13328 tablet_bootstrap.cc:492] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:07.826797 13330 tablet_bootstrap.cc:492] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:07.827364 13330 ts_tablet_manager.cc:1403] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.010s
I20260709 12:20:07.827519 13329 raft_consensus.cc:359] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.828678 13329 raft_consensus.cc:385] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.829064 13329 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.830711 13330 raft_consensus.cc:359] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.831821 13330 raft_consensus.cc:385] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.830848 13329 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.832257 13330 raft_consensus.cc:740] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.834033 13328 tablet_bootstrap.cc:654] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.833530 13330 consensus_queue.cc:260] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.836316 13330 ts_tablet_manager.cc:1434] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:20:07.836699 13329 ts_tablet_manager.cc:1434] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 12:20:07.840473 13328 tablet_bootstrap.cc:492] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:07.840937 13330 tablet_bootstrap.cc:492] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:07.840986 13328 ts_tablet_manager.cc:1403] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.007s
I20260709 12:20:07.841089 13329 tablet_bootstrap.cc:492] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.844944 13328 raft_consensus.cc:359] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.846078 13328 raft_consensus.cc:385] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.846653 13328 raft_consensus.cc:740] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.847774 13328 consensus_queue.cc:260] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.848861 13330 tablet_bootstrap.cc:654] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.849431 13329 tablet_bootstrap.cc:654] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.859730 13328 ts_tablet_manager.cc:1434] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:20:07.860954 13330 tablet_bootstrap.cc:492] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:07.860973 13328 tablet_bootstrap.cc:492] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:07.861683 13330 ts_tablet_manager.cc:1403] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.006s
I20260709 12:20:07.865562 13329 tablet_bootstrap.cc:492] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.865487 13330 raft_consensus.cc:359] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.866192 13329 ts_tablet_manager.cc:1403] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.011s
I20260709 12:20:07.866627 13330 raft_consensus.cc:385] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.867033 13330 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.868319 13328 tablet_bootstrap.cc:654] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.868041 13330 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.870112 13329 raft_consensus.cc:359] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.870995 13329 raft_consensus.cc:385] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.871095 13330 ts_tablet_manager.cc:1434] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:20:07.871369 13329 raft_consensus.cc:740] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.872611 13330 tablet_bootstrap.cc:492] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:07.872839 13329 consensus_queue.cc:260] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.877899 13329 ts_tablet_manager.cc:1434] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:20:07.879222 13329 tablet_bootstrap.cc:492] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.883102 13330 tablet_bootstrap.cc:654] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.887419 13329 tablet_bootstrap.cc:654] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.903093 13328 tablet_bootstrap.cc:492] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:07.903793 13328 ts_tablet_manager.cc:1403] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.043s	user 0.021s	sys 0.015s
I20260709 12:20:07.906023 13329 tablet_bootstrap.cc:492] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.906531 13329 ts_tablet_manager.cc:1403] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.007s
I20260709 12:20:07.906245 13328 raft_consensus.cc:359] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.907003 13328 raft_consensus.cc:385] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.907279 13328 raft_consensus.cc:740] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.907938 13328 consensus_queue.cc:260] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.909278 13329 raft_consensus.cc:359] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:07.909863 13329 raft_consensus.cc:385] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.909976 13328 ts_tablet_manager.cc:1434] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 12:20:07.910189 13329 raft_consensus.cc:740] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.911119 13328 tablet_bootstrap.cc:492] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:07.911172 13329 consensus_queue.cc:260] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:07.903090 13330 tablet_bootstrap.cc:492] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:07.914149 13329 ts_tablet_manager.cc:1434] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:07.914737 13330 ts_tablet_manager.cc:1403] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.042s	user 0.019s	sys 0.010s
I20260709 12:20:07.919394 13329 tablet_bootstrap.cc:492] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.919008 13330 raft_consensus.cc:359] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.921295 13328 tablet_bootstrap.cc:654] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.921649 13330 raft_consensus.cc:385] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.922101 13330 raft_consensus.cc:740] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.923205 13330 consensus_queue.cc:260] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.925782 13330 ts_tablet_manager.cc:1434] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.006s
I20260709 12:20:07.927556 13329 tablet_bootstrap.cc:654] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.940091 13328 tablet_bootstrap.cc:492] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:07.940757 13328 ts_tablet_manager.cc:1403] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.009s
I20260709 12:20:07.951165 13328 raft_consensus.cc:359] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.952260 13328 raft_consensus.cc:385] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.952661 13328 raft_consensus.cc:740] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.951494 13330 tablet_bootstrap.cc:492] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:07.954979 13328 consensus_queue.cc:260] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.958521 13329 tablet_bootstrap.cc:492] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.958722 13328 ts_tablet_manager.cc:1434] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:20:07.959136 13329 ts_tablet_manager.cc:1403] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.024s
I20260709 12:20:07.962634 13329 raft_consensus.cc:359] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.964113 13329 raft_consensus.cc:385] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:07.964669 13329 raft_consensus.cc:740] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:07.966234 13332 raft_consensus.cc:493] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.966862 13332 raft_consensus.cc:515] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.966509 13329 consensus_queue.cc:260] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.967419 13330 tablet_bootstrap.cc:654] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.972216 13329 ts_tablet_manager.cc:1434] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:20:07.973248 13337 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.973953 13329 tablet_bootstrap.cc:492] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:07.973867 13337 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.976897 13337 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:07.983723 13329 tablet_bootstrap.cc:654] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:07.985880 13332 leader_election.cc:290] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:07.987390 13332 raft_consensus.cc:493] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:07.989163 13332 raft_consensus.cc:515] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:07.993240 13332 leader_election.cc:290] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:07.998088 13329 tablet_bootstrap.cc:492] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:07.998906 13329 ts_tablet_manager.cc:1403] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.014s
I20260709 12:20:08.001644 13328 tablet_bootstrap.cc:492] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:08.002733 13329 raft_consensus.cc:359] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.004578 13329 raft_consensus.cc:385] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.005115 13329 raft_consensus.cc:740] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.006629 13329 consensus_queue.cc:260] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.025484 13328 tablet_bootstrap.cc:654] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.016227 13329 ts_tablet_manager.cc:1434] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:20:08.028597 13329 tablet_bootstrap.cc:492] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:08.035709 13332 raft_consensus.cc:493] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.051033 13332 raft_consensus.cc:515] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:08.054246 13332 leader_election.cc:290] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355)
I20260709 12:20:08.080585 13332 raft_consensus.cc:493] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.081349 13332 raft_consensus.cc:515] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.084472 13332 leader_election.cc:290] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.096231 13212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.097379 13212 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.100824 13212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.102061 13212 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.105737 13336 raft_consensus.cc:493] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.106340 13336 raft_consensus.cc:515] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.106670 13212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.107458 13212 raft_consensus.cc:2468] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.108717 13336 leader_election.cc:290] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.089144 13329 tablet_bootstrap.cc:654] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.114020 13211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.117681 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.118491 13286 raft_consensus.cc:2468] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 0.
I20260709 12:20:08.119249 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.119300 13252 leader_election.cc:304] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.120072 13137 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.120977 13358 raft_consensus.cc:2804] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.121852 13102 leader_election.cc:304] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:08.121382 13358 raft_consensus.cc:493] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.123242 13336 raft_consensus.cc:2804] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.123081 13135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.123627 13336 raft_consensus.cc:493] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.123555 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.123087 13358 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.124399 13136 raft_consensus.cc:2468] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.124922 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.125614 13336 raft_consensus.cc:3060] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.126838 13251 leader_election.cc:304] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:08.128158 13332 raft_consensus.cc:2804] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.128558 13332 raft_consensus.cc:493] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.128924 13332 raft_consensus.cc:3060] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.134339 13252 leader_election.cc:304] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.133823 13358 raft_consensus.cc:515] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.138609 13358 leader_election.cc:290] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.139139 13358 raft_consensus.cc:2804] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.139442 13358 raft_consensus.cc:493] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.139480 13329 tablet_bootstrap.cc:492] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:08.139770 13358 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.140208 13329 ts_tablet_manager.cc:1403] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.112s	user 0.018s	sys 0.008s
I20260709 12:20:08.144723 13329 raft_consensus.cc:359] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
W20260709 12:20:08.136587 13251 leader_election.cc:343] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.148942 13336 raft_consensus.cc:515] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.149928 13212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.150358 13135 raft_consensus.cc:2393] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 019deb94fe994e468a6407fa87791634 in current term 1: Already voted for candidate 20c2882af1cb46d6b9b683e1619db3c6 in this term.
I20260709 12:20:08.151304 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.152024 13137 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.152920 13336 leader_election.cc:290] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.151542 13328 tablet_bootstrap.cc:492] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:08.154476 13328 ts_tablet_manager.cc:1403] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.153s	user 0.017s	sys 0.000s
I20260709 12:20:08.155131 13211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
W20260709 12:20:08.157125 13103 leader_election.cc:343] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.158214 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:08.158823 13286 raft_consensus.cc:3060] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.157749 13328 raft_consensus.cc:359] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.160821 13328 raft_consensus.cc:385] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.145769 13329 raft_consensus.cc:385] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.161700 13329 raft_consensus.cc:740] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.161164 13328 raft_consensus.cc:740] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
W20260709 12:20:08.161109 13103 leader_election.cc:343] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.164430 13330 tablet_bootstrap.cc:492] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:08.163898 13328 consensus_queue.cc:260] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.165683 13329 consensus_queue.cc:260] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.168859 13135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.169852 13332 raft_consensus.cc:515] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:08.170918 13358 raft_consensus.cc:515] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.172303 13135 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 0.
I20260709 12:20:08.171984 13330 ts_tablet_manager.cc:1403] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.228s	user 0.025s	sys 0.035s
W20260709 12:20:08.172731 13252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:35085, user_credentials={real_user=slave}} blocked reactor thread for 50365.9us
I20260709 12:20:08.174070 13137 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.179291 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.180002 13136 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:08.181057 13252 leader_election.cc:343] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.181617 13252 leader_election.cc:304] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019deb94fe994e468a6407fa87791634; no voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.179332 13251 leader_election.cc:304] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:08.185313 13286 raft_consensus.cc:2468] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 1.
I20260709 12:20:08.186818 13102 leader_election.cc:304] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.187316 13329 ts_tablet_manager.cc:1434] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.004s
I20260709 12:20:08.188243 13336 raft_consensus.cc:2804] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.188500 13329 tablet_bootstrap.cc:492] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:08.178678 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.195647 13136 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.197121 13137 raft_consensus.cc:3060] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.198208 13330 raft_consensus.cc:359] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.199916 13363 raft_consensus.cc:493] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.205307 13363 raft_consensus.cc:515] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.208475 13363 leader_election.cc:290] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.200807 13177 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.202692 13358 leader_election.cc:290] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.203809 13330 raft_consensus.cc:385] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.175488 13332 leader_election.cc:290] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355)
I20260709 12:20:08.209695 13337 raft_consensus.cc:2804] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.211050 13337 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.209772 13358 raft_consensus.cc:2749] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:08.211470 13337 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.211321 13332 raft_consensus.cc:2804] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.213291 13332 raft_consensus.cc:697] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:08.212553 13358 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.215806 13358 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.209762 13330 raft_consensus.cc:740] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
W20260709 12:20:08.223688 13251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:45355, user_credentials={real_user=slave}} blocked reactor thread for 44452.6us
I20260709 12:20:08.219096 13332 consensus_queue.cc:237] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.231149 13336 raft_consensus.cc:697] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:08.238344 13212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.241313 13212 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.241683 13336 consensus_queue.cc:237] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.238853 13211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.239928 13358 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.240679 13329 tablet_bootstrap.cc:654] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.237360 13330 consensus_queue.cc:260] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.244506 13210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.246011 13211 raft_consensus.cc:3060] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.247579 13210 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.249253 13209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.250969 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.251673 13136 raft_consensus.cc:2468] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.254360 13328 ts_tablet_manager.cc:1434] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.099s	user 0.022s	sys 0.026s
I20260709 12:20:08.247654 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.256057 13286 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 0.
W20260709 12:20:08.256397 13252 leader_election.cc:343] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.260520 13328 tablet_bootstrap.cc:492] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:08.260177 13137 raft_consensus.cc:2468] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.260177 13211 raft_consensus.cc:2468] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.247552 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.263288 13252 leader_election.cc:304] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.264139 13251 leader_election.cc:304] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:08.267630 13332 raft_consensus.cc:2804] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.268268 13367 raft_consensus.cc:2804] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.268471 13332 raft_consensus.cc:697] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:08.268114 13251 leader_election.cc:304] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.268968 13367 raft_consensus.cc:697] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:08.269665 13358 raft_consensus.cc:2804] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.270136 13358 raft_consensus.cc:493] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.269912 13367 consensus_queue.cc:237] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.270707 13366 raft_consensus.cc:493] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.270711 13358 raft_consensus.cc:3060] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.272607 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.273392 13137 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.273567 13337 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.265782 13212 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.275914 13366 raft_consensus.cc:515] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.283037 13211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
W20260709 12:20:08.284924 13103 leader_election.cc:343] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.285039 13358 raft_consensus.cc:515] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.286377 13332 consensus_queue.cc:237] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:08.288456 13251 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:08.292744 13208 raft_consensus.cc:2393] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 019deb94fe994e468a6407fa87791634 in current term 1: Already voted for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in this term.
I20260709 12:20:08.292515 13369 raft_consensus.cc:493] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.293385 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.279362 13210 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.295451 13364 raft_consensus.cc:2804] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.296309 13364 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.295846 13369 raft_consensus.cc:515] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.290208 13366 leader_election.cc:290] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.304570 13286 raft_consensus.cc:2393] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20c2882af1cb46d6b9b683e1619db3c6 in current term 1: Already voted for candidate 019deb94fe994e468a6407fa87791634 in this term.
I20260709 12:20:08.305720 13329 tablet_bootstrap.cc:492] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:08.306286 13329 ts_tablet_manager.cc:1403] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.118s	user 0.006s	sys 0.014s
I20260709 12:20:08.297926 13364 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.306552 13102 leader_election.cc:304] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6; no voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.314389 13371 raft_consensus.cc:3060] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.316787 13330 ts_tablet_manager.cc:1434] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.137s	user 0.008s	sys 0.004s
I20260709 12:20:08.318147 13330 tablet_bootstrap.cc:492] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:08.318871 13328 tablet_bootstrap.cc:654] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.328491 13369 leader_election.cc:290] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.328635 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.330097 13208 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 0.
I20260709 12:20:08.330720 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.330830 13337 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.331382 13286 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 0.
I20260709 12:20:08.331550 13103 leader_election.cc:304] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.333458 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.333527 13358 leader_election.cc:290] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.334919 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.334952 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.337167 13369 raft_consensus.cc:2804] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.343240 13136 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.345413 13369 raft_consensus.cc:493] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.348074 13369 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:08.350517 13252 leader_election.cc:343] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.352715 13364 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.355072 13330 tablet_bootstrap.cc:654] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.345631 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:08.361696 13286 raft_consensus.cc:2393] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2a28ad919fa341a8b382a9c1b28b9af4 in current term 1: Already voted for candidate 019deb94fe994e468a6407fa87791634 in this term.
I20260709 12:20:08.357553 13135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.362190 13369 raft_consensus.cc:515] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.359628 13371 raft_consensus.cc:2749] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:08.360530 13329 raft_consensus.cc:359] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.366447 13329 raft_consensus.cc:385] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.366945 13329 raft_consensus.cc:740] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.367511 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.369944 13364 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.371663 13210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.372546 13210 raft_consensus.cc:2393] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 019deb94fe994e468a6407fa87791634 in current term 1: Already voted for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in this term.
I20260709 12:20:08.373317 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:08.374070 13252 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019deb94fe994e468a6407fa87791634; no voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.374353 13286 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.370105 13208 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.377871 13137 raft_consensus.cc:2468] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.380232 13035 catalog_manager.cc:5697] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 0 to 1, leader changed from <none> to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:08.382797 13364 raft_consensus.cc:2749] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:08.383005 13328 tablet_bootstrap.cc:492] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:08.383723 13328 ts_tablet_manager.cc:1403] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.005s
I20260709 12:20:08.382251 13035 catalog_manager.cc:5697] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 0 to 1, leader changed from <none> to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.385102 13251 leader_election.cc:304] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.385980 13369 leader_election.cc:290] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.386590 13364 raft_consensus.cc:2804] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.385862 13035 catalog_manager.cc:5697] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 0 to 1, leader changed from <none> to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:08.373291 13329 consensus_queue.cc:260] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.387101 13364 raft_consensus.cc:697] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:08.387990 13364 consensus_queue.cc:237] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.390429 13328 raft_consensus.cc:359] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.391108 13328 raft_consensus.cc:385] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.391393 13328 raft_consensus.cc:740] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.388958 13329 ts_tablet_manager.cc:1434] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.001s
I20260709 12:20:08.393319 13329 tablet_bootstrap.cc:492] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6: Bootstrap starting.
I20260709 12:20:08.387382 13032 catalog_manager.cc:5697] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.396363 13328 consensus_queue.cc:260] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.389093 13208 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 1.
I20260709 12:20:08.400687 13332 raft_consensus.cc:493] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.401180 13328 ts_tablet_manager.cc:1434] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 12:20:08.403546 13328 tablet_bootstrap.cc:492] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:08.402940 13332 raft_consensus.cc:515] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.411532 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.412199 13329 tablet_bootstrap.cc:654] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.412335 13137 raft_consensus.cc:2468] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
I20260709 12:20:08.411654 13136 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 1.
I20260709 12:20:08.405453 13103 leader_election.cc:304] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.413116 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.414291 13177 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 019deb94fe994e468a6407fa87791634
I20260709 12:20:08.414753 13330 tablet_bootstrap.cc:492] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:08.415318 13330 ts_tablet_manager.cc:1403] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.097s	user 0.017s	sys 0.011s
I20260709 12:20:08.415449 13251 leader_election.cc:304] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:08.415757 13374 raft_consensus.cc:2804] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.416332 13374 raft_consensus.cc:697] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Becoming Leader. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
W20260709 12:20:08.416690 13252 leader_election.cc:343] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.417520 13369 raft_consensus.cc:2804] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.417250 13374 consensus_queue.cc:237] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.418236 13369 raft_consensus.cc:697] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:08.419229 13330 raft_consensus.cc:359] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.418797 13332 leader_election.cc:290] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.420061 13330 raft_consensus.cc:385] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.420441 13330 raft_consensus.cc:740] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.420032 13328 tablet_bootstrap.cc:654] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.421092 13286 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 1.
I20260709 12:20:08.419796 13369 consensus_queue.cc:237] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.421665 13330 consensus_queue.cc:260] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.420918 13332 raft_consensus.cc:2804] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.428716 13332 raft_consensus.cc:493] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.429004 13330 ts_tablet_manager.cc:1434] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:20:08.429322 13332 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.430917 13330 tablet_bootstrap.cc:492] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:08.442904 13330 tablet_bootstrap.cc:654] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.429322 13328 tablet_bootstrap.cc:492] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:08.444103 13328 ts_tablet_manager.cc:1403] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.002s
I20260709 12:20:08.442993 13332 raft_consensus.cc:515] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.446730 13329 tablet_bootstrap.cc:492] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6: No bootstrap required, opened a new log
I20260709 12:20:08.447261 13329 ts_tablet_manager.cc:1403] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.000s
I20260709 12:20:08.449239 13328 raft_consensus.cc:359] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.450115 13328 raft_consensus.cc:385] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.450017 13329 raft_consensus.cc:359] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.450500 13328 raft_consensus.cc:740] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.450721 13329 raft_consensus.cc:385] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.451010 13329 raft_consensus.cc:740] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.451532 13328 consensus_queue.cc:260] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.451810 13329 consensus_queue.cc:260] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.453191 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.453887 13328 ts_tablet_manager.cc:1434] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:20:08.454423 13329 ts_tablet_manager.cc:1434] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:08.455283 13328 tablet_bootstrap.cc:492] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634: Bootstrap starting.
I20260709 12:20:08.454156 13136 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.459403 13332 leader_election.cc:290] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.461923 13328 tablet_bootstrap.cc:654] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.461812 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:08.463585 13136 raft_consensus.cc:2468] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
W20260709 12:20:08.463943 13252 leader_election.cc:343] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.464859 13251 leader_election.cc:304] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.466090 13332 raft_consensus.cc:2804] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.466342 13330 tablet_bootstrap.cc:492] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:08.466789 13332 raft_consensus.cc:697] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:08.467056 13330 ts_tablet_manager.cc:1403] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.000s
I20260709 12:20:08.467718 13332 consensus_queue.cc:237] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.471669 13330 raft_consensus.cc:359] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.473172 13330 raft_consensus.cc:385] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.473613 13330 raft_consensus.cc:740] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.474748 13330 consensus_queue.cc:260] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.478106 13330 ts_tablet_manager.cc:1434] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:20:08.479503 13330 tablet_bootstrap.cc:492] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:08.482376 13328 tablet_bootstrap.cc:492] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634: No bootstrap required, opened a new log
I20260709 12:20:08.482988 13328 ts_tablet_manager.cc:1403] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.015s
I20260709 12:20:08.486364 13328 raft_consensus.cc:359] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.487128 13330 tablet_bootstrap.cc:654] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.487421 13328 raft_consensus.cc:385] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.487913 13328 raft_consensus.cc:740] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.487754 13033 catalog_manager.cc:5697] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.489115 13328 consensus_queue.cc:260] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.492995 13328 ts_tablet_manager.cc:1434] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:20:08.502000 13332 raft_consensus.cc:493] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.502830 13332 raft_consensus.cc:515] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.507716 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b396c16c6d4f91a80832c50df6e865" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" is_pre_election: true
I20260709 12:20:08.508431 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b396c16c6d4f91a80832c50df6e865" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.509522 13136 raft_consensus.cc:2468] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 0.
W20260709 12:20:08.510186 13252 leader_election.cc:343] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.511435 13332 leader_election.cc:290] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.511531 13251 leader_election.cc:304] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.515880 13332 raft_consensus.cc:2804] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.516366 13332 raft_consensus.cc:493] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.516726 13332 raft_consensus.cc:3060] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.517416 13330 tablet_bootstrap.cc:492] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:08.517989 13330 ts_tablet_manager.cc:1403] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.039s	user 0.020s	sys 0.015s
I20260709 12:20:08.519505 13032 catalog_manager.cc:5697] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.521189 13330 raft_consensus.cc:359] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.521966 13330 raft_consensus.cc:385] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.522293 13330 raft_consensus.cc:740] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.523522 13330 consensus_queue.cc:260] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.525599 13332 raft_consensus.cc:515] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.526006 13330 ts_tablet_manager.cc:1434] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:20:08.527459 13330 tablet_bootstrap.cc:492] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:08.528817 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b396c16c6d4f91a80832c50df6e865" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.529477 13136 raft_consensus.cc:3060] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.534665 13330 tablet_bootstrap.cc:654] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.535218 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b396c16c6d4f91a80832c50df6e865" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
W20260709 12:20:08.536950 13252 leader_election.cc:343] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:08.537638 13332 leader_election.cc:290] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:08.538831 13136 raft_consensus.cc:2468] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 1.
I20260709 12:20:08.538955 13035 catalog_manager.cc:5697] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 0 to 1, leader changed from <none> to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:08.540669 13251 leader_election.cc:304] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.540894 13035 catalog_manager.cc:5697] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 0 to 1, leader changed from <none> to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:08.542654 13332 raft_consensus.cc:2804] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.543155 13332 raft_consensus.cc:697] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:08.543913 13330 tablet_bootstrap.cc:492] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:08.544479 13330 ts_tablet_manager.cc:1403] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:20:08.544000 13332 consensus_queue.cc:237] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.547469 13330 raft_consensus.cc:359] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.548426 13330 raft_consensus.cc:385] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.548759 13330 raft_consensus.cc:740] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.549495 13330 consensus_queue.cc:260] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.551560 13330 ts_tablet_manager.cc:1434] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:20:08.552521 13330 tablet_bootstrap.cc:492] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4: Bootstrap starting.
I20260709 12:20:08.557806 13330 tablet_bootstrap.cc:654] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:08.563959 13330 tablet_bootstrap.cc:492] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4: No bootstrap required, opened a new log
I20260709 12:20:08.564630 13330 ts_tablet_manager.cc:1403] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.001s
I20260709 12:20:08.563934 13035 catalog_manager.cc:5697] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 0 to 1, leader changed from <none> to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:08.567548 13330 raft_consensus.cc:359] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.568238 13330 raft_consensus.cc:385] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:08.568663 13330 raft_consensus.cc:740] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Initialized, Role: FOLLOWER
I20260709 12:20:08.569358 13330 consensus_queue.cc:260] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.571411 13330 ts_tablet_manager.cc:1434] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:20:08.645164 13374 raft_consensus.cc:493] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.645699 13374 raft_consensus.cc:515] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.648067 13374 leader_election.cc:290] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.649344 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a7557f7a94dab8d8e500facf13712" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.649540 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a7557f7a94dab8d8e500facf13712" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.650089 13286 raft_consensus.cc:2468] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 0.
I20260709 12:20:08.650089 13136 raft_consensus.cc:2468] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 0.
I20260709 12:20:08.651423 13177 leader_election.cc:304] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.652117 13374 raft_consensus.cc:2804] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:08.652453 13374 raft_consensus.cc:493] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:08.652731 13374 raft_consensus.cc:3060] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.658398 13374 raft_consensus.cc:515] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.659302 13337 raft_consensus.cc:493] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.659811 13337 raft_consensus.cc:515] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.660264 13374 leader_election.cc:290] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.661317 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a7557f7a94dab8d8e500facf13712" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:08.661535 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "512a7557f7a94dab8d8e500facf13712" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:08.662001 13136 raft_consensus.cc:3060] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.662137 13286 raft_consensus.cc:3060] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.662143 13337 leader_election.cc:290] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.663894 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.664259 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.664736 13285 raft_consensus.cc:2393] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a28ad919fa341a8b382a9c1b28b9af4 in current term 1: Already voted for candidate 20c2882af1cb46d6b9b683e1619db3c6 in this term.
I20260709 12:20:08.667776 13177 leader_election.cc:304] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:08.668671 13332 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.669024 13337 raft_consensus.cc:3060] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.671072 13136 raft_consensus.cc:2468] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 1.
I20260709 12:20:08.671820 13286 raft_consensus.cc:2468] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 1.
I20260709 12:20:08.672441 13177 leader_election.cc:304] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:08.674937 13374 raft_consensus.cc:2804] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:08.675657 13374 raft_consensus.cc:697] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Becoming Leader. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:08.676962 13374 consensus_queue.cc:237] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.678968 13337 raft_consensus.cc:2749] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:08.692672 13332 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.700484 13033 catalog_manager.cc:5697] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:08.703157 13369 consensus_queue.cc:1048] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.719290 13366 consensus_queue.cc:1048] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.747319 13374 raft_consensus.cc:493] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.747915 13374 raft_consensus.cc:515] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.749778 13374 leader_election.cc:290] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.751183 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.751302 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.752087 13136 raft_consensus.cc:2393] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a28ad919fa341a8b382a9c1b28b9af4 in current term 1: Already voted for candidate 019deb94fe994e468a6407fa87791634 in this term.
I20260709 12:20:08.753086 13177 leader_election.cc:304] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:08.753762 13374 raft_consensus.cc:3060] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.758926 13374 raft_consensus.cc:2749] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:08.781617 13332 consensus_queue.cc:1048] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.793536 13364 consensus_queue.cc:1048] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.797410 13374 raft_consensus.cc:493] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.798002 13374 raft_consensus.cc:515] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.801407 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.802291 13137 raft_consensus.cc:2393] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a28ad919fa341a8b382a9c1b28b9af4 in current term 1: Already voted for candidate 019deb94fe994e468a6407fa87791634 in this term.
I20260709 12:20:08.802767 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.804833 13177 leader_election.cc:304] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:08.805872 13397 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:08.813773 13374 leader_election.cc:290] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.815048 13397 raft_consensus.cc:2749] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:08.819839 13364 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.832429 13367 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.839676 13397 consensus_queue.cc:1048] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.851761 13367 consensus_queue.cc:1048] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.852643 13332 consensus_queue.cc:1048] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.895851 13397 consensus_queue.cc:1048] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.897835 13286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:08.898623 13286 raft_consensus.cc:606] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:08.909674 13366 consensus_queue.cc:1048] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.916440 13285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6b2ca002693479e8153b1d576421784"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:08.917125 13285 raft_consensus.cc:606] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:08.923588 13286 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:08.924403 13286 raft_consensus.cc:606] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:08.927718 13336 consensus_queue.cc:1048] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:08.943279 13285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:08.944016 13285 raft_consensus.cc:606] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Received request to transfer leadership to TS 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.950637 13285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:08.951254 13285 raft_consensus.cc:606] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Received request to transfer leadership to TS 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:08.990720 13367 consensus_queue.cc:1048] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:08.992767 13414 raft_consensus.cc:493] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:08.993302 13414 raft_consensus.cc:515] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:08.995865 13414 leader_election.cc:290] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:08.997889 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b396c16c6d4f91a80832c50df6e865" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019deb94fe994e468a6407fa87791634" is_pre_election: true
I20260709 12:20:08.997880 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80b396c16c6d4f91a80832c50df6e865" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6" is_pre_election: true
I20260709 12:20:08.999243 13137 raft_consensus.cc:2393] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a28ad919fa341a8b382a9c1b28b9af4 in current term 1: Already voted for candidate 019deb94fe994e468a6407fa87791634 in this term.
I20260709 12:20:09.001039 13367 raft_consensus.cc:993] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: : Instructing follower 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:09.000820 13177 leader_election.cc:304] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:09.001720 13364 raft_consensus.cc:1081] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Signalling peer 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:09.003054 13208 raft_consensus.cc:3060] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:09.006186 13210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:59074
I20260709 12:20:09.006839 13210 raft_consensus.cc:493] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:09.006892 13364 consensus_queue.cc:1048] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:09.007274 13210 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.012411 13337 raft_consensus.cc:2749] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:09.015197 13364 consensus_queue.cc:1048] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:09.020746 13210 raft_consensus.cc:515] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.024555 13210 leader_election.cc:290] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 2 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:09.026322 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:09.026651 13137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:09.027043 13285 raft_consensus.cc:3055] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:09.027261 13137 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.027459 13285 raft_consensus.cc:740] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:09.028519 13285 consensus_queue.cc:260] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.029932 13285 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.051817 13364 consensus_queue.cc:1048] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:09.054100 13285 raft_consensus.cc:2468] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 2.
I20260709 12:20:09.055402 13177 leader_election.cc:304] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:09.056844 13337 raft_consensus.cc:2804] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:09.055402 13137 raft_consensus.cc:2468] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 2.
I20260709 12:20:09.057614 13337 raft_consensus.cc:697] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Becoming Leader. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:09.058607 13337 consensus_queue.cc:237] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.070598 13035 catalog_manager.cc:5697] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 reported cstate change: term changed from 1 to 2, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.093047 13337 consensus_queue.cc:1048] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:09.105015 13414 consensus_queue.cc:1048] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:09.195555 13363 raft_consensus.cc:993] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:09.196074 13364 raft_consensus.cc:1081] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:09.198335 13136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6b2ca002693479e8153b1d576421784"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:20:09.198906 13136 raft_consensus.cc:493] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:09.199246 13136 raft_consensus.cc:3060] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.204485 13136 raft_consensus.cc:515] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.206472 13136 leader_election.cc:290] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:09.207442 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:09.207671 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6b2ca002693479e8153b1d576421784" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:09.208210 13208 raft_consensus.cc:3060] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.208484 13285 raft_consensus.cc:3055] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:09.209000 13285 raft_consensus.cc:740] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:09.209851 13285 consensus_queue.cc:260] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.211084 13285 raft_consensus.cc:3060] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.214535 13208 raft_consensus.cc:2468] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:09.216251 13103 leader_election.cc:304] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:09.217228 13285 raft_consensus.cc:2468] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:09.217286 13336 raft_consensus.cc:2804] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:09.218108 13336 raft_consensus.cc:697] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:09.219045 13336 consensus_queue.cc:237] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.220422 13367 raft_consensus.cc:993] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:09.220980 13367 raft_consensus.cc:1081] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:09.222770 13136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:20:09.223584 13136 raft_consensus.cc:493] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:09.224123 13136 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.227732 13035 catalog_manager.cc:5697] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 1 to 2, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.229900 13136 raft_consensus.cc:515] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.232299 13136 leader_election.cc:290] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:09.232980 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:09.232779 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:09.233755 13208 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.233577 13285 raft_consensus.cc:3055] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:09.234543 13285 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:09.235400 13285 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.236748 13285 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.241775 13208 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:09.243347 13103 leader_election.cc:304] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:09.244221 13336 raft_consensus.cc:2804] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:09.244503 13285 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:09.244725 13336 raft_consensus.cc:697] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:09.245755 13336 consensus_queue.cc:237] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.253074 13035 catalog_manager.cc:5697] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 1 to 2, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.301999 13364 raft_consensus.cc:993] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: : Instructing follower 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:09.302631 13367 raft_consensus.cc:1081] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Signalling peer 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:09.304515 13208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:59074
I20260709 12:20:09.305208 13208 raft_consensus.cc:493] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:09.305604 13208 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.312139 13208 raft_consensus.cc:515] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.314306 13208 leader_election.cc:290] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 2 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:09.315522 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:09.315374 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:09.316098 13285 raft_consensus.cc:3055] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:09.316325 13136 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.316455 13285 raft_consensus.cc:740] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:09.317353 13285 consensus_queue.cc:260] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.318403 13285 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.322067 13136 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 2.
I20260709 12:20:09.324186 13177 leader_election.cc:304] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:09.325075 13397 raft_consensus.cc:2804] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:09.325187 13285 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 2.
I20260709 12:20:09.325654 13397 raft_consensus.cc:697] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Becoming Leader. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:09.326812 13397 consensus_queue.cc:237] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:09.333658 13035 catalog_manager.cc:5697] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 reported cstate change: term changed from 1 to 2, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:09.432196 13364 raft_consensus.cc:993] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:09.432744 13367 raft_consensus.cc:1081] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:09.434530 13136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:20:09.435072 13136 raft_consensus.cc:493] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:09.435420 13136 raft_consensus.cc:3060] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.441049 13136 raft_consensus.cc:515] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:09.443408 13136 leader_election.cc:290] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Requested vote from peers 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:09.444741 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:09.444746 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ab74cddcf49ab870e1d92f7dc43c6" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:09.445513 13285 raft_consensus.cc:3055] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:09.445590 13208 raft_consensus.cc:3060] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.445935 13285 raft_consensus.cc:740] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:09.447016 13285 consensus_queue.cc:260] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:09.448431 13285 raft_consensus.cc:3060] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:09.452960 13208 raft_consensus.cc:2468] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:09.454437 13103 leader_election.cc:304] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:09.455339 13366 raft_consensus.cc:2804] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:09.455859 13366 raft_consensus.cc:697] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:09.456838 13285 raft_consensus.cc:2468] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:09.456705 13366 consensus_queue.cc:237] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } }
I20260709 12:20:09.463879 13035 catalog_manager.cc:5697] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 1 to 2, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:09.625818 13285 raft_consensus.cc:1275] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.627379 13366 consensus_queue.cc:1048] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.636744 13208 raft_consensus.cc:1275] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.638535 13369 consensus_queue.cc:1048] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.644542 13136 raft_consensus.cc:1275] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.646121 13397 consensus_queue.cc:1048] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.658529 13285 raft_consensus.cc:1275] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.660076 13397 consensus_queue.cc:1048] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.723613 13285 raft_consensus.cc:1275] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.723843 13286 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.725315 13397 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:09.725493 13336 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.729140 13208 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.731716 13369 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:09.739601 13136 raft_consensus.cc:1275] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.742415 13337 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.854856 13208 raft_consensus.cc:1275] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.856323 13336 consensus_queue.cc:1048] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:09.867295 13285 raft_consensus.cc:1275] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:09.868955 13429 consensus_queue.cc:1048] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:11.998227 12469 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:20:11.999059 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:12.018406 13136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
mode: GRACEFUL
new_leader_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:37892
I20260709 12:20:12.019228 13136 raft_consensus.cc:606] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Received request to transfer leadership to TS 019deb94fe994e468a6407fa87791634
I20260709 12:20:12.022154 13136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
mode: GRACEFUL
new_leader_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:37892
I20260709 12:20:12.022907 13136 raft_consensus.cc:606] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Received request to transfer leadership to TS 019deb94fe994e468a6407fa87791634
I20260709 12:20:12.024155 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:20:12.024626 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:12.415342 13429 raft_consensus.cc:993] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: : Instructing follower 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:12.415934 13431 raft_consensus.cc:1081] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Signalling peer 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:12.417412 13286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4"
dest_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:50010
I20260709 12:20:12.417886 13286 raft_consensus.cc:493] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:12.418184 13286 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:12.422879 13286 raft_consensus.cc:515] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:12.424644 13286 leader_election.cc:290] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 2 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:12.425681 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:12.425949 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:12.426347 13136 raft_consensus.cc:3055] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:12.426638 13208 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:12.426679 13136 raft_consensus.cc:740] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:12.427702 13136 consensus_queue.cc:260] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:12.428653 13136 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:12.432132 13208 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 2.
I20260709 12:20:12.433355 13252 leader_election.cc:304] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:12.433862 13136 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 2.
I20260709 12:20:12.434446 13363 raft_consensus.cc:2804] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:12.435823 13363 raft_consensus.cc:697] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:12.436585 13363 consensus_queue.cc:237] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:12.446844 13035 catalog_manager.cc:5697] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 1 to 2, leader changed from 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65) to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:12.466616 13429 raft_consensus.cc:993] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: : Instructing follower 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:12.467110 13431 raft_consensus.cc:1081] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Signalling peer 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:12.468647 13286 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:50010
I20260709 12:20:12.469116 13286 raft_consensus.cc:493] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:12.469378 13286 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:12.475448 13286 raft_consensus.cc:515] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:12.477421 13286 leader_election.cc:290] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 3 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:12.478482 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:12.478745 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:12.479493 13208 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:12.479430 13136 raft_consensus.cc:3055] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:12.480065 13136 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:12.480865 13136 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:12.482024 13136 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:12.486231 13208 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 3.
I20260709 12:20:12.487542 13252 leader_election.cc:304] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:12.488118 13136 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 3.
I20260709 12:20:12.488677 13363 raft_consensus.cc:2804] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:12.489240 13363 raft_consensus.cc:697] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:12.490237 13363 consensus_queue.cc:237] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:12.497849 13035 catalog_manager.cc:5697] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 2 to 3, leader changed from 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65) to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:12.909639 13136 raft_consensus.cc:1275] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:12.911159 13363 consensus_queue.cc:1048] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:12.922828 13208 raft_consensus.cc:1275] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:12.924176 13363 consensus_queue.cc:1048] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:13.025712 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:13.029637 13136 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:13.031144 13363 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:13.032591 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:13.033214 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:13.040727 13208 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:13.044020 13445 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:14.034376 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:14.040028 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:14.040422 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:15.041543 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:15.048692 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:15.049234 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:16.050515 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:16.056720 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:16.057181 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:17.058228 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:17.063503 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:17.063993 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:18.065142 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:18.072654 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:18.073247 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:19.074524 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:19.081715 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:19.082161 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:20.083261 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:20.090183 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:20.090737 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:21.091656 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:21.098764 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:21.099167 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:22.137626 12469 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:20:22.153303 13285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:22.154080 13285 raft_consensus.cc:606] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:22.160394 13285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4"
dest_uuid: "019deb94fe994e468a6407fa87791634"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:50024
I20260709 12:20:22.160935 13285 raft_consensus.cc:606] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:22.179719 13208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:47962
I20260709 12:20:22.180475 13208 raft_consensus.cc:606] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:22.185889 13208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:47962
I20260709 12:20:22.186445 13208 raft_consensus.cc:606] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:22.191725 13208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
mode: GRACEFUL
new_leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:47962
I20260709 12:20:22.192252 13208 raft_consensus.cc:606] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Received request to transfer leadership to TS 20c2882af1cb46d6b9b683e1619db3c6
I20260709 12:20:22.261420 13459 raft_consensus.cc:993] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.261889 13459 raft_consensus.cc:1081] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.263582 13137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37886
I20260709 12:20:22.264093 13137 raft_consensus.cc:493] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:22.264411 13137 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.269538 13137 raft_consensus.cc:515] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.271238 13137 leader_election.cc:290] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 3 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:22.272470 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:22.272495 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "306360dadd3244c9968d3eeb1c836413" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:22.273479 13285 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.273470 13208 raft_consensus.cc:3055] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:22.274034 13208 raft_consensus.cc:740] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:22.275014 13208 consensus_queue.cc:260] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.276409 13208 raft_consensus.cc:3060] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.281831 13285 raft_consensus.cc:2468] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 3.
I20260709 12:20:22.283496 13102 leader_election.cc:304] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:22.284524 13455 raft_consensus.cc:2804] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:22.284670 13208 raft_consensus.cc:2468] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 3.
I20260709 12:20:22.285936 13455 raft_consensus.cc:697] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:22.286916 13455 consensus_queue.cc:237] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.295249 13033 catalog_manager.cc:5697] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 2 to 3, leader changed from 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.468520 13462 raft_consensus.cc:993] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.469254 13458 raft_consensus.cc:993] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.469131 13459 raft_consensus.cc:1081] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.469976 13469 raft_consensus.cc:1081] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.471391 13136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37886
I20260709 12:20:22.471943 13136 raft_consensus.cc:493] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:22.471832 13137 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:20:22.472359 13136 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.472595 13137 raft_consensus.cc:493] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:22.473001 13137 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.477950 13137 raft_consensus.cc:515] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.478232 13136 raft_consensus.cc:515] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.479703 13137 leader_election.cc:290] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 3 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:22.480950 13136 leader_election.cc:290] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 3 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:22.480736 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:22.481814 13210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:22.481877 13208 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.482622 13210 raft_consensus.cc:3055] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:22.483047 13210 raft_consensus.cc:740] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:22.483851 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "331728bfdb514c50b772b2186b20a7a4" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:22.484508 13285 raft_consensus.cc:3055] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:22.483892 13210 consensus_queue.cc:260] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.484833 13285 raft_consensus.cc:740] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:22.485486 13210 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.485697 13285 consensus_queue.cc:260] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.487031 13285 raft_consensus.cc:3060] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.487666 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:22.488261 13286 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:22.490360 13459 raft_consensus.cc:993] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.490815 13459 raft_consensus.cc:1081] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.492574 13136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37886
I20260709 12:20:22.493111 13136 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:22.493451 13136 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:22.494184 13210 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 3.
I20260709 12:20:22.494860 13285 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 3.
I20260709 12:20:22.495563 13103 leader_election.cc:304] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:22.496153 13102 leader_election.cc:304] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:22.496479 13286 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 3.
I20260709 12:20:22.497072 13455 raft_consensus.cc:2804] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:22.498343 13463 raft_consensus.cc:2804] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:22.499639 13455 raft_consensus.cc:697] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:22.499749 13208 raft_consensus.cc:2468] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 3.
I20260709 12:20:22.500900 13463 raft_consensus.cc:697] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:22.500646 13455 consensus_queue.cc:237] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.501820 13463 consensus_queue.cc:237] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.509788 13136 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.513666 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:22.514006 13136 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:22.514284 13208 raft_consensus.cc:3055] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:22.514652 13208 raft_consensus.cc:740] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:22.515270 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:22.515811 13286 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:22.515602 13208 consensus_queue.cc:260] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.516782 13208 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:22.519107 13033 catalog_manager.cc:5697] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 2 to 3, leader changed from 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.520519 13033 catalog_manager.cc:5697] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 2 to 3, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.522089 13286 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:22.522611 13208 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 2.
I20260709 12:20:22.523514 13102 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 20c2882af1cb46d6b9b683e1619db3c6; no voters: 
I20260709 12:20:22.525856 13463 raft_consensus.cc:2804] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:22.526350 13463 raft_consensus.cc:697] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:22.527215 13463 consensus_queue.cc:237] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.528945 13458 raft_consensus.cc:993] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: : Instructing follower 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.529516 13469 raft_consensus.cc:1081] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 LEADER]: Signalling peer 20c2882af1cb46d6b9b683e1619db3c6 to start an election
I20260709 12:20:22.531572 13136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
 from {username='slave'} at 127.0.0.1:37890
I20260709 12:20:22.532754 13136 raft_consensus.cc:493] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:22.533077 13136 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:22.536783 13033 catalog_manager.cc:5697] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 1 to 2, leader changed from 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.538800 13136 raft_consensus.cc:515] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.540999 13136 leader_election.cc:290] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 4 election: Requested vote from peers 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:22.541683 13208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:22.542377 13208 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:22.544620 13286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "20c2882af1cb46d6b9b683e1619db3c6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:22.545177 13286 raft_consensus.cc:3055] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:22.545506 13286 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:22.546428 13286 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.547835 13286 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:22.548141 13208 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 4.
I20260709 12:20:22.549356 13103 leader_election.cc:304] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20c2882af1cb46d6b9b683e1619db3c6, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:22.550074 13463 raft_consensus.cc:2804] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:22.550635 13463 raft_consensus.cc:697] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 LEADER]: Becoming Leader. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:22.551592 13463 consensus_queue.cc:237] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:22.555514 13286 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20c2882af1cb46d6b9b683e1619db3c6 in term 4.
I20260709 12:20:22.560577 13033 catalog_manager.cc:5697] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 reported cstate change: term changed from 3 to 4, leader changed from 019deb94fe994e468a6407fa87791634 (127.12.45.67) to 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65). New cstate: current_term: 4 leader_uuid: "20c2882af1cb46d6b9b683e1619db3c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:22.816001 13208 raft_consensus.cc:1275] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:22.817384 13463 consensus_queue.cc:1048] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:22.828286 13286 raft_consensus.cc:1275] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:22.829691 13463 consensus_queue.cc:1048] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:22.970805 13286 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:22.972237 13455 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:22.982039 13208 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:22.983915 13471 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:22.985091 13208 raft_consensus.cc:1275] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:22.987412 13463 consensus_queue.cc:1048] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:22.996919 13208 raft_consensus.cc:1275] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:22.998435 13470 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:23.001905 13286 raft_consensus.cc:1275] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:23.003944 13463 consensus_queue.cc:1048] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:23.007704 13212 raft_consensus.cc:1275] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:23.009703 13485 consensus_queue.cc:1048] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:23.018932 13285 raft_consensus.cc:1275] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:23.021263 13485 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:23.053390 13285 raft_consensus.cc:1275] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 20c2882af1cb46d6b9b683e1619db3c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:23.055343 13484 consensus_queue.cc:1048] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:25.237453 12469 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:20:25.238610 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:25.247625 13136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
mode: GRACEFUL
new_leader_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:37892
I20260709 12:20:25.248638 13136 raft_consensus.cc:606] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 LEADER]: Received request to transfer leadership to TS 019deb94fe994e468a6407fa87791634
I20260709 12:20:25.251991 13136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
mode: GRACEFUL
new_leader_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:37892
I20260709 12:20:25.252532 13136 raft_consensus.cc:606] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Received request to transfer leadership to TS 019deb94fe994e468a6407fa87791634
I20260709 12:20:25.255477 13136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
mode: GRACEFUL
new_leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:37892
I20260709 12:20:25.256004 13136 raft_consensus.cc:606] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Received request to transfer leadership to TS 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:25.258953 13136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61"
dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
mode: GRACEFUL
new_leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:37892
I20260709 12:20:25.259434 13136 raft_consensus.cc:606] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Received request to transfer leadership to TS 2a28ad919fa341a8b382a9c1b28b9af4
I20260709 12:20:25.260653 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:20:25.261198 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:25.604225 13471 raft_consensus.cc:993] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: : Instructing follower 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:25.604846 13484 raft_consensus.cc:1081] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Signalling peer 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:25.606915 13285 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a"
dest_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:50010
I20260709 12:20:25.607690 13285 raft_consensus.cc:493] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:25.608120 13285 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:25.615070 13285 raft_consensus.cc:515] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.617615 13285 leader_election.cc:290] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 4 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:25.618959 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:25.618966 13210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d827b78a43c2417484eb0f0db56a2f0a" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:25.619779 13136 raft_consensus.cc:3055] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:20:25.619844 13210 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:25.620180 13136 raft_consensus.cc:740] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:25.621297 13136 consensus_queue.cc:260] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.622844 13136 raft_consensus.cc:3060] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:20:25.625761 13210 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 4.
I20260709 12:20:25.627223 13252 leader_election.cc:304] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:25.628089 13469 raft_consensus.cc:2804] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:20:25.629390 13469 raft_consensus.cc:697] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 4 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:25.630570 13469 consensus_queue.cc:237] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.631356 13136 raft_consensus.cc:2468] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 4.
I20260709 12:20:25.638718 13471 raft_consensus.cc:993] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: : Instructing follower 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:25.639307 13484 raft_consensus.cc:1081] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Signalling peer 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:25.639855 13033 catalog_manager.cc:5697] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 3 to 4, leader changed from 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65) to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 4 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:25.641331 13210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:59068
I20260709 12:20:25.641943 13210 raft_consensus.cc:493] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:25.642349 13210 raft_consensus.cc:3060] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:25.648103 13210 raft_consensus.cc:515] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.651298 13210 leader_election.cc:290] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 2 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:25.652361 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:25.652369 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e590b63ed3d84c04aa4413b90cfa83bb" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:25.653229 13136 raft_consensus.cc:3055] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:25.653596 13285 raft_consensus.cc:3060] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:25.653759 13136 raft_consensus.cc:740] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:25.654927 13136 consensus_queue.cc:260] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.656260 13136 raft_consensus.cc:3060] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:25.659629 13285 raft_consensus.cc:2468] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 2.
I20260709 12:20:25.661065 13177 leader_election.cc:304] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:25.662012 13459 raft_consensus.cc:2804] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:25.663563 13459 raft_consensus.cc:697] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Becoming Leader. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:25.664052 13471 raft_consensus.cc:993] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: : Instructing follower 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:25.664624 13484 raft_consensus.cc:1081] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 LEADER]: Signalling peer 019deb94fe994e468a6407fa87791634 to start an election
I20260709 12:20:25.664563 13459 consensus_queue.cc:237] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.664247 13136 raft_consensus.cc:2468] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 2.
I20260709 12:20:25.666589 13285 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d"
dest_uuid: "019deb94fe994e468a6407fa87791634"
 from {username='slave'} at 127.0.0.1:50010
I20260709 12:20:25.667032 13285 raft_consensus.cc:493] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:25.667328 13285 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:25.673084 13285 raft_consensus.cc:515] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.673879 13033 catalog_manager.cc:5697] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 reported cstate change: term changed from 1 to 2, leader changed from 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65) to 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:25.675207 13285 leader_election.cc:290] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 5 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085)
I20260709 12:20:25.676239 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:25.676309 13210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e3ef3c2f65044ca89fcec134898704d" candidate_uuid: "019deb94fe994e468a6407fa87791634" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
I20260709 12:20:25.677078 13136 raft_consensus.cc:3055] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:20:25.677098 13210 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:25.677513 13136 raft_consensus.cc:740] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:25.678234 13136 consensus_queue.cc:260] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.679194 13136 raft_consensus.cc:3060] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:20:25.683362 13210 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 5.
I20260709 12:20:25.684471 13252 leader_election.cc:304] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:25.685233 13469 raft_consensus.cc:2804] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:20:25.685700 13469 raft_consensus.cc:697] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 5 LEADER]: Becoming Leader. State: Replica: 019deb94fe994e468a6407fa87791634, State: Running, Role: LEADER
I20260709 12:20:25.686789 13136 raft_consensus.cc:2468] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019deb94fe994e468a6407fa87791634 in term 5.
I20260709 12:20:25.686679 13469 consensus_queue.cc:237] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.695570 13033 catalog_manager.cc:5697] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 reported cstate change: term changed from 4 to 5, leader changed from 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65) to 019deb94fe994e468a6407fa87791634 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "019deb94fe994e468a6407fa87791634" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:25.716262 13471 raft_consensus.cc:993] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: : Instructing follower 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:25.716812 13471 raft_consensus.cc:1081] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Signalling peer 2a28ad919fa341a8b382a9c1b28b9af4 to start an election
I20260709 12:20:25.718456 13210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61"
dest_uuid: "2a28ad919fa341a8b382a9c1b28b9af4"
 from {username='slave'} at 127.0.0.1:59068
I20260709 12:20:25.718972 13210 raft_consensus.cc:493] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:20:25.719427 13210 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:25.724586 13210 raft_consensus.cc:515] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.726163 13210 leader_election.cc:290] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 3 election: Requested vote from peers 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355), 019deb94fe994e468a6407fa87791634 (127.12.45.67:42297)
I20260709 12:20:25.727244 13136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20c2882af1cb46d6b9b683e1619db3c6"
I20260709 12:20:25.727429 13285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4198ae3f50142469c2ea4e969a89d61" candidate_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019deb94fe994e468a6407fa87791634"
I20260709 12:20:25.727973 13136 raft_consensus.cc:3055] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:25.728020 13285 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:25.728340 13136 raft_consensus.cc:740] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20c2882af1cb46d6b9b683e1619db3c6, State: Running, Role: LEADER
I20260709 12:20:25.729503 13136 consensus_queue.cc:260] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.730755 13136 raft_consensus.cc:3060] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:25.733435 13285 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 3.
I20260709 12:20:25.734730 13177 leader_election.cc:304] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019deb94fe994e468a6407fa87791634, 2a28ad919fa341a8b382a9c1b28b9af4; no voters: 
I20260709 12:20:25.735579 13459 raft_consensus.cc:2804] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:25.736141 13459 raft_consensus.cc:697] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 LEADER]: Becoming Leader. State: Replica: 2a28ad919fa341a8b382a9c1b28b9af4, State: Running, Role: LEADER
I20260709 12:20:25.737118 13459 consensus_queue.cc:237] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } }
I20260709 12:20:25.737900 13136 raft_consensus.cc:2468] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a28ad919fa341a8b382a9c1b28b9af4 in term 3.
I20260709 12:20:25.745122 13033 catalog_manager.cc:5697] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 reported cstate change: term changed from 2 to 3, leader changed from 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65) to 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:26.080530 13136 raft_consensus.cc:1275] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:26.082353 13459 consensus_queue.cc:1048] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:26.085832 13210 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 5 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:20:26.087757 13469 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:20:26.098102 13285 raft_consensus.cc:1275] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:26.100294 13459 consensus_queue.cc:1048] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:26.111954 13136 raft_consensus.cc:1275] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 5 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:20:26.113561 13469 consensus_queue.cc:1048] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:20:26.123405 13210 raft_consensus.cc:1275] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:26.125432 13507 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a28ad919fa341a8b382a9c1b28b9af4" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:26.140381 13136 raft_consensus.cc:1275] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Refusing update from remote peer 019deb94fe994e468a6407fa87791634: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:20:26.141974 13469 consensus_queue.cc:1048] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:26.201781 13285 raft_consensus.cc:1275] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:26.203670 13498 consensus_queue.cc:1048] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "019deb94fe994e468a6407fa87791634" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:26.213941 13136 raft_consensus.cc:1275] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Refusing update from remote peer 2a28ad919fa341a8b382a9c1b28b9af4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:26.215312 13498 consensus_queue.cc:1048] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c2882af1cb46d6b9b683e1619db3c6" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:26.262441 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:26.268203 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:26.268755 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:27.269697 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:27.275244 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:27.275732 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:28.276815 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:28.282560 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:28.283106 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:29.284322 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:29.290824 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:29.291365 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:30.292508 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:30.298527 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:30.299068 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:31.300237 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:31.305922 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:31.306501 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:32.307827 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:32.314625 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:32.315142 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:33.316154 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:33.322335 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:33.322798 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:34.323855 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:20:34.330610 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:20:34.331072 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:20:35.371060 12469 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:20:35.371842 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
W20260709 12:20:35.389024 13251 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260709 12:20:35.407042 13251 consensus_peers.cc:597] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 -> Peer 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355): Couldn't send request to peer 20c2882af1cb46d6b9b683e1619db3c6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:35.427677 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:35.428632 12469 tablet_replica.cc:333] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.429343 12469 raft_consensus.cc:2243] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.430176 12469 raft_consensus.cc:2272] T 80b396c16c6d4f91a80832c50df6e865 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.433282 12469 tablet_replica.cc:333] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
W20260709 12:20:35.433562 13251 consensus_peers.cc:597] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 -> Peer 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355): Couldn't send request to peer 20c2882af1cb46d6b9b683e1619db3c6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:35.433904 12469 raft_consensus.cc:2243] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.434973 12469 raft_consensus.cc:2272] T 912ab74cddcf49ab870e1d92f7dc43c6 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.437130 12469 tablet_replica.cc:333] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.437700 12469 raft_consensus.cc:2243] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.438203 12469 raft_consensus.cc:2272] T e590b63ed3d84c04aa4413b90cfa83bb P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.440267 12469 tablet_replica.cc:333] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.440842 12469 raft_consensus.cc:2243] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.441293 12469 raft_consensus.cc:2272] T 512a7557f7a94dab8d8e500facf13712 P 20c2882af1cb46d6b9b683e1619db3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.443523 12469 tablet_replica.cc:333] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.444106 12469 raft_consensus.cc:2243] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.445106 12469 raft_consensus.cc:2272] T 306360dadd3244c9968d3eeb1c836413 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.447000 12469 tablet_replica.cc:333] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.447584 12469 raft_consensus.cc:2243] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.448050 12469 raft_consensus.cc:2272] T 6e3ef3c2f65044ca89fcec134898704d P 20c2882af1cb46d6b9b683e1619db3c6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.450399 12469 tablet_replica.cc:333] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.450947 12469 raft_consensus.cc:2243] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.451406 12469 raft_consensus.cc:2272] T d827b78a43c2417484eb0f0db56a2f0a P 20c2882af1cb46d6b9b683e1619db3c6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.453822 12469 tablet_replica.cc:333] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.454456 12469 raft_consensus.cc:2243] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.455323 12469 raft_consensus.cc:2272] T c6b2ca002693479e8153b1d576421784 P 20c2882af1cb46d6b9b683e1619db3c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.457830 12469 tablet_replica.cc:333] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.458527 12469 raft_consensus.cc:2243] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.459416 12469 raft_consensus.cc:2272] T 331728bfdb514c50b772b2186b20a7a4 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.461506 12469 tablet_replica.cc:333] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6: stopping tablet replica
I20260709 12:20:35.462087 12469 raft_consensus.cc:2243] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.462669 12469 raft_consensus.cc:2272] T f4198ae3f50142469c2ea4e969a89d61 P 20c2882af1cb46d6b9b683e1619db3c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.496822 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
I20260709 12:20:35.520040 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
I20260709 12:20:35.555749 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:35.556591 12469 tablet_replica.cc:333] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.557524 12469 raft_consensus.cc:2243] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.558210 12469 raft_consensus.cc:2272] T 80b396c16c6d4f91a80832c50df6e865 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.560534 12469 tablet_replica.cc:333] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.561117 12469 raft_consensus.cc:2243] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.562033 12469 raft_consensus.cc:2272] T e590b63ed3d84c04aa4413b90cfa83bb P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.564116 12469 tablet_replica.cc:333] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.564736 12469 raft_consensus.cc:2243] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.565387 12469 raft_consensus.cc:2272] T c6b2ca002693479e8153b1d576421784 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.567468 12469 tablet_replica.cc:333] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.568177 12469 raft_consensus.cc:2243] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.568723 12469 raft_consensus.cc:2272] T d827b78a43c2417484eb0f0db56a2f0a P 2a28ad919fa341a8b382a9c1b28b9af4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.570861 12469 tablet_replica.cc:333] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.571440 12469 raft_consensus.cc:2243] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.571998 12469 raft_consensus.cc:2272] T 306360dadd3244c9968d3eeb1c836413 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.573916 12469 tablet_replica.cc:333] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.574544 12469 raft_consensus.cc:2243] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.575384 12469 raft_consensus.cc:2272] T 512a7557f7a94dab8d8e500facf13712 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.577320 12469 tablet_replica.cc:333] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.577836 12469 raft_consensus.cc:2243] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.578229 12469 raft_consensus.cc:2272] T 6e3ef3c2f65044ca89fcec134898704d P 2a28ad919fa341a8b382a9c1b28b9af4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.580052 12469 tablet_replica.cc:333] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.580539 12469 raft_consensus.cc:2243] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.580958 12469 raft_consensus.cc:2272] T 331728bfdb514c50b772b2186b20a7a4 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.582852 12469 tablet_replica.cc:333] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.583340 12469 raft_consensus.cc:2243] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.584218 12469 raft_consensus.cc:2272] T f4198ae3f50142469c2ea4e969a89d61 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.586077 12469 tablet_replica.cc:333] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4: stopping tablet replica
I20260709 12:20:35.586588 12469 raft_consensus.cc:2243] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.587039 12469 raft_consensus.cc:2272] T 912ab74cddcf49ab870e1d92f7dc43c6 P 2a28ad919fa341a8b382a9c1b28b9af4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.616395 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:20:35.637216 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
W20260709 12:20:35.656294 13251 consensus_peers.cc:597] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 -> Peer 20c2882af1cb46d6b9b683e1619db3c6 (127.12.45.65:45355): Couldn't send request to peer 20c2882af1cb46d6b9b683e1619db3c6. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45355: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:35.674664 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:20:35.675566 12469 tablet_replica.cc:333] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.676265 12469 raft_consensus.cc:2243] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.677188 12469 raft_consensus.cc:2272] T 80b396c16c6d4f91a80832c50df6e865 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.679307 12469 tablet_replica.cc:333] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.679841 12469 raft_consensus.cc:2243] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.680303 12469 raft_consensus.cc:2272] T f4198ae3f50142469c2ea4e969a89d61 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.682242 12469 tablet_replica.cc:333] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.682794 12469 raft_consensus.cc:2243] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.683220 12469 raft_consensus.cc:2272] T 331728bfdb514c50b772b2186b20a7a4 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.685427 12469 tablet_replica.cc:333] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.685983 12469 raft_consensus.cc:2243] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.686843 12469 raft_consensus.cc:2272] T d827b78a43c2417484eb0f0db56a2f0a P 019deb94fe994e468a6407fa87791634 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.688817 12469 tablet_replica.cc:333] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
W20260709 12:20:35.689255 13252 consensus_peers.cc:597] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 -> Peer 2a28ad919fa341a8b382a9c1b28b9af4 (127.12.45.66:35085): Couldn't send request to peer 2a28ad919fa341a8b382a9c1b28b9af4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:35085: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:35.689349 12469 raft_consensus.cc:2243] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.689926 12469 raft_consensus.cc:2272] T c6b2ca002693479e8153b1d576421784 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.691776 12469 tablet_replica.cc:333] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.692346 12469 raft_consensus.cc:2243] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.692791 12469 raft_consensus.cc:2272] T 512a7557f7a94dab8d8e500facf13712 P 019deb94fe994e468a6407fa87791634 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.694679 12469 tablet_replica.cc:333] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.695205 12469 raft_consensus.cc:2243] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.695644 12469 raft_consensus.cc:2272] T 306360dadd3244c9968d3eeb1c836413 P 019deb94fe994e468a6407fa87791634 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.697387 12469 tablet_replica.cc:333] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.698112 12469 raft_consensus.cc:2243] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.699266 12469 raft_consensus.cc:2272] T 6e3ef3c2f65044ca89fcec134898704d P 019deb94fe994e468a6407fa87791634 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.701203 12469 tablet_replica.cc:333] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.701771 12469 raft_consensus.cc:2243] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.702204 12469 raft_consensus.cc:2272] T e590b63ed3d84c04aa4413b90cfa83bb P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.704399 12469 tablet_replica.cc:333] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634: stopping tablet replica
I20260709 12:20:35.704975 12469 raft_consensus.cc:2243] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:35.705458 12469 raft_consensus.cc:2272] T 912ab74cddcf49ab870e1d92f7dc43c6 P 019deb94fe994e468a6407fa87791634 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.739238 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:20:35.764123 12469 master.cc:562] Master@127.12.45.126:33649 shutting down...
I20260709 12:20:35.796662 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:20:35.797308 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:35.797641 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P cdc4b94fd5c84a07ba5248ac42e09786: stopping tablet replica
I20260709 12:20:35.820565 12469 master.cc:584] Master@127.12.45.126:33649 shutdown complete.
019deb94fe994e468a6407fa87791634  1
20c2882af1cb46d6b9b683e1619db3c6  5
2a28ad919fa341a8b382a9c1b28b9af4  4
019deb94fe994e468a6407fa87791634  3
20c2882af1cb46d6b9b683e1619db3c6  3
2a28ad919fa341a8b382a9c1b28b9af4  4
019deb94fe994e468a6407fa87791634  1
20c2882af1cb46d6b9b683e1619db3c6  8
2a28ad919fa341a8b382a9c1b28b9af4  1
019deb94fe994e468a6407fa87791634  3
20c2882af1cb46d6b9b683e1619db3c6  4
2a28ad919fa341a8b382a9c1b28b9af4  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29234 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:20:35.867070 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:44051
I20260709 12:20:35.868237 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:35.875078 13525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:35.875299 13526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:35.875814 13528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:35.877936 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:35.879019 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:35.879212 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:35.879441 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599635879423 us; error 0 us; skew 500 ppm
I20260709 12:20:35.880030 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:35.883309 12469 webserver.cc:533] Webserver started at http://127.12.45.126:44881/ using document root <none> and password file <none>
I20260709 12:20:35.883842 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:35.884037 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:35.884356 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:35.885536 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "d5e54c6d2c954fbf93d5136276b7b78c"
format_stamp: "Formatted at 2026-07-09 12:20:35 on dist-test-slave-90bm"
I20260709 12:20:35.890411 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:20:35.894167 13533 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:35.895373 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:20:35.895681 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "d5e54c6d2c954fbf93d5136276b7b78c"
format_stamp: "Formatted at 2026-07-09 12:20:35 on dist-test-slave-90bm"
I20260709 12:20:35.896106 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:35.912511 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:35.913909 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:35.961562 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:44051
I20260709 12:20:35.961835 13584 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:44051 every 8 connection(s)
I20260709 12:20:35.966146 13585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:35.978569 13585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Bootstrap starting.
I20260709 12:20:35.983959 13585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:35.988794 13585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: No bootstrap required, opened a new log
I20260709 12:20:35.991643 13585 raft_consensus.cc:359] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" member_type: VOTER }
I20260709 12:20:35.992184 13585 raft_consensus.cc:385] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:35.992419 13585 raft_consensus.cc:740] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e54c6d2c954fbf93d5136276b7b78c, State: Initialized, Role: FOLLOWER
I20260709 12:20:35.993037 13585 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" member_type: VOTER }
I20260709 12:20:35.993837 13585 raft_consensus.cc:399] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:20:35.994125 13585 raft_consensus.cc:493] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:20:35.994509 13585 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:35.999815 13585 raft_consensus.cc:515] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" member_type: VOTER }
I20260709 12:20:36.000438 13585 leader_election.cc:304] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5e54c6d2c954fbf93d5136276b7b78c; no voters: 
I20260709 12:20:36.001794 13585 leader_election.cc:290] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:20:36.002276 13588 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:36.003671 13588 raft_consensus.cc:697] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 1 LEADER]: Becoming Leader. State: Replica: d5e54c6d2c954fbf93d5136276b7b78c, State: Running, Role: LEADER
I20260709 12:20:36.004441 13588 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" member_type: VOTER }
I20260709 12:20:36.005131 13585 sys_catalog.cc:565] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:20:36.009315 13590 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5e54c6d2c954fbf93d5136276b7b78c. Latest consensus state: current_term: 1 leader_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" member_type: VOTER } }
I20260709 12:20:36.009341 13589 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e54c6d2c954fbf93d5136276b7b78c" member_type: VOTER } }
I20260709 12:20:36.010192 13589 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:36.010938 13590 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [sys.catalog]: This master's current role is: LEADER
I20260709 12:20:36.015115 13596 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:20:36.024024 13596 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:20:36.025233 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:20:36.034247 13596 catalog_manager.cc:1383] Generated new cluster ID: f35849a8594847fab6236d811311760c
I20260709 12:20:36.034633 13596 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:20:36.071856 13596 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:20:36.073256 13596 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:20:36.093976 13596 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Generated new TSK 0
I20260709 12:20:36.094713 13596 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:20:36.157078 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:36.164482 13606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:36.165913 13607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:36.168807 12469 server_base.cc:1061] running on GCE node
W20260709 12:20:36.169627 13609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:36.170706 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:36.170897 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:36.171028 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599636171016 us; error 0 us; skew 500 ppm
I20260709 12:20:36.171597 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:36.174631 12469 webserver.cc:533] Webserver started at http://127.12.45.65:39349/ using document root <none> and password file <none>
I20260709 12:20:36.175189 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:36.175371 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:36.175657 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:36.176941 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "4dafe08a4f404502822a0732fe570de4"
format_stamp: "Formatted at 2026-07-09 12:20:36 on dist-test-slave-90bm"
I20260709 12:20:36.183029 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.007s
I20260709 12:20:36.187480 13614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.188469 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:20:36.188835 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "4dafe08a4f404502822a0732fe570de4"
format_stamp: "Formatted at 2026-07-09 12:20:36 on dist-test-slave-90bm"
I20260709 12:20:36.189196 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:36.221926 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:36.223490 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:36.225382 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:36.228116 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:36.228379 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.228693 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:36.228911 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.278651 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:33075
I20260709 12:20:36.278837 13676 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:33075 every 8 connection(s)
I20260709 12:20:36.284374 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:36.293646 13681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:36.294827 13682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:36.297464 13684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:36.299749 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:36.301717 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:36.302101 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:36.302515 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599636302484 us; error 0 us; skew 500 ppm
I20260709 12:20:36.303447 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:36.304524 13677 heartbeater.cc:344] Connected to a master server at 127.12.45.126:44051
I20260709 12:20:36.305277 13677 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:36.306452 13677 heartbeater.cc:507] Master 127.12.45.126:44051 requested a full tablet report, sending...
I20260709 12:20:36.309775 12469 webserver.cc:533] Webserver started at http://127.12.45.66:41709/ using document root <none> and password file <none>
I20260709 12:20:36.309783 13550 ts_manager.cc:194] Registered new tserver with Master: 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:20:36.310726 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:36.311018 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:36.311327 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:36.312569 13550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45090
I20260709 12:20:36.313022 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "b4f6bfcda63148e4965ca02c639336f1"
format_stamp: "Formatted at 2026-07-09 12:20:36 on dist-test-slave-90bm"
I20260709 12:20:36.318797 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260709 12:20:36.322410 13689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.323225 12469 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:20:36.323568 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "b4f6bfcda63148e4965ca02c639336f1"
format_stamp: "Formatted at 2026-07-09 12:20:36 on dist-test-slave-90bm"
I20260709 12:20:36.323884 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:36.335867 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:36.337345 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:36.339466 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:36.342063 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:36.342347 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:20:36.342619 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:36.342799 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.395478 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:41841
I20260709 12:20:36.395720 13751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:41841 every 8 connection(s)
I20260709 12:20:36.401353 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:36.409828 13755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:36.411810 13756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:36.413516 13758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:36.415017 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:36.416585 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:36.416904 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:36.417124 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599636417103 us; error 0 us; skew 500 ppm
I20260709 12:20:36.417752 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:36.418756 13752 heartbeater.cc:344] Connected to a master server at 127.12.45.126:44051
I20260709 12:20:36.419384 13752 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:36.420367 13752 heartbeater.cc:507] Master 127.12.45.126:44051 requested a full tablet report, sending...
I20260709 12:20:36.421180 12469 webserver.cc:533] Webserver started at http://127.12.45.67:41665/ using document root <none> and password file <none>
I20260709 12:20:36.421813 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:36.422037 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:36.422405 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:36.423300 13550 ts_manager.cc:194] Registered new tserver with Master: b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:36.424022 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "a4188193a10c4d50a555d76ea85c60b7"
format_stamp: "Formatted at 2026-07-09 12:20:36 on dist-test-slave-90bm"
I20260709 12:20:36.425287 13550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45092
I20260709 12:20:36.430076 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:20:36.434332 13763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.435714 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 12:20:36.436066 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "a4188193a10c4d50a555d76ea85c60b7"
format_stamp: "Formatted at 2026-07-09 12:20:36 on dist-test-slave-90bm"
I20260709 12:20:36.436350 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:36.488988 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:36.490384 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:36.491925 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:36.494791 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:36.495080 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.495340 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:36.495536 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:36.549464 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:40415
I20260709 12:20:36.549794 13825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:40415 every 8 connection(s)
I20260709 12:20:36.566529 13826 heartbeater.cc:344] Connected to a master server at 127.12.45.126:44051
I20260709 12:20:36.567122 13826 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:36.567957 13826 heartbeater.cc:507] Master 127.12.45.126:44051 requested a full tablet report, sending...
I20260709 12:20:36.570580 13550 ts_manager.cc:194] Registered new tserver with Master: a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:36.571228 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0177893s
I20260709 12:20:36.572391 13550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45106
I20260709 12:20:36.604583 13550 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45112:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:20:36.611478 13550 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:20:36.847298 13717 tablet_service.cc:1511] Processing CreateTablet for tablet a9e96f5f06294edba36ea19efe454394 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:36.847615 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff03c62343d402fa469da8080d33a6d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:36.849058 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e96f5f06294edba36ea19efe454394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.848687 13642 tablet_service.cc:1511] Processing CreateTablet for tablet a9e96f5f06294edba36ea19efe454394 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:36.849702 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff03c62343d402fa469da8080d33a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.850272 13642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e96f5f06294edba36ea19efe454394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.856698 13641 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff03c62343d402fa469da8080d33a6d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:36.858613 13641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff03c62343d402fa469da8080d33a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.865003 13715 tablet_service.cc:1511] Processing CreateTablet for tablet de32481a3be147a5b854251468fc667b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:36.868914 13640 tablet_service.cc:1511] Processing CreateTablet for tablet de32481a3be147a5b854251468fc667b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:36.870924 13640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32481a3be147a5b854251468fc667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.873104 13713 tablet_service.cc:1511] Processing CreateTablet for tablet 80eb1d112d05410a917177e781e466ed (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:36.868963 13714 tablet_service.cc:1511] Processing CreateTablet for tablet 30d1d057032c4ae0ad5ae21d374a2baa (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:36.869495 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32481a3be147a5b854251468fc667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.878772 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb1d112d05410a917177e781e466ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.881003 13639 tablet_service.cc:1511] Processing CreateTablet for tablet 30d1d057032c4ae0ad5ae21d374a2baa (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:36.881539 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30d1d057032c4ae0ad5ae21d374a2baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.882599 13639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30d1d057032c4ae0ad5ae21d374a2baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.892697 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 0a0048f5294f4012ad1519c0bc8e9e5a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:36.892808 13638 tablet_service.cc:1511] Processing CreateTablet for tablet 80eb1d112d05410a917177e781e466ed (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:36.894784 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb1d112d05410a917177e781e466ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.895463 13712 tablet_service.cc:1511] Processing CreateTablet for tablet 28452187c3924de9940235e9b965477d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:36.897529 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28452187c3924de9940235e9b965477d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.899401 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a0048f5294f4012ad1519c0bc8e9e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.900970 13637 tablet_service.cc:1511] Processing CreateTablet for tablet 0a0048f5294f4012ad1519c0bc8e9e5a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:36.902637 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a0048f5294f4012ad1519c0bc8e9e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.908841 13791 tablet_service.cc:1511] Processing CreateTablet for tablet a9e96f5f06294edba36ea19efe454394 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:20:36.910466 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e96f5f06294edba36ea19efe454394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.911391 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 80eb1d112d05410a917177e781e466ed (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:20:36.912962 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb1d112d05410a917177e781e466ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.914840 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 28452187c3924de9940235e9b965477d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:36.915604 13790 tablet_service.cc:1511] Processing CreateTablet for tablet 7ff03c62343d402fa469da8080d33a6d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:20:36.916360 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28452187c3924de9940235e9b965477d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.917711 13790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff03c62343d402fa469da8080d33a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.917618 13785 tablet_service.cc:1511] Processing CreateTablet for tablet 28452187c3924de9940235e9b965477d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:20:36.918960 13785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28452187c3924de9940235e9b965477d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.921602 13789 tablet_service.cc:1511] Processing CreateTablet for tablet de32481a3be147a5b854251468fc667b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:20:36.925516 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32481a3be147a5b854251468fc667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.923460 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 30d1d057032c4ae0ad5ae21d374a2baa (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:20:36.931332 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30d1d057032c4ae0ad5ae21d374a2baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.915608 13786 tablet_service.cc:1511] Processing CreateTablet for tablet 0a0048f5294f4012ad1519c0bc8e9e5a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:20:36.936676 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:36.938199 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.939067 13786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a0048f5294f4012ad1519c0bc8e9e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.939209 13711 tablet_service.cc:1511] Processing CreateTablet for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:36.940692 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.957870 13844 tablet_bootstrap.cc:492] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:36.962512 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:20:36.961750 13634 tablet_service.cc:1511] Processing CreateTablet for tablet f136fb5f4e574b6db1336e9434176d83 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:36.964121 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.964285 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f136fb5f4e574b6db1336e9434176d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.970418 13845 tablet_bootstrap.cc:492] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:36.972688 13717 tablet_service.cc:1511] Processing CreateTablet for tablet f136fb5f4e574b6db1336e9434176d83 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:36.976850 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f136fb5f4e574b6db1336e9434176d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.975251 13783 tablet_service.cc:1511] Processing CreateTablet for tablet f136fb5f4e574b6db1336e9434176d83 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:20:36.977964 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f136fb5f4e574b6db1336e9434176d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:36.985556 13844 tablet_bootstrap.cc:654] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:36.991603 13639 tablet_service.cc:1511] Processing CreateTablet for tablet af56607c7c564ce4bda1487902e25c7f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:36.993065 13639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af56607c7c564ce4bda1487902e25c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.000404 13711 tablet_service.cc:1511] Processing CreateTablet for tablet af56607c7c564ce4bda1487902e25c7f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:37.001850 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af56607c7c564ce4bda1487902e25c7f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:37.017379 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:37.021265 13782 tablet_service.cc:1511] Processing CreateTablet for tablet af56607c7c564ce4bda1487902e25c7f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:20:37.022874 13782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af56607c7c564ce4bda1487902e25c7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.023468 13714 tablet_service.cc:1511] Processing CreateTablet for tablet b1d64b0184dc44a1afc1b5885669d80d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:37.024789 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d64b0184dc44a1afc1b5885669d80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.026422 13845 tablet_bootstrap.cc:654] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.028492 13642 tablet_service.cc:1511] Processing CreateTablet for tablet b1d64b0184dc44a1afc1b5885669d80d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:37.030406 13642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d64b0184dc44a1afc1b5885669d80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.045558 13788 tablet_service.cc:1511] Processing CreateTablet for tablet b1d64b0184dc44a1afc1b5885669d80d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:20:37.059149 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d64b0184dc44a1afc1b5885669d80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.069595 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:37.071411 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.086023 13845 tablet_bootstrap.cc:492] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:37.086673 13845 ts_tablet_manager.cc:1403] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.117s	user 0.013s	sys 0.004s
I20260709 12:20:37.089972 13845 raft_consensus.cc:359] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.093101 13845 raft_consensus.cc:385] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.095734 13845 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.097393 13844 tablet_bootstrap.cc:492] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:37.098030 13844 ts_tablet_manager.cc:1403] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.141s	user 0.011s	sys 0.008s
I20260709 12:20:37.100212 13847 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:37.101333 13844 raft_consensus.cc:359] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.096846 13845 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.100090 13713 tablet_service.cc:1511] Processing CreateTablet for tablet ac3728586e264940a93f76650edf9ad3 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:37.102066 13844 raft_consensus.cc:385] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.102483 13844 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.103098 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3728586e264940a93f76650edf9ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.103434 13844 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.100265 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:37.112100 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.115658 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:20:37.117692 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.118551 13789 tablet_service.cc:1511] Processing CreateTablet for tablet ac3728586e264940a93f76650edf9ad3 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:37.119405 13639 tablet_service.cc:1511] Processing CreateTablet for tablet ac3728586e264940a93f76650edf9ad3 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:20:37.120482 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3728586e264940a93f76650edf9ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.121066 13639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3728586e264940a93f76650edf9ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.126528 13642 tablet_service.cc:1511] Processing CreateTablet for tablet 4613215991134c66b270c6024bec9d73 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:37.128211 13642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4613215991134c66b270c6024bec9d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.134817 13714 tablet_service.cc:1511] Processing CreateTablet for tablet 4613215991134c66b270c6024bec9d73 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:37.137516 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 4613215991134c66b270c6024bec9d73 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:20:37.139110 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4613215991134c66b270c6024bec9d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.143575 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 8debd745c407406c88fc03ec3a21f4db (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:37.145076 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8debd745c407406c88fc03ec3a21f4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.150604 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4613215991134c66b270c6024bec9d73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.150910 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 8debd745c407406c88fc03ec3a21f4db (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:37.153649 13783 tablet_service.cc:1511] Processing CreateTablet for tablet 8debd745c407406c88fc03ec3a21f4db (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:20:37.159696 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8debd745c407406c88fc03ec3a21f4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.163355 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 3934ae8ea567487da7a989adece218c2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:37.165522 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3934ae8ea567487da7a989adece218c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.167608 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8debd745c407406c88fc03ec3a21f4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.168838 13847 tablet_bootstrap.cc:654] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.175810 13844 ts_tablet_manager.cc:1434] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.077s	user 0.028s	sys 0.039s
I20260709 12:20:37.181819 13752 heartbeater.cc:499] Master 127.12.45.126:44051 was elected leader, sending a full tablet report...
I20260709 12:20:37.184466 13712 tablet_service.cc:1511] Processing CreateTablet for tablet 3934ae8ea567487da7a989adece218c2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:37.186028 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3934ae8ea567487da7a989adece218c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.186882 13844 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:37.187304 13638 tablet_service.cc:1511] Processing CreateTablet for tablet 52052a89b7534d0488f41d7f8e905a14 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:37.188947 13786 tablet_service.cc:1511] Processing CreateTablet for tablet 3934ae8ea567487da7a989adece218c2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:20:37.189843 13785 tablet_service.cc:1511] Processing CreateTablet for tablet 52052a89b7534d0488f41d7f8e905a14 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:37.190690 13786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3934ae8ea567487da7a989adece218c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.194785 13785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52052a89b7534d0488f41d7f8e905a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.195955 13715 tablet_service.cc:1511] Processing CreateTablet for tablet 52052a89b7534d0488f41d7f8e905a14 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:20:37.197360 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52052a89b7534d0488f41d7f8e905a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.206702 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52052a89b7534d0488f41d7f8e905a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.207270 13845 ts_tablet_manager.cc:1434] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.120s	user 0.012s	sys 0.007s
I20260709 12:20:37.208505 13845 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:37.210649 13677 heartbeater.cc:499] Master 127.12.45.126:44051 was elected leader, sending a full tablet report...
I20260709 12:20:37.217751 13845 tablet_bootstrap.cc:654] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.217864 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 79b849c107424cc79f4cbd223feeeb47 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:37.219544 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79b849c107424cc79f4cbd223feeeb47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.256269 13850 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.257061 13850 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.265722 13850 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.278952 13844 tablet_bootstrap.cc:654] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.306887 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 79b849c107424cc79f4cbd223feeeb47 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:37.307498 13845 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:37.308298 13845 ts_tablet_manager.cc:1403] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.100s	user 0.044s	sys 0.032s
I20260709 12:20:37.308512 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79b849c107424cc79f4cbd223feeeb47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.314308 13638 tablet_service.cc:1511] Processing CreateTablet for tablet bdd4114edc6d43c08d41462fd399480c (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:37.314498 13783 tablet_service.cc:1511] Processing CreateTablet for tablet 79b849c107424cc79f4cbd223feeeb47 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:20:37.316032 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd4114edc6d43c08d41462fd399480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.316540 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79b849c107424cc79f4cbd223feeeb47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.317811 13849 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.318372 13849 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.326709 13849 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.333989 13716 tablet_service.cc:1511] Processing CreateTablet for tablet bdd4114edc6d43c08d41462fd399480c (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:37.335879 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd4114edc6d43c08d41462fd399480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.347798 13789 tablet_service.cc:1511] Processing CreateTablet for tablet bdd4114edc6d43c08d41462fd399480c (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:20:37.349742 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd4114edc6d43c08d41462fd399480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.352491 13845 raft_consensus.cc:359] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.356324 13845 raft_consensus.cc:385] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.356774 13845 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.356555 13712 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:37.357987 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.357726 13845 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.360205 13845 ts_tablet_manager.cc:1434] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.001s
I20260709 12:20:37.360466 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb6d086379f4190b73fc7f692fb490a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:37.361328 13845 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:37.361769 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb6d086379f4190b73fc7f692fb490a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.351154 13642 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:37.367375 13844 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:37.368140 13844 ts_tablet_manager.cc:1403] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.182s	user 0.016s	sys 0.025s
I20260709 12:20:37.405900 13714 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:37.407588 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.407714 13711 tablet_service.cc:1511] Processing CreateTablet for tablet 46d57087aee64ae9a498d3d60b92ab92 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:37.408990 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46d57087aee64ae9a498d3d60b92ab92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.406154 13713 tablet_service.cc:1511] Processing CreateTablet for tablet e2603f7eea4f49ab89f0ebb1304ee239 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:37.411870 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2603f7eea4f49ab89f0ebb1304ee239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.411641 13710 tablet_service.cc:1511] Processing CreateTablet for tablet 3e52c8b752b54bd9b959b2363df455a4 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:37.413136 13710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e52c8b752b54bd9b959b2363df455a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.413527 13642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.417248 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 46d57087aee64ae9a498d3d60b92ab92 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:37.417273 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e68c4d38b4465937dee955f980255 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:37.417738 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:37.419363 13639 tablet_service.cc:1511] Processing CreateTablet for tablet e2603f7eea4f49ab89f0ebb1304ee239 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:37.421797 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e68c4d38b4465937dee955f980255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.419368 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:37.418691 13844 raft_consensus.cc:359] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.429240 13785 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb6d086379f4190b73fc7f692fb490a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:37.429584 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:20:37.430845 13785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb6d086379f4190b73fc7f692fb490a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.431111 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:20:37.431432 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 46d57087aee64ae9a498d3d60b92ab92 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:20:37.432381 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.432101 13791 tablet_service.cc:1511] Processing CreateTablet for tablet 3e52c8b752b54bd9b959b2363df455a4 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:37.433388 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e52c8b752b54bd9b959b2363df455a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.433552 13786 tablet_service.cc:1511] Processing CreateTablet for tablet e2603f7eea4f49ab89f0ebb1304ee239 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:20:37.435853 13847 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:37.435906 13786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2603f7eea4f49ab89f0ebb1304ee239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.436434 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46d57087aee64ae9a498d3d60b92ab92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:37.436921 13618 leader_election.cc:343] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.438356 13847 ts_tablet_manager.cc:1403] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.339s	user 0.015s	sys 0.011s
I20260709 12:20:37.438846 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.440912 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46d57087aee64ae9a498d3d60b92ab92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.438402 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.446070 13641 tablet_service.cc:1511] Processing CreateTablet for tablet 3e52c8b752b54bd9b959b2363df455a4 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:20:37.446312 13637 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e68c4d38b4465937dee955f980255 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:37.447736 13641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e52c8b752b54bd9b959b2363df455a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.448249 13845 tablet_bootstrap.cc:654] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.450796 13639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2603f7eea4f49ab89f0ebb1304ee239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.451974 13715 tablet_service.cc:1511] Processing CreateTablet for tablet 3cb6d086379f4190b73fc7f692fb490a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:20:37.454591 13709 tablet_service.cc:1511] Processing CreateTablet for tablet e1d1b82fa8024303a24cb00ab8e7f845 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:37.454840 13640 tablet_service.cc:1511] Processing CreateTablet for tablet e1d1b82fa8024303a24cb00ab8e7f845 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:37.456918 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e68c4d38b4465937dee955f980255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.457526 13640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1d1b82fa8024303a24cb00ab8e7f845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.422982 13844 raft_consensus.cc:385] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.478658 13844 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.480209 13849 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.479943 13844 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.480732 13849 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.482769 13844 ts_tablet_manager.cc:1434] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.114s	user 0.009s	sys 0.003s
I20260709 12:20:37.483074 13849 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.485186 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:37.486097 13709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1d1b82fa8024303a24cb00ab8e7f845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.493680 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb6d086379f4190b73fc7f692fb490a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.495385 13844 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:37.442085 13847 raft_consensus.cc:359] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.496510 13847 raft_consensus.cc:385] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.496876 13847 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.498198 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:37.499061 13652 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:37.499410 13782 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e68c4d38b4465937dee955f980255 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:20:37.500918 13782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e68c4d38b4465937dee955f980255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.501034 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:37.502133 13850 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:37.502590 13850 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.503005 13850 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.516621 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:37.518476 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:37.520323 13617 leader_election.cc:343] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.524622 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:37.535266 13847 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.536528 13801 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:37.541451 13618 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:37.542824 13849 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:37.543344 13849 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:37.543774 13849 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.566023 13844 tablet_bootstrap.cc:654] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:37.570734 13537 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45783.2us
I20260709 12:20:37.571664 13845 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:37.572257 13845 ts_tablet_manager.cc:1403] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.211s	user 0.010s	sys 0.011s
I20260709 12:20:37.574636 13850 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.575294 13638 tablet_service.cc:1511] Processing CreateTablet for tablet 8916aaed46c543b59ce09ffca942c33d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:37.576000 13727 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:37.575639 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 8916aaed46c543b59ce09ffca942c33d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:37.576866 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8916aaed46c543b59ce09ffca942c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.577008 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8916aaed46c543b59ce09ffca942c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.577992 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:37.578606 13826 heartbeater.cc:499] Master 127.12.45.126:44051 was elected leader, sending a full tablet report...
I20260709 12:20:37.579891 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:37.580514 13652 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.589712 13652 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:37.596786 13863 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.602473 13849 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.603359 13860 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.605857 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:20:37.608125 13617 leader_election.cc:343] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.603992 13860 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:37.616237 13693 leader_election.cc:343] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.625449 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:37.617151 13845 raft_consensus.cc:359] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.626200 13801 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:37.626529 13845 raft_consensus.cc:385] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.626893 13845 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.627501 13860 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.624033 13849 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.631343 13791 tablet_service.cc:1511] Processing CreateTablet for tablet e1d1b82fa8024303a24cb00ab8e7f845 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:20:37.633472 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1d1b82fa8024303a24cb00ab8e7f845. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.634701 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:37.634969 13712 tablet_service.cc:1511] Processing CreateTablet for tablet cd4fd9484f184d2c994c269eb10d6ec1 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:37.636391 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4fd9484f184d2c994c269eb10d6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.636138 13635 tablet_service.cc:1511] Processing CreateTablet for tablet cd4fd9484f184d2c994c269eb10d6ec1 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:37.635983 13845 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.637260 13847 ts_tablet_manager.cc:1434] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.198s	user 0.005s	sys 0.006s
I20260709 12:20:37.637455 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4fd9484f184d2c994c269eb10d6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.638574 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:37.639865 13850 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.640240 13860 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:37.644104 13860 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:37.646086 13860 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:37.639825 13693 leader_election.cc:343] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.655354 13844 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:37.655961 13844 ts_tablet_manager.cc:1403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.161s	user 0.015s	sys 0.003s
I20260709 12:20:37.659459 13844 raft_consensus.cc:359] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.660233 13844 raft_consensus.cc:385] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.660552 13844 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.661521 13844 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.663853 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcbdced6d0a46b394cae3d101fbbf40 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:37.665375 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbdced6d0a46b394cae3d101fbbf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.666785 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:37.668632 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:37.670163 13783 tablet_service.cc:1511] Processing CreateTablet for tablet cd4fd9484f184d2c994c269eb10d6ec1 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:20:37.671396 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4fd9484f184d2c994c269eb10d6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:37.686673 13693 leader_election.cc:343] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.687299 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:37.695039 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 8916aaed46c543b59ce09ffca942c33d (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:20:37.697165 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8916aaed46c543b59ce09ffca942c33d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.649718 13845 ts_tablet_manager.cc:1434] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.002s
I20260709 12:20:37.699318 13845 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:37.688719 13847 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:37.716789 13850 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.720276 13801 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:37.733480 13618 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:37.735138 13849 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:37.735690 13849 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:37.736716 13849 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.752591 13844 ts_tablet_manager.cc:1434] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.096s	user 0.022s	sys 0.027s
I20260709 12:20:37.759698 13782 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcbdced6d0a46b394cae3d101fbbf40 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:37.762531 13847 tablet_bootstrap.cc:654] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.762394 13715 tablet_service.cc:1511] Processing CreateTablet for tablet f22eab7d8372432799f894a161194acd (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 12:20:37.764144 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77522.3us
W20260709 12:20:37.765015 13693 leader_election.cc:343] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.765096 13713 tablet_service.cc:1511] Processing CreateTablet for tablet 2dcbdced6d0a46b394cae3d101fbbf40 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:20:37.767187 13782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbdced6d0a46b394cae3d101fbbf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.773769 13845 tablet_bootstrap.cc:654] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:37.777931 13548 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:37.781070 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22eab7d8372432799f894a161194acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.784993 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 48612be3c56f482c81533c30abebe60b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:37.787070 13638 tablet_service.cc:1511] Processing CreateTablet for tablet 48612be3c56f482c81533c30abebe60b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:37.787739 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbdced6d0a46b394cae3d101fbbf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.788694 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48612be3c56f482c81533c30abebe60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.782601 13783 tablet_service.cc:1511] Processing CreateTablet for tablet f22eab7d8372432799f894a161194acd (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 12:20:37.791147 13678 tablet.cc:2406] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:20:37.791514 13634 tablet_service.cc:1511] Processing CreateTablet for tablet f22eab7d8372432799f894a161194acd (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:20:37.791862 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22eab7d8372432799f894a161194acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.800185 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48612be3c56f482c81533c30abebe60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.800872 13844 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:37.835264 13550 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:37.840948 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 15d97b0b20fd480e8f728a8d38f52e03 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:37.842500 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d97b0b20fd480e8f728a8d38f52e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.842082 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 15d97b0b20fd480e8f728a8d38f52e03 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:37.843019 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f22eab7d8372432799f894a161194acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.844925 13791 tablet_service.cc:1511] Processing CreateTablet for tablet 48612be3c56f482c81533c30abebe60b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:20:37.840951 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 56f25c6fe9f245d99ae467bdd3948582 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:37.851233 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48612be3c56f482c81533c30abebe60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.851233 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f25c6fe9f245d99ae467bdd3948582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.842969 13710 tablet_service.cc:1511] Processing CreateTablet for tablet 15d97b0b20fd480e8f728a8d38f52e03 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:20:37.854467 13710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d97b0b20fd480e8f728a8d38f52e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.861912 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 56f25c6fe9f245d99ae467bdd3948582 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:37.864400 13714 tablet_service.cc:1511] Processing CreateTablet for tablet 56f25c6fe9f245d99ae467bdd3948582 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:20:37.865198 13637 tablet_service.cc:1511] Processing CreateTablet for tablet a2dcf256ec7a4ea6be04f95be4459f59 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:37.865868 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f25c6fe9f245d99ae467bdd3948582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.866511 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2dcf256ec7a4ea6be04f95be4459f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.868707 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f25c6fe9f245d99ae467bdd3948582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.877087 13709 tablet_service.cc:1511] Processing CreateTablet for tablet a2dcf256ec7a4ea6be04f95be4459f59 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:37.878757 13709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2dcf256ec7a4ea6be04f95be4459f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.885188 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d97b0b20fd480e8f728a8d38f52e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.893719 13786 tablet_service.cc:1511] Processing CreateTablet for tablet a2dcf256ec7a4ea6be04f95be4459f59 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:20:37.894142 13641 tablet_service.cc:1511] Processing CreateTablet for tablet 45dc2f34593649778ec4b80d29638572 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:37.895516 13786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2dcf256ec7a4ea6be04f95be4459f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.896210 13641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dc2f34593649778ec4b80d29638572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.910940 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 45dc2f34593649778ec4b80d29638572 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:37.911990 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 45dc2f34593649778ec4b80d29638572 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:20:37.912509 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dc2f34593649778ec4b80d29638572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.913535 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dc2f34593649778ec4b80d29638572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.934134 13866 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:37.934759 13866 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.935185 13847 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:37.936792 13847 ts_tablet_manager.cc:1403] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.296s	user 0.011s	sys 0.020s
I20260709 12:20:37.936894 13866 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:37.939409 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:37.940395 13844 tablet_bootstrap.cc:654] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:37.941085 13693 leader_election.cc:343] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.940477 13847 raft_consensus.cc:359] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.949652 13847 raft_consensus.cc:385] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.950155 13847 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.951287 13847 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.945122 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:37.955660 13638 tablet_service.cc:1511] Processing CreateTablet for tablet 54361b2742ab4613b55749cbb699113b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:37.956980 13847 ts_tablet_manager.cc:1434] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:20:37.948334 13845 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
W20260709 12:20:37.957223 13693 leader_election.cc:343] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:37.958462 13845 ts_tablet_manager.cc:1403] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.259s	user 0.037s	sys 0.017s
I20260709 12:20:37.958715 13693 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:37.959540 13866 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:37.957188 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54361b2742ab4613b55749cbb699113b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.962047 13845 raft_consensus.cc:359] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:37.963973 13845 raft_consensus.cc:385] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:37.965813 13845 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:37.964980 13847 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:37.977682 13711 tablet_service.cc:1511] Processing CreateTablet for tablet 54361b2742ab4613b55749cbb699113b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:37.983492 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54361b2742ab4613b55749cbb699113b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:37.971362 13845 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.003149 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab361e956dd4342b80045629d159a2b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:38.003387 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab361e956dd4342b80045629d159a2b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:38.020133 13845 ts_tablet_manager.cc:1434] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
W20260709 12:20:38.021066 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:38.021397 13845 tablet_bootstrap.cc:492] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.022316 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab361e956dd4342b80045629d159a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.023630 13637 tablet_service.cc:1511] Processing CreateTablet for tablet 7d21a9fba1484f4985407f4cde83b77c (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:38.026525 13849 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.016430 13791 tablet_service.cc:1511] Processing CreateTablet for tablet 54361b2742ab4613b55749cbb699113b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:20:38.027910 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab361e956dd4342b80045629d159a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.027560 13849 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.028851 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54361b2742ab4613b55749cbb699113b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.030535 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d21a9fba1484f4985407f4cde83b77c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.031934 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:38.035986 13617 leader_election.cc:343] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.036842 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:38.038545 13801 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:38.040155 13618 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:38.036175 13849 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.041055 13863 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:38.041766 13863 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.042158 13863 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.035208 13715 tablet_service.cc:1511] Processing CreateTablet for tablet 8ab361e956dd4342b80045629d159a2b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:20:38.043695 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ab361e956dd4342b80045629d159a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.055049 13716 tablet_service.cc:1511] Processing CreateTablet for tablet 7d21a9fba1484f4985407f4cde83b77c (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:38.058244 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d21a9fba1484f4985407f4cde83b77c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.055929 13844 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:38.082729 13844 ts_tablet_manager.cc:1403] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.282s	user 0.011s	sys 0.039s
I20260709 12:20:38.083094 13710 tablet_service.cc:1511] Processing CreateTablet for tablet 77fe434d99534108934a1cdcf44e68ac (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:38.086942 13844 raft_consensus.cc:359] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.087854 13844 raft_consensus.cc:385] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.088168 13844 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.089161 13844 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.091542 13715 tablet_service.cc:1511] Processing CreateTablet for tablet 990ed59e36454f16b81d57d0e4f59c4e (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:38.092139 13844 ts_tablet_manager.cc:1434] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:20:38.066567 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 990ed59e36454f16b81d57d0e4f59c4e (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 12:20:38.092679 13693 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:38.093598 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990ed59e36454f16b81d57d0e4f59c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.067813 13863 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.098177 13845 tablet_bootstrap.cc:654] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.101385 13863 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.101614 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:38.102381 13801 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.105504 13786 tablet_service.cc:1511] Processing CreateTablet for tablet 7d21a9fba1484f4985407f4cde83b77c (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:20:38.106722 13710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fe434d99534108934a1cdcf44e68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.107244 13786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d21a9fba1484f4985407f4cde83b77c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.110623 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:38.111333 13727 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.111941 13715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990ed59e36454f16b81d57d0e4f59c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.111760 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 77fe434d99534108934a1cdcf44e68ac (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:38.113168 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fe434d99534108934a1cdcf44e68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.122784 13709 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8b916ed93d478c990f668818f43952 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:38.098106 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 990ed59e36454f16b81d57d0e4f59c4e (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:20:38.126364 13714 tablet_service.cc:1511] Processing CreateTablet for tablet 89717fd329da4af68d87ae1ef1b104fb (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:38.127388 13801 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:38.133500 13618 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:38.128711 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990ed59e36454f16b81d57d0e4f59c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.124678 13727 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:38.111333 13844 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:38.135349 13863 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:38.136075 13863 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:38.130097 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 89717fd329da4af68d87ae1ef1b104fb (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:38.130506 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 89717fd329da4af68d87ae1ef1b104fb (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:20:38.129276 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8b916ed93d478c990f668818f43952 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:38.147037 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89717fd329da4af68d87ae1ef1b104fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.126353 13641 tablet_service.cc:1511] Processing CreateTablet for tablet 77fe434d99534108934a1cdcf44e68ac (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:20:38.153386 13641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77fe434d99534108934a1cdcf44e68ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.135807 13637 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8b916ed93d478c990f668818f43952 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:20:38.081835 13847 tablet_bootstrap.cc:654] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.127671 13709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8b916ed93d478c990f668818f43952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.143648 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89717fd329da4af68d87ae1ef1b104fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.153730 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 0a27ddca262a4e0e8c46331cd36be4d1 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:38.172268 13866 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.165489 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8b916ed93d478c990f668818f43952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.191027 13845 tablet_bootstrap.cc:492] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:38.191865 13845 ts_tablet_manager.cc:1403] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.003s
I20260709 12:20:38.196874 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a27ddca262a4e0e8c46331cd36be4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.165750 13863 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.212792 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8b916ed93d478c990f668818f43952. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.223919 13638 tablet_service.cc:1511] Processing CreateTablet for tablet 0a27ddca262a4e0e8c46331cd36be4d1 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:38.225412 13638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a27ddca262a4e0e8c46331cd36be4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.225046 13711 tablet_service.cc:1511] Processing CreateTablet for tablet 6966d62b938845429a9aa8feb8fe2b9b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:38.231021 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89717fd329da4af68d87ae1ef1b104fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:38.249711 13617 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:38.261982 13791 tablet_service.cc:1511] Processing CreateTablet for tablet 0a27ddca262a4e0e8c46331cd36be4d1 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:20:38.263926 13791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a27ddca262a4e0e8c46331cd36be4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.270228 13713 tablet_service.cc:1511] Processing CreateTablet for tablet 068a8cc7309648e3a661d4d056f11b7a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:38.274619 13845 raft_consensus.cc:359] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.275506 13845 raft_consensus.cc:385] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.275720 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6966d62b938845429a9aa8feb8fe2b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.275833 13845 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.277511 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068a8cc7309648e3a661d4d056f11b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.277544 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 6966d62b938845429a9aa8feb8fe2b9b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:38.281601 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6966d62b938845429a9aa8feb8fe2b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.281716 13782 tablet_service.cc:1511] Processing CreateTablet for tablet 068a8cc7309648e3a661d4d056f11b7a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:38.283092 13782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068a8cc7309648e3a661d4d056f11b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.280061 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 068a8cc7309648e3a661d4d056f11b7a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:20:38.285316 13712 tablet_service.cc:1511] Processing CreateTablet for tablet 255f10165e844f2cb50d654996d639cf (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:38.288625 13783 tablet_service.cc:1511] Processing CreateTablet for tablet 6966d62b938845429a9aa8feb8fe2b9b (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:20:38.290071 13783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6966d62b938845429a9aa8feb8fe2b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.293025 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068a8cc7309648e3a661d4d056f11b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.293869 13640 tablet_service.cc:1511] Processing CreateTablet for tablet 255f10165e844f2cb50d654996d639cf (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:38.295308 13640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255f10165e844f2cb50d654996d639cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.299384 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255f10165e844f2cb50d654996d639cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.315801 13847 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:38.316416 13847 ts_tablet_manager.cc:1403] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.352s	user 0.025s	sys 0.011s
I20260709 12:20:38.333699 13844 tablet_bootstrap.cc:654] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.334499 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 255f10165e844f2cb50d654996d639cf (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:20:38.335968 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 255f10165e844f2cb50d654996d639cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.336054 13639 tablet_service.cc:1511] Processing CreateTablet for tablet 3ff310d454634aa0bbc38f2b233ccb8e (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:38.337345 13639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ff310d454634aa0bbc38f2b233ccb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.347389 13710 tablet_service.cc:1511] Processing CreateTablet for tablet 3ff310d454634aa0bbc38f2b233ccb8e (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:38.349222 13710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ff310d454634aa0bbc38f2b233ccb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.328650 13845 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.353637 13845 ts_tablet_manager.cc:1434] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.161s	user 0.008s	sys 0.003s
I20260709 12:20:38.359001 13863 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.374691 13845 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.360056 13550 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.367641 13847 raft_consensus.cc:359] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.382365 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 3ff310d454634aa0bbc38f2b233ccb8e (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:20:38.386361 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ff310d454634aa0bbc38f2b233ccb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.407948 13637 tablet_service.cc:1511] Processing CreateTablet for tablet e9c30191f356495b9732659f925abc9f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:38.417577 13863 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.418185 13863 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.424646 13863 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.427142 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:38.428025 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:38.428607 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:38.429630 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:38.430649 13863 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:38.430540 13711 tablet_service.cc:1511] Processing CreateTablet for tablet e9c30191f356495b9732659f925abc9f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:38.431123 13863 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.431594 13863 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.432011 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c30191f356495b9732659f925abc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.432403 13845 tablet_bootstrap.cc:654] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.440232 13863 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.443434 13784 tablet_service.cc:1511] Processing CreateTablet for tablet e9c30191f356495b9732659f925abc9f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:20:38.444823 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c30191f356495b9732659f925abc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.444909 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:38.385035 13847 raft_consensus.cc:385] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.445574 13727 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.445715 13847 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.446642 13847 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.410454 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c30191f356495b9732659f925abc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.448935 13800 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:38.454695 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:38.455550 13799 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.459825 13714 tablet_service.cc:1511] Processing CreateTablet for tablet b5c65e93a0c1420ca348d0fc678031ca (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:38.459911 13635 tablet_service.cc:1511] Processing CreateTablet for tablet b5c65e93a0c1420ca348d0fc678031ca (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:38.461593 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5c65e93a0c1420ca348d0fc678031ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.471532 13788 tablet_service.cc:1511] Processing CreateTablet for tablet b5c65e93a0c1420ca348d0fc678031ca (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:20:38.473482 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5c65e93a0c1420ca348d0fc678031ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.476027 13847 ts_tablet_manager.cc:1434] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.159s	user 0.009s	sys 0.001s
I20260709 12:20:38.480906 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5c65e93a0c1420ca348d0fc678031ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.493654 13847 tablet_bootstrap.cc:492] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:38.465554 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:38.499608 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:38.500614 13849 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:38.505085 13782 tablet_service.cc:1511] Processing CreateTablet for tablet 749228d3459a444a9382bc38bbfcbbee (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:38.506830 13844 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:38.507325 13782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 749228d3459a444a9382bc38bbfcbbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.507877 13799 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:38.507388 13844 ts_tablet_manager.cc:1403] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.396s	user 0.026s	sys 0.002s
I20260709 12:20:38.514974 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 684ee7a163da446aba11bf15d10d691a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:38.516573 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684ee7a163da446aba11bf15d10d691a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.520169 13639 tablet_service.cc:1511] Processing CreateTablet for tablet 749228d3459a444a9382bc38bbfcbbee (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:38.521639 13639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 749228d3459a444a9382bc38bbfcbbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.523093 13847 tablet_bootstrap.cc:654] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.509550 13712 tablet_service.cc:1511] Processing CreateTablet for tablet 749228d3459a444a9382bc38bbfcbbee (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:20:38.466606 13863 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.527779 13849 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:38.528740 13712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 749228d3459a444a9382bc38bbfcbbee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.534652 13845 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:38.535279 13845 ts_tablet_manager.cc:1403] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.161s	user 0.005s	sys 0.013s
I20260709 12:20:38.528860 13849 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.543823 13845 raft_consensus.cc:359] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.544656 13845 raft_consensus.cc:385] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.545001 13845 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.537339 13844 raft_consensus.cc:359] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.562724 13845 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.563984 13844 raft_consensus.cc:385] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.564415 13844 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.569391 13845 ts_tablet_manager.cc:1434] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.002s
I20260709 12:20:38.584910 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 63b331138ada4d5e96b6136417e740f7 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:38.586395 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63b331138ada4d5e96b6136417e740f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.592027 13847 tablet_bootstrap.cc:492] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:38.565385 13844 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.592777 13847 ts_tablet_manager.cc:1403] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.008s
I20260709 12:20:38.597721 13844 ts_tablet_manager.cc:1434] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.002s
I20260709 12:20:38.590031 13550 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.590551 13711 tablet_service.cc:1511] Processing CreateTablet for tablet 684ee7a163da446aba11bf15d10d691a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:38.589425 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 63b331138ada4d5e96b6136417e740f7 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:38.610512 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63b331138ada4d5e96b6136417e740f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.611039 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 0b7a0a5ab87d4adbb517d125624bc8ad (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:38.611215 13784 tablet_service.cc:1511] Processing CreateTablet for tablet 684ee7a163da446aba11bf15d10d691a (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:20:38.603029 13845 tablet_bootstrap.cc:492] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.612507 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b7a0a5ab87d4adbb517d125624bc8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.612761 13636 tablet_service.cc:1511] Processing CreateTablet for tablet 0b7a0a5ab87d4adbb517d125624bc8ad (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:38.604575 13847 raft_consensus.cc:359] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.614218 13636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b7a0a5ab87d4adbb517d125624bc8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.614696 13847 raft_consensus.cc:385] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.616484 13847 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.617014 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 435988495fcf4cf49f65293e95b5ceb5 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:38.611799 13844 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:38.618677 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435988495fcf4cf49f65293e95b5ceb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.620828 13714 tablet_service.cc:1511] Processing CreateTablet for tablet 63b331138ada4d5e96b6136417e740f7 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:20:38.617492 13847 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.623544 13713 tablet_service.cc:1511] Processing CreateTablet for tablet 0b7a0a5ab87d4adbb517d125624bc8ad (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:20:38.628193 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684ee7a163da446aba11bf15d10d691a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.635984 13847 ts_tablet_manager.cc:1434] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 12:20:38.638458 13849 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:38.647035 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63b331138ada4d5e96b6136417e740f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.647904 13713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b7a0a5ab87d4adbb517d125624bc8ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.652139 13710 tablet_service.cc:1511] Processing CreateTablet for tablet 435988495fcf4cf49f65293e95b5ceb5 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:38.653681 13710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435988495fcf4cf49f65293e95b5ceb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.662042 13787 tablet_service.cc:1511] Processing CreateTablet for tablet 435988495fcf4cf49f65293e95b5ceb5 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:20:38.669608 13787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435988495fcf4cf49f65293e95b5ceb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.672171 13784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 684ee7a163da446aba11bf15d10d691a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.674594 13637 tablet_service.cc:1511] Processing CreateTablet for tablet 20dc442f4fae4cf49e336a3d350b7760 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:38.676255 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20dc442f4fae4cf49e336a3d350b7760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.664347 13844 tablet_bootstrap.cc:654] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.662022 13847 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:38.699380 13845 tablet_bootstrap.cc:654] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.700773 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 20dc442f4fae4cf49e336a3d350b7760 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:38.706249 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20dc442f4fae4cf49e336a3d350b7760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.716562 13847 tablet_bootstrap.cc:654] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.731833 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 20dc442f4fae4cf49e336a3d350b7760 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:20:38.733171 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20dc442f4fae4cf49e336a3d350b7760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.741986 13849 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.742638 13849 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.749786 13849 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.751425 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:38.752787 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:38.753604 13634 tablet_service.cc:1511] Processing CreateTablet for tablet 4ce599ec23ec4ac0b358f4e47922bc3f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
W20260709 12:20:38.754925 13617 leader_election.cc:343] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.755178 13634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce599ec23ec4ac0b358f4e47922bc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.753679 13788 tablet_service.cc:1511] Processing CreateTablet for tablet 4ce599ec23ec4ac0b358f4e47922bc3f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:38.755311 13782 tablet_service.cc:1511] Processing CreateTablet for tablet e71fd62c156c4baf990c9e212f69ecc2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:38.756317 13788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce599ec23ec4ac0b358f4e47922bc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.757174 13782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71fd62c156c4baf990c9e212f69ecc2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:20:38.757602 13618 leader_election.cc:343] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:38.758098 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:38.729609 13637 tablet_service.cc:1511] Processing CreateTablet for tablet e71fd62c156c4baf990c9e212f69ecc2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:38.759526 13637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71fd62c156c4baf990c9e212f69ecc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.760738 13885 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:38.768620 13635 tablet_service.cc:1511] Processing CreateTablet for tablet 85fee8a18ff4429cb870f59aa46b3892 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:38.771698 13635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fee8a18ff4429cb870f59aa46b3892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.774464 13849 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.791375 13714 tablet_service.cc:1511] Processing CreateTablet for tablet e71fd62c156c4baf990c9e212f69ecc2 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:20:38.793134 13714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71fd62c156c4baf990c9e212f69ecc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.795740 13717 tablet_service.cc:1511] Processing CreateTablet for tablet 85fee8a18ff4429cb870f59aa46b3892 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:38.795740 13711 tablet_service.cc:1511] Processing CreateTablet for tablet 4ce599ec23ec4ac0b358f4e47922bc3f (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:20:38.798686 13844 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:38.798378 13847 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:38.799387 13844 ts_tablet_manager.cc:1403] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.192s	user 0.015s	sys 0.003s
I20260709 12:20:38.800443 13789 tablet_service.cc:1511] Processing CreateTablet for tablet 85fee8a18ff4429cb870f59aa46b3892 (DEFAULT_TABLE table=test-workload [id=5df2a3b9258a41f6b50be2d77e637387]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:20:38.799423 13847 ts_tablet_manager.cc:1403] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.138s	user 0.021s	sys 0.020s
I20260709 12:20:38.801819 13789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fee8a18ff4429cb870f59aa46b3892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.820698 13717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fee8a18ff4429cb870f59aa46b3892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.822656 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce599ec23ec4ac0b358f4e47922bc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:38.827806 13847 raft_consensus.cc:359] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.828629 13847 raft_consensus.cc:385] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.828955 13847 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.832499 13847 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.836006 13726 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.836171 13847 ts_tablet_manager.cc:1434] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:20:38.842366 13845 tablet_bootstrap.cc:492] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:38.842888 13847 tablet_bootstrap.cc:492] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:38.842952 13845 ts_tablet_manager.cc:1403] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.240s	user 0.019s	sys 0.015s
I20260709 12:20:38.846192 13845 raft_consensus.cc:359] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.847092 13845 raft_consensus.cc:385] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.847383 13845 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.848256 13845 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.856278 13844 raft_consensus.cc:359] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.856938 13845 ts_tablet_manager.cc:1434] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 12:20:38.857117 13844 raft_consensus.cc:385] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.858426 13845 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.857498 13844 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.862190 13844 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.865931 13845 tablet_bootstrap.cc:654] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.867316 13885 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:38.867835 13885 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.870882 13885 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.872525 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:38.875136 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:38.876672 13847 tablet_bootstrap.cc:654] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.876113 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:38.873594 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:38.878332 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:38.879312 13885 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:38.879673 13885 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:38.880003 13885 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.873049 13844 ts_tablet_manager.cc:1434] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.073s	user 0.009s	sys 0.000s
I20260709 12:20:38.885481 13844 tablet_bootstrap.cc:492] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:38.886693 13845 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:38.887200 13845 ts_tablet_manager.cc:1403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.011s
I20260709 12:20:38.889931 13885 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.890117 13845 raft_consensus.cc:359] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.892010 13845 raft_consensus.cc:385] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.892294 13845 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.892563 13844 tablet_bootstrap.cc:654] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.893013 13845 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.895141 13845 ts_tablet_manager.cc:1434] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:20:38.896136 13845 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.896878 13885 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:38.900053 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:38.901172 13847 tablet_bootstrap.cc:492] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:38.896886 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:38.901371 13799 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.902992 13845 tablet_bootstrap.cc:654] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.901741 13847 ts_tablet_manager.cc:1403] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.004s
I20260709 12:20:38.901832 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:38.908851 13847 raft_consensus.cc:359] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.909654 13847 raft_consensus.cc:385] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.910004 13847 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.912243 13844 tablet_bootstrap.cc:492] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:38.912772 13844 ts_tablet_manager.cc:1403] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.009s
I20260709 12:20:38.912878 13845 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:38.913390 13845 ts_tablet_manager.cc:1403] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.001s
I20260709 12:20:38.915807 13847 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.916277 13844 raft_consensus.cc:359] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.917062 13844 raft_consensus.cc:385] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.917356 13844 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.917673 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:38.918510 13847 ts_tablet_manager.cc:1434] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:20:38.918098 13844 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.919981 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:38.920717 13844 ts_tablet_manager.cc:1434] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:20:38.920893 13885 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:38.921341 13885 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:38.921995 13845 raft_consensus.cc:359] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.922515 13847 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:38.922150 13885 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.922751 13845 raft_consensus.cc:385] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.923039 13845 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.923683 13845 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.925818 13845 ts_tablet_manager.cc:1434] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:20:38.926898 13845 tablet_bootstrap.cc:492] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.930675 13847 tablet_bootstrap.cc:654] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.931600 13849 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.933094 13845 tablet_bootstrap.cc:654] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.951418 13844 tablet_bootstrap.cc:492] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:38.969136 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:38.971927 13845 tablet_bootstrap.cc:492] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:38.972532 13845 ts_tablet_manager.cc:1403] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.046s	user 0.030s	sys 0.012s
I20260709 12:20:38.975697 13845 raft_consensus.cc:359] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.976543 13845 raft_consensus.cc:385] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.976831 13845 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.976086 13550 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:38.977569 13845 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.978965 13847 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:38.979465 13847 ts_tablet_manager.cc:1403] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.028s
I20260709 12:20:38.979859 13845 ts_tablet_manager.cc:1434] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:38.980983 13845 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:38.982668 13847 raft_consensus.cc:359] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.983408 13847 raft_consensus.cc:385] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:38.983693 13847 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:38.985276 13847 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:38.987634 13847 ts_tablet_manager.cc:1434] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:38.987789 13845 tablet_bootstrap.cc:654] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:38.988687 13847 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:38.996034 13847 tablet_bootstrap.cc:654] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.000161 13885 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:38.977231 13844 tablet_bootstrap.cc:654] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.013931 13861 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.014729 13861 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.024922 13847 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.026706 13847 ts_tablet_manager.cc:1403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.014s
I20260709 12:20:39.031116 13847 raft_consensus.cc:359] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.032652 13847 raft_consensus.cc:385] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.033038 13847 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.039091 13847 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.031821 13861 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:20:39.060653 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:39.061911 13844 tablet_bootstrap.cc:492] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.062500 13844 ts_tablet_manager.cc:1403] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.111s	user 0.013s	sys 0.015s
I20260709 12:20:39.063987 13845 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.064553 13845 ts_tablet_manager.cc:1403] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.084s	user 0.039s	sys 0.035s
I20260709 12:20:39.068375 13845 raft_consensus.cc:359] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.069180 13845 raft_consensus.cc:385] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.069515 13845 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.070298 13845 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.072386 13845 ts_tablet_manager.cc:1434] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:39.073619 13845 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.080462 13845 tablet_bootstrap.cc:654] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.083981 13844 raft_consensus.cc:359] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.084808 13844 raft_consensus.cc:385] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.085104 13844 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.085704 13844 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.087066 13861 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.087599 13844 ts_tablet_manager.cc:1434] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.002s
I20260709 12:20:39.087610 13861 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.095146 13861 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.099803 13847 ts_tablet_manager.cc:1434] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.073s	user 0.012s	sys 0.011s
I20260709 12:20:39.100872 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.101313 13847 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.101661 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:39.102421 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.103263 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.104146 13651 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:39.105067 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.106009 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
W20260709 12:20:39.108198 13766 leader_election.cc:343] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.109026 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:39.110154 13912 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.110774 13912 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.111209 13912 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.111419 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.112875 13861 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.113310 13861 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.113724 13861 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.120041 13844 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.123845 13847 tablet_bootstrap.cc:654] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.126938 13861 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.128533 13912 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.130730 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:39.131304 13844 tablet_bootstrap.cc:654] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.131393 13651 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.132843 13912 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.134778 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:39.135461 13652 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.137635 13845 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.138132 13845 ts_tablet_manager.cc:1403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.028s
I20260709 12:20:39.138571 13847 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.139127 13847 ts_tablet_manager.cc:1403] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.005s
I20260709 12:20:39.140154 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:39.140995 13727 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.141050 13845 raft_consensus.cc:359] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.141847 13845 raft_consensus.cc:385] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.142153 13845 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.141969 13847 raft_consensus.cc:359] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.142638 13847 raft_consensus.cc:385] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.143057 13847 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.142905 13845 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.143776 13847 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.145426 13845 ts_tablet_manager.cc:1434] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:39.146332 13847 ts_tablet_manager.cc:1434] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:20:39.147344 13847 tablet_bootstrap.cc:492] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.147344 13845 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.153914 13847 tablet_bootstrap.cc:654] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.156076 13845 tablet_bootstrap.cc:654] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.162366 13866 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.162981 13866 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.165509 13866 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.167577 13912 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.169004 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.169131 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.174947 13651 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.175268 13650 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:39.177054 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:39.169951 13912 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.183481 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:39.184003 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.182150 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.183270 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:39.186992 13866 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.187399 13866 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.187458 13861 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.187870 13866 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.188112 13861 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.189752 13912 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.190793 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.191717 13651 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:39.191957 13912 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.195987 13912 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:39.195729 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:39.191738 13861 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:39.201193 13861 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:39.201946 13693 leader_election.cc:343] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.200378 13866 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.202657 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:39.204402 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.197245 13912 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.201809 13847 tablet_bootstrap.cc:492] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.204893 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:39.205087 13866 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:39.205231 13766 leader_election.cc:343] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.206771 13847 ts_tablet_manager.cc:1403] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.007s
I20260709 12:20:39.207065 13919 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.207458 13919 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.207688 13919 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.208598 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:39.209790 13766 leader_election.cc:343] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.209934 13847 raft_consensus.cc:359] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.210639 13847 raft_consensus.cc:385] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.210831 13844 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.210951 13847 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
W20260709 12:20:39.211133 13693 leader_election.cc:343] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.211416 13844 ts_tablet_manager.cc:1403] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.022s
I20260709 12:20:39.200316 13845 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.212275 13845 ts_tablet_manager.cc:1403] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.019s
I20260709 12:20:39.206789 13651 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.214425 13844 raft_consensus.cc:359] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.215157 13844 raft_consensus.cc:385] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.215442 13844 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.216146 13844 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.216660 13919 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.217407 13918 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.217895 13918 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.219235 13885 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.219743 13885 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.221108 13919 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.221256 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:39.221858 13885 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.221905 13652 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.225317 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.222559 13847 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.223147 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.228157 13651 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:39.228703 13799 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:39.229775 13847 ts_tablet_manager.cc:1434] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 12:20:39.225359 13844 ts_tablet_manager.cc:1434] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:20:39.225749 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:39.230031 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:39.230403 13727 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.230974 13885 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.231063 13918 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.231374 13885 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.231758 13885 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.231725 13726 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:39.218048 13845 raft_consensus.cc:359] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.226853 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.229568 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:39.231223 13844 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.226106 13550 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.233973 13845 raft_consensus.cc:385] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.234691 13847 tablet_bootstrap.cc:492] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.234529 13650 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:39.239961 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:39.238080 13866 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.238564 13845 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.242045 13845 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.244256 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.246376 13727 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.242205 13912 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.248052 13912 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.248415 13912 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:39.250293 13766 leader_election.cc:343] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.250516 13652 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.251997 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:39.253535 13918 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.254110 13918 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:39.255144 13918 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.246690 13845 ts_tablet_manager.cc:1434] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:20:39.261656 13845 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.270486 13844 tablet_bootstrap.cc:654] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.266826 13866 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:39.272454 13866 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.275733 13885 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.279597 13847 tablet_bootstrap.cc:654] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.282400 13845 tablet_bootstrap.cc:654] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.282721 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:39.283654 13727 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.283362 13912 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.285944 13912 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.287060 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:39.287660 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:39.287964 13652 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:39.301803 13766 leader_election.cc:343] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.303380 13885 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.300444 13727 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:39.306551 13617 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:39.307488 13885 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.309499 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:39.312727 13885 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:39.313763 13885 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.312731 13799 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.321165 13847 tablet_bootstrap.cc:492] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.321920 13847 ts_tablet_manager.cc:1403] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.000s
I20260709 12:20:39.323607 13550 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.331115 13845 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.333241 13845 ts_tablet_manager.cc:1403] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.072s	user 0.026s	sys 0.009s
I20260709 12:20:39.337774 13652 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.341962 13844 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.342557 13844 ts_tablet_manager.cc:1403] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.112s	user 0.018s	sys 0.021s
I20260709 12:20:39.343092 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.344347 13918 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.344861 13918 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:39.345660 13844 raft_consensus.cc:359] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.348496 13844 raft_consensus.cc:385] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.348810 13844 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.349663 13844 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.351294 13844 ts_tablet_manager.cc:1434] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:39.352869 13844 tablet_bootstrap.cc:492] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.358551 13845 raft_consensus.cc:359] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.359305 13845 raft_consensus.cc:385] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.359584 13845 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.360136 13845 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.361191 13918 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.363462 13847 raft_consensus.cc:359] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.363966 13845 ts_tablet_manager.cc:1434] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 12:20:39.364151 13847 raft_consensus.cc:385] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.364408 13847 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.364971 13845 tablet_bootstrap.cc:492] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.365193 13847 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.367249 13847 ts_tablet_manager.cc:1434] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.001s
I20260709 12:20:39.368513 13847 tablet_bootstrap.cc:492] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.370535 13547 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.374511 13547 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.374554 13549 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:39.379113 13799 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:39.382357 13844 tablet_bootstrap.cc:654] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.384096 13885 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.385090 13885 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.389276 13885 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.390566 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:39.393220 13617 leader_election.cc:343] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.398016 13845 tablet_bootstrap.cc:654] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.399138 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.399854 13847 tablet_bootstrap.cc:654] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:39.400823 13618 leader_election.cc:343] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.401317 13618 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.402823 13885 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.408668 13844 tablet_bootstrap.cc:492] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.409178 13844 ts_tablet_manager.cc:1403] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.002s
I20260709 12:20:39.410120 13845 tablet_bootstrap.cc:492] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.410682 13845 ts_tablet_manager.cc:1403] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.008s
I20260709 12:20:39.412042 13844 raft_consensus.cc:359] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.412611 13844 raft_consensus.cc:385] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.412848 13844 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.413489 13845 raft_consensus.cc:359] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.413527 13844 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.414378 13845 raft_consensus.cc:385] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.414695 13845 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.416013 13847 tablet_bootstrap.cc:492] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.416175 13844 ts_tablet_manager.cc:1434] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:20:39.415925 13845 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.416638 13847 ts_tablet_manager.cc:1403] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.000s
I20260709 12:20:39.417541 13844 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.418558 13845 ts_tablet_manager.cc:1434] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:39.419986 13845 tablet_bootstrap.cc:492] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.420163 13847 raft_consensus.cc:359] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.420943 13847 raft_consensus.cc:385] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.421241 13847 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.422149 13847 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.414754 13547 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:39.441215 13847 ts_tablet_manager.cc:1434] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 12:20:39.443022 13847 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.446830 13844 tablet_bootstrap.cc:654] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.446831 13845 tablet_bootstrap.cc:654] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.454900 13799 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.457532 13847 tablet_bootstrap.cc:654] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.462208 13845 tablet_bootstrap.cc:492] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.462813 13845 ts_tablet_manager.cc:1403] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.004s
I20260709 12:20:39.465794 13845 raft_consensus.cc:359] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.467509 13845 raft_consensus.cc:385] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.467890 13845 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.479928 13863 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:20:39.469321 13845 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.479281 13885 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.483788 13885 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.484620 13886 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.487874 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.489746 13885 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.493278 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.494469 13844 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.494833 13847 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.494967 13844 ts_tablet_manager.cc:1403] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.078s	user 0.030s	sys 0.016s
I20260709 12:20:39.495285 13847 ts_tablet_manager.cc:1403] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.021s
W20260709 12:20:39.495107 13618 leader_election.cc:343] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.498090 13844 raft_consensus.cc:359] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.498854 13844 raft_consensus.cc:385] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.499186 13844 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.500010 13844 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.500325 13847 raft_consensus.cc:359] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:39.500744 13617 leader_election.cc:343] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.500982 13847 raft_consensus.cc:385] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.501313 13847 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.501356 13617 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.502278 13885 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.502592 13844 ts_tablet_manager.cc:1434] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:39.502909 13847 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.503536 13844 tablet_bootstrap.cc:492] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.505177 13847 ts_tablet_manager.cc:1434] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:20:39.506199 13847 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.511441 13844 tablet_bootstrap.cc:654] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.514533 13845 ts_tablet_manager.cc:1434] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:20:39.517733 13845 tablet_bootstrap.cc:492] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.522104 13847 tablet_bootstrap.cc:654] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.523241 13844 tablet_bootstrap.cc:492] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.523769 13844 ts_tablet_manager.cc:1403] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.006s
I20260709 12:20:39.527060 13844 raft_consensus.cc:359] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.527776 13844 raft_consensus.cc:385] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.528143 13844 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.529474 13844 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.532732 13844 ts_tablet_manager.cc:1434] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:20:39.534085 13844 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.543339 13844 tablet_bootstrap.cc:654] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.543802 13845 tablet_bootstrap.cc:654] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.549947 13918 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.550527 13918 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.554895 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.554730 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.555850 13652 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
W20260709 12:20:39.557024 13766 leader_election.cc:343] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.557730 13918 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.557981 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.559468 13918 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.559926 13918 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.560833 13918 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.564534 13844 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.565508 13844 ts_tablet_manager.cc:1403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.004s
I20260709 12:20:39.573992 13844 raft_consensus.cc:359] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.574824 13844 raft_consensus.cc:385] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.575119 13844 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.576004 13844 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.578572 13844 ts_tablet_manager.cc:1434] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 12:20:39.579419 13844 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.579576 13847 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.580062 13847 ts_tablet_manager.cc:1403] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.074s	user 0.015s	sys 0.040s
I20260709 12:20:39.583194 13847 raft_consensus.cc:359] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.583966 13847 raft_consensus.cc:385] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.584275 13847 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.586750 13844 tablet_bootstrap.cc:654] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.591778 13918 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.588502 13847 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.596320 13845 tablet_bootstrap.cc:492] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.596936 13845 ts_tablet_manager.cc:1403] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.003s
I20260709 12:20:39.597697 13847 ts_tablet_manager.cc:1434] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 12:20:39.599313 13847 tablet_bootstrap.cc:492] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.600250 13845 raft_consensus.cc:359] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.601110 13845 raft_consensus.cc:385] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.601444 13845 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.602401 13845 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.604771 13845 ts_tablet_manager.cc:1434] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:39.606215 13845 tablet_bootstrap.cc:492] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.611455 13847 tablet_bootstrap.cc:654] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.618144 13918 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.618486 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:39.619138 13652 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.619850 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:20:39.621757 13766 leader_election.cc:343] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.623090 13844 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:39.623768 13844 ts_tablet_manager.cc:1403] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.020s
I20260709 12:20:39.627250 13844 raft_consensus.cc:359] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.627985 13844 raft_consensus.cc:385] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.628258 13844 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.628650 13847 tablet_bootstrap.cc:492] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.629247 13847 ts_tablet_manager.cc:1403] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.004s
I20260709 12:20:39.629011 13844 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.632052 13844 ts_tablet_manager.cc:1434] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:20:39.635923 13847 raft_consensus.cc:359] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.636787 13847 raft_consensus.cc:385] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.637027 13845 tablet_bootstrap.cc:654] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.637097 13847 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.638350 13847 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.639305 13652 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:39.640959 13847 ts_tablet_manager.cc:1434] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 12:20:39.641443 13844 tablet_bootstrap.cc:492] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:39.641988 13847 tablet_bootstrap.cc:492] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:39.641585 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.643328 13918 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:39.644050 13918 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:39.649112 13844 tablet_bootstrap.cc:654] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.649112 13847 tablet_bootstrap.cc:654] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:39.645295 13918 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.653915 13866 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.654537 13866 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.656941 13866 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.662851 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.663944 13652 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:39.665304 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.668772 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:39.673851 13866 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.677393 13886 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.677974 13886 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.680523 13886 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.681856 13912 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.682327 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.682425 13912 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.683044 13727 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:39.684314 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:39.684638 13912 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.685292 13886 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:39.685717 13886 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:39.686138 13886 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.687368 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.688102 13801 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:39.688198 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.688999 13652 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:39.690397 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.692708 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.694522 13912 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:39.724522 13861 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.726610 13922 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.728384 13885 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.729154 13885 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.738961 13863 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.740676 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:39.737464 13885 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:39.742642 13618 leader_election.cc:343] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.746369 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:39.749925 13919 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.751916 13919 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:39.752200 13617 leader_election.cc:343] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:39.752892 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:39.754671 13885 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.814435 13845 tablet_bootstrap.cc:492] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:39.817862 13845 ts_tablet_manager.cc:1403] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.212s	user 0.012s	sys 0.015s
W20260709 12:20:39.821745 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69610.8us
I20260709 12:20:39.824337 13861 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:20:39.840914 13861 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:39.836300 13922 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.842608 13922 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.841662 13845 raft_consensus.cc:359] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.846333 13845 raft_consensus.cc:385] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:39.846673 13845 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:39.847451 13845 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.837610 13912 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.882515 13922 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.894786 13845 ts_tablet_manager.cc:1434] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.075s	user 0.008s	sys 0.000s
I20260709 12:20:39.895954 13845 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:39.874745 13866 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:39.892586 13951 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.899184 13951 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.902997 13922 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.918874 13922 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.929508 13922 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.931090 13951 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.932775 13886 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.940558 13886 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.903971 13885 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 12:20:39.940377 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.965049 13651 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:39.970727 13847 tablet_bootstrap.cc:492] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:39.971900 13847 ts_tablet_manager.cc:1403] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.330s	user 0.051s	sys 0.095s
W20260709 12:20:39.973150 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52922.4us
I20260709 12:20:39.979458 13912 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:39.981033 13919 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.981524 13919 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.983685 13919 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.986358 13922 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.986933 13922 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.989055 13922 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:39.990513 13918 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:39.992514 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.992645 13918 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:39.993249 13651 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:39.994347 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:39.994917 13918 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:39.995543 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.010841 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.011008 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:40.024668 13844 tablet_bootstrap.cc:492] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.025142 13844 ts_tablet_manager.cc:1403] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.391s	user 0.062s	sys 0.128s
I20260709 12:20:40.019773 13800 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.023062 13885 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.019869 13799 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:40.023886 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:40.008803 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.041620 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.051383 13725 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.056193 13844 raft_consensus.cc:359] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.058338 13844 raft_consensus.cc:385] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.062836 13844 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.066836 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.067713 13726 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:40.070492 13726 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.037655 13885 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.051841 13650 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
W20260709 12:20:40.052160 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49922.5us
I20260709 12:20:40.102041 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.102658 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.103037 13723 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:40.119475 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.127034 13960 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.127501 13960 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.127854 13960 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.130322 13844 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.132405 13847 raft_consensus.cc:359] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.133522 13847 raft_consensus.cc:385] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.122417 13919 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:20:40.136300 13956 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.136791 13956 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.137178 13956 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.133946 13847 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.141366 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.140489 13847 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.145478 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.126118 13885 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.146486 13795 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:40.140429 13766 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.165117 13800 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:40.168991 13912 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.160008 13961 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.169283 13618 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:40.176625 13847 ts_tablet_manager.cc:1434] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.204s	user 0.011s	sys 0.002s
I20260709 12:20:40.189283 13961 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.190299 13845 tablet_bootstrap.cc:654] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.190064 13961 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.191524 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:40.191931 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.193725 13725 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:40.146486 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:40.194571 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.169601 13912 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.199011 13886 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.199311 13918 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.200676 13550 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:40.217623 13847 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:40.197829 13885 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.221799 13844 ts_tablet_manager.cc:1434] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.196s	user 0.012s	sys 0.012s
I20260709 12:20:40.205049 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.212777 13912 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:40.239251 13693 leader_election.cc:343] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.212273 13886 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.203756 13960 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.217634 13652 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:40.237373 13885 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:40.242581 13886 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.254159 13885 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.259526 13922 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.259701 13693 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:20:40.261819 13966 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:20:40.264590 13844 tablet_bootstrap.cc:492] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
W20260709 12:20:40.262228 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:40.274982 13956 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.284540 13922 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.288012 13960 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.289081 13886 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.293110 13886 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.300559 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:40.301397 13801 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.301731 13956 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:40.303220 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:40.304435 13844 tablet_bootstrap.cc:654] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.307684 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.308391 13651 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.308529 13723 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.316977 13847 tablet_bootstrap.cc:654] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.308897 13912 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.318928 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:40.320679 13795 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:40.320971 13722 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:40.322137 13845 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.323439 13845 ts_tablet_manager.cc:1403] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.428s	user 0.014s	sys 0.014s
I20260709 12:20:40.327416 13912 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:40.331386 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.332213 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.333110 13650 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.320034 13956 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:20:40.342887 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:40.356453 13724 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.365236 13650 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:40.367767 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:40.369083 13918 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.369628 13918 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:40.369171 13919 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.378664 13918 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.380688 13801 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:40.381989 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:40.386739 13885 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.387269 13885 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:40.382145 13693 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.385545 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:40.390890 13798 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:40.388191 13961 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.388160 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.366202 13651 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:40.403862 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:40.405602 13919 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.406210 13919 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:40.407230 13919 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.388293 13885 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.421907 13961 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:40.431668 13617 leader_election.cc:343] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.434957 13845 raft_consensus.cc:359] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.435672 13845 raft_consensus.cc:385] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.435936 13845 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.441138 13845 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:40.442910 13617 leader_election.cc:343] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.451246 13845 ts_tablet_manager.cc:1434] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.127s	user 0.009s	sys 0.003s
I20260709 12:20:40.452514 13845 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.455132 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:40.457670 13960 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.460805 13845 tablet_bootstrap.cc:654] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.463652 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.464337 13651 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.473635 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:40.475939 13693 leader_election.cc:343] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.480581 13651 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:40.484306 13724 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:40.485453 13922 raft_consensus.cc:2764] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:20:40.489318 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:40.490466 13922 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.490993 13922 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:40.493048 13922 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.531790 13886 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.532720 13886 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.534720 13847 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:40.535286 13847 ts_tablet_manager.cc:1403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.322s	user 0.014s	sys 0.016s
I20260709 12:20:40.536722 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.537384 13886 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:40.538548 13617 leader_election.cc:343] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.540400 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:40.543937 13618 leader_election.cc:343] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.544515 13618 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:40.545936 13844 tablet_bootstrap.cc:492] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.546984 13844 ts_tablet_manager.cc:1403] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.283s	user 0.016s	sys 0.018s
I20260709 12:20:40.548646 13886 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.549831 13847 raft_consensus.cc:359] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.550501 13847 raft_consensus.cc:385] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.550748 13847 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.551451 13847 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.556121 13550 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:40.558919 13844 raft_consensus.cc:359] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.559731 13844 raft_consensus.cc:385] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.560041 13844 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.559937 13550 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:40.561173 13844 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.563916 13845 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.564517 13845 ts_tablet_manager.cc:1403] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.013s
I20260709 12:20:40.564036 13847 ts_tablet_manager.cc:1434] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:20:40.567934 13847 tablet_bootstrap.cc:492] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:40.567658 13845 raft_consensus.cc:359] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.568432 13845 raft_consensus.cc:385] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.568723 13845 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.569446 13845 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.573441 13844 ts_tablet_manager.cc:1434] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.002s
I20260709 12:20:40.574932 13844 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:40.574983 13845 ts_tablet_manager.cc:1434] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:20:40.576570 13845 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.578516 13847 tablet_bootstrap.cc:654] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.583350 13845 tablet_bootstrap.cc:654] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.587725 13844 tablet_bootstrap.cc:654] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.588261 13549 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:40.594883 13549 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:40.610522 13886 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.611326 13886 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.616035 13886 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.616114 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.616591 13547 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:40.618428 13617 leader_election.cc:343] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.616146 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:40.621031 13618 leader_election.cc:343] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.621521 13618 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:40.622380 13845 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.622890 13845 ts_tablet_manager.cc:1403] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.047s	user 0.017s	sys 0.025s
I20260709 12:20:40.622926 13844 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.622984 13886 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.624264 13844 ts_tablet_manager.cc:1403] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.004s
I20260709 12:20:40.626104 13845 raft_consensus.cc:359] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.626917 13845 raft_consensus.cc:385] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.627194 13845 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.627548 13844 raft_consensus.cc:359] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.628284 13844 raft_consensus.cc:385] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.627918 13845 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.628674 13844 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.629645 13844 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.630363 13845 ts_tablet_manager.cc:1434] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:40.631376 13845 tablet_bootstrap.cc:492] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.632207 13844 ts_tablet_manager.cc:1434] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:20:40.633381 13844 tablet_bootstrap.cc:492] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:40.638833 13845 tablet_bootstrap.cc:654] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.644192 13844 tablet_bootstrap.cc:654] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.655401 13845 tablet_bootstrap.cc:492] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.656023 13845 ts_tablet_manager.cc:1403] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.000s
I20260709 12:20:40.659839 13845 raft_consensus.cc:359] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.660735 13845 raft_consensus.cc:385] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.661059 13845 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.661878 13845 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.663552 13845 ts_tablet_manager.cc:1434] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:40.664146 13847 tablet_bootstrap.cc:492] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:40.664613 13845 tablet_bootstrap.cc:492] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.664705 13847 ts_tablet_manager.cc:1403] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.051s
I20260709 12:20:40.666210 13844 tablet_bootstrap.cc:492] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.666749 13844 ts_tablet_manager.cc:1403] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.001s
I20260709 12:20:40.668160 13847 raft_consensus.cc:359] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.669075 13847 raft_consensus.cc:385] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.669600 13847 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.669884 13844 raft_consensus.cc:359] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.670667 13844 raft_consensus.cc:385] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.670948 13844 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.670842 13847 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.672155 13845 tablet_bootstrap.cc:654] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.671828 13844 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.679486 13847 ts_tablet_manager.cc:1434] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:20:40.680955 13847 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:40.683207 13844 ts_tablet_manager.cc:1434] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:20:40.684254 13844 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:40.689085 13886 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.689661 13886 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.691875 13886 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.694039 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.694711 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:40.696173 13617 leader_election.cc:343] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:40.696869 13618 leader_election.cc:343] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.697228 13618 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:40.697918 13886 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.702594 13847 tablet_bootstrap.cc:654] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.702594 13844 tablet_bootstrap.cc:654] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.703686 13845 tablet_bootstrap.cc:492] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.704241 13845 ts_tablet_manager.cc:1403] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.000s
I20260709 12:20:40.707665 13845 raft_consensus.cc:359] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.708668 13845 raft_consensus.cc:385] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.709038 13845 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.710000 13845 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.710762 13886 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.714596 13847 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:40.715291 13847 ts_tablet_manager.cc:1403] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.010s
I20260709 12:20:40.716817 13845 ts_tablet_manager.cc:1434] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:20:40.718952 13847 raft_consensus.cc:359] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.719736 13847 raft_consensus.cc:385] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.720023 13847 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.720748 13847 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.724040 13847 ts_tablet_manager.cc:1434] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:20:40.724788 13844 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.727605 13844 ts_tablet_manager.cc:1403] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.009s
I20260709 12:20:40.725087 13847 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:40.731925 13844 raft_consensus.cc:359] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.732764 13844 raft_consensus.cc:385] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.732961 13845 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.733114 13844 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.734030 13844 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.737145 13847 tablet_bootstrap.cc:654] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.745571 13845 tablet_bootstrap.cc:654] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.746325 13844 ts_tablet_manager.cc:1434] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:20:40.751365 13885 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.752063 13844 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:40.751904 13885 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.753449 13847 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:40.753947 13847 ts_tablet_manager.cc:1403] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.006s
I20260709 12:20:40.754318 13885 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.758221 13847 raft_consensus.cc:359] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.759074 13844 tablet_bootstrap.cc:654] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.759779 13847 raft_consensus.cc:385] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.760524 13847 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.759081 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.761241 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:40.762315 13618 leader_election.cc:343] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.764238 13847 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.767870 13847 ts_tablet_manager.cc:1434] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:20:40.768507 13922 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.769340 13922 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.772078 13845 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.772414 13922 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.772886 13845 ts_tablet_manager.cc:1403] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.003s
I20260709 12:20:40.776984 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:20:40.777529 13617 leader_election.cc:343] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.776937 13845 raft_consensus.cc:359] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.779156 13845 raft_consensus.cc:385] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.778638 13886 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.779366 13845 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.780107 13845 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.778206 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:40.781646 13886 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.781956 13845 ts_tablet_manager.cc:1434] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:20:40.782691 13922 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.783187 13922 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.783660 13845 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.785467 13922 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.791080 13845 tablet_bootstrap.cc:654] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.794626 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.795429 13652 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:40.797827 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.798776 13961 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.799144 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.799930 13847 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:40.800022 13795 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:40.799170 13961 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.801697 13961 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.802706 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:40.804454 13960 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.799175 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:40.806514 13693 leader_election.cc:343] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.813809 13847 tablet_bootstrap.cc:654] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.813705 13961 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.820587 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.821715 13652 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.827713 13922 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.828346 13922 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.830051 13960 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:40.830850 13922 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.831921 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.832703 13651 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:40.833277 13844 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.833793 13844 ts_tablet_manager.cc:1403] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.082s	user 0.029s	sys 0.033s
I20260709 12:20:40.835421 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:40.836223 13798 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:40.837159 13844 raft_consensus.cc:359] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.838038 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.838949 13922 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.839313 13922 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.839623 13922 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.840596 13652 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:40.843657 13961 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.846185 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.846376 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:40.838091 13844 raft_consensus.cc:385] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.847324 13961 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.847461 13844 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.847922 13961 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:40.848239 13844 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.848804 13961 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.850982 13922 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.851581 13844 ts_tablet_manager.cc:1434] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 12:20:40.852412 13847 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:40.852605 13844 tablet_bootstrap.cc:492] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:40.853086 13847 ts_tablet_manager.cc:1403] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.000s
I20260709 12:20:40.853305 13922 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.856014 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:40.856658 13798 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.857224 13845 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.857739 13845 ts_tablet_manager.cc:1403] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.074s	user 0.003s	sys 0.011s
I20260709 12:20:40.860297 13844 tablet_bootstrap.cc:654] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.860500 13845 raft_consensus.cc:359] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.861275 13845 raft_consensus.cc:385] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.861546 13845 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.862440 13845 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.864552 13798 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:40.864806 13845 ts_tablet_manager.cc:1434] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
W20260709 12:20:40.866132 13693 leader_election.cc:343] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.867230 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.863061 13847 raft_consensus.cc:359] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.868466 13922 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:40.871378 13845 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:40.872529 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.873096 13652 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.874315 13922 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:40.868744 13847 raft_consensus.cc:385] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.876905 13847 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.875774 13922 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.884398 13847 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.887398 13847 ts_tablet_manager.cc:1434] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.000s
I20260709 12:20:40.888535 13847 tablet_bootstrap.cc:492] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:40.888662 13863 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:40.893615 13861 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.891423 13550 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:40.899621 13861 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.900532 13847 tablet_bootstrap.cc:654] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.900379 13861 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.904915 13861 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:40.910272 13886 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:40.913544 13861 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.914971 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.915699 13651 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:40.916981 13767 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:40.918300 13912 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.918730 13912 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.919070 13912 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.920884 13976 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.921841 13976 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.922070 13970 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:40.910960 13845 tablet_bootstrap.cc:654] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.924153 13976 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:40.911212 13652 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:40.911228 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:40.929069 13766 leader_election.cc:343] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.931087 13844 tablet_bootstrap.cc:492] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:40.931633 13844 ts_tablet_manager.cc:1403] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.016s
I20260709 12:20:40.933841 13912 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.934396 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.935071 13724 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:40.935413 13844 raft_consensus.cc:359] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.936126 13844 raft_consensus.cc:385] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.936203 13912 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:40.936968 13844 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.936820 13847 tablet_bootstrap.cc:492] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:40.937892 13847 ts_tablet_manager.cc:1403] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.050s	user 0.009s	sys 0.004s
I20260709 12:20:40.937748 13844 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.940582 13844 ts_tablet_manager.cc:1434] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:20:40.941165 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:40.941965 13844 tablet_bootstrap.cc:492] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:40.942378 13912 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:40.946390 13845 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:40.947187 13845 ts_tablet_manager.cc:1403] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.001s
I20260709 12:20:40.949297 13844 tablet_bootstrap.cc:654] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:40.949689 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:40.950491 13652 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:40.951160 13845 raft_consensus.cc:359] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.951931 13845 raft_consensus.cc:385] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.952275 13845 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.953055 13845 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.954310 13912 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:40.955174 13912 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.914561 13861 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.955725 13847 raft_consensus.cc:359] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.957031 13863 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:40.957748 13847 raft_consensus.cc:385] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:40.958194 13847 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:40.976512 13847 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.979532 13847 ts_tablet_manager.cc:1434] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 12:20:40.983507 13912 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.958796 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:40.963721 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:40.968021 13861 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:20:40.991353 13766 leader_election.cc:343] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:40.968250 13863 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:40.969909 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:40.996158 13650 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:40.997728 13863 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:40.972867 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.004118 13912 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:41.004523 13649 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:40.999223 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:41.000485 13861 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.001971 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.003559 13847 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
W20260709 12:20:41.012800 13618 leader_election.cc:343] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.014758 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:41.022704 13617 leader_election.cc:343] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.023458 13617 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.024936 13886 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.031442 13650 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:41.034415 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:41.035681 13795 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.036206 13649 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.036652 13845 ts_tablet_manager.cc:1434] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.089s	user 0.004s	sys 0.004s
I20260709 12:20:41.037717 13845 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:41.043085 13922 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.043656 13922 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.046226 13922 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.047601 13861 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.048137 13922 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.051914 13976 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.051959 13724 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.052969 13976 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.065209 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.065408 13649 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:41.066008 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.066706 13767 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.066817 13723 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:41.068311 13767 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.068547 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:41.069820 13918 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:41.070601 13976 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:41.071954 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.075994 13886 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.076586 13919 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.077095 13919 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.077487 13919 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.081435 13970 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.076673 13886 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:41.084213 13693 leader_election.cc:343] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.085752 13886 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.086246 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.087730 13863 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.090755 13863 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.095638 13919 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.097795 13798 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:41.100034 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.099881 13550 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:41.100903 13801 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:41.102684 13618 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:41.104503 13958 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.104887 13958 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.105235 13958 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.105630 13863 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.069942 13845 tablet_bootstrap.cc:654] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.114575 13958 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.121927 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:41.122829 13798 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.122634 13958 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:41.123203 13693 leader_election.cc:343] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.123992 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:41.089849 13847 tablet_bootstrap.cc:654] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.134404 13918 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:41.136016 13918 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:41.139065 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 69521.8us
I20260709 12:20:41.140307 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:41.142060 13912 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:41.142928 13912 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:41.144171 13912 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.152783 13798 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:41.154306 13618 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:41.155144 13958 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:41.155668 13958 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:41.156536 13958 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.157544 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.158278 13651 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:41.162143 13919 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:41.164211 13922 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.168427 13960 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:41.168702 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:41.169376 13651 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.170234 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:41.171976 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:41.172569 13722 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.174649 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.168618 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.182785 13651 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
W20260709 12:20:41.185627 13617 leader_election.cc:343] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.185994 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:41.186942 13844 tablet_bootstrap.cc:492] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:41.187117 14017 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:41.187496 13844 ts_tablet_manager.cc:1403] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.246s	user 0.039s	sys 0.078s
I20260709 12:20:41.187768 14017 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:20:41.188148 13617 leader_election.cc:343] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.188755 14017 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.191339 13844 raft_consensus.cc:359] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.192085 13844 raft_consensus.cc:385] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.192350 13844 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.193037 13844 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.193804 13722 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:41.193631 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.195683 13844 ts_tablet_manager.cc:1434] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:41.196326 13863 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.197151 13844 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:41.204464 13844 tablet_bootstrap.cc:654] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.207548 13845 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:41.208765 13845 ts_tablet_manager.cc:1403] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.171s	user 0.014s	sys 0.018s
I20260709 12:20:41.212081 13845 raft_consensus.cc:359] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.212970 13845 raft_consensus.cc:385] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.213341 13845 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.214386 13845 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.221134 13845 ts_tablet_manager.cc:1434] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:20:41.222337 13845 tablet_bootstrap.cc:492] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:41.154155 13724 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:41.233767 13847 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:41.237349 13847 ts_tablet_manager.cc:1403] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.234s	user 0.015s	sys 0.005s
I20260709 12:20:41.246563 13847 raft_consensus.cc:359] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.247941 13847 raft_consensus.cc:385] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.248334 13847 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.249303 13847 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.250483 13549 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.251924 13549 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.252447 13847 ts_tablet_manager.cc:1434] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.008s
I20260709 12:20:41.254313 13847 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:41.266736 13548 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:41.296731 13844 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:41.297312 13844 ts_tablet_manager.cc:1403] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.100s	user 0.024s	sys 0.007s
I20260709 12:20:41.299782 13922 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.300297 13844 raft_consensus.cc:359] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.301092 13844 raft_consensus.cc:385] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.301399 13844 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.302107 13844 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.304224 13844 ts_tablet_manager.cc:1434] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:41.305454 13844 tablet_bootstrap.cc:492] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:41.313905 13847 tablet_bootstrap.cc:654] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.318099 13845 tablet_bootstrap.cc:654] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.329499 13960 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.337886 13550 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:41.351383 13795 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.353981 13844 tablet_bootstrap.cc:654] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.377771 13844 tablet_bootstrap.cc:492] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:41.378496 13844 ts_tablet_manager.cc:1403] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.000s
W20260709 12:20:41.385008 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:41.385921 13863 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.386571 13863 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.386902 13922 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:41.391335 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.392395 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:41.394394 13617 leader_election.cc:343] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:41.395218 13618 leader_election.cc:343] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.395725 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.397145 13886 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.401928 13844 raft_consensus.cc:359] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.402726 13844 raft_consensus.cc:385] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.403024 13844 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.401546 13863 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.403697 13844 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.418747 13844 ts_tablet_manager.cc:1434] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 12:20:41.420238 13844 tablet_bootstrap.cc:492] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:41.424885 13847 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:41.425063 13845 tablet_bootstrap.cc:492] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:41.425491 13845 ts_tablet_manager.cc:1403] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.203s	user 0.013s	sys 0.010s
I20260709 12:20:41.425606 13847 ts_tablet_manager.cc:1403] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.172s	user 0.026s	sys 0.008s
I20260709 12:20:41.429284 13844 tablet_bootstrap.cc:654] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.429051 13845 raft_consensus.cc:359] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.429925 13845 raft_consensus.cc:385] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.430351 13845 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.430754 13922 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:41.431167 13845 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.433827 13845 ts_tablet_manager.cc:1434] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:20:41.438635 13847 raft_consensus.cc:359] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.439664 13847 raft_consensus.cc:385] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.439966 13847 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.441989 13847 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.447820 13847 ts_tablet_manager.cc:1434] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:20:41.449084 13847 tablet_bootstrap.cc:492] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:41.453290 13844 tablet_bootstrap.cc:492] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:41.453826 13844 ts_tablet_manager.cc:1403] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.008s
I20260709 12:20:41.458674 13845 tablet_bootstrap.cc:492] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:41.461632 13863 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.462314 13863 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.465443 13863 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.466562 13845 tablet_bootstrap.cc:654] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.471077 13847 tablet_bootstrap.cc:654] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.474939 13844 raft_consensus.cc:359] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.476210 13844 raft_consensus.cc:385] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.476457 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.476589 13844 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
W20260709 12:20:41.478808 13618 leader_election.cc:343] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.478477 13844 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.482523 13844 ts_tablet_manager.cc:1434] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 12:20:41.483592 13922 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.484083 13844 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:41.484225 13922 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.486917 13922 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.487582 13845 tablet_bootstrap.cc:492] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:41.488124 13845 ts_tablet_manager.cc:1403] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.009s
I20260709 12:20:41.491576 13845 raft_consensus.cc:359] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.492313 13845 raft_consensus.cc:385] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.492601 13845 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.492779 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.493547 13724 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:41.495493 13844 tablet_bootstrap.cc:654] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.496510 13845 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.500294 13863 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.500495 13845 ts_tablet_manager.cc:1434] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:20:41.500830 13863 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.501552 13845 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:41.503113 13863 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.515691 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:41.519907 13618 leader_election.cc:343] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.521420 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.523970 13844 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:41.524757 13844 ts_tablet_manager.cc:1403] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.008s
I20260709 12:20:41.531195 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:41.533219 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.533994 13651 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:41.534973 13885 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:20:41.535102 13617 leader_election.cc:343] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.535349 13885 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.535756 13885 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.538738 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.539465 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:41.535636 13617 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.540709 14033 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.540982 13958 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.541085 14033 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.541678 14033 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:41.586956 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51874.6us
I20260709 12:20:41.588292 13844 raft_consensus.cc:359] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.589033 13844 raft_consensus.cc:385] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.589313 13844 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.590029 13844 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.591364 13847 tablet_bootstrap.cc:492] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:41.591948 13847 ts_tablet_manager.cc:1403] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.143s	user 0.022s	sys 0.023s
I20260709 12:20:41.591934 13844 ts_tablet_manager.cc:1434] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.067s	user 0.034s	sys 0.014s
I20260709 12:20:41.593454 13844 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:41.596968 13847 raft_consensus.cc:359] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.597817 13847 raft_consensus.cc:385] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.598140 13847 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.599931 13847 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:41.601536 13693 leader_election.cc:343] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.617498 13885 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.621687 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:41.622454 13798 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.629742 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:41.630093 13798 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:41.631440 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:41.632344 13886 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:41.632923 13886 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:41.634397 13845 tablet_bootstrap.cc:654] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.633874 13886 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.638809 13847 ts_tablet_manager.cc:1434] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.001s
I20260709 12:20:41.643165 14017 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.643693 14017 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.645732 13844 tablet_bootstrap.cc:654] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.646418 13861 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.646999 13885 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.646870 13861 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.614063 14033 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.649093 13861 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:41.649340 13722 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:41.650343 14033 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.651204 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:41.653488 13766 leader_election.cc:343] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.655543 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:41.659412 13724 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.661234 13845 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:41.661826 13845 ts_tablet_manager.cc:1403] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.161s	user 0.006s	sys 0.021s
I20260709 12:20:41.664610 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:41.646014 14017 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:41.665512 13798 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:41.665172 13845 raft_consensus.cc:359] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.666039 13845 raft_consensus.cc:385] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.666407 13845 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.667215 13845 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.669293 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:41.669896 13845 ts_tablet_manager.cc:1434] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:41.670526 14033 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:41.671065 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.671823 13722 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:41.673259 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:41.671429 13845 tablet_bootstrap.cc:492] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:41.674194 13918 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.674757 13918 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.675216 13918 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.682534 13845 tablet_bootstrap.cc:654] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.682386 13958 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.682735 13844 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:41.683362 13844 ts_tablet_manager.cc:1403] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.090s	user 0.029s	sys 0.007s
I20260709 12:20:41.683171 13958 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.684677 13918 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.646893 13847 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:41.685722 13958 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.688709 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:41.692265 13618 leader_election.cc:343] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.695189 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:41.696234 13722 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:41.698097 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:41.699832 13958 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.700240 13958 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.700603 13958 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.700969 13844 raft_consensus.cc:359] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.701707 13844 raft_consensus.cc:385] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.701990 13844 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
W20260709 12:20:41.691833 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86941.7us
I20260709 12:20:41.702881 13844 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.712062 13847 tablet_bootstrap.cc:654] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:41.710564 13918 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:41.710270 14017 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.762038 13845 tablet_bootstrap.cc:492] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:41.763077 13845 ts_tablet_manager.cc:1403] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.092s	user 0.029s	sys 0.022s
I20260709 12:20:41.719162 13549 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:41.767510 13844 ts_tablet_manager.cc:1434] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.084s	user 0.009s	sys 0.000s
I20260709 12:20:41.768735 13844 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:41.715085 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:41.717775 13922 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.773665 13722 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.716519 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.785224 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.786633 14016 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:41.791440 13722 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:20:41.792522 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:41.793655 14016 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:41.794703 13886 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:41.794955 14016 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:41.798718 14016 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.777755 13958 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.795855 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:41.780103 13922 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:41.833268 13617 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:41.840965 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.841761 13649 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:41.844481 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:41.848742 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:41.851153 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:41.851811 13722 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.855011 13918 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:41.856967 13845 raft_consensus.cc:359] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.857796 13845 raft_consensus.cc:385] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:41.859112 13922 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.860702 13845 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:41.861951 13958 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.867524 13845 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.872750 13961 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:41.873750 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:41.876327 13861 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.876849 13861 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.877354 13861 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.875046 14017 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:20:41.878029 13961 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:20:41.879835 13961 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:41.881615 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:41.887650 13845 ts_tablet_manager.cc:1434] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.124s	user 0.001s	sys 0.007s
W20260709 12:20:41.889621 13618 leader_election.cc:343] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:41.890239 13845 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:41.903448 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:41.914317 13847 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:41.915388 13847 ts_tablet_manager.cc:1403] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.269s	user 0.025s	sys 0.002s
I20260709 12:20:41.922456 13922 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:41.923089 13922 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:41.925314 13922 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:41.927215 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:41.928267 13651 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:41.929797 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:41.930687 13922 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:41.931084 13922 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:41.931427 13922 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:41.949066 13844 tablet_bootstrap.cc:654] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:41.953929 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47776.3us
I20260709 12:20:41.980739 13960 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.002676 14052 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.000479 13960 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.009838 13960 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.029896 13847 raft_consensus.cc:359] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.033034 13847 raft_consensus.cc:385] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.033349 13847 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.034132 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:20:41.995855 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84420.7us
I20260709 12:20:42.016629 13722 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:42.022989 13922 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.068862 13847 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.064302 13922 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.021407 13861 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.065037 13652 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:42.080330 13847 ts_tablet_manager.cc:1434] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.165s	user 0.008s	sys 0.002s
I20260709 12:20:42.081830 13847 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:42.082825 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:42.090943 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:42.093544 14033 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.093686 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.094660 13798 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:42.094842 14033 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.095348 14033 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.095991 13861 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:42.095918 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.097627 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:42.098438 13800 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.098559 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:42.099501 13651 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.091307 13958 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.105621 13958 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:42.107139 13958 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.110755 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:42.111478 13650 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.112679 13886 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.113271 13886 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.116526 13886 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.116801 13845 tablet_bootstrap.cc:654] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.103417 13844 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.118126 13844 ts_tablet_manager.cc:1403] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.350s	user 0.031s	sys 0.011s
W20260709 12:20:42.120203 13693 leader_election.cc:343] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.141232 13800 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:42.145406 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:42.146138 13725 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.139354 14033 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.121313 13886 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.148479 13886 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.155400 14033 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.156512 13651 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:42.155671 13650 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:42.158850 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:42.159487 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.159669 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:42.160418 13650 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.168324 13886 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.169239 13724 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:42.180009 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.183269 13844 raft_consensus.cc:359] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.184643 13844 raft_consensus.cc:385] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.185014 13844 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.186936 13844 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.172679 14017 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:20:42.176208 14033 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.171922 13725 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:42.195201 13844 ts_tablet_manager.cc:1434] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.002s
I20260709 12:20:42.178556 13961 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:42.197721 13617 leader_election.cc:343] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.198745 13885 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:20:42.203917 13885 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:42.204335 13885 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:42.205605 13727 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:42.205963 13885 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.207417 13885 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.209473 13650 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:42.210155 13961 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:42.210668 13618 leader_election.cc:343] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.211148 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:42.213491 13863 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.199733 14033 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:42.220995 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.221746 13725 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:42.221877 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:42.222895 14033 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.224517 14061 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.226171 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:42.227138 13886 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.227531 13886 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.227926 13886 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.217808 13550 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.229472 13845 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:42.229091 14061 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:42.233805 13798 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:42.236619 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.242594 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:42.229738 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:42.241212 13845 ts_tablet_manager.cc:1403] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.351s	user 0.015s	sys 0.023s
I20260709 12:20:42.227499 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:42.245832 13844 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:42.243924 14061 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.252954 13961 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.253973 13961 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:42.255259 13961 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:42.257457 13618 leader_election.cc:343] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.270526 13845 raft_consensus.cc:359] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.271391 13845 raft_consensus.cc:385] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.271682 13845 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.277112 13845 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.278918 13845 ts_tablet_manager.cc:1434] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:20:42.279891 13845 tablet_bootstrap.cc:492] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:42.279811 13886 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.282078 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:42.283077 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:42.283653 13727 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.293417 13844 tablet_bootstrap.cc:654] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.296036 14061 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.297367 14061 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:42.297757 14061 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:20:42.298127 13693 leader_election.cc:343] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.304047 13722 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:42.309581 13547 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.312247 13847 tablet_bootstrap.cc:654] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.312170 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:42.305990 13727 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:42.319741 13651 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:20:42.320725 13651 pending_rounds.cc:85] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 0
I20260709 12:20:42.321085 13651 pending_rounds.cc:107] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:20:42.321436 13651 raft_consensus.cc:2889] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:42.320740 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:42.323660 13885 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:42.324172 13885 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:42.315289 14053 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:20:42.325050 13885 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.326633 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:42.328603 13618 leader_election.cc:343] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.331496 14053 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.331825 13548 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.333590 13722 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.315932 13961 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:20:42.317561 14061 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.347142 13845 tablet_bootstrap.cc:654] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.320237 13886 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.351578 14061 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.369858 13550 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.383692 13550 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.401366 13845 tablet_bootstrap.cc:492] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:42.402065 13845 ts_tablet_manager.cc:1403] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.122s	user 0.016s	sys 0.004s
I20260709 12:20:42.405350 13847 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:42.405222 13845 raft_consensus.cc:359] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.406014 13847 ts_tablet_manager.cc:1403] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.324s	user 0.016s	sys 0.031s
I20260709 12:20:42.406183 13845 raft_consensus.cc:385] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.406786 13845 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.407481 13845 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.409162 13847 raft_consensus.cc:359] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.411216 13847 raft_consensus.cc:385] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.411520 13847 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.412237 13847 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.417268 13845 ts_tablet_manager.cc:1434] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:20:42.418406 13845 tablet_bootstrap.cc:492] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:42.414539 13847 ts_tablet_manager.cc:1434] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:42.419615 13847 tablet_bootstrap.cc:492] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:42.429984 13547 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:42.440987 13845 tablet_bootstrap.cc:654] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.449975 13847 tablet_bootstrap.cc:654] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.453357 13844 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.453914 13844 ts_tablet_manager.cc:1403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.213s	user 0.010s	sys 0.013s
I20260709 12:20:42.465278 13845 tablet_bootstrap.cc:492] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:42.465392 13886 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.465973 13845 ts_tablet_manager.cc:1403] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.048s	user 0.006s	sys 0.006s
I20260709 12:20:42.466238 13886 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.469755 13844 raft_consensus.cc:359] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.471370 13844 raft_consensus.cc:385] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.471658 13844 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.472504 13844 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.475718 13844 ts_tablet_manager.cc:1434] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.001s
I20260709 12:20:42.477519 13844 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:42.479018 13847 tablet_bootstrap.cc:492] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:42.479568 13847 ts_tablet_manager.cc:1403] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.006s
I20260709 12:20:42.484982 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.486491 13886 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:42.486831 13618 leader_election.cc:343] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.489708 13844 tablet_bootstrap.cc:654] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.487553 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:42.492408 13617 leader_election.cc:343] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.492914 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:20:42.493256 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:42.493707 13845 raft_consensus.cc:359] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.494345 13845 raft_consensus.cc:385] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.494707 13845 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.494822 13847 raft_consensus.cc:359] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.495190 13886 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.495631 13847 raft_consensus.cc:385] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.495491 13845 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.496307 13847 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.497217 13847 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.498653 13845 ts_tablet_manager.cc:1434] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:20:42.499574 13845 tablet_bootstrap.cc:492] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:42.507434 13844 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.508040 13844 ts_tablet_manager.cc:1403] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.007s
I20260709 12:20:42.509256 13845 tablet_bootstrap.cc:654] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.510871 13844 raft_consensus.cc:359] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.511593 13844 raft_consensus.cc:385] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.511861 13844 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.512814 13844 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.519793 13844 ts_tablet_manager.cc:1434] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:20:42.520975 13844 tablet_bootstrap.cc:492] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:42.520920 13847 ts_tablet_manager.cc:1434] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.003s
I20260709 12:20:42.522573 13847 tablet_bootstrap.cc:492] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:42.529651 13847 tablet_bootstrap.cc:654] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.532019 13845 tablet_bootstrap.cc:492] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:42.532634 13845 ts_tablet_manager.cc:1403] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.005s
I20260709 12:20:42.536041 13845 raft_consensus.cc:359] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.542003 13844 tablet_bootstrap.cc:654] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.537209 13845 raft_consensus.cc:385] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.547134 13845 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.547788 13845 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.550410 13845 ts_tablet_manager.cc:1434] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 12:20:42.550328 13886 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.551254 13886 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.551898 14033 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.552762 13845 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:42.552397 14033 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.554670 14033 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.554653 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.555864 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.557207 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:20:42.557420 13617 leader_election.cc:343] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.558245 13693 leader_election.cc:343] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:42.559015 13693 leader_election.cc:343] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.559175 13847 tablet_bootstrap.cc:492] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:42.559408 13693 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:42.560020 13847 ts_tablet_manager.cc:1403] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.006s
I20260709 12:20:42.560397 13845 tablet_bootstrap.cc:654] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.560441 13886 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.564359 13844 tablet_bootstrap.cc:492] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.565232 13844 ts_tablet_manager.cc:1403] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.007s
I20260709 12:20:42.567206 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.568828 14033 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.571214 14050 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.571755 14050 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.572134 13847 raft_consensus.cc:359] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.572938 13847 raft_consensus.cc:385] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.574434 13847 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.575317 13847 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.575845 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:42.573699 13844 raft_consensus.cc:359] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.576490 13844 raft_consensus.cc:385] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.576182 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.576913 13844 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
W20260709 12:20:42.577402 13618 leader_election.cc:343] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.578047 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:42.577771 13844 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.580554 13886 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.580054 14050 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:42.580247 13847 ts_tablet_manager.cc:1434] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:20:42.585088 13847 tablet_bootstrap.cc:492] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:42.582384 13844 ts_tablet_manager.cc:1434] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:20:42.577418 13727 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:42.589591 13844 tablet_bootstrap.cc:492] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:42.591013 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:42.592788 14050 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.594863 13847 tablet_bootstrap.cc:654] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.598798 13844 tablet_bootstrap.cc:654] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.639577 13847 tablet_bootstrap.cc:492] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:42.644819 13798 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.650614 13847 ts_tablet_manager.cc:1403] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.007s
I20260709 12:20:42.655921 13847 raft_consensus.cc:359] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.656946 13847 raft_consensus.cc:385] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.658316 13886 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.657301 13847 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.662083 13847 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.663861 13844 tablet_bootstrap.cc:492] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.664371 13844 ts_tablet_manager.cc:1403] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.075s	user 0.027s	sys 0.033s
I20260709 12:20:42.664913 13847 ts_tablet_manager.cc:1434] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:20:42.667284 13844 raft_consensus.cc:359] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.668375 13844 raft_consensus.cc:385] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.668708 13844 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.669509 13844 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.670037 14061 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:42.671615 13844 ts_tablet_manager.cc:1434] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:42.683368 13801 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.697986 13844 tablet_bootstrap.cc:492] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:42.699488 14061 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.707576 13847 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:42.711184 13886 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.711961 13886 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.716133 14061 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.716858 14061 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.714035 13886 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.720753 14050 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.723753 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:42.724524 13649 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:42.725857 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:42.727738 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.725564 14061 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:42.728487 14050 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:42.730835 14050 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.731246 14050 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.738060 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.739203 13844 tablet_bootstrap.cc:654] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.750160 13922 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:20:42.747097 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.764477 13958 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:42.765025 13617 leader_election.cc:343] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.766960 13845 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:42.767532 13845 ts_tablet_manager.cc:1403] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.215s	user 0.064s	sys 0.114s
W20260709 12:20:42.767580 13618 leader_election.cc:343] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.768160 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:42.768684 13844 tablet_bootstrap.cc:492] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.769145 13844 ts_tablet_manager.cc:1403] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.000s
I20260709 12:20:42.772672 13844 raft_consensus.cc:359] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.773432 13844 raft_consensus.cc:385] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.773535 13885 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:42.773854 13844 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.774662 13845 raft_consensus.cc:359] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.775617 13845 raft_consensus.cc:385] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.775938 13845 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.759981 13847 tablet_bootstrap.cc:654] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.762032 13886 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.776774 13845 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.762646 14050 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.774740 13844 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.781862 14050 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:42.784094 13845 ts_tablet_manager.cc:1434] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:20:42.785084 13845 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:42.786427 14033 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:42.787741 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:42.788486 13800 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:20:42.788515 13649 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.791200 13844 ts_tablet_manager.cc:1434] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:20:42.792479 13844 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:42.791308 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:42.805472 13922 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:20:42.816766 14033 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:42.817209 14033 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:42.818075 14033 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.820742 14033 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:42.834816 13844 tablet_bootstrap.cc:654] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.840147 13886 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.848294 13922 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:20:42.864320 13649 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:20:42.865254 13651 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:20:42.873334 13961 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:42.874912 13845 tablet_bootstrap.cc:654] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.877632 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:42.879024 14050 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:42.879617 14050 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:42.880528 14050 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.883023 13847 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:42.883543 13847 ts_tablet_manager.cc:1403] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.176s	user 0.020s	sys 0.047s
I20260709 12:20:42.886770 13847 raft_consensus.cc:359] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.890895 13847 raft_consensus.cc:385] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.891283 13847 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.892378 13844 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:42.892486 13847 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.893600 13844 ts_tablet_manager.cc:1403] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.101s	user 0.017s	sys 0.006s
I20260709 12:20:42.901616 13847 ts_tablet_manager.cc:1434] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:20:42.903015 13847 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:42.908896 13845 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:42.911229 13845 ts_tablet_manager.cc:1403] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.126s	user 0.017s	sys 0.000s
I20260709 12:20:42.915416 13845 raft_consensus.cc:359] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.917220 13845 raft_consensus.cc:385] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.917585 13845 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.921679 13845 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.924486 13845 ts_tablet_manager.cc:1434] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:20:42.927945 13922 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.928514 13922 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.927695 13547 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:42.931255 13922 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.932847 13845 tablet_bootstrap.cc:492] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:42.933614 13886 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.933547 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:42.934454 13651 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:42.934171 13886 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.935968 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:42.936623 13886 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.938697 13922 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.938874 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.939128 13922 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.939527 13922 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.939558 13798 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:42.942744 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.944541 13886 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.946784 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.954304 13725 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
W20260709 12:20:42.950006 13618 leader_election.cc:343] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.954480 13886 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.956287 13617 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:42.957321 13845 tablet_bootstrap.cc:654] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:42.957473 13885 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:42.957867 13885 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:42.960146 13885 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.961453 13844 raft_consensus.cc:359] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.962246 13844 raft_consensus.cc:385] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:42.962559 13844 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:42.964159 13844 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:42.967486 13844 ts_tablet_manager.cc:1434] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.000s
I20260709 12:20:42.969718 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:42.970338 13886 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:42.977232 13847 tablet_bootstrap.cc:654] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:42.973136 13618 leader_election.cc:343] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:42.984992 14050 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:42.989197 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:42.995182 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:42.990178 14050 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.002943 13961 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.003315 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:43.003546 13961 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.005288 13958 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.005662 13958 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.010924 13844 tablet_bootstrap.cc:492] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.006108 13958 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:42.997709 13922 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.005712 13961 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.025003 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.025774 13649 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:43.026937 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.027945 14098 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.028451 14098 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.028894 14098 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.029644 13922 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.033113 13961 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.036837 13961 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.038538 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:43.039047 13961 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.039247 13800 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.041157 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:43.041745 13649 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.045374 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.055090 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.055913 13798 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
W20260709 12:20:43.058641 13693 leader_election.cc:343] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.061205 13649 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:43.045455 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.063131 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:43.065142 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:43.065143 13885 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.067417 13961 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.068130 13961 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.070152 13800 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:43.078896 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.068619 13961 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.076639 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.072779 14050 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.089968 13844 tablet_bootstrap.cc:654] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.099143 14033 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.099771 14033 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:43.070562 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:43.093092 13958 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.100728 14033 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.102906 13958 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:43.103762 13618 leader_election.cc:343] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.108458 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:43.128922 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:43.073247 13885 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.110924 13847 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:43.130015 13847 ts_tablet_manager.cc:1403] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.227s	user 0.010s	sys 0.007s
I20260709 12:20:43.130558 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:43.131258 13725 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.133023 13847 raft_consensus.cc:359] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.133735 13847 raft_consensus.cc:385] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.133996 13847 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.113426 13845 tablet_bootstrap.cc:492] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:43.113583 13863 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.113804 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.134800 13845 ts_tablet_manager.cc:1403] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.210s	user 0.018s	sys 0.017s
I20260709 12:20:43.113776 14098 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.120303 13961 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.135272 13652 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:43.129676 13727 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.135125 13863 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.120905 13844 tablet_bootstrap.cc:492] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.137759 13844 ts_tablet_manager.cc:1403] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.006s
I20260709 12:20:43.138586 13863 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.143568 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:43.144199 13652 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.144424 14098 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.144737 14033 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.145020 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:43.145233 13722 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:43.145269 14033 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.147614 14033 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:43.148387 13693 leader_election.cc:343] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.148792 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:43.149405 13649 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.148420 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.150692 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.151957 13648 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:43.153093 13845 raft_consensus.cc:359] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.153855 13845 raft_consensus.cc:385] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:20:43.142411 13618 leader_election.cc:343] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.155862 13548 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.137818 13961 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:43.159375 13617 leader_election.cc:343] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.154502 13847 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.159528 13649 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:43.160458 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.161262 13961 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.161470 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.161720 13961 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.162281 14033 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.162853 14033 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:43.163199 13847 ts_tablet_manager.cc:1434] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:20:43.163691 13845 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.163748 14033 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.164558 13845 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.167047 13847 tablet_bootstrap.cc:492] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:43.167804 13961 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.139535 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:43.174319 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.174862 14053 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.175396 14053 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.175959 14053 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.181042 13727 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:43.182236 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.182600 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:43.183454 13617 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:43.183835 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:43.184568 13693 leader_election.cc:343] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.184810 13885 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.185612 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:43.185973 13885 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:43.186903 13863 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.187425 13863 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:43.187247 13885 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.188618 13847 tablet_bootstrap.cc:654] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.188328 13863 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:43.184765 13618 leader_election.cc:343] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.192417 13618 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:43.193265 14101 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.193846 13961 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.199857 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:43.200521 13648 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.201817 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:43.205281 13693 leader_election.cc:343] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.206247 13961 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.175341 13844 raft_consensus.cc:359] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.207491 13844 raft_consensus.cc:385] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.207757 13844 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.210584 14053 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.211863 13800 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:43.214284 13845 ts_tablet_manager.cc:1434] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.079s	user 0.004s	sys 0.002s
I20260709 12:20:43.215421 13845 tablet_bootstrap.cc:492] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:43.172719 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:43.221077 13648 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:43.220667 13844 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.230580 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:43.223402 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:43.232909 14033 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.233444 14033 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:43.227356 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:43.234323 14033 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.235185 13652 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:43.236430 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:43.237295 14050 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.240546 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:43.241667 14033 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:43.242206 14033 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:43.243104 14033 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.247381 14053 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.257627 13844 ts_tablet_manager.cc:1434] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.000s
I20260709 12:20:43.257071 13548 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.260293 13844 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.260566 13845 tablet_bootstrap.cc:654] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.260627 13548 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.262626 13547 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.266109 13547 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.268169 13847 tablet_bootstrap.cc:492] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:43.267644 13844 tablet_bootstrap.cc:654] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.271067 14061 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.271637 14061 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.274008 14061 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.275635 13844 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.276150 13844 ts_tablet_manager.cc:1403] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.003s
I20260709 12:20:43.276420 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.276150 13847 ts_tablet_manager.cc:1403] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.109s	user 0.025s	sys 0.039s
I20260709 12:20:43.281255 13847 raft_consensus.cc:359] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.282011 13847 raft_consensus.cc:385] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.282305 13847 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.282310 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.283280 13725 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:43.283032 13847 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.283907 13844 raft_consensus.cc:359] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.284554 13844 raft_consensus.cc:385] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.284803 13844 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.284935 13766 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:43.285522 13844 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.286435 14053 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.288332 13844 ts_tablet_manager.cc:1434] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:20:43.289463 13844 tablet_bootstrap.cc:492] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.294831 14053 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.295974 13847 ts_tablet_manager.cc:1434] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 12:20:43.297436 13847 tablet_bootstrap.cc:492] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:43.296126 13844 tablet_bootstrap.cc:654] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.299608 13845 tablet_bootstrap.cc:492] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:43.300140 13845 ts_tablet_manager.cc:1403] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.085s	user 0.011s	sys 0.000s
I20260709 12:20:43.302872 13845 raft_consensus.cc:359] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.303679 13845 raft_consensus.cc:385] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.303982 13845 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.304824 13845 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.308595 13847 tablet_bootstrap.cc:654] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.320276 13845 ts_tablet_manager.cc:1434] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:20:43.320919 13844 tablet_bootstrap.cc:492] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.321375 13844 ts_tablet_manager.cc:1403] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.005s
I20260709 12:20:43.321509 13845 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:43.323661 13547 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:43.324319 13844 raft_consensus.cc:359] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.325181 13844 raft_consensus.cc:385] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.325479 13844 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.326331 13844 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.327937 13844 ts_tablet_manager.cc:1434] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:20:43.328819 13844 tablet_bootstrap.cc:492] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.347916 13847 tablet_bootstrap.cc:492] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:43.348495 13847 ts_tablet_manager.cc:1403] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.005s
I20260709 12:20:43.349735 13844 tablet_bootstrap.cc:654] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.351415 13847 raft_consensus.cc:359] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.352133 13847 raft_consensus.cc:385] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.352404 13847 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.353163 13847 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.355363 13847 ts_tablet_manager.cc:1434] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 12:20:43.357666 13845 tablet_bootstrap.cc:654] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.358453 13847 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:43.363729 13844 tablet_bootstrap.cc:492] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.364306 13844 ts_tablet_manager.cc:1403] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.010s
I20260709 12:20:43.366762 13847 tablet_bootstrap.cc:654] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.367823 13844 raft_consensus.cc:359] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.368526 13844 raft_consensus.cc:385] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.368860 13844 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.369676 13844 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.372105 13844 ts_tablet_manager.cc:1434] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:43.381192 13845 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:43.381764 13845 ts_tablet_manager.cc:1403] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.012s
I20260709 12:20:43.384677 13845 raft_consensus.cc:359] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.385396 13845 raft_consensus.cc:385] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.385665 13845 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.387049 13844 tablet_bootstrap.cc:492] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.388126 13845 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.390522 13845 ts_tablet_manager.cc:1434] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 12:20:43.391472 13845 tablet_bootstrap.cc:492] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:43.398998 13844 tablet_bootstrap.cc:654] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.404942 13845 tablet_bootstrap.cc:654] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.405228 13725 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:20:43.405575 13725 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 0
I20260709 12:20:43.405869 13725 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:20:43.406169 13725 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:43.409250 14050 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.412498 13648 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:43.414856 14050 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.417066 13845 tablet_bootstrap.cc:492] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:43.417591 13845 ts_tablet_manager.cc:1403] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.026s	user 0.005s	sys 0.011s
I20260709 12:20:43.420447 13845 raft_consensus.cc:359] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.422405 13845 raft_consensus.cc:385] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.422701 13845 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.423604 13845 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.425268 13847 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:43.425958 13847 ts_tablet_manager.cc:1403] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.068s	user 0.020s	sys 0.028s
I20260709 12:20:43.426394 13845 ts_tablet_manager.cc:1434] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.006s
I20260709 12:20:43.427430 13845 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:43.430401 13847 raft_consensus.cc:359] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.431864 13847 raft_consensus.cc:385] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.432240 13847 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.438635 13847 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.445533 13847 ts_tablet_manager.cc:1434] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:20:43.446923 13847 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:43.454075 13845 tablet_bootstrap.cc:654] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.455016 13844 tablet_bootstrap.cc:492] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.455648 13844 ts_tablet_manager.cc:1403] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.018s
I20260709 12:20:43.459214 13844 raft_consensus.cc:359] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.460343 13847 tablet_bootstrap.cc:654] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.460228 13844 raft_consensus.cc:385] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.460739 13844 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.461867 13844 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.465286 13844 ts_tablet_manager.cc:1434] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:20:43.466675 13844 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.472775 13847 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:43.473348 13847 ts_tablet_manager.cc:1403] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.003s
I20260709 12:20:43.476724 13847 raft_consensus.cc:359] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.477169 13845 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:43.477727 13845 ts_tablet_manager.cc:1403] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.010s
I20260709 12:20:43.478518 13844 tablet_bootstrap.cc:654] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.480501 13845 raft_consensus.cc:359] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.481228 13845 raft_consensus.cc:385] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.481485 13845 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.482321 13845 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.484576 13845 ts_tablet_manager.cc:1434] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:20:43.489797 13847 raft_consensus.cc:385] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.490204 13847 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.491710 13845 tablet_bootstrap.cc:492] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:43.491740 14033 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.492468 14033 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.498822 13845 tablet_bootstrap.cc:654] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.498489 13847 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.500574 13847 ts_tablet_manager.cc:1434] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:20:43.501660 13847 tablet_bootstrap.cc:492] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:43.508292 13847 tablet_bootstrap.cc:654] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.528716 14033 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.538087 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.539405 13652 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:43.542532 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.544868 13961 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.545648 13961 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.548826 13961 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.549784 14101 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.550606 14101 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.553045 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.554540 13961 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.556758 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.557090 13961 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.557864 13961 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:43.560436 13618 leader_election.cc:343] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.564800 14101 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.584301 13863 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.585150 14050 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.585075 13863 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.585750 14050 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.588651 14050 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.589219 13844 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.589775 13844 ts_tablet_manager.cc:1403] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.123s	user 0.010s	sys 0.003s
I20260709 12:20:43.590692 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.592826 13844 raft_consensus.cc:359] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.593528 13844 raft_consensus.cc:385] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:43.593780 13844 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:43.594587 13844 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.596186 13844 ts_tablet_manager.cc:1434] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.005s
I20260709 12:20:43.596374 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.597532 13722 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:43.601585 13844 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:43.602846 13885 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.603341 13885 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.605679 13863 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.606865 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:43.615847 13618 leader_election.cc:343] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.618387 13844 tablet_bootstrap.cc:654] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:43.618755 14050 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.623060 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:43.625533 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.619328 14050 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.627805 13724 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:43.628240 13885 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.626575 14053 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.630200 14050 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.632113 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.622816 14061 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.640424 14061 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:43.640785 13618 leader_election.cc:343] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:43.632522 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.645527 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:43.645758 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.647517 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.648964 14057 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.649365 14057 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.653668 14057 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.649718 13725 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:43.657888 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:43.663103 14050 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:43.663527 14050 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:43.663841 14050 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:43.671912 14050 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.682530 14101 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:20:43.683985 14101 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.687487 14101 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.657829 14061 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.688764 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.695163 13800 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:20:43.705157 13618 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:43.706880 13885 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:43.707442 13885 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:20:43.707890 13885 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:43.710731 13863 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:43.728721 13885 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.746938 14053 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:43.748818 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.941138 13885 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:43.953645 13844 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:43.954625 13844 ts_tablet_manager.cc:1403] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.353s	user 0.013s	sys 0.033s
I20260709 12:20:43.757364 14050 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:43.753067 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:43.798928 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.747064 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:43.758193 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:43.975970 14050 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:43.977594 13725 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:44.006666 13961 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:43.977217 13652 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:43.975629 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:43.977444 13863 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.010695 14098 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.020850 13844 raft_consensus.cc:359] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.050036 13844 raft_consensus.cc:385] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.050613 13844 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.010159 14057 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:44.028138 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 264806us
I20260709 12:20:44.051571 13844 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:44.035310 13847 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.098s	sys 0.169s
W20260709 12:20:44.062223 13847 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.098s	sys 0.170s
I20260709 12:20:44.105613 13844 ts_tablet_manager.cc:1434] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.150s	user 0.009s	sys 0.000s
I20260709 12:20:44.120925 13844 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:44.064337 13722 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:44.049467 13723 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.029897 14050 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:44.057116 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 174480us
I20260709 12:20:44.060484 13863 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.137104 13795 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:44.063087 14098 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.138625 14033 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.159708 13847 tablet_bootstrap.cc:492] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:44.160439 13847 ts_tablet_manager.cc:1403] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.659s	user 0.109s	sys 0.171s
W20260709 12:20:44.165096 13693 leader_election.cc:343] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:44.180672 13845 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.095s	sys 0.168s
W20260709 12:20:44.181383 13845 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.095s	sys 0.168s
I20260709 12:20:44.150530 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:44.155194 14033 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.196357 13863 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.200379 13885 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:44.209288 13847 raft_consensus.cc:359] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.185799 13961 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.210227 13847 raft_consensus.cc:385] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.219301 13847 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.221310 13845 tablet_bootstrap.cc:492] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:44.221722 13845 ts_tablet_manager.cc:1403] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.730s	user 0.101s	sys 0.170s
I20260709 12:20:44.226526 13845 raft_consensus.cc:359] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.227372 13845 raft_consensus.cc:385] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.230993 13845 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.231639 13845 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.233320 13845 ts_tablet_manager.cc:1434] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:20:44.234985 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:44.214668 14033 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.201288 14135 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.219969 14057 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:20:44.223579 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80065.4us
I20260709 12:20:44.247494 14146 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:20:44.249325 13617 leader_election.cc:343] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.236537 14139 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.255357 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:44.257316 14139 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.249948 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:44.244390 14098 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.265267 13863 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.250597 14135 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.269227 14139 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.267540 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:44.276341 13961 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:20:44.287762 13886 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.296487 13886 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.296643 13844 tablet_bootstrap.cc:654] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.296967 13886 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.275112 14135 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.311455 13845 tablet_bootstrap.cc:492] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:44.285980 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:44.297772 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:44.288739 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:44.383760 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105068us
I20260709 12:20:44.299221 13863 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.076s
I20260709 12:20:44.316112 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:44.373086 13961 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:44.408265 13693 leader_election.cc:343] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.409601 13886 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.411252 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:44.412113 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:44.412575 13886 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.372304 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:44.422143 13618 leader_election.cc:343] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.431082 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:44.417985 13722 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:44.408504 13961 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.449880 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:44.448961 13723 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:44.402866 13647 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
W20260709 12:20:44.445129 13693 leader_election.cc:343] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:44.453703 13618 leader_election.cc:343] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.456959 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:44.463598 14139 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.440867 13721 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
W20260709 12:20:44.467810 13693 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:44.460155 13650 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:44.468608 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:44.473634 14098 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:44.467242 13961 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.463680 13648 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.478773 14145 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:44.481936 13725 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:44.483819 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:44.477761 14150 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:44.484359 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:44.488623 14150 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:44.492751 13723 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.512930 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:44.513959 13643 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:20:44.514197 13863 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:44.519563 13845 tablet_bootstrap.cc:654] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.522528 13844 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:44.490935 13961 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:20:44.501245 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:44.517724 14150 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.541826 13793 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:20:44.547960 13847 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.505380 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:44.553537 13844 ts_tablet_manager.cc:1403] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.433s	user 0.033s	sys 0.032s
I20260709 12:20:44.570393 13847 ts_tablet_manager.cc:1434] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.410s	user 0.010s	sys 0.006s
I20260709 12:20:44.573971 13847 tablet_bootstrap.cc:492] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
W20260709 12:20:44.568238 13693 leader_election.cc:343] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:44.583042 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:44.598373 14132 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.598992 14132 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.599100 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:44.603031 13652 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.568486 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:44.606027 13646 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.568807 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:44.599390 14132 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.615262 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:44.616299 13794 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:20:44.570506 14135 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.624497 14135 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.632788 13647 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:44.635100 14149 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.656507 13847 tablet_bootstrap.cc:654] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.664512 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:44.668382 13844 raft_consensus.cc:359] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.668983 13844 raft_consensus.cc:385] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.669214 13844 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.655652 13723 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:44.671010 13844 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.677302 14141 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:20:44.677810 14141 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.678872 13844 ts_tablet_manager.cc:1434] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.111s	user 0.007s	sys 0.001s
I20260709 12:20:44.679450 13618 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:44.680038 13844 tablet_bootstrap.cc:492] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:44.680262 13863 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:44.680719 13863 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:44.681607 13863 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.683458 13845 tablet_bootstrap.cc:492] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:44.684089 13845 ts_tablet_manager.cc:1403] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.373s	user 0.011s	sys 0.014s
I20260709 12:20:44.690058 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:44.678169 14141 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:44.635411 14050 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:44.656682 13648 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:44.667958 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:44.697257 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:44.676494 14139 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:44.708676 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:44.701568 13725 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:44.704852 14139 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:44.710398 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:44.722496 14143 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:20:44.684988 14149 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.712949 14139 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:44.733932 13652 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:44.731797 14139 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:44.753772 13618 leader_election.cc:343] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:44.787482 14157 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:20:44.786103 13844 tablet_bootstrap.cc:654] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:44.810564 14139 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.815423 14139 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.827289 13845 raft_consensus.cc:359] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.828305 13845 raft_consensus.cc:385] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.828652 13845 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.836887 13845 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.818594 13646 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:44.865262 13844 tablet_bootstrap.cc:492] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:44.866812 14149 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:20:44.867666 14158 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:20:44.866048 13844 ts_tablet_manager.cc:1403] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.186s	user 0.011s	sys 0.005s
I20260709 12:20:44.841060 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:44.904397 14163 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:20:44.934691 13847 tablet_bootstrap.cc:492] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:44.947469 13847 ts_tablet_manager.cc:1403] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.374s	user 0.041s	sys 0.039s
I20260709 12:20:44.850844 14132 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.882376 13845 ts_tablet_manager.cc:1434] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.198s	user 0.010s	sys 0.002s
I20260709 12:20:44.950350 13844 raft_consensus.cc:359] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.919278 14141 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.951216 13847 raft_consensus.cc:359] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:44.952692 13845 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:44.953006 13847 raft_consensus.cc:385] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:44.953446 13847 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:44.953868 14141 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:44.954183 14132 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:44.902156 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 205011us
W20260709 12:20:45.000833 13693 leader_election.cc:343] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.001830 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.005376 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:45.008352 13647 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:45.010833 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.014160 13550 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.029175 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:45.032749 14168 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.037209 14101 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:44.934456 13725 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:44.951308 13844 raft_consensus.cc:385] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.041498 13844 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.048112 14147 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:44.954411 13847 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.051666 14141 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.051800 13767 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.052698 14141 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:45.034628 14168 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.038244 14101 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.066427 14137 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:20:45.068545 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46690.9us
I20260709 12:20:45.067335 14137 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.074092 14131 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.058475 13844 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.077337 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.078449 14132 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.079336 14132 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.080415 14101 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.083086 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.083364 14132 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.088054 13799 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.056293 14141 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.094017 14137 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:45.100315 13618 leader_election.cc:343] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.106817 13844 ts_tablet_manager.cc:1434] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.160s	user 0.005s	sys 0.004s
I20260709 12:20:45.107422 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:45.107594 14053 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:45.110632 14168 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.075554 14131 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:45.127544 13844 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:45.134553 14131 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.138688 13847 ts_tablet_manager.cc:1434] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.191s	user 0.007s	sys 0.004s
I20260709 12:20:45.141091 13847 tablet_bootstrap.cc:492] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
W20260709 12:20:45.144583 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61934.4us
I20260709 12:20:45.146073 14061 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.150389 13648 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:45.151124 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.158475 14140 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.159230 14140 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:45.162855 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.163221 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.163640 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:45.164793 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.168882 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.174575 14168 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:20:45.176523 14168 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.170630 14050 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.160456 14140 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.248680 14141 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:20:45.249544 14141 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.257167 13847 tablet_bootstrap.cc:654] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.242086 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:45.268196 14147 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:20:45.278813 14147 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.282250 14132 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:45.282444 14147 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.282971 14132 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:20:45.283481 14132 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:20:45.278218 13618 leader_election.cc:343] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.311368 13845 tablet_bootstrap.cc:654] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:45.317754 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 149034us
I20260709 12:20:45.321482 14141 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.322095 14050 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.325862 14101 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.326730 14101 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.333452 14101 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.347240 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:45.347249 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:45.348809 13649 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:45.361622 14168 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.383982 14137 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:20:45.385073 13844 tablet_bootstrap.cc:654] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.384711 14137 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.385084 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:45.387569 14137 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.376550 14132 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.399040 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:45.400197 13726 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:45.403262 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:45.404772 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.405149 13723 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:20:45.405277 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.416705 13845 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:45.417665 13845 ts_tablet_manager.cc:1403] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.466s	user 0.077s	sys 0.096s
I20260709 12:20:45.427747 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.431538 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:45.438277 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.439244 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:45.441011 14180 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.120s	user 0.002s	sys 0.000s
I20260709 12:20:45.440898 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.449452 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:45.450790 14137 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:45.451242 13797 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.451506 14137 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:45.469105 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:45.482510 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:45.483160 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.483536 13652 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
W20260709 12:20:45.482547 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44462.8us
I20260709 12:20:45.470391 13845 raft_consensus.cc:359] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.496380 13845 raft_consensus.cc:385] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.497773 13845 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.474685 14132 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:45.504562 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64668.2us
I20260709 12:20:45.504741 14141 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:45.457685 14137 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.494571 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:45.511516 14149 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.492262 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:45.512215 14149 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:45.567341 13693 leader_election.cc:343] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.517354 13845 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.547408 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:45.610597 13845 ts_tablet_manager.cc:1434] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.193s	user 0.013s	sys 0.002s
I20260709 12:20:45.611014 13794 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.613193 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.614228 14149 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:20:45.615473 13618 leader_election.cc:343] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.569329 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.528801 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:45.625042 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55914.1us
I20260709 12:20:45.625221 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.625480 14133 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.626750 14168 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.629153 13847 tablet_bootstrap.cc:492] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:45.629994 13847 ts_tablet_manager.cc:1403] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.490s	user 0.019s	sys 0.019s
I20260709 12:20:45.614775 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.632318 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.633491 13845 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:45.619700 14149 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:20:45.643273 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:45.654237 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.667266 13797 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:20:45.670797 13795 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:20:45.691753 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47949.9us
I20260709 12:20:45.692394 13847 raft_consensus.cc:359] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.693718 14101 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:45.693763 13847 raft_consensus.cc:385] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.694582 13847 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.694603 14101 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.695538 13847 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.703306 13794 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:20:45.704269 14101 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.703696 13548 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.710078 13845 tablet_bootstrap.cc:654] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.712271 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:20:45.715552 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51513us
I20260709 12:20:45.715955 13847 ts_tablet_manager.cc:1434] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.001s
I20260709 12:20:45.714576 14140 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:45.718178 14140 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:45.716327 14132 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:20:45.718899 13844 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:45.719451 13844 ts_tablet_manager.cc:1403] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.592s	user 0.021s	sys 0.026s
I20260709 12:20:45.719187 14140 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.723235 13844 raft_consensus.cc:359] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.719457 14132 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:20:45.724102 14132 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:45.724372 13847 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:45.725332 14132 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.726601 14132 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:20:45.727802 13844 raft_consensus.cc:385] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:45.728072 13844 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:45.730929 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.732201 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:45.732622 14168 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:20:45.739442 13618 leader_election.cc:343] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.739620 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:45.739939 13844 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:45.747581 13617 leader_election.cc:343] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.750070 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.751401 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.752312 13720 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:45.765583 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.771165 14137 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:45.745010 14168 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.769579 14149 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:20:45.774994 14168 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.778033 14168 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.779066 13550 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:45.779871 14133 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:45.780619 14133 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:45.781035 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.781855 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.791178 13794 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:20:45.781002 14133 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.797777 13844 ts_tablet_manager.cc:1434] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.001s
I20260709 12:20:45.820479 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:45.795329 14189 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:45.805863 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:45.808831 14140 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:45.831314 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:45.833510 13845 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:45.836022 13845 ts_tablet_manager.cc:1403] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.224s	user 0.017s	sys 0.016s
I20260709 12:20:45.833707 13651 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:20:45.821964 14137 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:45.838726 13844 tablet_bootstrap.cc:492] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:45.846771 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:20:45.864646 13693 leader_election.cc:343] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:45.865682 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.840014 13727 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:45.866845 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:45.859560 14205 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:45.868913 14168 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.871075 14168 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:45.875254 14205 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:20:45.846027 14133 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:45.882273 14208 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:20:45.889436 14197 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:20:45.919028 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46520.3us
W20260709 12:20:45.918184 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45736.8us
I20260709 12:20:45.918740 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:20:45.930348 13618 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:45.932651 13847 tablet_bootstrap.cc:654] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.928838 14147 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:20:45.930403 14198 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:20:45.934548 14147 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:20:45.934995 14147 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:45.934773 14128 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:20:45.917941 13548 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:45.934577 14198 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:45.946661 14198 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.935899 14147 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:45.967432 14147 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:45.970561 13844 tablet_bootstrap.cc:654] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:45.958565 14150 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:45.970582 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43797.3us
I20260709 12:20:45.974609 14150 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.006799 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:46.000995 14210 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:20:46.004478 13845 raft_consensus.cc:359] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.011675 13845 raft_consensus.cc:385] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.012141 13845 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
W20260709 12:20:46.017302 13693 leader_election.cc:343] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:46.017866 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 196017us
I20260709 12:20:46.010834 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:46.002485 14141 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:20:46.015136 14189 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.024845 13845 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.016597 14133 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:46.027161 13649 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.027484 13845 ts_tablet_manager.cc:1434] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.191s	user 0.006s	sys 0.002s
I20260709 12:20:46.014705 13796 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.032176 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:46.033849 14133 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.036135 13845 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:46.036948 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:46.037580 13725 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.041224 13617 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:20:46.046811 13618 leader_election.cc:343] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.056047 13649 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:46.057380 13725 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:46.058588 14137 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.059126 14137 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.059568 14137 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.099964 14137 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.112730 14137 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:46.122989 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53914.8us
I20260709 12:20:46.124178 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.158826 14202 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.159614 14202 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:46.160964 14202 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.182593 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:46.187633 14150 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:46.195971 13847 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:46.196542 13847 ts_tablet_manager.cc:1403] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.472s	user 0.011s	sys 0.021s
W20260709 12:20:46.197399 13618 leader_election.cc:343] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.201325 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.202209 14137 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.202847 14137 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:46.203781 14137 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.207973 13847 raft_consensus.cc:359] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.208762 13847 raft_consensus.cc:385] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.209033 13847 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.209797 13847 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.211760 13847 ts_tablet_manager.cc:1434] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 12:20:46.212922 13847 tablet_bootstrap.cc:492] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:46.224953 13845 tablet_bootstrap.cc:654] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.258618 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:46.305928 13844 tablet_bootstrap.cc:492] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:46.306540 13844 ts_tablet_manager.cc:1403] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.468s	user 0.014s	sys 0.020s
I20260709 12:20:46.314178 13844 raft_consensus.cc:359] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.317570 13844 raft_consensus.cc:385] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.317930 13844 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.318727 13844 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.343220 14147 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.382558 13844 ts_tablet_manager.cc:1434] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.003s
I20260709 12:20:46.385779 13844 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:46.385918 13847 tablet_bootstrap.cc:654] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:46.387194 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 142085us
I20260709 12:20:46.404763 14132 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.410629 13845 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:46.411227 13845 ts_tablet_manager.cc:1403] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.381s	user 0.015s	sys 0.014s
I20260709 12:20:46.411423 13844 tablet_bootstrap.cc:654] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.414455 13845 raft_consensus.cc:359] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.415340 13845 raft_consensus.cc:385] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.417815 13845 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.418598 13845 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.420341 13845 ts_tablet_manager.cc:1434] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:20:46.421478 13845 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:46.428072 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:46.428822 13724 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.429287 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:46.431146 13796 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:46.443841 14204 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.444587 14204 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.451172 14204 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:46.445683 14227 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.458096 14133 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.462987 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:46.464766 13845 tablet_bootstrap.cc:654] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:46.464107 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:46.474151 14228 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.474856 14228 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.464169 14050 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.481792 13847 tablet_bootstrap.cc:492] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:46.482367 13847 ts_tablet_manager.cc:1403] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.270s	user 0.026s	sys 0.012s
I20260709 12:20:46.459947 13724 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:46.496084 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:46.497735 13693 leader_election.cc:343] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.500068 13725 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:20:46.503186 13847 raft_consensus.cc:359] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.503451 13617 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.504166 13847 raft_consensus.cc:385] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.504475 13847 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.505841 13847 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.508805 13550 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.502532 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:46.513125 13793 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:20:46.514431 14203 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:20:46.516654 13847 ts_tablet_manager.cc:1434] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 12:20:46.510533 14133 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.519028 14133 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:46.521765 14133 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.523329 14228 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:46.514066 13648 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:46.524621 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:46.537851 13847 tablet_bootstrap.cc:492] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
W20260709 12:20:46.541109 13618 leader_election.cc:343] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.534860 14228 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.551862 14228 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.596796 14189 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.598410 13844 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:46.598963 13844 ts_tablet_manager.cc:1403] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.213s	user 0.038s	sys 0.021s
I20260709 12:20:46.600780 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.607919 14198 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.597487 14189 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.608345 14198 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.612720 14204 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:46.614429 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:46.615321 14150 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.615629 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:46.615866 14189 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:46.616338 13727 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:46.616974 14198 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.615913 14150 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:46.619582 13617 leader_election.cc:343] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.619980 13617 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:46.621353 14227 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.622897 13845 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:46.623422 13845 ts_tablet_manager.cc:1403] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.202s	user 0.020s	sys 0.032s
I20260709 12:20:46.623991 13617 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:46.620611 13547 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.627414 13547 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.630609 13547 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:46.632182 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:46.627488 14137 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.635672 14137 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.636072 14137 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.641413 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:46.643513 13618 leader_election.cc:343] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.645632 13844 raft_consensus.cc:359] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.646582 13844 raft_consensus.cc:385] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.646895 13844 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.647642 13844 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.638648 14228 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:46.650944 13847 tablet_bootstrap.cc:654] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.652984 13649 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:20:46.653462 13649 pending_rounds.cc:85] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 1
I20260709 12:20:46.653743 13649 pending_rounds.cc:107] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:20:46.654076 13649 raft_consensus.cc:2889] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:20:46.647657 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:46.655372 14197 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:20:46.662513 14197 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:46.665510 14150 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:46.667585 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:46.668617 14150 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.671494 13844 ts_tablet_manager.cc:1434] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.002s
I20260709 12:20:46.672477 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:46.672914 13844 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:46.681738 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:46.690833 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:46.692579 14189 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:46.694994 14137 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.697989 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:46.698729 13719 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.704007 13799 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.715111 13845 raft_consensus.cc:359] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.716871 13845 raft_consensus.cc:385] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.717296 13845 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.718086 13845 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.721361 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:46.719518 14137 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:46.692971 14198 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.727960 14133 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:46.731269 13618 leader_election.cc:343] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.739876 13719 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:46.741734 13845 ts_tablet_manager.cc:1434] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.000s
I20260709 12:20:46.741987 13617 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.744570 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:46.745138 14137 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.745211 13845 tablet_bootstrap.cc:492] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:46.745261 13649 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.747509 14137 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:46.751196 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:46.766497 14137 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.743745 14198 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:46.772135 13693 leader_election.cc:343] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.772743 14141 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:46.790190 13844 tablet_bootstrap.cc:654] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.795420 13847 tablet_bootstrap.cc:492] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:46.795886 13796 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.796068 13847 ts_tablet_manager.cc:1403] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.263s	user 0.015s	sys 0.010s
I20260709 12:20:46.803740 13649 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:46.811143 13847 raft_consensus.cc:359] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.812913 13847 raft_consensus.cc:385] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:46.813179 13847 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:46.813707 13847 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.812510 14101 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:46.814970 14101 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.819020 14101 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:46.827757 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:46.831259 13847 ts_tablet_manager.cc:1434] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
W20260709 12:20:46.843149 13618 leader_election.cc:343] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:46.852316 13794 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.864838 13845 tablet_bootstrap.cc:654] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.866490 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.877611 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:46.878461 13725 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:46.879454 14197 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:46.880787 14197 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:46.884207 13847 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:46.888048 13617 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:46.894743 14224 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:46.913196 14224 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:46.913622 14224 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:46.915192 14197 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:46.943011 14133 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:46.959574 13847 tablet_bootstrap.cc:654] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:46.994728 13844 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:46.995643 13844 ts_tablet_manager.cc:1403] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.323s	user 0.014s	sys 0.021s
I20260709 12:20:47.122133 13844 raft_consensus.cc:359] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.125555 13844 raft_consensus.cc:385] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.125826 13844 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.132706 13845 tablet_bootstrap.cc:492] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:47.133322 13845 ts_tablet_manager.cc:1403] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.388s	user 0.010s	sys 0.027s
I20260709 12:20:47.137537 13844 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.160065 13847 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.160669 13847 ts_tablet_manager.cc:1403] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.277s	user 0.040s	sys 0.043s
W20260709 12:20:47.312376 13618 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:47.332108 13847 raft_consensus.cc:359] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.332855 13847 raft_consensus.cc:385] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.333101 13847 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.334048 13847 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.346414 14228 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.347616 13847 ts_tablet_manager.cc:1434] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.187s	user 0.010s	sys 0.002s
I20260709 12:20:47.348975 13847 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:47.361855 13847 tablet_bootstrap.cc:654] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.368247 13847 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.368760 13847 ts_tablet_manager.cc:1403] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.002s
I20260709 12:20:47.372044 13847 raft_consensus.cc:359] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.372933 13847 raft_consensus.cc:385] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.373224 13847 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.373996 13847 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.376614 13847 ts_tablet_manager.cc:1434] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:20:47.377908 13847 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:47.384883 13847 tablet_bootstrap.cc:654] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.391099 13847 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.391599 13847 ts_tablet_manager.cc:1403] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:20:47.393072 13844 ts_tablet_manager.cc:1434] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.397s	user 0.015s	sys 0.000s
I20260709 12:20:47.394181 13844 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:47.394964 13847 raft_consensus.cc:359] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.395700 13847 raft_consensus.cc:385] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.395967 13847 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.396625 13847 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.398762 13847 ts_tablet_manager.cc:1434] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:20:47.399816 13847 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:47.406919 13847 tablet_bootstrap.cc:654] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.408481 13844 tablet_bootstrap.cc:654] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.414856 13845 raft_consensus.cc:359] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.415706 13845 raft_consensus.cc:385] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.415992 13845 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.416666 13845 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.418447 13845 ts_tablet_manager.cc:1434] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.285s	user 0.003s	sys 0.012s
I20260709 12:20:47.164229 14224 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.425263 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:47.426183 13845 tablet_bootstrap.cc:492] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:47.423633 14224 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:47.429260 13618 leader_election.cc:343] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.431391 13844 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:47.432106 13844 ts_tablet_manager.cc:1403] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.002s
I20260709 12:20:47.446777 13844 raft_consensus.cc:359] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.447918 13844 raft_consensus.cc:385] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.448273 13844 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.449229 13844 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.451678 13845 tablet_bootstrap.cc:654] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.461163 13844 ts_tablet_manager.cc:1434] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.029s	user 0.011s	sys 0.000s
I20260709 12:20:47.466995 13844 tablet_bootstrap.cc:492] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
W20260709 12:20:47.467947 13618 consensus_peers.cc:597] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:47.484926 13550 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.489720 13844 tablet_bootstrap.cc:654] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.509115 13844 tablet_bootstrap.cc:492] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:47.509845 13844 ts_tablet_manager.cc:1403] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.000s
I20260709 12:20:47.517673 13844 raft_consensus.cc:359] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.524385 13844 raft_consensus.cc:385] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.524879 13844 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.525955 13844 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.533605 13844 ts_tablet_manager.cc:1434] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.023s	user 0.012s	sys 0.000s
I20260709 12:20:47.539625 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:47.540365 13719 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.541200 13844 tablet_bootstrap.cc:492] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:47.552776 14197 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.553624 14197 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.557246 14197 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.558120 14132 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.558872 14132 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.562578 13844 tablet_bootstrap.cc:654] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.562619 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:47.563457 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57221.7us
W20260709 12:20:47.564558 13618 leader_election.cc:343] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.569476 14132 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.557528 13719 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:47.579617 14189 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.580295 14189 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.582149 14189 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:47.582729 13844 tablet_bootstrap.cc:492] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:47.583453 13844 ts_tablet_manager.cc:1403] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.004s
I20260709 12:20:47.585124 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:20:47.593701 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:47.597043 13845 tablet_bootstrap.cc:492] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:47.597674 13845 ts_tablet_manager.cc:1403] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.172s	user 0.019s	sys 0.043s
I20260709 12:20:47.598356 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:47.599211 13847 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.599828 13847 ts_tablet_manager.cc:1403] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.200s	user 0.096s	sys 0.095s
I20260709 12:20:47.606073 13847 raft_consensus.cc:359] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.606997 13847 raft_consensus.cc:385] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.606809 13845 raft_consensus.cc:359] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.607275 13847 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.607527 13845 raft_consensus.cc:385] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.607785 13845 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.608060 13792 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.607985 13847 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.608582 13845 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.608151 13766 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:47.610206 13847 ts_tablet_manager.cc:1434] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:20:47.611616 13847 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:47.611689 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:47.612654 13720 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:47.612803 14197 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:47.612221 13844 raft_consensus.cc:359] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.615706 13844 raft_consensus.cc:385] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.616073 13844 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.618196 14132 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:47.618832 13844 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.620332 13847 tablet_bootstrap.cc:654] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.620728 13845 ts_tablet_manager.cc:1434] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:20:47.625056 13844 ts_tablet_manager.cc:1434] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 12:20:47.625311 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:47.626457 13648 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:47.627452 13845 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:47.628973 13844 tablet_bootstrap.cc:492] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:47.629319 13617 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:47.631852 14132 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.632402 14132 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:47.634564 13847 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.635176 13847 ts_tablet_manager.cc:1403] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.004s
I20260709 12:20:47.635341 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:47.637854 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:47.633337 14132 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:47.644457 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:47.645440 14189 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.645694 13847 raft_consensus.cc:359] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.646828 13847 raft_consensus.cc:385] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.647265 13847 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.647557 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:47.648589 14198 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.648125 13847 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.649062 14198 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.649000 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:47.649616 14198 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.650360 14137 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:47.650763 14137 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.651109 14137 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.652629 13847 ts_tablet_manager.cc:1434] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 12:20:47.654346 13847 tablet_bootstrap.cc:492] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:47.661897 14198 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.701387 13845 tablet_bootstrap.cc:654] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.682191 14137 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.713069 14197 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.735378 14197 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.737309 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:47.737702 14197 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.737979 13719 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.683630 14189 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.738893 14137 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.738667 14189 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.729357 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:47.741107 13649 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.747026 14189 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:47.747146 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:47.749276 13651 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:47.745909 13719 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:47.755052 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:47.755654 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:47.730592 13847 tablet_bootstrap.cc:654] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.748313 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:47.756510 13793 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.757118 14197 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.766812 14197 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.768929 14204 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.769222 14197 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.731442 13844 tablet_bootstrap.cc:654] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.755815 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:47.769659 14204 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.771600 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:47.772571 14204 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.767093 14254 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.774015 14254 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:47.775487 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:47.775336 14254 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.776929 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:47.781140 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:47.781993 13650 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:47.787231 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:47.788193 13792 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:47.733683 14198 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.789742 13549 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:47.791184 13693 leader_election.cc:343] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:47.793493 13618 leader_election.cc:343] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.797765 13649 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:47.757293 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:20:47.799619 13693 leader_election.cc:343] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.792379 13549 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:47.806653 13844 tablet_bootstrap.cc:492] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:47.806936 13847 tablet_bootstrap.cc:492] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.807592 13844 ts_tablet_manager.cc:1403] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.182s	user 0.011s	sys 0.005s
I20260709 12:20:47.807744 13847 ts_tablet_manager.cc:1403] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.007s
I20260709 12:20:47.799851 13719 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:47.810225 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:47.811882 13847 raft_consensus.cc:359] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.812348 13766 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:47.812659 13847 raft_consensus.cc:385] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.813318 14243 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:47.812460 13693 leader_election.cc:343] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.813184 14198 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:20:47.814252 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:47.814409 14198 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:47.814919 14198 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:47.815106 14204 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.813694 13847 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.818118 13847 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.821043 13847 ts_tablet_manager.cc:1434] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:20:47.822235 13847 tablet_bootstrap.cc:492] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:47.823868 13844 raft_consensus.cc:359] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.839609 13844 raft_consensus.cc:385] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.840212 13844 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.841040 13844 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.841566 13845 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:47.841457 14244 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.829752 13847 tablet_bootstrap.cc:654] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.830585 14243 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.842538 14244 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.844125 13844 ts_tablet_manager.cc:1434] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:20:47.845218 13844 tablet_bootstrap.cc:492] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
W20260709 12:20:47.816872 13693 leader_election.cc:343] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:47.845929 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:47.846104 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:47.847800 14204 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.825363 14198 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.842140 13845 ts_tablet_manager.cc:1403] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.220s	user 0.030s	sys 0.045s
I20260709 12:20:47.848706 14244 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:47.832243 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:47.850611 14198 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:47.804630 13549 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:47.851861 13845 raft_consensus.cc:359] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.852751 13845 raft_consensus.cc:385] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.852921 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:47.853219 13845 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.853968 13793 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:47.854050 13845 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.856038 13845 ts_tablet_manager.cc:1434] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:20:47.857199 13845 tablet_bootstrap.cc:492] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:47.847808 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:47.859267 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:47.861152 14204 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:47.861704 14204 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:47.862636 14204 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.864348 13845 tablet_bootstrap.cc:654] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:47.915144 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:47.916328 13649 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:47.919680 13847 tablet_bootstrap.cc:492] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:47.920267 13847 ts_tablet_manager.cc:1403] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.018s
I20260709 12:20:47.956477 13845 tablet_bootstrap.cc:492] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:47.957209 13845 ts_tablet_manager.cc:1403] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.100s	user 0.016s	sys 0.011s
I20260709 12:20:47.942587 14244 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:47.981871 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:20:47.976657 14244 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.988780 14244 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:47.991950 14243 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:47.994329 13847 raft_consensus.cc:359] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:47.995137 13847 raft_consensus.cc:385] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:47.995450 13847 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:47.996354 13847 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.001202 13847 ts_tablet_manager.cc:1434] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.001s
I20260709 12:20:47.985525 13795 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.025611 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:48.028055 14198 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:20:48.028753 14198 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:48.029915 14198 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.034248 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:48.035179 13724 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:48.037883 13845 raft_consensus.cc:359] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.038982 13845 raft_consensus.cc:385] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.039335 13845 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.040261 13845 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.061494 13844 tablet_bootstrap.cc:654] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.062019 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:48.062952 13799 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:20:48.073323 13845 ts_tablet_manager.cc:1434] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.116s	user 0.010s	sys 0.000s
I20260709 12:20:48.102620 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:48.106611 13847 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:48.107424 14133 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.107851 13799 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:20:48.108234 14133 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:48.124014 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 123463us
I20260709 12:20:48.098587 13547 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.177250 14133 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:48.211282 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:20:48.176131 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080003e860 after lost signal to thread 12472
W20260709 12:20:48.214664 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080003bac0 after lost signal to thread 13584
W20260709 12:20:48.216023 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08001d14a0 after lost signal to thread 13676
I20260709 12:20:48.219455 14285 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.222368 14204 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.222970 14204 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.225080 14204 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:48.225286 14254 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.225857 14254 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.226485 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:48.227780 14254 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:48.228498 14254 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:48.228533 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08000eae20 after lost signal to thread 13751
W20260709 12:20:48.233110 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080026f1a0 after lost signal to thread 13825
I20260709 12:20:48.231071 14132 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:48.234864 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50702.3us
I20260709 12:20:48.235595 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:48.237128 13845 tablet_bootstrap.cc:492] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:48.229044 14254 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.250326 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:48.235008 13617 leader_election.cc:343] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:48.253131 13617 leader_election.cc:343] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:20:48.251731 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080026f4a0 after lost signal to thread 14174
I20260709 12:20:48.263626 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:48.271539 13847 tablet_bootstrap.cc:654] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.281538 14254 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:20:48.261251 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080026f4c0 after lost signal to thread 14185
I20260709 12:20:48.274893 14285 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.277318 13844 tablet_bootstrap.cc:492] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:48.284847 13844 ts_tablet_manager.cc:1403] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.440s	user 0.015s	sys 0.008s
W20260709 12:20:48.287463 13693 leader_election.cc:343] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.291023 13844 raft_consensus.cc:359] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.293227 13547 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:20:48.300669 13693 leader_election.cc:343] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.303215 14254 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:48.362108 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:48.311543 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:48.363045 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62432.5us
I20260709 12:20:48.312310 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:48.363533 13547 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 2, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.319278 13847 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:48.366714 13847 ts_tablet_manager.cc:1403] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.260s	user 0.013s	sys 0.002s
I20260709 12:20:48.367110 14198 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.318979 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:48.339959 14189 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.368692 14189 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.370442 13801 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.370973 14189 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:48.356750 14224 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:48.372490 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:48.373203 13720 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:48.377542 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:48.380033 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:48.370023 13847 raft_consensus.cc:359] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.383267 13847 raft_consensus.cc:385] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.383656 13847 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.384511 13847 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.385646 13800 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:48.387156 13847 ts_tablet_manager.cc:1434] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:20:48.388329 13847 tablet_bootstrap.cc:492] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:48.388553 13845 tablet_bootstrap.cc:654] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.394404 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:20:48.397055 13618 leader_election.cc:343] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.361614 13844 raft_consensus.cc:385] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.399016 13844 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.392302 13549 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.400038 13844 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.398921 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:48.398135 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:48.401512 14228 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.402138 14228 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.402516 14228 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.402839 14228 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:48.404034 13618 leader_election.cc:343] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.415663 13844 ts_tablet_manager.cc:1434] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.130s	user 0.008s	sys 0.000s
I20260709 12:20:48.408599 13549 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.422791 13844 tablet_bootstrap.cc:492] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:48.423401 14243 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.424880 14228 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.426687 13847 tablet_bootstrap.cc:654] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.404495 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:48.435725 14140 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:48.441404 14289 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:48.438683 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:48.444682 13723 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.437294 14283 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:20:48.486370 14228 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:48.525090 13845 tablet_bootstrap.cc:492] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:48.525978 13845 ts_tablet_manager.cc:1403] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.289s	user 0.023s	sys 0.001s
I20260709 12:20:48.533370 13844 tablet_bootstrap.cc:654] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.567561 13847 tablet_bootstrap.cc:492] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:48.568135 13847 ts_tablet_manager.cc:1403] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.180s	user 0.013s	sys 0.005s
I20260709 12:20:48.574869 13845 raft_consensus.cc:359] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.575676 13845 raft_consensus.cc:385] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.575901 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:48.576421 14298 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:20:48.575930 13845 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.576658 13800 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.553980 13723 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:48.582540 13845 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.590006 13845 ts_tablet_manager.cc:1434] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.000s
I20260709 12:20:48.599442 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:48.613281 13845 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:48.613950 14132 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.652655 13844 tablet_bootstrap.cc:492] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:48.653224 13844 ts_tablet_manager.cc:1403] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.237s	user 0.027s	sys 0.004s
W20260709 12:20:48.654011 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:48.631505 14204 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:20:48.641878 14132 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:48.661074 14132 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.666328 13649 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:48.657565 13844 raft_consensus.cc:359] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.667544 13844 raft_consensus.cc:385] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.667847 13844 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.668488 13844 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.641395 13800 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:48.679734 14302 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.680306 14302 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.682528 14302 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:48.676123 14283 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:20:48.688199 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:48.688925 13643 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:48.689316 13793 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:20:48.693154 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:48.693547 13845 tablet_bootstrap.cc:654] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.694767 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:48.695387 13844 ts_tablet_manager.cc:1434] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.002s
I20260709 12:20:48.695724 13847 raft_consensus.cc:359] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.696482 13847 raft_consensus.cc:385] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.696844 13847 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
W20260709 12:20:48.697126 13693 leader_election.cc:343] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.700290 14197 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:48.699338 13847 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.720857 13844 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:48.704779 14283 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.744933 13844 tablet_bootstrap.cc:654] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.723472 13847 ts_tablet_manager.cc:1434] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.155s	user 0.007s	sys 0.004s
I20260709 12:20:48.727190 14283 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.777858 13847 tablet_bootstrap.cc:492] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:48.752108 14283 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.790454 13845 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:48.791039 13845 ts_tablet_manager.cc:1403] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.178s	user 0.009s	sys 0.013s
W20260709 12:20:48.805588 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61913.9us
I20260709 12:20:48.818854 13845 raft_consensus.cc:359] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.820274 13845 raft_consensus.cc:385] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.820636 13845 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.848600 13845 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.859413 14192 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:48.860087 14192 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.864122 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:48.865346 13549 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:48.868933 14192 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:48.869264 13844 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:48.869899 13844 ts_tablet_manager.cc:1403] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.163s	user 0.018s	sys 0.009s
I20260709 12:20:48.830125 14283 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.870199 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:48.864863 13643 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:48.876446 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:48.882982 14192 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:48.883425 14192 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:48.874627 13719 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:48.883795 14192 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.892423 13845 ts_tablet_manager.cc:1434] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.002s
I20260709 12:20:48.892298 14192 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.893226 14283 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:48.877988 13844 raft_consensus.cc:359] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.904172 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:48.907176 13643 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.905992 13847 tablet_bootstrap.cc:654] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:48.906215 13844 raft_consensus.cc:385] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:48.909368 13844 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:48.904029 13845 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:48.904403 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:48.918978 14192 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:48.920646 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:48.921556 13650 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.924165 13643 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:48.912921 13844 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.925666 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:48.941107 13723 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:48.934860 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:20:48.951134 13693 leader_election.cc:343] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:48.953374 14302 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.954607 14302 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:48.955657 14302 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.957715 13723 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:48.959342 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:48.960094 14192 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:48.960619 14192 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:48.961692 14192 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:48.962433 13844 ts_tablet_manager.cc:1434] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.092s	user 0.004s	sys 0.006s
I20260709 12:20:48.963738 13844 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:48.966078 13650 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:48.994772 13845 tablet_bootstrap.cc:654] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.001875 13550 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:20:49.004024 13549 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:49.019389 13847 tablet_bootstrap.cc:492] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:49.020183 13847 ts_tablet_manager.cc:1403] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.249s	user 0.015s	sys 0.017s
I20260709 12:20:49.034906 13847 raft_consensus.cc:359] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.035737 13847 raft_consensus.cc:385] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.036046 13847 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.036705 13847 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.038552 13845 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:49.039661 13847 ts_tablet_manager.cc:1434] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:20:49.041421 13845 ts_tablet_manager.cc:1403] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.011s
I20260709 12:20:49.033265 13844 tablet_bootstrap.cc:654] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.051272 13847 tablet_bootstrap.cc:492] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:49.061298 13845 raft_consensus.cc:359] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.068715 13847 tablet_bootstrap.cc:654] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.062243 13845 raft_consensus.cc:385] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.070459 13845 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.072165 13845 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.075618 14140 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.076216 14140 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.078905 14140 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.080448 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:49.083920 13618 leader_election.cc:343] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.082783 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:49.084630 13720 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:49.084995 13845 ts_tablet_manager.cc:1434] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.003s
I20260709 12:20:49.090102 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:49.090528 13845 tablet_bootstrap.cc:492] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:49.091620 14312 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.092109 14312 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.092586 14312 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.108069 13844 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:49.108616 13844 ts_tablet_manager.cc:1403] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.145s	user 0.009s	sys 0.006s
I20260709 12:20:49.111882 13844 raft_consensus.cc:359] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.112560 13844 raft_consensus.cc:385] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.112846 13844 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.113525 13844 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.113739 14307 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:49.107137 14312 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.118399 13844 ts_tablet_manager.cc:1434] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:20:49.124338 13847 tablet_bootstrap.cc:492] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:49.133635 13847 ts_tablet_manager.cc:1403] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.002s
I20260709 12:20:49.137092 13847 raft_consensus.cc:359] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.137903 13847 raft_consensus.cc:385] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.139183 14307 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.139856 14307 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.143783 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:49.144220 14312 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.144515 13720 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.125124 13845 tablet_bootstrap.cc:654] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.147606 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:49.126214 13844 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:49.138217 13847 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.158118 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:49.159310 13719 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:49.163242 13847 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.164897 13617 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:49.167423 13800 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.170228 13847 ts_tablet_manager.cc:1434] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.036s	user 0.012s	sys 0.000s
I20260709 12:20:49.169866 14140 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.170702 14140 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.171129 14140 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.171640 13847 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:49.174304 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:20:49.176278 13618 leader_election.cc:343] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.182650 13844 tablet_bootstrap.cc:654] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.165411 14307 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.192967 13720 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:49.198974 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:49.197746 14286 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.202903 14286 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.202474 14307 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.210315 13845 tablet_bootstrap.cc:492] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:49.212406 13845 ts_tablet_manager.cc:1403] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.008s
I20260709 12:20:49.216073 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:49.219372 13847 tablet_bootstrap.cc:654] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.212093 13800 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:49.221273 14286 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:20:49.222115 13767 leader_election.cc:343] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.239440 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:49.240517 13720 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 0.
I20260709 12:20:49.220749 14307 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:49.242306 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:20:49.237545 13845 raft_consensus.cc:359] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.243739 13845 raft_consensus.cc:385] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.243235 14286 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.244091 13845 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.244282 14286 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.244648 14286 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.243109 14307 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.245573 13845 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.249598 14140 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.248238 14312 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.255975 14312 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.259904 13845 ts_tablet_manager.cc:1434] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 12:20:49.258250 14312 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.268926 13845 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:49.276081 13844 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:49.276677 13844 ts_tablet_manager.cc:1403] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.151s	user 0.003s	sys 0.016s
I20260709 12:20:49.268621 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:49.268438 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:49.281221 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:49.278445 14283 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.284685 14283 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.283751 13847 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:49.286657 13847 ts_tablet_manager.cc:1403] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.115s	user 0.011s	sys 0.013s
W20260709 12:20:49.289354 13618 leader_election.cc:343] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.291299 13726 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:49.300920 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:49.302978 14132 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.303547 14132 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.303895 14132 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.304935 14283 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.306593 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:49.307297 13650 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:49.308408 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:49.309839 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:49.312353 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:49.313234 14283 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.313323 13720 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.313751 14283 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.314291 14283 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.305285 14286 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.316659 14228 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:20:49.314858 13693 leader_election.cc:343] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.317569 13795 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:49.320317 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:49.278569 14140 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.321738 13844 raft_consensus.cc:359] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.325942 13844 raft_consensus.cc:385] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.326380 13844 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.327075 13844 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.340126 13844 ts_tablet_manager.cc:1434] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.063s	user 0.008s	sys 0.001s
I20260709 12:20:49.348511 13847 raft_consensus.cc:359] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.349681 13847 raft_consensus.cc:385] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.350024 13847 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.350822 13847 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.355593 13844 tablet_bootstrap.cc:492] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:49.356369 14286 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:49.360014 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:49.360664 13726 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.358007 13549 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:49.371498 13847 ts_tablet_manager.cc:1434] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.084s	user 0.005s	sys 0.002s
I20260709 12:20:49.364916 14132 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.387931 13847 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:49.395031 13650 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:49.400421 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:49.402531 14302 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.385735 14283 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.402933 14132 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.403077 14302 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.409732 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:49.410509 13650 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.412488 14302 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.412590 14283 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.413265 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:49.410228 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:20:49.415593 13693 leader_election.cc:343] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.417048 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:49.389062 13726 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:20:49.423534 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:49.426887 13792 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 1: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:20:49.428602 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:49.429850 14140 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:20:49.419415 13693 leader_election.cc:343] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.431329 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:49.439698 14204 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:49.442134 13845 tablet_bootstrap.cc:654] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
W20260709 12:20:49.445595 13693 leader_election.cc:343] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:49.452351 13720 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:49.454525 14243 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:49.460517 13650 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:49.467549 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:20:49.473186 14243 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.473538 13844 tablet_bootstrap.cc:654] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.475682 14243 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:20:49.475814 13617 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:49.476791 14243 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.478960 14140 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.479488 14140 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:49.480427 14140 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.484534 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:20:49.487311 13693 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:49.489532 14204 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:49.531764 13847 tablet_bootstrap.cc:654] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.514904 14204 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:49.553225 14204 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.561193 14283 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:20:49.582034 13845 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:49.582643 13845 ts_tablet_manager.cc:1403] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.317s	user 0.011s	sys 0.025s
I20260709 12:20:49.585223 14286 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:49.589419 13845 raft_consensus.cc:359] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.590570 13845 raft_consensus.cc:385] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.591699 13845 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.593741 13845 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.596782 13550 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:20:49.604233 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49559.8us
I20260709 12:20:49.607179 13845 ts_tablet_manager.cc:1434] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.003s
I20260709 12:20:49.608484 13845 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
I20260709 12:20:49.655100 13844 tablet_bootstrap.cc:492] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
W20260709 12:20:49.655328 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70201.3us
I20260709 12:20:49.655723 13844 ts_tablet_manager.cc:1403] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.300s	user 0.007s	sys 0.019s
I20260709 12:20:49.687505 13845 tablet_bootstrap.cc:654] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.704679 13844 raft_consensus.cc:359] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.705562 13844 raft_consensus.cc:385] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.705863 13844 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.706924 13844 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.719920 13844 ts_tablet_manager.cc:1434] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.003s
I20260709 12:20:49.720082 13847 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:49.720753 13847 ts_tablet_manager.cc:1403] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.333s	user 0.023s	sys 0.001s
I20260709 12:20:49.733914 13547 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:49.734143 13847 raft_consensus.cc:359] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.737125 13847 raft_consensus.cc:385] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.737413 13847 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.740208 13847 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.757197 14335 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.757784 14335 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.760396 14335 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:49.736375 14140 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:20:49.762775 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:49.776518 13847 ts_tablet_manager.cc:1434] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:20:49.796424 13847 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:49.827658 13844 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Bootstrap starting.
I20260709 12:20:49.844480 13845 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:49.845091 13845 ts_tablet_manager.cc:1403] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.237s	user 0.035s	sys 0.028s
I20260709 12:20:49.856395 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:49.863189 13726 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:20:49.864882 13847 tablet_bootstrap.cc:654] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.864882 13844 tablet_bootstrap.cc:654] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.871073 13766 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:49.876027 13845 raft_consensus.cc:359] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.880599 13845 raft_consensus.cc:385] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.880927 13845 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.881620 13845 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.873783 14243 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.875613 14132 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.873667 14140 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:20:49.914769 14132 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.917738 14346 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:49.918938 14346 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.932910 14346 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.937706 14243 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:49.950747 14243 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:20:49.951709 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 72151.8us
I20260709 12:20:49.954349 13845 ts_tablet_manager.cc:1434] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.109s	user 0.005s	sys 0.007s
I20260709 12:20:49.955418 13845 tablet_bootstrap.cc:492] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Bootstrap starting.
W20260709 12:20:49.957971 13598 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:20:49.959542 14132 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:49.971414 13847 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:49.972110 13847 ts_tablet_manager.cc:1403] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.189s	user 0.010s	sys 0.008s
I20260709 12:20:49.974632 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:49.975358 13726 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:49.975199 13847 raft_consensus.cc:359] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.976071 13847 raft_consensus.cc:385] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:49.976409 13847 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:49.977165 13847 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.951398 14243 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:49.979736 13847 ts_tablet_manager.cc:1434] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:20:49.980918 13847 tablet_bootstrap.cc:492] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:49.975210 13844 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: No bootstrap required, opened a new log
I20260709 12:20:49.983606 13844 ts_tablet_manager.cc:1403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Time spent bootstrapping tablet: real 0.175s	user 0.012s	sys 0.011s
I20260709 12:20:49.983394 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:49.985831 13793 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:49.987567 13847 tablet_bootstrap.cc:654] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:49.992187 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:49.993515 13651 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:49.993453 13618 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:20:49.996238 14340 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:49.997014 14340 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:49.997463 14340 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:49.998499 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:50.001842 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:50.004087 14132 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:50.004426 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:50.009630 13844 raft_consensus.cc:359] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.016050 13844 raft_consensus.cc:385] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.016396 13844 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.016880 13767 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:50.017094 13844 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.017628 14335 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.027060 14243 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:50.034335 13844 ts_tablet_manager.cc:1434] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 12:20:50.042927 14286 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.061945 14243 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.068722 13845 tablet_bootstrap.cc:654] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.074524 14335 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:20:50.075637 13693 leader_election.cc:343] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.078322 14340 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.092036 14132 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.113339 13847 tablet_bootstrap.cc:492] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:50.114112 13847 ts_tablet_manager.cc:1403] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.133s	user 0.019s	sys 0.015s
W20260709 12:20:50.118103 13693 leader_election.cc:343] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.118675 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:50.117765 13847 raft_consensus.cc:359] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.124228 13847 raft_consensus.cc:385] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.129282 14348 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:50.134718 13847 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.135125 14204 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.144654 14204 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.147646 14204 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:50.147478 14346 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.149852 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:50.151893 13647 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 0.
I20260709 12:20:50.157891 13847 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.165902 14197 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.165964 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:20:50.168126 14332 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:50.168525 14332 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.168874 14332 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.173420 13847 ts_tablet_manager.cc:1434] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 12:20:50.174731 13847 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:50.173753 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:50.183524 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:50.184621 13801 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.195277 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:50.196384 13726 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:20:50.202980 13693 leader_election.cc:343] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.178510 13845 tablet_bootstrap.cc:492] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: No bootstrap required, opened a new log
I20260709 12:20:50.204218 13845 ts_tablet_manager.cc:1403] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent bootstrapping tablet: real 0.249s	user 0.014s	sys 0.003s
I20260709 12:20:50.177064 14340 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:50.243767 13549 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.247879 13845 raft_consensus.cc:359] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.248809 13845 raft_consensus.cc:385] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.251961 13845 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.254846 13845 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.248595 14332 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.267220 13845 ts_tablet_manager.cc:1434] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent starting tablet: real 0.063s	user 0.014s	sys 0.005s
I20260709 12:20:50.260824 13847 tablet_bootstrap.cc:654] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.268555 14332 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:50.269286 13726 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:50.269603 13793 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.285758 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:20:50.286535 13643 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.286573 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:50.292018 13617 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:20:50.293144 13693 leader_election.cc:343] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.293330 14228 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:50.293897 14228 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:50.294797 14228 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.259379 13801 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:50.302591 14334 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.303275 14334 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.312218 13847 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:50.312916 13847 ts_tablet_manager.cc:1403] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.138s	user 0.015s	sys 0.010s
I20260709 12:20:50.318712 14334 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:50.321942 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:50.322742 13726 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 0.
I20260709 12:20:50.323638 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:20:50.324040 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:20:50.327633 13618 leader_election.cc:343] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.328567 14132 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:20:50.329005 14132 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:20:50.329339 14132 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.333127 13643 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:20:50.334630 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:50.334653 13847 raft_consensus.cc:359] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.335457 14197 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:50.335522 13847 raft_consensus.cc:385] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.336040 14197 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:20:50.336050 13847 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.341588 14132 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.344602 13847 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.345942 14132 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:50.346808 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:20:50.346884 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:20:50.347483 13726 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.337005 14197 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:20:50.357028 13618 leader_election.cc:343] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:20:50.370368 13726 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:20:50.371460 13549 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.377707 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:20:50.379458 13847 ts_tablet_manager.cc:1434] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.002s
I20260709 12:20:50.380612 13847 tablet_bootstrap.cc:492] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Bootstrap starting.
I20260709 12:20:50.381031 14132 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:20:50.381500 14132 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:20:50.383343 14132 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.374136 13550 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.420264 13847 tablet_bootstrap.cc:654] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Neither blocks nor log segments found. Creating new log.
I20260709 12:20:50.445161 13550 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:50.478734 13847 tablet_bootstrap.cc:492] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: No bootstrap required, opened a new log
I20260709 12:20:50.479331 13847 ts_tablet_manager.cc:1403] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent bootstrapping tablet: real 0.099s	user 0.010s	sys 0.007s
I20260709 12:20:50.482460 13847 raft_consensus.cc:359] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.483299 13847 raft_consensus.cc:385] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:20:50.483596 13847 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Initialized, Role: FOLLOWER
I20260709 12:20:50.484334 13847 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.486097 13847 ts_tablet_manager.cc:1434] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:20:50.566872 14335 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:20:50.567416 14335 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:20:50.574388 14335 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:50.576375 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:20:50.577399 13643 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 1: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:20:50.577301 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:20:50.579252 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:20:50.580363 14335 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.593195 14335 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:20:50.743320 14133 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:50.771356 14133 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:50.772961 14346 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.801599 13801 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:20:50.834409 14132 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:50.851414 14332 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:20:50.855496 14133 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:20:51.538121 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:20:51.583519 14378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:51.588063 14379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:20:51.606541 14381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:20:51.608156 12469 server_base.cc:1061] running on GCE node
I20260709 12:20:51.610544 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:20:51.610817 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:20:51.611095 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783599651611063 us; error 0 us; skew 500 ppm
I20260709 12:20:51.612078 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:20:51.617010 12469 webserver.cc:533] Webserver started at http://127.12.45.68:41517/ using document root <none> and password file <none>
I20260709 12:20:51.617868 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:20:51.618251 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:20:51.618742 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:20:51.620716 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-3-root/instance:
uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
format_stamp: "Formatted at 2026-07-09 12:20:51 on dist-test-slave-90bm"
I20260709 12:20:51.628885 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.003s
I20260709 12:20:51.641131 14387 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:51.642781 12469 fs_manager.cc:730] Time spent opening block manager: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:20:51.645236 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-3-root
uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
format_stamp: "Formatted at 2026-07-09 12:20:51 on dist-test-slave-90bm"
I20260709 12:20:51.645709 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:20:51.669804 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:20:51.672205 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:20:51.677984 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:20:51.681574 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:20:51.681849 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:20:51.682173 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:20:51.682498 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:20:51.791113 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.68:42965
I20260709 12:20:51.791293 14449 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.68:42965 every 8 connection(s)
I20260709 12:20:51.798434 12469 auto_leader_rebalancer-test.cc:439] add a tserver: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:51.823256 14450 heartbeater.cc:344] Connected to a master server at 127.12.45.126:44051
I20260709 12:20:51.823925 14450 heartbeater.cc:461] Registering TS with master...
I20260709 12:20:51.824992 14450 heartbeater.cc:507] Master 127.12.45.126:44051 requested a full tablet report, sending...
I20260709 12:20:51.827430 13550 ts_manager.cc:194] Registered new tserver with Master: b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:20:51.830670 13550 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43972
I20260709 12:20:52.188395 13726 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:52.231777 13795 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:52.234071 14332 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:52.234966 13647 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:20:52.238523 14332 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:52.267989 14302 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.268975 13801 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
W20260709 12:20:52.278467 13690 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3cb6d086379f4190b73fc7f692fb490a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.289803 13718 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:52.313898 13549 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.328403 13795 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:20:52.332854 13690 consensus_peers.cc:597] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1866ffc09d7a4cd3b2f2883e9baf96fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.337185 14302 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.338883 13647 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.357024 13643 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:52.361891 14302 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.370321 14463 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.379143 13801 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.410519 13647 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.443723 13550 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.499404 13795 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:52.518406 13719 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:52.518993 13651 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:52.520810 14470 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:52.521806 14386 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.543762 14286 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
W20260709 12:20:52.550151 13764 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8debd745c407406c88fc03ec3a21f4db. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.554723 13719 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.553501 13651 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.577728 13547 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:52.605783 13643 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:52.624080 13795 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:52.626487 13719 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:52.626545 14471 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:52.628356 14228 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:20:52.645059 13615 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac3728586e264940a93f76650edf9ad3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:52.652802 14228 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.654768 13719 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.677675 13550 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.690827 13795 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:52.783666 14482 ts_tablet_manager.cc:933] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:52.799093 14482 tablet_copy_client.cc:323] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:20:52.842463 14450 heartbeater.cc:499] Master 127.12.45.126:44051 was elected leader, sending a full tablet report...
I20260709 12:20:52.844964 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 3cb6d086379f4190b73fc7f692fb490a from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:20:52.845716 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 3cb6d086379f4190b73fc7f692fb490a from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-3cb6d086379f4190b73fc7f692fb490a
I20260709 12:20:52.863421 14486 ts_tablet_manager.cc:933] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:52.867741 13737 tablet_copy_source_session.cc:215] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:52.875705 14482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cb6d086379f4190b73fc7f692fb490a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:52.879225 14486 tablet_copy_client.cc:323] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:20:52.894932 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:20:52.897302 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 1866ffc09d7a4cd3b2f2883e9baf96fe from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-1866ffc09d7a4cd3b2f2883e9baf96fe
I20260709 12:20:52.918887 13737 tablet_copy_source_session.cc:215] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:52.933591 14486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:52.942642 14482 tablet_copy_client.cc:806] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:52.943843 14482 tablet_copy_client.cc:670] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:52.949342 14482 tablet_copy_client.cc:538] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:52.957285 14486 tablet_copy_client.cc:806] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:52.958124 14486 tablet_copy_client.cc:670] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:52.959877 14482 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:52.962617 14486 tablet_copy_client.cc:538] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:53.021224 14486 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:53.041688 14489 ts_tablet_manager.cc:933] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:53.053813 14489 tablet_copy_client.cc:323] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:20:53.069028 14490 ts_tablet_manager.cc:933] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:20:53.083791 14482 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:53.087430 14490 tablet_copy_client.cc:323] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:20:53.095482 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 8debd745c407406c88fc03ec3a21f4db from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:20:53.096156 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 8debd745c407406c88fc03ec3a21f4db from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-8debd745c407406c88fc03ec3a21f4db
I20260709 12:20:53.114810 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet ac3728586e264940a93f76650edf9ad3 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:20:53.115278 14482 tablet_bootstrap.cc:492] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:53.115602 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet ac3728586e264940a93f76650edf9ad3 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-ac3728586e264940a93f76650edf9ad3
I20260709 12:20:53.116223 14482 ts_tablet_manager.cc:1403] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.157s	user 0.053s	sys 0.029s
I20260709 12:20:53.120113 14482 raft_consensus.cc:359] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.121381 14482 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:53.122299 14482 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.125620 13662 tablet_copy_source_session.cc:215] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:53.127249 13811 tablet_copy_source_session.cc:215] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:53.138069 14490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac3728586e264940a93f76650edf9ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:53.145190 14489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8debd745c407406c88fc03ec3a21f4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:53.176025 14486 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:53.177199 14486 tablet_bootstrap.cc:492] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:53.177841 14489 tablet_copy_client.cc:806] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:53.179051 14486 ts_tablet_manager.cc:1403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.158s	user 0.039s	sys 0.016s
I20260709 12:20:53.183498 14486 raft_consensus.cc:359] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.189265 14489 tablet_copy_client.cc:670] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:53.182607 14490 tablet_copy_client.cc:806] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:53.193174 14486 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:53.194545 14490 tablet_copy_client.cc:670] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:53.192682 14482 ts_tablet_manager.cc:1434] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.015s
I20260709 12:20:53.194984 14486 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.199880 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-3cb6d086379f4190b73fc7f692fb490a received from {username='slave'} at 127.0.0.1:50270
I20260709 12:20:53.200156 14486 ts_tablet_manager.cc:1434] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:20:53.200505 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-3cb6d086379f4190b73fc7f692fb490a on tablet 3cb6d086379f4190b73fc7f692fb490a with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:53.202132 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-1866ffc09d7a4cd3b2f2883e9baf96fe received from {username='slave'} at 127.0.0.1:50270
I20260709 12:20:53.202639 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-1866ffc09d7a4cd3b2f2883e9baf96fe on tablet 1866ffc09d7a4cd3b2f2883e9baf96fe with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:53.203001 14489 tablet_copy_client.cc:538] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:53.215358 14490 tablet_copy_client.cc:538] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:53.223289 14489 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:53.227834 14490 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:53.260195 14490 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:53.263625 14490 tablet_bootstrap.cc:492] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:53.264868 14490 ts_tablet_manager.cc:1403] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.012s
I20260709 12:20:53.275874 14489 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:53.283960 14489 tablet_bootstrap.cc:492] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:53.284865 14489 ts_tablet_manager.cc:1403] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.012s
I20260709 12:20:53.285353 14424 raft_consensus.cc:1217] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:20:53.269793 14490 raft_consensus.cc:359] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.287034 14490 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:53.287936 14490 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.292097 14490 ts_tablet_manager.cc:1434] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 12:20:53.295560 13662 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-ac3728586e264940a93f76650edf9ad3 received from {username='slave'} at 127.0.0.1:60832
I20260709 12:20:53.295250 14489 raft_consensus.cc:359] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.296092 13662 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-ac3728586e264940a93f76650edf9ad3 on tablet ac3728586e264940a93f76650edf9ad3 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:53.296217 14489 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:53.296978 14489 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:53.299803 14489 ts_tablet_manager.cc:1434] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:20:53.301780 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-8debd745c407406c88fc03ec3a21f4db received from {username='slave'} at 127.0.0.1:44068
I20260709 12:20:53.302240 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-8debd745c407406c88fc03ec3a21f4db on tablet 8debd745c407406c88fc03ec3a21f4db with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:53.400606 14424 raft_consensus.cc:1217] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:53.435570 14424 raft_consensus.cc:1217] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:53.624456 14424 raft_consensus.cc:1217] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:53.865478 14463 raft_consensus.cc:1064] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:53.877872 14463 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:53.939922 14424 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:20:53.941960 14377 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:53.968547 13795 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:20:53.974674 14463 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:53.977102 14503 raft_consensus.cc:1064] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:53.980450 14503 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.006677 14424 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.009924 14504 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:54.015530 14496 raft_consensus.cc:1064] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:54.019840 14504 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.020963 14496 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.023444 13795 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.024902 13800 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.033252 14503 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.034782 13643 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.056221 14425 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.057834 14503 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:54.053121 13547 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.065580 13647 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:20:54.077291 13718 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.078351 14463 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:54.091813 14476 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.090838 14377 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.096438 14470 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.098062 14424 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.102921 13647 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.100760 14425 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.115046 14496 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:54.316978 13643 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:20:54.347414 14511 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:20:54.398970 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103954us
I20260709 12:20:54.423230 13795 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.436571 14476 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.440665 13643 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.464879 14525 raft_consensus.cc:1064] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:54.465031 13647 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.474510 13719 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.468758 14525 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.481045 14425 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.517961 13548 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.533481 14424 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.534063 13793 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.536134 13719 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:54.536317 14507 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:54.537413 14525 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.538427 14334 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:54.565603 13547 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.591661 13795 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.599323 13647 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:20:54.599582 14425 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:20:54.602344 14470 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:54.604090 14476 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:54.610200 14228 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.613703 14518 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.624558 13793 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.627213 13722 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.627496 13719 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.634063 14425 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:20:54.635955 14519 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:54.641840 14424 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:20:54.643757 13795 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:20:54.644789 14534 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:20:54.652505 14302 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.666530 14522 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:20:54.675863 14424 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.683063 14425 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.688935 14500 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.694029 14423 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.702477 13719 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.704262 14422 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.706727 13647 raft_consensus.cc:2955] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.718011 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:54.722316 13649 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:20:54.746235 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8debd745c407406c88fc03ec3a21f4db with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:54.748065 13793 raft_consensus.cc:2955] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.750468 14302 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:20:54.753638 13550 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.714604 13549 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:54.774740 13649 raft_consensus.cc:2955] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.794160 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3cb6d086379f4190b73fc7f692fb490a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:54.803278 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 1866ffc09d7a4cd3b2f2883e9baf96fe with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:20:54.825420 13550 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:54.840286 13548 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 4 to 5, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.844357 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 8debd745c407406c88fc03ec3a21f4db with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:20:54.859894 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet 3cb6d086379f4190b73fc7f692fb490a with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:60478
I20260709 12:20:54.865337 13647 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:54.852345 14541 tablet_replica.cc:333] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:20:54.877719 14425 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:20:54.880159 14530 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:20:54.880709 14542 tablet_replica.cc:333] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:20:54.881764 14542 raft_consensus.cc:2243] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:54.882406 13718 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:20:54.883127 14543 tablet_replica.cc:333] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:20:54.884159 14543 raft_consensus.cc:2243] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:54.884641 14334 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:20:54.894333 14542 raft_consensus.cc:2272] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:54.894586 14543 raft_consensus.cc:2272] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:54.900439 14334 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.917553 14543 ts_tablet_manager.cc:1916] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:54.902922 14425 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.924585 13718 raft_consensus.cc:2955] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:54.878708 14541 raft_consensus.cc:2243] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:54.927371 14541 raft_consensus.cc:2272] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:54.939888 14541 ts_tablet_manager.cc:1916] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:54.978003 14542 ts_tablet_manager.cc:1916] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:55.102111 14541 ts_tablet_manager.cc:1929] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:20:55.102900 14541 log.cc:1199] T 1866ffc09d7a4cd3b2f2883e9baf96fe P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/1866ffc09d7a4cd3b2f2883e9baf96fe
I20260709 12:20:55.189209 14543 ts_tablet_manager.cc:1929] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:20:55.191094 14543 log.cc:1199] T 3cb6d086379f4190b73fc7f692fb490a P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/3cb6d086379f4190b73fc7f692fb490a
I20260709 12:20:55.193501 14542 ts_tablet_manager.cc:1929] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:20:55.194581 14542 log.cc:1199] T 8debd745c407406c88fc03ec3a21f4db P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/8debd745c407406c88fc03ec3a21f4db
I20260709 12:20:55.300144 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 8debd745c407406c88fc03ec3a21f4db (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:20:55.431352 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet 3cb6d086379f4190b73fc7f692fb490a (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:20:55.447424 13548 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:20:55.487901 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ac3728586e264940a93f76650edf9ad3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:20:55.489269 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 1866ffc09d7a4cd3b2f2883e9baf96fe (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
W20260709 12:20:55.505735 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 101358us
I20260709 12:20:55.658895 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet ac3728586e264940a93f76650edf9ad3 with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:20:55.660269 14543 tablet_replica.cc:333] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:20:55.661693 14543 raft_consensus.cc:2243] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:20:55.662916 14543 raft_consensus.cc:2272] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:20:55.677027 14543 ts_tablet_manager.cc:1916] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:20:55.701716 14543 ts_tablet_manager.cc:1929] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:20:55.702214 14543 log.cc:1199] T ac3728586e264940a93f76650edf9ad3 P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/ac3728586e264940a93f76650edf9ad3
I20260709 12:20:55.704025 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet ac3728586e264940a93f76650edf9ad3 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
W20260709 12:20:56.422338 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800197b40 after lost signal to thread 12472
W20260709 12:20:56.423880 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8fc0 after lost signal to thread 13584
W20260709 12:20:56.425405 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08003157e0 after lost signal to thread 13676
W20260709 12:20:56.426698 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800314fa0 after lost signal to thread 13751
W20260709 12:20:56.428228 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800317c40 after lost signal to thread 13825
W20260709 12:20:56.429493 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08003185a0 after lost signal to thread 14449
I20260709 12:20:56.862608 13722 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:20:57.182564 13690 consensus_peers.cc:597] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 435988495fcf4cf49f65293e95b5ceb5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.218040 13647 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:57.220927 13800 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:57.241700 14558 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:57.253654 14558 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:57.283561 14534 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.288442 13799 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.322770 13650 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.337709 13647 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:57.359146 13548 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.432461 13799 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:20:57.436913 13615 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a27ddca262a4e0e8c46331cd36be4d1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.437516 14562 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.460372 13722 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:57.462229 14562 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:20:57.480939 13799 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.477190 14563 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.547704 13647 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:57.562665 13549 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:57.592000 13722 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.617956 13799 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:57.619882 14530 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:20:57.622351 13615 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e52c8b752b54bd9b959b2363df455a4. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:20:57.647161 14573 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:20:57.647251 13720 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:57.653077 14530 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:20:57.671684 14562 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.678020 13799 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.731865 13719 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.739054 13546 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.774125 13795 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:57.848068 13722 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:20:57.848237 13764 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f136fb5f4e574b6db1336e9434176d83. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:20:57.852589 13650 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:20:57.855926 14547 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:57.896704 14547 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:20:57.930541 14560 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.937958 13643 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:57.969130 14577 ts_tablet_manager.cc:933] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:20:57.989766 14582 ts_tablet_manager.cc:933] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:20:57.994810 13546 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:20:58.018070 14577 tablet_copy_client.cc:323] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:20:58.021999 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 435988495fcf4cf49f65293e95b5ceb5 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:20:58.022693 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 435988495fcf4cf49f65293e95b5ceb5 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-435988495fcf4cf49f65293e95b5ceb5
I20260709 12:20:58.026760 14582 tablet_copy_client.cc:323] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:20:58.029563 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 0a27ddca262a4e0e8c46331cd36be4d1 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:20:58.030176 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 0a27ddca262a4e0e8c46331cd36be4d1 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-0a27ddca262a4e0e8c46331cd36be4d1
I20260709 12:20:58.012432 13719 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:20:58.039842 13662 tablet_copy_source_session.cc:215] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:58.043702 14582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a27ddca262a4e0e8c46331cd36be4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:58.052205 13737 tablet_copy_source_session.cc:215] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:58.057670 14577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 435988495fcf4cf49f65293e95b5ceb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:58.156814 14582 tablet_copy_client.cc:806] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:58.157688 14582 tablet_copy_client.cc:670] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:58.163136 14582 tablet_copy_client.cc:538] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:58.193573 14586 ts_tablet_manager.cc:933] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:20:58.199487 14582 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:58.205852 14577 tablet_copy_client.cc:806] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:58.206684 14577 tablet_copy_client.cc:670] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:58.211278 14577 tablet_copy_client.cc:538] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:58.218771 14586 tablet_copy_client.cc:323] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:20:58.220855 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 3e52c8b752b54bd9b959b2363df455a4 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:20:58.221503 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 3e52c8b752b54bd9b959b2363df455a4 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-3e52c8b752b54bd9b959b2363df455a4
I20260709 12:20:58.224838 14577 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:58.242961 13662 tablet_copy_source_session.cc:215] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:58.246923 14586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e52c8b752b54bd9b959b2363df455a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:58.280623 14582 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:58.281993 14582 tablet_bootstrap.cc:492] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:58.282404 14586 tablet_copy_client.cc:806] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:58.283044 14582 ts_tablet_manager.cc:1403] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.084s	user 0.019s	sys 0.009s
I20260709 12:20:58.284372 14577 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:58.285424 14577 tablet_bootstrap.cc:492] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:58.286232 14577 ts_tablet_manager.cc:1403] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.062s	user 0.028s	sys 0.005s
I20260709 12:20:58.283077 14586 tablet_copy_client.cc:670] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:58.292235 14577 raft_consensus.cc:359] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.296530 14586 tablet_copy_client.cc:538] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:58.302551 14577 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:58.307343 14577 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.315478 14577 ts_tablet_manager.cc:1434] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.003s
I20260709 12:20:58.318352 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-435988495fcf4cf49f65293e95b5ceb5 received from {username='slave'} at 127.0.0.1:50270
I20260709 12:20:58.297505 14582 raft_consensus.cc:359] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.318897 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-435988495fcf4cf49f65293e95b5ceb5 on tablet 435988495fcf4cf49f65293e95b5ceb5 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:58.319139 14582 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:58.319872 14582 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.392938 14582 ts_tablet_manager.cc:1434] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.001s
I20260709 12:20:58.397917 13662 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-0a27ddca262a4e0e8c46331cd36be4d1 received from {username='slave'} at 127.0.0.1:60832
I20260709 12:20:58.398449 13662 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-0a27ddca262a4e0e8c46331cd36be4d1 on tablet 0a27ddca262a4e0e8c46331cd36be4d1 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:58.398432 14586 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:58.443964 14582 ts_tablet_manager.cc:933] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:20:58.464339 14582 tablet_copy_client.cc:323] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:20:58.466976 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet f136fb5f4e574b6db1336e9434176d83 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:20:58.467759 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet f136fb5f4e574b6db1336e9434176d83 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-f136fb5f4e574b6db1336e9434176d83
I20260709 12:20:58.479063 13811 tablet_copy_source_session.cc:215] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:20:58.484535 14582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f136fb5f4e574b6db1336e9434176d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:20:58.542433 14582 tablet_copy_client.cc:806] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:20:58.543190 14582 tablet_copy_client.cc:670] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:20:58.544266 14586 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:58.545828 14586 tablet_bootstrap.cc:492] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:58.547698 14582 tablet_copy_client.cc:538] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:20:58.565161 14586 ts_tablet_manager.cc:1403] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.167s	user 0.038s	sys 0.011s
I20260709 12:20:58.571094 14586 raft_consensus.cc:359] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.578979 14586 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:58.577574 14582 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:20:58.584975 14586 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.595019 14586 ts_tablet_manager.cc:1434] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 12:20:58.605814 13662 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-3e52c8b752b54bd9b959b2363df455a4 received from {username='slave'} at 127.0.0.1:60832
I20260709 12:20:58.606365 13662 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-3e52c8b752b54bd9b959b2363df455a4 on tablet 3e52c8b752b54bd9b959b2363df455a4 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:58.649551 14582 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:20:58.650995 14582 tablet_bootstrap.cc:492] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:20:58.651907 14582 ts_tablet_manager.cc:1403] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.075s	user 0.018s	sys 0.014s
I20260709 12:20:58.685250 14582 raft_consensus.cc:359] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.693707 14425 raft_consensus.cc:1217] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:58.695657 14582 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:20:58.696461 14582 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:20:58.700353 14582 ts_tablet_manager.cc:1434] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.002s
I20260709 12:20:58.703003 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-f136fb5f4e574b6db1336e9434176d83 received from {username='slave'} at 127.0.0.1:44068
I20260709 12:20:58.703490 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-f136fb5f4e574b6db1336e9434176d83 on tablet f136fb5f4e574b6db1336e9434176d83 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:20:58.727070 14425 raft_consensus.cc:1217] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:58.822775 14425 raft_consensus.cc:1217] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:58.933015 14425 raft_consensus.cc:1217] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:20:59.292348 14546 raft_consensus.cc:1064] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:59.305999 14546 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:59.364672 14562 raft_consensus.cc:1064] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:59.367568 14562 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:59.439071 14547 raft_consensus.cc:1064] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:59.441793 14547 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:59.479436 14534 raft_consensus.cc:1064] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:20:59.485441 14425 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.487485 14563 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:59.485632 13799 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.496045 14530 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:59.510994 14425 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.517614 14581 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:59.534123 13720 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.542845 13799 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.545434 14563 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:59.549019 13643 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.550022 14425 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.549537 14563 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.553064 13795 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.553889 14530 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:59.577378 14604 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:20:59.593447 14534 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:20:59.623338 13720 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:20:59.627831 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75575.8us
I20260709 12:20:59.611941 14547 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:20:59.632090 13720 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.665232 14425 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:20:59.710621 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80767.4us
I20260709 12:20:59.717724 14422 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.719931 14596 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:20:59.726814 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 114064us
I20260709 12:20:59.738416 13549 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.761852 14562 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.777091 13793 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:20:59.819279 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:20:59.843887 14603 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.845090 14424 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.862843 13643 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.916209 14424 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.919425 13651 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:20:59.927266 13548 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:20:59.937181 14597 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:20:59.953202 13546 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:20:59.989776 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91100.4us
I20260709 12:20:59.994165 13651 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:00.000310 13793 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:00.001489 14425 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.001983 13795 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.004175 14562 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.011083 13795 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:00.018631 13647 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.022423 14610 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.033187 14547 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.035629 13650 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.038455 14608 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.055289 14562 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.063067 14422 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:00.019881 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 324165us
I20260709 12:21:00.100154 14597 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:00.112664 14425 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.117070 14610 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.119449 13548 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.132340 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet f136fb5f4e574b6db1336e9434176d83 with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:21:00.140744 13795 raft_consensus.cc:2955] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.161484 14597 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.166461 13546 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.126822 14425 raft_consensus.cc:2955] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:00.144899 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47591.2us
I20260709 12:21:00.172226 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a27ddca262a4e0e8c46331cd36be4d1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:00.148907 14631 tablet_replica.cc:333] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:00.184043 14530 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:00.195502 13643 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.200848 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a27ddca262a4e0e8c46331cd36be4d1 with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:21:00.207502 13722 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.253922 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f136fb5f4e574b6db1336e9434176d83 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:00.300899 13549 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.348695 14422 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:00.364190 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56406us
I20260709 12:21:00.470765 14631 raft_consensus.cc:2243] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.471573 14631 pending_rounds.cc:70] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Trying to abort 1 pending ops.
I20260709 12:21:00.471879 14631 pending_rounds.cc:77] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305624205067931648 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "f136fb5f4e574b6db1336e9434176d83" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } } }
I20260709 12:21:00.473198 14631 raft_consensus.cc:2917] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 12:21:00.473640 14631 raft_consensus.cc:2889] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:21:00.474035 14631 raft_consensus.cc:2272] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.478219 14631 ts_tablet_manager.cc:1916] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:00.485662 13722 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:00.489857 13793 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.536636 14425 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.540643 14596 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.555823 13795 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.558882 14593 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.564779 14631 ts_tablet_manager.cc:1929] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:00.565320 14631 log.cc:1199] T f136fb5f4e574b6db1336e9434176d83 P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/f136fb5f4e574b6db1336e9434176d83
I20260709 12:21:00.567266 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet f136fb5f4e574b6db1336e9434176d83 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:00.577452 14425 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.567296 14631 tablet_replica.cc:333] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:00.589695 14631 raft_consensus.cc:2243] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.590560 14631 raft_consensus.cc:2272] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.594014 14631 ts_tablet_manager.cc:1916] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:00.610172 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 435988495fcf4cf49f65293e95b5ceb5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:00.610241 14596 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:21:00.612443 13650 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:00.637768 14422 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.638600 14631 ts_tablet_manager.cc:1929] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260709 12:21:00.639091 14631 log.cc:1199] T 0a27ddca262a4e0e8c46331cd36be4d1 P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/0a27ddca262a4e0e8c46331cd36be4d1
I20260709 12:21:00.640892 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 0a27ddca262a4e0e8c46331cd36be4d1 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:00.642030 14616 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.660523 13792 raft_consensus.cc:2955] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.664788 14605 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.682507 14422 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.730810 13549 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:00.737095 13546 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.747852 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 435988495fcf4cf49f65293e95b5ceb5 with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:00.750730 13725 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:00.754081 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e52c8b752b54bd9b959b2363df455a4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:00.791123 14633 tablet_replica.cc:333] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:00.794091 14563 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:00.806528 14633 raft_consensus.cc:2243] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.815016 14633 raft_consensus.cc:2272] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.828599 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e52c8b752b54bd9b959b2363df455a4 with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:00.831314 14633 ts_tablet_manager.cc:1916] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:00.830196 13720 raft_consensus.cc:2955] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:00.852833 14634 tablet_replica.cc:333] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:00.854137 14634 raft_consensus.cc:2243] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:00.856176 14634 raft_consensus.cc:2272] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:00.859896 14634 ts_tablet_manager.cc:1916] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:00.889887 14634 ts_tablet_manager.cc:1929] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:00.890441 14634 log.cc:1199] T 3e52c8b752b54bd9b959b2363df455a4 P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/3e52c8b752b54bd9b959b2363df455a4
I20260709 12:21:00.892832 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet 3e52c8b752b54bd9b959b2363df455a4 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:00.907271 14633 ts_tablet_manager.cc:1929] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:00.921233 14633 log.cc:1199] T 435988495fcf4cf49f65293e95b5ceb5 P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/435988495fcf4cf49f65293e95b5ceb5
I20260709 12:21:00.923559 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 435988495fcf4cf49f65293e95b5ceb5 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:01.940536 13793 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:01.965575 13724 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:01.968055 14638 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:01.969867 13643 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:01.972922 13764 consensus_peers.cc:597] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45dc2f34593649778ec4b80d29638572. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:01.976405 14636 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:02.006539 14636 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.009014 13724 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.012135 13643 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.028539 13720 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.032470 13546 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:02.070580 13690 consensus_peers.cc:597] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 068a8cc7309648e3a661d4d056f11b7a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.075011 13792 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:02.076516 13643 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:02.077986 14596 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:02.080224 14635 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:02.102116 14596 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.108083 13643 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.117777 13792 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.148396 13795 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.167662 13549 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:02.213958 13764 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1d64b0184dc44a1afc1b5885669d80d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.214723 13724 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:02.218952 14636 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:02.223366 13643 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:02.231432 14610 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:02.243673 14610 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.257323 13724 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.262555 13651 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.290781 13549 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.295305 13724 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:21:02.308207 13690 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c6e2c2ada9f4af79430ba4cd2c6558b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:02.312034 13795 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:02.315038 14635 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:02.325922 13651 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:02.328207 14635 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:02.334134 13795 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.334779 14648 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.379359 13651 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.385835 13546 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:02.427580 14655 ts_tablet_manager.cc:933] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:02.430557 14655 tablet_copy_client.cc:323] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:21:02.433104 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 45dc2f34593649778ec4b80d29638572 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:21:02.433712 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 45dc2f34593649778ec4b80d29638572 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-45dc2f34593649778ec4b80d29638572
I20260709 12:21:02.442437 13811 tablet_copy_source_session.cc:215] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:02.452737 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45dc2f34593649778ec4b80d29638572. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:02.478037 14655 tablet_copy_client.cc:806] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:02.478858 14655 tablet_copy_client.cc:670] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:02.483489 14655 tablet_copy_client.cc:538] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:02.493655 14655 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:02.552734 14655 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:02.554339 14655 tablet_bootstrap.cc:492] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:02.555290 14655 ts_tablet_manager.cc:1403] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.062s	user 0.028s	sys 0.000s
I20260709 12:21:02.559170 14655 raft_consensus.cc:359] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.560657 14655 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:02.561784 14655 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.574155 14655 ts_tablet_manager.cc:1434] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:21:02.591998 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-45dc2f34593649778ec4b80d29638572 received from {username='slave'} at 127.0.0.1:44068
I20260709 12:21:02.592505 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-45dc2f34593649778ec4b80d29638572 on tablet 45dc2f34593649778ec4b80d29638572 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:02.671960 14655 ts_tablet_manager.cc:933] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:02.674420 14655 tablet_copy_client.cc:323] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:02.676647 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 068a8cc7309648e3a661d4d056f11b7a from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:02.677268 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 068a8cc7309648e3a661d4d056f11b7a from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-068a8cc7309648e3a661d4d056f11b7a
I20260709 12:21:02.706528 13737 tablet_copy_source_session.cc:215] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:02.711616 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 068a8cc7309648e3a661d4d056f11b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:02.735325 14655 tablet_copy_client.cc:806] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:02.736032 14655 tablet_copy_client.cc:670] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:02.741922 14655 tablet_copy_client.cc:538] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:02.760219 14655 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:02.822503 14655 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:02.823616 14655 tablet_bootstrap.cc:492] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:02.824481 14655 ts_tablet_manager.cc:1403] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.010s
I20260709 12:21:02.837037 14655 raft_consensus.cc:359] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.838034 14655 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:02.838922 14655 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.843080 14663 ts_tablet_manager.cc:933] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:02.851046 14663 tablet_copy_client.cc:323] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:21:02.855852 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet b1d64b0184dc44a1afc1b5885669d80d from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:21:02.856581 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet b1d64b0184dc44a1afc1b5885669d80d from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-b1d64b0184dc44a1afc1b5885669d80d
I20260709 12:21:02.865895 13811 tablet_copy_source_session.cc:215] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:02.870842 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d64b0184dc44a1afc1b5885669d80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:02.876816 14655 ts_tablet_manager.cc:1434] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.052s	user 0.011s	sys 0.000s
I20260709 12:21:02.881738 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-068a8cc7309648e3a661d4d056f11b7a received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:02.882288 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-068a8cc7309648e3a661d4d056f11b7a on tablet 068a8cc7309648e3a661d4d056f11b7a with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:02.887044 14655 ts_tablet_manager.cc:933] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:02.901139 14655 tablet_copy_client.cc:323] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:02.903213 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:02.903785 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 6c6e2c2ada9f4af79430ba4cd2c6558b from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-6c6e2c2ada9f4af79430ba4cd2c6558b
I20260709 12:21:02.912084 14663 tablet_copy_client.cc:806] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:02.913058 14663 tablet_copy_client.cc:670] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:02.923017 14663 tablet_copy_client.cc:538] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:02.923207 13737 tablet_copy_source_session.cc:215] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:02.926703 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:02.935073 14663 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:02.948885 14425 raft_consensus.cc:1217] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:02.970481 14655 tablet_copy_client.cc:806] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:02.971145 14655 tablet_copy_client.cc:670] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:02.976563 14655 tablet_copy_client.cc:538] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:02.980922 14663 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:02.982203 14663 tablet_bootstrap.cc:492] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:02.983502 14663 ts_tablet_manager.cc:1403] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.049s	user 0.021s	sys 0.012s
I20260709 12:21:02.987282 14663 raft_consensus.cc:359] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.988291 14663 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:02.989029 14663 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:02.992699 14663 ts_tablet_manager.cc:1434] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:21:03.000473 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-b1d64b0184dc44a1afc1b5885669d80d received from {username='slave'} at 127.0.0.1:44068
I20260709 12:21:03.002041 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-b1d64b0184dc44a1afc1b5885669d80d on tablet b1d64b0184dc44a1afc1b5885669d80d with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:02.998914 14655 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:03.057824 14655 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:03.058853 14655 tablet_bootstrap.cc:492] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:03.059541 14655 ts_tablet_manager.cc:1403] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.013s
I20260709 12:21:03.062641 14655 raft_consensus.cc:359] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:03.063491 14655 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:03.064133 14655 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:03.066828 14655 ts_tablet_manager.cc:1434] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:21:03.068881 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-6c6e2c2ada9f4af79430ba4cd2c6558b received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:03.069363 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-6c6e2c2ada9f4af79430ba4cd2c6558b on tablet 6c6e2c2ada9f4af79430ba4cd2c6558b with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:03.210410 14422 raft_consensus.cc:1217] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:03.260267 14422 raft_consensus.cc:1217] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:03.446650 14422 raft_consensus.cc:1217] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:03.572238 14610 raft_consensus.cc:1064] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:03.575139 14610 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:03.585350 14422 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.588819 14636 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:03.589986 13649 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.592088 14638 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:21:03.608276 13724 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.614949 14638 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:03.652144 14610 raft_consensus.cc:1064] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:03.654879 14610 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:03.668608 13724 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.665395 14674 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.683462 13651 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.697558 14648 raft_consensus.cc:1064] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:03.700517 14648 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:03.720932 14422 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.722638 14670 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:03.736271 13548 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.752233 13720 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:21:03.783041 14675 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:21:03.793625 14610 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:03.799314 13649 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.813079 14425 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.814504 14422 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.824717 14682 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:21:03.840308 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 64273.1us
I20260709 12:21:03.841710 14676 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:21:03.847522 13793 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.851861 14674 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.859117 13643 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:03.867915 14425 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.891654 13651 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.898984 13720 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.938048 14682 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:03.950680 14681 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:03.960338 13548 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:03.972476 13797 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:03.984114 14425 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:03.989159 13723 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:03.994060 14676 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:03.996733 14610 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:04.005561 14670 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.015039 14425 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.023705 14681 raft_consensus.cc:1064] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:04.022423 13723 raft_consensus.cc:2955] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.031102 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 45dc2f34593649778ec4b80d29638572 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.034858 14681 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:04.084391 14422 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:04.086298 14682 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:04.111155 13797 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:04.113092 13537 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 45dc2f34593649778ec4b80d29638572 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 12:21:04.119369 14424 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:04.124886 14670 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:04.131335 13643 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:04.137938 13549 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:04.140327 14610 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:04.142750 14674 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.151170 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 45dc2f34593649778ec4b80d29638572 with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:04.160363 14648 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.165601 14694 tablet_replica.cc:333] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:04.167016 14694 raft_consensus.cc:2243] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.168059 14694 raft_consensus.cc:2272] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.179373 14694 ts_tablet_manager.cc:1916] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.185310 13651 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.196554 14425 raft_consensus.cc:2955] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.200601 13793 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:04.205107 14424 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.208976 13643 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:04.217866 13650 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.223104 13797 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:04.211834 14693 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:21:04.235495 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b1d64b0184dc44a1afc1b5885669d80d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.244625 13548 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:04.254420 14648 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:04.267194 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet b1d64b0184dc44a1afc1b5885669d80d with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:21:04.279672 14682 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:04.284345 13550 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.288014 14694 ts_tablet_manager.cc:1929] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:04.294009 14694 log.cc:1199] T 45dc2f34593649778ec4b80d29638572 P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/45dc2f34593649778ec4b80d29638572
I20260709 12:21:04.298833 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 45dc2f34593649778ec4b80d29638572 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:04.301934 14696 tablet_replica.cc:333] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:04.303494 14696 raft_consensus.cc:2243] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.304431 14696 raft_consensus.cc:2272] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.310110 14696 ts_tablet_manager.cc:1916] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.325344 13720 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:04.333832 14424 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:04.335559 14682 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:04.345011 14680 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.347374 14425 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.371510 13549 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.390233 13793 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:04.406539 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 068a8cc7309648e3a661d4d056f11b7a with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:04.407935 14694 tablet_replica.cc:333] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:04.410127 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 068a8cc7309648e3a661d4d056f11b7a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.431337 14648 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.433822 14694 raft_consensus.cc:2243] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.434670 14694 raft_consensus.cc:2272] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.439744 14694 ts_tablet_manager.cc:1916] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.442181 14635 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:04.453861 14424 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.470863 13549 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.476130 13793 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.482699 14696 ts_tablet_manager.cc:1929] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:04.483191 14696 log.cc:1199] T b1d64b0184dc44a1afc1b5885669d80d P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/b1d64b0184dc44a1afc1b5885669d80d
I20260709 12:21:04.485173 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet b1d64b0184dc44a1afc1b5885669d80d (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:04.489778 13651 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.520279 14694 ts_tablet_manager.cc:1929] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:04.520891 14694 log.cc:1199] T 068a8cc7309648e3a661d4d056f11b7a P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/068a8cc7309648e3a661d4d056f11b7a
I20260709 12:21:04.522914 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 068a8cc7309648e3a661d4d056f11b7a (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:04.540829 13797 raft_consensus.cc:2955] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.620355 13726 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:04.626183 14424 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:04.628185 14635 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:04.628129 13651 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:04.634093 14635 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:04.635830 14648 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.637935 14425 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.645795 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:04.653477 13651 raft_consensus.cc:2955] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.657099 13546 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:04.674192 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet 6c6e2c2ada9f4af79430ba4cd2c6558b with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:04.678488 14699 tablet_replica.cc:333] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:04.680163 14699 raft_consensus.cc:2243] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:04.680791 14699 raft_consensus.cc:2272] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:04.683928 14699 ts_tablet_manager.cc:1916] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:04.698036 14699 ts_tablet_manager.cc:1929] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:04.698568 14699 log.cc:1199] T 6c6e2c2ada9f4af79430ba4cd2c6558b P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/6c6e2c2ada9f4af79430ba4cd2c6558b
I20260709 12:21:04.700202 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet 6c6e2c2ada9f4af79430ba4cd2c6558b (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:05.757310 13726 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:05.765753 13793 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:05.765910 13651 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:05.766978 13690 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bdd4114edc6d43c08d41462fd399480c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:05.767498 14703 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:05.768455 14648 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:05.778347 14703 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.780493 13650 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.783025 13793 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.798893 13726 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:05.821848 13546 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:05.832630 13690 consensus_peers.cc:597] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcbdced6d0a46b394cae3d101fbbf40. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:05.832921 13799 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:05.835813 14680 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:05.836455 13650 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:05.840507 14703 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:05.852052 14635 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.858773 13793 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.860643 13650 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.881349 13797 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:05.898250 13651 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:05.900285 14707 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:05.902063 13726 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:05.904623 14670 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:05.905047 13764 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0d7e68c4d38b4465937dee955f980255. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:05.909003 13549 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.960902 14610 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.963429 13720 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.986461 13549 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.991407 13651 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:05.992733 13650 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.006556 13726 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:06.007858 13615 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 30d1d057032c4ae0ad5ae21d374a2baa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:06.008773 14530 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:06.010232 13793 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:06.013787 14704 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:06.025671 14704 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:06.027268 13793 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:06.032840 13726 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:06.051735 13550 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:06.269594 14725 ts_tablet_manager.cc:933] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:06.273106 14725 tablet_copy_client.cc:323] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:06.276458 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 2dcbdced6d0a46b394cae3d101fbbf40 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:06.277235 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 2dcbdced6d0a46b394cae3d101fbbf40 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-2dcbdced6d0a46b394cae3d101fbbf40
I20260709 12:21:06.285775 13737 tablet_copy_source_session.cc:215] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:06.289794 14725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dcbdced6d0a46b394cae3d101fbbf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:06.318223 14725 tablet_copy_client.cc:806] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:06.319010 14725 tablet_copy_client.cc:670] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:06.323930 14725 tablet_copy_client.cc:538] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:06.333261 14725 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:06.357872 14725 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.359002 14725 tablet_bootstrap.cc:492] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:06.359817 14725 ts_tablet_manager.cc:1403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.027s	user 0.021s	sys 0.005s
I20260709 12:21:06.363582 14725 raft_consensus.cc:359] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.364758 14725 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:06.365538 14725 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.369825 14725 ts_tablet_manager.cc:1434] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.003s
I20260709 12:21:06.371951 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-2dcbdced6d0a46b394cae3d101fbbf40 received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:06.372479 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-2dcbdced6d0a46b394cae3d101fbbf40 on tablet 2dcbdced6d0a46b394cae3d101fbbf40 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:06.389503 14725 ts_tablet_manager.cc:933] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:06.391584 14725 tablet_copy_client.cc:323] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:06.393175 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet bdd4114edc6d43c08d41462fd399480c from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:06.393946 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet bdd4114edc6d43c08d41462fd399480c from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-bdd4114edc6d43c08d41462fd399480c
I20260709 12:21:06.402465 13737 tablet_copy_source_session.cc:215] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:06.406093 14725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdd4114edc6d43c08d41462fd399480c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:06.443482 14725 tablet_copy_client.cc:806] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:06.444262 14725 tablet_copy_client.cc:670] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:06.450639 14725 tablet_copy_client.cc:538] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:06.467152 14725 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:06.517386 14725 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.518778 14725 tablet_bootstrap.cc:492] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:06.519539 14725 ts_tablet_manager.cc:1403] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.053s	user 0.031s	sys 0.012s
I20260709 12:21:06.522671 14725 raft_consensus.cc:359] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.523506 14725 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:06.524148 14725 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.529690 14725 ts_tablet_manager.cc:1434] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:21:06.531755 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-bdd4114edc6d43c08d41462fd399480c received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:06.533216 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-bdd4114edc6d43c08d41462fd399480c on tablet bdd4114edc6d43c08d41462fd399480c with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:06.542435 14731 ts_tablet_manager.cc:933] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:06.545465 14731 tablet_copy_client.cc:323] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:21:06.547605 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 0d7e68c4d38b4465937dee955f980255 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:21:06.548064 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 0d7e68c4d38b4465937dee955f980255 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-0d7e68c4d38b4465937dee955f980255
I20260709 12:21:06.556383 14725 ts_tablet_manager.cc:933] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:06.557185 13811 tablet_copy_source_session.cc:215] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:06.558704 14725 tablet_copy_client.cc:323] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:06.561681 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 30d1d057032c4ae0ad5ae21d374a2baa from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:06.562503 14731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e68c4d38b4465937dee955f980255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:06.562424 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 30d1d057032c4ae0ad5ae21d374a2baa from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-30d1d057032c4ae0ad5ae21d374a2baa
I20260709 12:21:06.572085 13662 tablet_copy_source_session.cc:215] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:06.577987 14725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30d1d057032c4ae0ad5ae21d374a2baa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:06.592499 14731 tablet_copy_client.cc:806] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:06.593585 14731 tablet_copy_client.cc:670] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:06.605963 14725 tablet_copy_client.cc:806] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:06.606613 14725 tablet_copy_client.cc:670] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:06.610867 14731 tablet_copy_client.cc:538] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:06.612318 14725 tablet_copy_client.cc:538] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:06.622162 14731 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:06.626366 14725 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:06.655968 14731 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.657073 14725 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:06.657224 14731 tablet_bootstrap.cc:492] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:06.658032 14725 tablet_bootstrap.cc:492] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:06.658166 14731 ts_tablet_manager.cc:1403] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.011s
I20260709 12:21:06.658746 14725 ts_tablet_manager.cc:1403] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.006s
I20260709 12:21:06.661680 14725 raft_consensus.cc:359] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.662417 14731 raft_consensus.cc:359] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.663064 14725 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:06.663515 14731 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:06.664424 14731 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.664182 14725 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:06.671231 14731 ts_tablet_manager.cc:1434] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:21:06.672145 14725 ts_tablet_manager.cc:1434] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:21:06.673599 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-0d7e68c4d38b4465937dee955f980255 received from {username='slave'} at 127.0.0.1:44068
I20260709 12:21:06.674106 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-0d7e68c4d38b4465937dee955f980255 on tablet 0d7e68c4d38b4465937dee955f980255 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:06.675088 13662 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-30d1d057032c4ae0ad5ae21d374a2baa received from {username='slave'} at 127.0.0.1:60832
I20260709 12:21:06.675675 13662 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-30d1d057032c4ae0ad5ae21d374a2baa on tablet 30d1d057032c4ae0ad5ae21d374a2baa with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:06.791951 14424 raft_consensus.cc:1217] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:06.859525 14424 raft_consensus.cc:1217] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:06.949555 14425 raft_consensus.cc:1217] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:07.153425 14425 raft_consensus.cc:1217] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:07.246443 14703 raft_consensus.cc:1064] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:07.248747 14703 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:07.256665 14425 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.258479 14648 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.259145 13651 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.259375 13793 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.261121 14648 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.261940 14703 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.273464 14648 raft_consensus.cc:1064] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:07.276185 14648 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:07.288444 14425 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.290140 14703 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:07.296403 13651 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.300714 13799 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.301970 14703 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.308683 13651 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.310467 14635 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.324371 14635 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.331399 14425 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.353282 13549 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.391386 14635 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.405843 13799 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.406520 13797 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.407272 13651 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.438177 14720 raft_consensus.cc:1064] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:07.441581 14720 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:07.455065 14425 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.479159 13546 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.498637 13651 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.499320 14425 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.501004 13726 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.501248 14610 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.503374 14610 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.512327 14754 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:07.520589 13720 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:07.531196 13724 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:07.541031 14738 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.590165 13799 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:07.589243 13720 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.605019 14425 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:07.614509 14635 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:07.617894 14425 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:07.619292 14635 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:07.627399 14648 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:07.629640 14703 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.643729 14424 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.649230 13651 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:07.655398 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2dcbdced6d0a46b394cae3d101fbbf40 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:07.659200 14648 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.664211 13548 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:07.668102 14425 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.671228 13651 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.685122 13550 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.699250 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 2dcbdced6d0a46b394cae3d101fbbf40 with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:07.699884 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bdd4114edc6d43c08d41462fd399480c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:07.703092 14759 tablet_replica.cc:333] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:07.707885 13550 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.722246 14425 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.733274 14635 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:07.740692 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet bdd4114edc6d43c08d41462fd399480c with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:07.752190 13651 raft_consensus.cc:2955] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.754303 14759 raft_consensus.cc:2243] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.755117 14759 raft_consensus.cc:2272] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.755553 14761 tablet_replica.cc:333] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:07.757181 14761 raft_consensus.cc:2243] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:07.758213 14761 raft_consensus.cc:2272] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:07.763034 14761 ts_tablet_manager.cc:1916] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:07.767187 13793 raft_consensus.cc:2955] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.772876 14759 ts_tablet_manager.cc:1916] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:07.775363 14734 raft_consensus.cc:1064] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:07.779390 14734 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:07.816919 14759 ts_tablet_manager.cc:1929] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:07.834523 14759 log.cc:1199] T 2dcbdced6d0a46b394cae3d101fbbf40 P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/2dcbdced6d0a46b394cae3d101fbbf40
I20260709 12:21:07.837476 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 2dcbdced6d0a46b394cae3d101fbbf40 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:07.840016 14761 ts_tablet_manager.cc:1929] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:07.840643 14761 log.cc:1199] T bdd4114edc6d43c08d41462fd399480c P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/bdd4114edc6d43c08d41462fd399480c
I20260709 12:21:07.842504 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet bdd4114edc6d43c08d41462fd399480c (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:07.854507 14425 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.858349 13799 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.859166 14530 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:07.862116 14705 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:07.869246 13726 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:07.909991 14765 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:07.909472 14425 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.906776 14763 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.929301 13548 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.940496 13720 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:07.949972 13799 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.006199 13799 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:08.013406 14425 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:08.015312 14757 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:08.025836 13650 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:08.025089 14754 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.028581 14610 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:08.032795 14425 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.042330 13651 raft_consensus.cc:2955] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.047072 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0d7e68c4d38b4465937dee955f980255 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:08.058871 13548 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:08.081590 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 0d7e68c4d38b4465937dee955f980255 with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:21:08.090144 14769 tablet_replica.cc:333] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:08.091758 14769 raft_consensus.cc:2243] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.092773 14769 raft_consensus.cc:2272] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.097951 14769 ts_tablet_manager.cc:1916] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:08.119839 14769 ts_tablet_manager.cc:1929] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:08.120379 14769 log.cc:1199] T 0d7e68c4d38b4465937dee955f980255 P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/0d7e68c4d38b4465937dee955f980255
I20260709 12:21:08.122422 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 0d7e68c4d38b4465937dee955f980255 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:08.383558 13650 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:08.393484 14425 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:08.395370 14734 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:08.395830 13720 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:08.397863 14734 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:08.407594 13720 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.407902 14425 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.418015 14705 raft_consensus.cc:2955] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.433908 13548 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:08.444725 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 30d1d057032c4ae0ad5ae21d374a2baa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:08.446560 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet 30d1d057032c4ae0ad5ae21d374a2baa with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:08.457868 14770 tablet_replica.cc:333] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:08.459853 14770 raft_consensus.cc:2243] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:08.460705 14770 raft_consensus.cc:2272] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:08.464776 14770 ts_tablet_manager.cc:1916] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:08.486191 14770 ts_tablet_manager.cc:1929] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:08.486617 14770 log.cc:1199] T 30d1d057032c4ae0ad5ae21d374a2baa P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/30d1d057032c4ae0ad5ae21d374a2baa
I20260709 12:21:08.488343 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet 30d1d057032c4ae0ad5ae21d374a2baa (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:09.526400 13650 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:21:09.535095 13615 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 20dc442f4fae4cf49e336a3d350b7760. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.536973 13797 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.538617 13720 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.538735 14734 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:09.540583 14705 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.548431 14734 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.550136 13799 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.565467 13720 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.572131 13722 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:09.574717 13549 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.604530 13799 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:09.604334 13690 consensus_peers.cc:597] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ce599ec23ec4ac0b358f4e47922bc3f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.606678 13651 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.608301 14771 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.609652 14703 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.630769 14703 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.630571 13799 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.670986 13651 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:09.686663 13650 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
W20260709 12:21:09.692389 13615 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de32481a3be147a5b854251468fc667b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.695369 13720 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.691759 13548 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.695510 13799 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.697885 14777 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.699455 14778 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.722191 14705 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.726890 13797 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.733322 13720 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.740325 13650 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:09.761045 13549 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:09.763497 13615 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ff310d454634aa0bbc38f2b233ccb8e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:09.765931 13799 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.768194 14734 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.771555 13720 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:09.773831 14734 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:09.787909 13799 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.792351 14777 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.819914 13722 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:09.844790 13548 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:09.875000 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54789us
I20260709 12:21:09.996042 14796 ts_tablet_manager.cc:933] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:09.998567 14796 tablet_copy_client.cc:323] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:10.000660 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 20dc442f4fae4cf49e336a3d350b7760 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:10.001322 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 20dc442f4fae4cf49e336a3d350b7760 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-20dc442f4fae4cf49e336a3d350b7760
I20260709 12:21:10.010484 13662 tablet_copy_source_session.cc:215] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:10.014530 14796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20dc442f4fae4cf49e336a3d350b7760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:10.033854 14796 tablet_copy_client.cc:806] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:10.034571 14796 tablet_copy_client.cc:670] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:10.042723 14796 tablet_copy_client.cc:538] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:10.053754 14796 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:10.076779 14796 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:10.077903 14796 tablet_bootstrap.cc:492] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:10.078717 14796 ts_tablet_manager.cc:1403] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.025s	user 0.023s	sys 0.004s
I20260709 12:21:10.082065 14796 raft_consensus.cc:359] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:10.082791 14796 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:10.083374 14796 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:10.088447 14796 ts_tablet_manager.cc:1434] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:21:10.091109 13662 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-20dc442f4fae4cf49e336a3d350b7760 received from {username='slave'} at 127.0.0.1:60832
I20260709 12:21:10.091611 13662 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-20dc442f4fae4cf49e336a3d350b7760 on tablet 20dc442f4fae4cf49e336a3d350b7760 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:10.183044 14796 ts_tablet_manager.cc:933] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:10.312705 14796 tablet_copy_client.cc:323] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:10.380420 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 3ff310d454634aa0bbc38f2b233ccb8e from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:10.381220 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 3ff310d454634aa0bbc38f2b233ccb8e from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-3ff310d454634aa0bbc38f2b233ccb8e
W20260709 12:21:10.389024 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 181972us
I20260709 12:21:10.392202 13662 tablet_copy_source_session.cc:215] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:10.396343 14804 ts_tablet_manager.cc:933] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:10.443751 14796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ff310d454634aa0bbc38f2b233ccb8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:10.514822 14811 ts_tablet_manager.cc:933] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:10.539083 14424 raft_consensus.cc:1217] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:21:10.593145 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44595us
I20260709 12:21:10.587630 14811 tablet_copy_client.cc:323] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:10.636376 14804 tablet_copy_client.cc:323] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:10.638584 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 4ce599ec23ec4ac0b358f4e47922bc3f from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:10.639246 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 4ce599ec23ec4ac0b358f4e47922bc3f from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-4ce599ec23ec4ac0b358f4e47922bc3f
W20260709 12:21:10.645226 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55395.7us
I20260709 12:21:10.662807 13662 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet de32481a3be147a5b854251468fc667b from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:10.663635 13662 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet de32481a3be147a5b854251468fc667b from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-de32481a3be147a5b854251468fc667b
I20260709 12:21:10.690012 13737 tablet_copy_source_session.cc:215] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:10.713397 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58277.5us
I20260709 12:21:10.715571 14804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ce599ec23ec4ac0b358f4e47922bc3f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:21:10.730518 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77047.4us
W20260709 12:21:10.743242 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54513.1us
I20260709 12:21:10.772122 14796 tablet_copy_client.cc:806] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:10.786769 14796 tablet_copy_client.cc:670] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:10.787765 13662 tablet_copy_source_session.cc:215] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:10.793159 14811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de32481a3be147a5b854251468fc667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:10.798429 14796 tablet_copy_client.cc:538] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:10.841634 14796 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:10.843453 14804 tablet_copy_client.cc:806] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:10.848776 14804 tablet_copy_client.cc:670] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:10.859949 14804 tablet_copy_client.cc:538] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:10.860368 14811 tablet_copy_client.cc:806] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:10.866887 14811 tablet_copy_client.cc:670] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:10.889489 14811 tablet_copy_client.cc:538] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:10.913672 14811 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:10.914455 14804 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:10.930882 14796 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:10.932070 14796 tablet_bootstrap.cc:492] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:10.932940 14796 ts_tablet_manager.cc:1403] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.092s	user 0.036s	sys 0.018s
I20260709 12:21:10.938108 14796 raft_consensus.cc:359] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:10.939394 14796 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:10.940486 14796 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:10.947347 14796 ts_tablet_manager.cc:1434] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:21:10.949451 13661 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-3ff310d454634aa0bbc38f2b233ccb8e received from {username='slave'} at 127.0.0.1:60832
I20260709 12:21:10.949971 13661 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-3ff310d454634aa0bbc38f2b233ccb8e on tablet 3ff310d454634aa0bbc38f2b233ccb8e with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:11.061538 14804 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:11.062949 14804 tablet_bootstrap.cc:492] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:11.064060 14804 ts_tablet_manager.cc:1403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.181s	user 0.031s	sys 0.005s
I20260709 12:21:11.067945 14804 raft_consensus.cc:359] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:11.078970 14804 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:11.079968 14804 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:11.082775 14804 ts_tablet_manager.cc:1434] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:21:11.068210 14811 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:11.084081 14811 tablet_bootstrap.cc:492] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:11.084957 14811 ts_tablet_manager.cc:1403] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.172s	user 0.030s	sys 0.011s
I20260709 12:21:11.099272 14811 raft_consensus.cc:359] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:11.100550 14811 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:11.099974 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-4ce599ec23ec4ac0b358f4e47922bc3f received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:11.101378 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-4ce599ec23ec4ac0b358f4e47922bc3f on tablet 4ce599ec23ec4ac0b358f4e47922bc3f with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:11.101425 14811 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:11.104967 14811 ts_tablet_manager.cc:1434] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:21:11.109372 13661 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-de32481a3be147a5b854251468fc667b received from {username='slave'} at 127.0.0.1:60832
I20260709 12:21:11.109880 13661 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-de32481a3be147a5b854251468fc667b on tablet de32481a3be147a5b854251468fc667b with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:11.147969 14777 raft_consensus.cc:1064] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:11.151484 14777 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:11.164330 14425 raft_consensus.cc:1217] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:11.187882 14425 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.189465 13797 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.189474 13726 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.192157 14778 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:11.196475 14778 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:11.205418 14777 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:11.247728 14425 raft_consensus.cc:1217] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:11.250276 14810 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.261090 13792 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.270349 14425 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.285709 14423 raft_consensus.cc:1217] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:11.289939 13548 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.313512 13726 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.356653 13643 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:11.365181 14425 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:11.365927 13792 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:11.366786 14778 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:11.368499 14810 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:11.379189 14777 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.389238 13793 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.391650 14425 raft_consensus.cc:2955] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.410681 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 20dc442f4fae4cf49e336a3d350b7760 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:11.417263 13549 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.439697 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 20dc442f4fae4cf49e336a3d350b7760 with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
W20260709 12:21:11.456084 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8500 after lost signal to thread 12472
W20260709 12:21:11.457660 13748 debug-util.cc:398] Leaking SignalData structure 0x7b080012cbc0 after lost signal to thread 13584
W20260709 12:21:11.459266 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800322c00 after lost signal to thread 13676
W20260709 12:21:11.460485 13748 debug-util.cc:398] Leaking SignalData structure 0x7b080032bd40 after lost signal to thread 13751
W20260709 12:21:11.461680 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08002c7900 after lost signal to thread 13825
W20260709 12:21:11.463099 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08002f01e0 after lost signal to thread 14449
I20260709 12:21:11.527339 14831 tablet_replica.cc:333] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:11.528795 14831 raft_consensus.cc:2243] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:11.534412 14831 raft_consensus.cc:2272] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:11.538234 14831 ts_tablet_manager.cc:1916] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:11.570284 14831 ts_tablet_manager.cc:1929] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:11.570874 14831 log.cc:1199] T 20dc442f4fae4cf49e336a3d350b7760 P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/20dc442f4fae4cf49e336a3d350b7760
I20260709 12:21:11.580910 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 20dc442f4fae4cf49e336a3d350b7760 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:11.671309 14810 raft_consensus.cc:1064] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:11.673863 14810 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:11.690949 14423 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.692822 13792 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.697443 14778 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:11.698964 14777 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:11.703827 14746 raft_consensus.cc:1064] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:11.709172 14746 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:11.712345 13726 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.717351 14778 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:11.769395 14778 raft_consensus.cc:1064] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:11.773936 14778 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:11.803010 14423 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.805517 13793 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.805737 14425 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:11.808537 14778 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:11.807740 14816 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:11.815642 14810 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:11.880848 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:11.922927 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69681.1us
I20260709 12:21:11.963766 14810 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:11.985988 13650 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:12.035964 13792 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.047003 13550 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:12.054592 14777 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.062916 13792 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.079841 13797 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:12.081655 13724 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:12.090178 14423 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.111268 13724 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:12.120081 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47512us
I20260709 12:21:12.235280 14777 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:12.239169 14703 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:12.208598 14423 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.253484 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:12.254427 13650 raft_consensus.cc:606] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:12.273052 13651 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.289103 14854 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:12.300022 13549 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:12.315661 13718 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:12.324477 14810 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:12.343667 13724 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.348210 14423 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:12.361907 14852 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:12.366461 13643 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.369246 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:12.370831 13651 raft_consensus.cc:606] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:12.375626 13792 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:12.377343 14853 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:12.387364 14423 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:12.387681 14854 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.399091 14423 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.399881 14849 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:12.410768 13792 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.419421 14808 raft_consensus.cc:993] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:12.420280 14808 raft_consensus.cc:1081] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:12.429303 14425 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.436534 13797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:12.404858 14851 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.470007 13793 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.464546 14424 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.440888 14808 raft_consensus.cc:2955] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.504185 13726 raft_consensus.cc:2955] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.509279 13549 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.518386 13797 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:12.518920 13797 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.523415 13649 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.539943 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ff310d454634aa0bbc38f2b233ccb8e with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:21:12.542486 14849 raft_consensus.cc:993] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:12.543466 14809 raft_consensus.cc:1081] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:12.545636 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:12.546427 13800 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:12.546819 13800 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.554104 13797 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.559633 13549 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.568495 13797 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:12.569531 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:12.576414 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:12.577167 14425 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.586931 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet de32481a3be147a5b854251468fc667b with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:12.596259 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet de32481a3be147a5b854251468fc667b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:12.593662 13549 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:12.601897 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ff310d454634aa0bbc38f2b233ccb8e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:12.616993 13649 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:12.617595 13649 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:12.618685 13649 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.620358 13649 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.627521 14425 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:12.629082 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:12.630228 14817 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:12.630291 13649 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:12.630868 14817 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:12.631779 14817 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.627504 14859 tablet_replica.cc:333] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:12.639997 13726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:12.640667 13726 raft_consensus.cc:606] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:12.613426 13800 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:12.664948 14860 tablet_replica.cc:333] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:12.666160 14860 raft_consensus.cc:2243] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.667079 14860 raft_consensus.cc:2272] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.667725 14859 raft_consensus.cc:2243] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.668450 14859 raft_consensus.cc:2272] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.670619 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:12.672048 13643 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:12.672946 13643 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:12.674090 13800 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:12.679325 14859 ts_tablet_manager.cc:1916] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:12.683509 13726 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.685601 13546 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:12.694519 14860 ts_tablet_manager.cc:1916] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:12.699662 13643 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:12.702201 13643 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.703236 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:12.705782 13718 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.721692 14425 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:12.727787 14703 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:12.726020 13643 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:12.747465 14803 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.747087 14424 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.747099 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:12.763055 14720 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:12.763851 14720 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:12.764731 14720 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:12.770033 13718 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:12.795617 13800 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:12.818136 14746 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:12.836086 13800 raft_consensus.cc:2955] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.864962 13548 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:12.869488 14859 ts_tablet_manager.cc:1929] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:12.872265 14859 log.cc:1199] T 3ff310d454634aa0bbc38f2b233ccb8e P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/3ff310d454634aa0bbc38f2b233ccb8e
I20260709 12:21:12.878589 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 3ff310d454634aa0bbc38f2b233ccb8e (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:12.882642 13546 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:12.892398 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:12.893343 13792 raft_consensus.cc:606] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:12.898622 14860 ts_tablet_manager.cc:1929] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:12.900250 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4ce599ec23ec4ac0b358f4e47922bc3f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:12.899175 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ce599ec23ec4ac0b358f4e47922bc3f with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:12.901007 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:12.908150 13643 raft_consensus.cc:606] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:12.910686 14864 tablet_replica.cc:333] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:12.912835 14864 raft_consensus.cc:2243] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:12.913558 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:12.914494 13643 raft_consensus.cc:606] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:12.916715 14864 raft_consensus.cc:2272] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:12.921303 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:12.922071 13643 raft_consensus.cc:606] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:12.924039 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:21:12.926153 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:12.931747 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:12.941763 14864 ts_tablet_manager.cc:1916] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:12.942133 14860 log.cc:1199] T de32481a3be147a5b854251468fc667b P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/de32481a3be147a5b854251468fc667b
I20260709 12:21:12.944878 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet de32481a3be147a5b854251468fc667b (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:12.960933 14803 raft_consensus.cc:993] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:12.961508 14803 raft_consensus.cc:1081] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:12.963624 14423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48712
I20260709 12:21:12.964280 14423 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:12.964711 14423 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.970148 14864 ts_tablet_manager.cc:1929] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:12.973959 14864 log.cc:1199] T 4ce599ec23ec4ac0b358f4e47922bc3f P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/4ce599ec23ec4ac0b358f4e47922bc3f
I20260709 12:21:12.978328 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 4ce599ec23ec4ac0b358f4e47922bc3f (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:12.981720 14423 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.986019 14423 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:12.987085 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:12.987752 13792 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:12.989281 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:12.990082 13718 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:12.990451 13718 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:12.991547 13718 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:12.992957 13718 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.000316 14808 raft_consensus.cc:993] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:13.000942 14808 raft_consensus.cc:1081] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:13.003371 13724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:13.003933 13724 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.004297 13724 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.005126 13718 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:13.006462 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:13.007536 14850 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.012689 13792 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:13.018688 13724 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.023571 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:13.024264 13643 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:13.024580 13643 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:13.024977 14850 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:13.025353 13643 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.026335 13643 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.026007 14850 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:13.035148 13724 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:13.035471 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:13.036165 13792 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.047336 13643 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:13.051950 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:13.053207 14854 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.053874 14854 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:13.054955 14854 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.056344 13792 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:13.061910 13548 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.073455 13546 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.207755 14817 raft_consensus.cc:993] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:13.208926 13650 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:13.209909 14817 raft_consensus.cc:1081] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:13.210824 14720 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:13.218151 13724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:21:13.218973 13724 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.219368 13724 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.222626 14425 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:13.227407 13718 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.231498 13724 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.237073 14849 raft_consensus.cc:993] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:13.237668 14849 raft_consensus.cc:1081] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:13.237577 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:13.238281 13650 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.239238 13724 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:13.227195 14840 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:13.240928 14840 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.253417 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:13.254205 13800 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:13.254547 13800 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:13.255436 13800 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.257114 13793 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:13.257843 13793 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.258328 13793 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.270035 13800 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.281774 13793 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.286314 13793 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:13.288182 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:13.294747 13718 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.299213 13800 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:13.300554 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:13.301437 14803 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.302107 14803 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:13.303038 14803 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.305346 13718 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:13.308000 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:13.310645 14720 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.328671 14845 raft_consensus.cc:993] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:13.329296 14845 raft_consensus.cc:1081] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:13.339347 14720 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:13.340775 14720 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.290123 13650 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
W20260709 12:21:13.381289 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47054.6us
I20260709 12:21:13.383625 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:13.384538 13651 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:13.385215 13651 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:13.386493 13651 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:13.393671 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58180.5us
I20260709 12:21:13.428210 13548 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.430962 13800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:13.431751 13800 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:13.432189 13800 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.439036 13650 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.442649 13651 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.442762 13800 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.453590 14877 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:13.454752 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71960.1us
I20260709 12:21:13.476919 13724 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:13.479055 14850 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:13.486235 13800 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:13.486155 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:13.487447 13650 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:13.487834 13650 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:13.488641 13650 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.501132 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:13.502434 13718 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.510664 13718 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:13.512341 13766 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:13.512491 13650 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:13.513386 14878 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:13.514294 14878 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:13.517658 14878 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:13.543759 13650 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:13.568076 13546 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:13.584241 13792 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.590348 14854 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:13.627547 13649 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.632244 14703 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.635445 13651 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:13.666194 13548 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:13.677309 13800 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:13.686349 14882 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:13.759007 13792 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.760550 14746 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.761358 13651 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.763409 14877 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:13.779419 13643 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.784083 14703 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:13.871008 13718 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:13.871950 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 63389.5us
I20260709 12:21:13.885272 14875 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:13.962980 13720 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:13.971171 14878 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:14.004014 13643 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:14.008046 14878 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:14.034181 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:14.103922 13720 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:21:14.114353 13690 consensus_peers.cc:597] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89717fd329da4af68d87ae1ef1b104fb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:14.119568 13792 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:14.120891 13643 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:14.119531 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:14.122185 13650 raft_consensus.cc:606] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:14.123361 14862 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:14.124646 14703 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:14.126861 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:14.127534 13650 raft_consensus.cc:606] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:14.138198 14746 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.150766 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:14.152165 13650 raft_consensus.cc:606] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:14.157753 13650 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.187479 13549 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.193837 13720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48612be3c56f482c81533c30abebe60b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:14.197232 13720 raft_consensus.cc:606] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:14.210587 13718 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:14.262105 14854 raft_consensus.cc:993] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:14.262988 14854 raft_consensus.cc:1081] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
W20260709 12:21:14.269766 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50894.9us
W20260709 12:21:14.271221 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90684.8us
I20260709 12:21:14.273918 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:14.274603 13793 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:14.281256 13647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48612be3c56f482c81533c30abebe60b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54756
I20260709 12:21:14.281952 13647 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.282349 13647 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.282502 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:14.283160 13643 raft_consensus.cc:606] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:14.288785 13793 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.291160 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de32481a3be147a5b854251468fc667b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:14.292048 13643 raft_consensus.cc:606] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:14.294059 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:21:14.296258 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:21:14.303928 13690 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6966d62b938845429a9aa8feb8fe2b9b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:14.312929 13647 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:14.317276 13800 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:14.321276 13647 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:14.324692 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:14.342449 13800 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.327745 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:14.330690 14862 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:14.336205 13647 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:14.352458 14862 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:14.357527 14849 raft_consensus.cc:993] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.358120 14849 raft_consensus.cc:1081] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.336412 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:14.360015 13720 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.360726 13720 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:14.367357 14425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:14.369568 14425 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.370249 14425 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.361626 13720 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:14.372979 13720 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.385900 14899 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.401575 13799 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.411446 13720 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:14.417413 14425 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.426028 13647 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.428361 14425 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:14.368112 13800 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:14.436877 13792 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:14.441157 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:14.442510 14808 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.435355 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:14.443054 14808 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.443229 13650 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.443655 13650 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.444514 13650 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.444978 14808 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:14.464785 14851 raft_consensus.cc:993] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.465333 14851 raft_consensus.cc:1081] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.468295 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:14.468791 14425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:14.472319 13720 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.472788 14425 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.473712 14425 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:14.471325 13598 auto_rebalancer.cc:629] Failed to schedule move for tablet 749228d3459a444a9382bc38bbfcbbee: Remote error: Service unavailable: leader transfer in progress
I20260709 12:21:14.445734 13650 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.479598 13718 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:14.486841 13546 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.489562 13720 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.491585 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:14.494745 14900 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.495587 14900 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:14.496650 14900 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.498638 14425 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.497915 13650 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.514300 14425 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:14.518221 14851 raft_consensus.cc:993] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.518777 14851 raft_consensus.cc:1081] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.520071 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:14.520756 13792 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.525408 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:14.526176 13650 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.526579 13650 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.529904 13650 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.531883 13650 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.535187 13800 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:14.536199 13651 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:14.538389 14746 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:14.539480 14899 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:21:14.546692 13690 consensus_peers.cc:597] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f8f1b91eeaf4fc28fa45d4b45b907cd. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:14.561475 14853 raft_consensus.cc:993] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.562495 14849 raft_consensus.cc:1081] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.565127 14422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de32481a3be147a5b854251468fc667b"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:14.565830 14422 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.566293 14422 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.572811 13792 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.574146 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:14.584359 14425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:14.567427 14853 raft_consensus.cc:993] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.596331 14853 raft_consensus.cc:1081] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:14.598968 14424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:14.599591 14424 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.599979 14424 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.601049 13550 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.602478 14422 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.604997 13549 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:14.608405 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:14.609040 13647 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.609354 13647 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.591050 14425 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.610399 13647 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.612887 14425 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.613534 13647 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.617717 13650 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.589919 14900 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.646087 14424 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.650633 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:14.651340 13650 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.651705 13650 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.652484 13650 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.653600 13650 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.650735 14425 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.656651 13651 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.659022 14422 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:14.657565 14899 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.646916 14900 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:14.661007 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:14.661665 13720 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.669004 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:14.669723 13643 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.670033 13643 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.670849 13643 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.671957 13643 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.661626 14900 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.682972 13643 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.684196 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:14.685115 14882 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.685825 14882 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:14.686690 14882 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.687909 13720 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.689245 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:14.690430 14903 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.702044 13650 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.703409 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:14.705154 14425 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:14.714329 14424 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:14.716385 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.717837 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:14.718494 13720 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.723724 14918 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.726735 14918 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:14.737294 14919 ts_tablet_manager.cc:933] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:14.736209 14918 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.741406 14903 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:14.744725 14903 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:14.747278 13546 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.759215 13799 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.767814 13548 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:14.783310 13720 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.810791 14919 tablet_copy_client.cc:323] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:14.840664 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:14.841566 13800 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.849318 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 89717fd329da4af68d87ae1ef1b104fb from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:14.850054 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 89717fd329da4af68d87ae1ef1b104fb from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-89717fd329da4af68d87ae1ef1b104fb
I20260709 12:21:14.857887 13800 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:14.872380 14897 raft_consensus.cc:993] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:14.873023 14897 raft_consensus.cc:1081] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:14.876734 13643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:14.877390 13643 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:14.877928 13643 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.879257 13737 tablet_copy_source_session.cc:215] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:14.883267 14919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89717fd329da4af68d87ae1ef1b104fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:14.870110 13550 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.898624 13550 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.903101 13643 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:14.901624 13550 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:14.909096 13643 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:14.917711 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:14.918596 13800 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:14.919155 13800 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:14.922652 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 151182us
I20260709 12:21:14.923982 13800 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:14.926442 13800 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.935050 13800 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:14.941370 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:14.954568 14923 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:21:14.956670 14922 ts_tablet_manager.cc:933] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:14.961616 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:14.962353 13718 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:14.951638 14808 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:14.970701 14808 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:14.971765 14808 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:14.972230 14919 tablet_copy_client.cc:806] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:14.973048 14919 tablet_copy_client.cc:670] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:14.976507 13718 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:14.985980 14919 tablet_copy_client.cc:538] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:14.992147 13726 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:15.002084 13550 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:15.003469 14919 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:15.019441 14922 tablet_copy_client.cc:323] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:21:15.021821 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 6966d62b938845429a9aa8feb8fe2b9b from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:15.022537 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 6966d62b938845429a9aa8feb8fe2b9b from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-6966d62b938845429a9aa8feb8fe2b9b
I20260709 12:21:15.043826 14808 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:15.055032 13799 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:15.057224 14925 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:15.064807 13720 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.072995 14918 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:15.074489 13737 tablet_copy_source_session.cc:215] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:15.076879 13643 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.080206 14903 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:15.092702 14922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6966d62b938845429a9aa8feb8fe2b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:15.194432 13799 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.235512 14935 ts_tablet_manager.cc:933] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:15.322438 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 190740us
I20260709 12:21:15.254209 14916 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
W20260709 12:21:15.444590 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57651.8us
I20260709 12:21:15.446918 13726 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.450543 14935 tablet_copy_client.cc:323] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
W20260709 12:21:15.494338 14919 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:21:15.549882 13726 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.551769 14918 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:15.571870 14916 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:21:15.583724 14936 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.121s	user 0.000s	sys 0.005s
I20260709 12:21:15.600492 14919 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:15.601925 14919 tablet_bootstrap.cc:492] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:15.606369 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:21:15.607577 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-2f8f1b91eeaf4fc28fa45d4b45b907cd
I20260709 12:21:15.628346 14919 ts_tablet_manager.cc:1403] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.625s	user 0.041s	sys 0.028s
I20260709 12:21:15.646831 14922 tablet_copy_client.cc:806] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:15.648279 14922 tablet_copy_client.cc:670] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:15.633517 14919 raft_consensus.cc:359] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:15.668216 14919 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:15.669261 14919 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:15.672816 14919 ts_tablet_manager.cc:1434] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.044s	user 0.010s	sys 0.000s
I20260709 12:21:15.622089 13737 tablet_copy_source_session.cc:215] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:15.822242 14903 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.009s	sys 0.023s
W20260709 12:21:15.823550 14903 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.010s	sys 0.023s
I20260709 12:21:15.834137 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:15.835414 14853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.001s	sys 0.030s
W20260709 12:21:15.846372 14853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.002s	sys 0.031s
W20260709 12:21:15.840924 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 427706us
W20260709 12:21:15.837416 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 419458us
W20260709 12:21:15.859673 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 427439us
I20260709 12:21:15.873129 13643 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.874872 13652 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.878810 13651 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.873202 13647 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:15.904922 13795 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:21:15.909647 14388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.007s	sys 0.005s
W20260709 12:21:15.910584 14388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.008s	sys 0.005s
I20260709 12:21:15.943962 14941 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:21:15.945273 14956 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:21:15.945211 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.002s	sys 0.021s
W20260709 12:21:15.952977 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.003s	sys 0.022s
W20260709 12:21:16.003661 14959 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.144s	user 0.000s	sys 0.003s
I20260709 12:21:16.024209 13797 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.047254 14956 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:16.060590 14956 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:16.062394 14941 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:16.090870 14949 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:16.091115 14935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:16.091895 14949 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:16.095932 14949 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:16.102177 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49305us
W20260709 12:21:16.110483 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52862.4us
I20260709 12:21:16.141314 14924 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:16.142174 14924 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:16.145005 14425 raft_consensus.cc:1217] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:21:16.146190 14924 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:16.172559 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44153.5us
I20260709 12:21:16.182327 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-89717fd329da4af68d87ae1ef1b104fb received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:16.183131 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-89717fd329da4af68d87ae1ef1b104fb on tablet 89717fd329da4af68d87ae1ef1b104fb with peer b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:16.220299 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76335us
W20260709 12:21:16.254915 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67416.7us
I20260709 12:21:16.317523 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:16.319476 14922 tablet_copy_client.cc:538] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:16.319810 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:16.328769 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:16.335093 14970 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 12:21:16.355698 14972 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.077s	user 0.001s	sys 0.002s
I20260709 12:21:16.404435 14925 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:21:16.426097 13721 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:16.436439 14935 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48712 (request call id 264) took 1301 ms. Trace:
W20260709 12:21:16.452711 14976 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.187s	user 0.002s	sys 0.000s
I20260709 12:21:16.476145 14914 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:16.437057 14935 rpcz_store.cc:276] 0709 12:21:15.135401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:15.136343 (+   942us) service_pool.cc:224] Handling call
0709 12:21:15.236004 (+ 99661us) ts_tablet_manager.cc:934] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
0709 12:21:16.088803 (+852799us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:16.219891 (+131088us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:16.220048 (+   157us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:16.435434 (+215386us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:16.435805 (+   371us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:16.436188 (+   383us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:16.436309 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":421,"tablet-copy.queue_time_us":36149,"thread_start_us":202848,"threads_started":2}
I20260709 12:21:16.478686 14935 tablet_copy_client.cc:806] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:16.476953 14914 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:16.480075 14935 tablet_copy_client.cc:670] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:16.492996 14914 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:16.513592 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:16.541685 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:16.554572 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55168.6us
I20260709 12:21:16.566762 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:16.569397 14954 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:21:16.573438 14949 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:16.585429 14935 tablet_copy_client.cc:538] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:16.593587 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:16.594501 13722 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:16.585440 14922 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:16.612113 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:16.617431 14899 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:16.678131 14935 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:16.716691 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:16.735213 14899 raft_consensus.cc:1064] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:16.737262 14897 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:16.737741 14897 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:16.738125 14897 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:16.739478 14899 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:16.795953 14423 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:16.798365 14854 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:16.801544 13793 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:16.802480 13649 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:16.766399 14897 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:16.804484 14899 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:21:16.814129 14854 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:21:16.830742 14897 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:16.841787 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:16.844163 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:16.847987 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:16.851442 14877 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:16.900151 14935 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:16.901247 14935 tablet_bootstrap.cc:492] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:16.902002 14935 ts_tablet_manager.cc:1403] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.224s	user 0.028s	sys 0.033s
I20260709 12:21:16.906805 14935 raft_consensus.cc:359] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:16.907738 14935 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:16.908463 14935 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:16.911451 14935 ts_tablet_manager.cc:1434] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:21:16.914249 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-2f8f1b91eeaf4fc28fa45d4b45b907cd received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:16.914729 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-2f8f1b91eeaf4fc28fa45d4b45b907cd on tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:16.933089 14949 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:16.945326 13649 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:16.950199 14922 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:16.983856 14922 tablet_bootstrap.cc:492] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:16.984905 14922 ts_tablet_manager.cc:1403] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.400s	user 0.031s	sys 0.006s
I20260709 12:21:16.989113 14922 raft_consensus.cc:359] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:16.990829 13727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:16.991608 13727 raft_consensus.cc:606] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:16.990047 14922 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:16.993623 14922 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:16.998101 14922 ts_tablet_manager.cc:1434] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 12:21:16.998175 13647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:16.999042 13647 raft_consensus.cc:606] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:17.001116 13737 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-6966d62b938845429a9aa8feb8fe2b9b received from {username='slave'} at 127.0.0.1:50270
I20260709 12:21:17.002462 13737 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-6966d62b938845429a9aa8feb8fe2b9b on tablet 6966d62b938845429a9aa8feb8fe2b9b with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:17.018043 14423 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:17.020009 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:17.023005 13794 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:17.024046 13793 raft_consensus.cc:606] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:17.029407 13550 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.038437 14987 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:17.039743 14954 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:17.044308 14954 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:17.045269 14954 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.046800 14954 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:17.049208 13720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:17.050232 13720 raft_consensus.cc:606] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:17.056936 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:17.062508 13796 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:17.057713 13794 raft_consensus.cc:606] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:17.074826 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:17.088749 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:17.091481 14853 raft_consensus.cc:993] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:17.094743 14853 raft_consensus.cc:1081] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:17.107900 14423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:17.108600 14423 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.108965 14423 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.124233 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:17.140184 14854 raft_consensus.cc:993] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:17.140775 14854 raft_consensus.cc:1081] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:17.141485 14423 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:17.143052 13794 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:17.143675 13794 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.144061 13794 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.153488 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:17.153761 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:17.154369 13720 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.155548 14423 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:17.154386 13650 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:17.158766 13650 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:17.159698 13650 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:17.161270 13650 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.161289 13794 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.164867 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:17.165550 13649 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.166411 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:17.167003 13727 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:17.167273 13727 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:17.168140 13727 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.169276 13727 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.172086 13720 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:17.173629 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:17.177919 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:17.180173 14903 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:17.164500 13794 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:17.198689 14903 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:17.199939 14903 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:17.187669 14423 raft_consensus.cc:1217] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:21:17.206728 14877 raft_consensus.cc:993] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:17.207370 14877 raft_consensus.cc:1081] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:17.211067 13651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:17.211685 13651 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.212042 13651 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.217972 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:17.220166 14897 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:17.220927 14914 raft_consensus.cc:993] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:17.221633 14937 raft_consensus.cc:1081] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:17.221525 14897 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:17.223758 14897 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.225399 13643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:17.226006 13643 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.226454 13643 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.229946 13649 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:17.242043 13650 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:17.246701 14423 raft_consensus.cc:1217] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:17.245743 13651 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.242429 13643 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.270017 13643 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:17.270195 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:17.270952 13720 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.275700 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:17.276413 13727 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.278673 13651 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:17.284832 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:17.285557 13794 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:17.285902 13794 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:17.286826 13794 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.288272 13794 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.280205 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:17.291021 13793 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:17.291340 13793 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:17.292161 13793 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.293452 13793 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.311786 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:17.311650 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:17.317507 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:17.327004 14985 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:17.327673 14985 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:17.328725 14985 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.338563 13722 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:17.344664 13793 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:17.346097 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:17.346936 14423 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:17.347342 14954 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:17.347918 14954 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:17.351280 14955 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:17.321571 13545 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:17.327337 13549 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:17.348971 14954 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:17.370095 14949 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:17.375102 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:17.384953 14425 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:17.419724 13793 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:17.425148 14854 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:17.435992 13794 raft_consensus.cc:2955] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:17.452217 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 89717fd329da4af68d87ae1ef1b104fb with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:17.441025 13549 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.478484 13545 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:17.509306 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 89717fd329da4af68d87ae1ef1b104fb with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:17.515977 14949 raft_consensus.cc:993] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:17.516793 14949 raft_consensus.cc:1081] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:17.521389 13794 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:17.522104 13794 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:17.522545 13794 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.530674 13794 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:17.528712 15005 tablet_replica.cc:333] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:17.532948 15005 raft_consensus.cc:2243] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:17.533824 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:17.534765 13651 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.535722 13547 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.537074 15005 raft_consensus.cc:2272] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:17.543985 15005 ts_tablet_manager.cc:1916] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:17.554159 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:17.557898 13720 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:17.558231 13720 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:17.559111 13720 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:17.560633 13720 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:17.556090 13794 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:17.557219 13651 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:17.578742 13767 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:17.579764 14877 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:17.580248 14877 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:17.581282 14877 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:17.600646 15005 ts_tablet_manager.cc:1929] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:21:17.601673 15005 log.cc:1199] T 89717fd329da4af68d87ae1ef1b104fb P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/89717fd329da4af68d87ae1ef1b104fb
I20260709 12:21:17.604598 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 89717fd329da4af68d87ae1ef1b104fb (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:17.637949 13720 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:17.646873 13547 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:17.859547 13720 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.865947 14987 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:17.869937 13796 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.874464 13720 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:17.875638 14954 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:17.907593 14903 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:17.911794 13720 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.920084 14924 raft_consensus.cc:1064] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:17.925936 13794 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.961640 14853 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:17.963281 13643 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:17.939769 14877 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:17.954056 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 219929us
I20260709 12:21:17.983052 13723 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:17.986238 13651 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:18.026882 14966 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:18.028998 13720 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:18.037034 14897 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:18.056589 14937 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:21:18.116142 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48123.3us
I20260709 12:21:18.122960 15023 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:18.056348 14924 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:18.199753 14425 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Advancing to term 2
I20260709 12:21:18.212273 14424 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:18.214596 15007 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:21:18.212349 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83423.3us
I20260709 12:21:18.232486 13650 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:18.236071 13794 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:18.236500 14425 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:18.260306 13643 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:18.279258 14937 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:18.307121 15027 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:21:18.313411 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46105.3us
I20260709 12:21:18.317030 14924 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:18.328848 14854 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:18.341092 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:18.345767 15024 raft_consensus.cc:1064] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:18.349022 15024 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:18.383399 14424 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:18.384521 13722 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:18.385301 14897 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:18.386317 14937 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:18.398172 14854 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:18.399039 14854 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.403386 14854 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.413223 14937 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.414943 15023 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:18.415761 15023 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.443482 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:18.444247 13651 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:18.419315 15029 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:18.457425 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:18.464424 14425 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.472807 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:18.473552 13727 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:18.474201 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:18.481367 14924 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:18.481834 14924 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:18.462124 15023 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.484719 14924 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:18.548805 13649 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:18.548215 15010 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:18.535514 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50300.8us
I20260709 12:21:18.545935 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:18.567955 13796 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.461534 15029 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.579836 13799 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:18.583847 15029 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:18.598454 13720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:18.599445 13720 raft_consensus.cc:606] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:18.606338 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144492us
I20260709 12:21:18.608245 13550 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.610170 14924 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.612687 14924 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.614441 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:18.614835 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 127773us
I20260709 12:21:18.633224 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:18.635119 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:18.635864 13720 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:18.637473 13720 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.646924 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:18.647697 13651 raft_consensus.cc:606] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:18.658401 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:18.659413 13794 raft_consensus.cc:606] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:18.660295 15041 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:18.639382 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:18.687927 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:18.688757 13799 raft_consensus.cc:606] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:18.690539 14425 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.695077 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:18.696079 13650 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:18.697722 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:18.699853 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:18.700780 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:18.726481 13545 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:18.738613 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:18.753312 14937 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:18.753962 14937 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:18.754431 14937 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:18.766707 13649 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:18.767350 14985 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:18.769675 14985 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:18.770051 14985 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:18.784049 14985 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.785697 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:18.786326 14985 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.788374 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:18.790901 13650 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.795955 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:18.797122 14924 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:18.797698 14924 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:18.799039 14924 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.828387 14877 raft_consensus.cc:993] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:18.828536 15021 raft_consensus.cc:993] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:18.828992 14877 raft_consensus.cc:1081] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:18.829217 15021 raft_consensus.cc:1081] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:18.828685 13651 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.832461 15023 raft_consensus.cc:993] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:18.833323 15023 raft_consensus.cc:1081] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:18.846961 14937 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.848429 13793 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:18.851702 13651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:18.852425 13651 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:18.852804 13651 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:18.859472 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:18.860538 13649 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:18.872957 14937 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:18.874583 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:18.879184 13720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:18.879794 13720 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:18.880164 13720 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:18.880954 13722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:18.881557 13722 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:18.881908 13722 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:18.885181 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:18.886566 14985 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:18.889051 13651 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.891211 13651 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.892230 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:18.892851 13793 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:18.893150 13793 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:18.893879 13793 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.894649 13720 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.895613 13793 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:18.896607 13720 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.903972 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:18.904656 13651 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:18.904994 13651 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:18.908306 13651 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.909683 13651 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:18.919282 13722 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.917431 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:18.920459 13793 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:18.924352 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:18.921617 13723 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:18.925743 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:18.926275 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:18.927273 15040 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:18.927711 15040 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:18.928561 15040 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.931771 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:18.932396 13796 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:18.935266 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:18.935977 13649 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:18.936265 13649 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:18.937031 13649 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.938402 13799 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:18.938086 13649 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:18.945901 14424 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:18.952862 15024 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:18.963037 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:18.965406 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:18.964349 14897 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:18.971669 13723 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:18.979609 13718 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:18.980559 14854 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:18.981041 14854 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:18.985616 13722 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:18.984539 13796 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:18.980448 14424 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:18.992383 14897 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:18.988312 14854 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:18.969041 14953 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.013836 13649 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:18.989384 13549 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.028251 15007 raft_consensus.cc:993] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:19.028788 15007 raft_consensus.cc:1081] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:19.037541 13722 raft_consensus.cc:2955] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.056173 15028 raft_consensus.cc:993] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:19.056787 15028 raft_consensus.cc:1081] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:19.059689 13649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:19.060326 13649 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:19.060746 13649 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:19.075210 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51089.7us
I20260709 12:21:19.079169 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:19.080042 13793 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:19.083308 13794 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:19.084256 13794 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:19.084710 13794 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:19.086199 13649 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:19.087428 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:19.089465 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:19.091280 15049 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:19.091833 15049 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:19.094051 13793 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:19.091491 13550 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.097410 15049 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:19.116842 13794 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:19.119326 13537 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 4 but the committed config has opid_index of 5
I20260709 12:21:19.120242 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:19.120883 14425 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:19.120527 13722 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:19.132709 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:19.133492 14423 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:19.136580 13649 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:19.133570 13549 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.138484 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:19.140568 13794 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:19.147861 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:19.148862 13796 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:19.149288 13796 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:19.168915 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:19.169564 13718 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:19.169847 13718 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:19.136916 14425 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:19.171528 14424 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:19.171934 13718 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:19.175628 13718 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:19.177485 14423 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:19.183344 15053 tablet_replica.cc:333] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:19.184469 15053 raft_consensus.cc:2243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.185374 15053 raft_consensus.cc:2272] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.200253 13718 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:19.201558 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:19.198882 13796 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:19.201565 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:19.204960 14423 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:19.206578 14853 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:19.209656 13649 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:19.210124 14853 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:19.211236 14853 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:19.213858 14854 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:21:19.216487 14897 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:19.219244 14897 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:19.220232 14897 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:19.222239 15053 ts_tablet_manager.cc:1916] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:19.234496 13796 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:19.243394 14854 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:21:19.252628 13796 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:19.272262 13794 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:19.282534 15007 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.277971 13549 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.297048 13549 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.314967 15050 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:19.316011 15050 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:19.316279 13651 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:19.316352 15050 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:19.317811 13643 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.314967 14423 raft_consensus.cc:2955] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.317714 15050 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:19.341584 14854 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:19.362793 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6966d62b938845429a9aa8feb8fe2b9b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:19.363080 13547 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:19.401038 15050 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:19.413501 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet 6966d62b938845429a9aa8feb8fe2b9b with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:19.454519 15057 tablet_replica.cc:333] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:19.459266 15057 raft_consensus.cc:2243] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:19.460541 15057 raft_consensus.cc:2272] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:19.461988 13549 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.464461 15057 ts_tablet_manager.cc:1916] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:19.463653 13547 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:19.489657 15053 ts_tablet_manager.cc:1929] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:21:19.490609 15053 log.cc:1199] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/2f8f1b91eeaf4fc28fa45d4b45b907cd
I20260709 12:21:19.495618 13722 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:19.497093 15057 ts_tablet_manager.cc:1929] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:21:19.497674 15057 log.cc:1199] T 6966d62b938845429a9aa8feb8fe2b9b P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/6966d62b938845429a9aa8feb8fe2b9b
I20260709 12:21:19.497541 14987 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:19.499919 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet 6966d62b938845429a9aa8feb8fe2b9b (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:19.511059 13794 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:19.513649 14987 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.515486 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 2f8f1b91eeaf4fc28fa45d4b45b907cd (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:19.534077 13796 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:19.535904 15050 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.613505 13649 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:19.616915 14949 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:19.625550 13793 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:19.628307 15054 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:19.639523 13799 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:19.646087 15023 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:21:19.662302 13650 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:19.676005 15009 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:21:19.686369 14425 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:19.696177 14985 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:19.719937 13722 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:19.722767 14937 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:19.811676 14423 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:19.815321 15027 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:20.027395 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:20.084046 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:20.084952 13799 raft_consensus.cc:606] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:20.092319 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:20.093029 13793 raft_consensus.cc:606] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:20.100165 13718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:20.100796 13718 raft_consensus.cc:606] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:20.105448 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:20.106078 13650 raft_consensus.cc:606] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:20.109551 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:20.110062 13650 raft_consensus.cc:606] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:20.111764 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:20.113960 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:20.115340 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:20.287024 14985 raft_consensus.cc:993] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:20.287608 15023 raft_consensus.cc:1081] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:20.289724 13796 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:20.290532 13796 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.290920 13796 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.304594 13796 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:20.326967 13796 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:20.330025 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:20.330904 13718 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.332355 15023 raft_consensus.cc:993] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:20.333480 15023 raft_consensus.cc:1081] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:20.335638 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:20.336328 13649 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:20.336634 13649 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:20.339901 14424 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:20.340548 14424 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.341271 14424 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.343173 13649 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:20.344497 13649 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:20.349766 15027 raft_consensus.cc:993] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:20.350414 15027 raft_consensus.cc:1081] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:20.357467 14424 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.361491 14424 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:20.368067 13650 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:20.397176 13650 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.397581 13650 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.376549 13718 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:20.395468 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:20.412563 13799 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.396371 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:20.416239 13651 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:20.416618 13651 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:20.417392 13651 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.418975 13651 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.424151 13799 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:20.424973 15074 raft_consensus.cc:993] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:20.426440 15074 raft_consensus.cc:1081] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:20.425679 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:20.407001 13650 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.431097 13649 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:20.439591 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:20.431442 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:20.431573 13643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54756
I20260709 12:21:20.440665 14897 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:20.441035 13643 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.441323 14897 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:20.441510 13643 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.442391 13650 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:20.444255 13799 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:20.444691 13799 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:20.445556 13799 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.447041 13799 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.449080 15040 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:20.449676 15040 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:20.442396 14897 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:20.452039 15040 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:20.455104 15073 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:20.432088 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:20.461812 14424 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.463860 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:20.464609 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:20.464761 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:20.469177 13799 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:20.474658 13727 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:20.472059 13643 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.470796 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:20.493031 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:20.490945 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:20.505798 13643 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:20.523003 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:20.523739 13727 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:20.524075 13727 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:20.527158 13727 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.528633 13727 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.535117 13651 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:20.505173 14985 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:20.521196 15073 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:20.562656 13727 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:20.564062 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:20.554445 14985 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:20.583824 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108670us
I20260709 12:21:20.584820 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:20.585536 14424 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.561322 15040 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:20.595477 13547 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:20.604192 14424 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:20.569787 14985 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:20.584713 15023 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:20.618737 15040 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:21:20.674832 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87784.9us
I20260709 12:21:20.643100 15073 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:20.738875 15078 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:21:20.742997 15040 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.752926 14985 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:20.755447 14985 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:20.767642 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:20.768361 13718 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:20.762898 15023 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:20.800675 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:20.801425 13799 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:20.806852 15023 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.812975 13549 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 3 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:20.817684 13651 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:20.829151 13718 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:20.813974 13799 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
W20260709 12:21:20.878350 13615 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15d97b0b20fd480e8f728a8d38f52e03. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:20.884083 13722 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:20.885195 15092 raft_consensus.cc:993] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:20.886106 15092 raft_consensus.cc:1081] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:20.887883 15090 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:20.892727 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46706.4us
I20260709 12:21:20.898437 13718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:21:20.902611 13718 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:20.903204 13718 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.912498 15083 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:20.916786 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:20.920027 15075 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:20.920573 15075 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:20.923576 13547 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:20.921420 15075 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:20.914639 13722 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:20.935837 13793 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:20.938323 13718 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.949316 13718 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:20.951329 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:20.951952 14425 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.954695 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:20.955513 13799 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:20.955854 13799 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:20.956694 13799 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.959286 13799 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:20.960811 13651 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:20.964226 14425 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:20.967854 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:20.967927 15105 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:20.968825 15091 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:20.969355 15091 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:20.970291 15091 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:20.976599 13547 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:20.988379 13720 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:20.991163 15029 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:20.994311 13797 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:21:20.999528 13722 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:21:21.003677 13615 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e9c30191f356495b9732659f925abc9f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:21.007167 15105 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:21.011318 15040 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:21.027732 13799 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:21.041008 13793 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:21.064884 15009 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:21.078696 13643 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:21.065567 15009 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:21.087759 15095 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:21:21.110795 15009 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:21.105708 13550 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.107882 13547 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.118693 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:21.120074 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:21.119536 13797 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:21.116377 13547 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 5 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:21.134451 13643 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:21.139364 15077 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:21.145871 13797 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:21.156847 15077 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:21.160955 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:21.168803 15075 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:21.175390 13793 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:21.173179 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:21.181988 15098 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:21.182555 15098 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:21.182910 15098 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:21.234910 13797 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:21:21.272967 15115 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:21:21.268894 13549 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:21.288640 15098 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:21.290037 14425 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:21.277609 13796 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:21.300323 15105 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:21:21.375139 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64526.2us
I20260709 12:21:21.375556 15023 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:21:21.428807 13718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:21.429512 13718 raft_consensus.cc:606] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:21.429795 13727 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:21.442423 15086 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:21:21.448179 13796 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:21:21.458710 15086 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:21:21.469512 15098 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:21.474287 14423 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:21.479609 15087 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260709 12:21:21.486558 13764 consensus_peers.cc:597] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 85fee8a18ff4429cb870f59aa46b3892. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:21.488507 13727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:21.489173 13727 raft_consensus.cc:606] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:21.525328 13649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:21.526160 13649 raft_consensus.cc:606] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:21.570276 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 135646us
I20260709 12:21:21.615655 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:21.687615 13718 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:21.626690 15134 ts_tablet_manager.cc:933] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:21.721175 13797 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:21.722302 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:21.783727 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:21.784600 13797 raft_consensus.cc:606] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:21.834049 13722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:21.834775 13722 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:21.839586 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:21.840660 14425 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:21.841773 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:21.842772 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:21.851649 15136 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:21:21.856705 13722 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:21.899093 13722 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:21.912308 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:21.919667 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 95147.7us
I20260709 12:21:21.916251 15098 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:21:21.922737 15136 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:21:21.923377 15023 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:21.924397 15144 ts_tablet_manager.cc:933] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:21.925710 15023 consensus_queue.cc:1243] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:21:21.926687 14985 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:21:21.915680 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 296014us
I20260709 12:21:21.927140 14985 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:21.929034 14985 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:21.949811 14985 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:21.953048 13649 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:21.963920 15134 tablet_copy_client.cc:323] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:21.974890 15027 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:21:21.997187 15090 raft_consensus.cc:993] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:21.997789 15090 raft_consensus.cc:1081] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:22.003141 13661 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 15d97b0b20fd480e8f728a8d38f52e03 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:22.003160 13792 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:22.004103 13792 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.004457 13792 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.004114 13661 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 15d97b0b20fd480e8f728a8d38f52e03 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-15d97b0b20fd480e8f728a8d38f52e03
W20260709 12:21:22.020294 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47924.1us
I20260709 12:21:22.030921 13661 tablet_copy_source_session.cc:215] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:22.032907 15138 raft_consensus.cc:993] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:22.033382 15138 raft_consensus.cc:1081] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:22.041394 15144 tablet_copy_client.cc:323] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:22.044718 13792 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.042611 15134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d97b0b20fd480e8f728a8d38f52e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:22.072328 13651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:22.080231 13792 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:22.081261 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:22.081875 13649 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:22.082195 13649 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:22.079161 13651 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.084482 13651 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.086849 13661 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet e9c30191f356495b9732659f925abc9f from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:22.087423 13661 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet e9c30191f356495b9732659f925abc9f from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-e9c30191f356495b9732659f925abc9f
I20260709 12:21:22.087572 15147 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:21:22.097820 13649 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.099540 13649 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.099601 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:22.101027 13727 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.120694 15054 raft_consensus.cc:993] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:22.121445 15091 raft_consensus.cc:1081] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:22.127556 15054 raft_consensus.cc:993] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:22.128108 15054 raft_consensus.cc:1081] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:22.134745 13651 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.145099 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:22.145802 13792 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:22.146128 13792 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:22.147122 13792 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.148561 13792 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.154735 13651 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:22.161612 13649 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:22.165302 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:22.170123 13792 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:22.171365 15149 ts_tablet_manager.cc:933] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:22.173347 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:22.171123 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:22.179553 15027 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:22.180099 15027 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:22.179059 15138 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
W20260709 12:21:22.176910 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58413.7us
I20260709 12:21:22.183552 13643 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:22.188494 13801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:22.189129 13801 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.189481 13801 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.194720 13649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54756
I20260709 12:21:22.195523 13649 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.195937 13649 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:22.181550 15105 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:22.197082 15105 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:22.204836 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:22.207571 13718 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.181074 15027 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.222757 13718 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:22.226320 15127 raft_consensus.cc:993] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:22.226951 15009 raft_consensus.cc:1081] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:22.227725 15105 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.241250 13801 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:22.244692 13796 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:22.245276 13796 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:22.245669 13796 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:22.248322 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:22.248950 13718 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:22.249260 13718 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:22.251668 15138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:44060 (request call id 6) took 1046 ms. Trace:
I20260709 12:21:22.257383 13649 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.261360 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:22.262172 13720 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:22.262567 13720 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:22.263604 13720 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.265185 13720 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:22.268076 13649 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:22.269429 13661 tablet_copy_source_session.cc:215] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:22.269905 15157 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:21:22.275009 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:22.278730 13794 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:22.250079 13718 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:22.276324 13722 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:22.283229 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:22.283926 14423 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.286423 13801 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:22.290704 13718 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:22.290195 13796 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:22.291787 13792 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:22.300483 13794 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:22.300483 13720 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:22.304338 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:22.304800 14423 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:22.306705 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:22.307220 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:22.308132 13727 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:22.309186 13727 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:22.310484 13727 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:22.311803 13727 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:22.314622 15095 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:22.315215 15095 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:22.316170 15095 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:22.318914 15040 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:22.319634 15040 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:22.319113 13545 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:22.320813 15040 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:22.323912 13796 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:22.324159 13727 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:22.252071 15138 rpcz_store.cc:276] 0709 12:21:21.205518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:21.205647 (+   129us) service_pool.cc:224] Handling call
0709 12:21:21.251548 (+ 45901us) log.cc:844] Serialized 499 byte log entry
0709 12:21:21.871081 (+619533us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b540040b8a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6d142 00007f5393b4deaa 00007f5393b54d53 00007f5393b54cd9 00007f5393b54c69 00007f5393b54c31 00007f5393b53dfd 0000000000388634 0000000000388479 00007f5390b07475
0709 12:21:22.251574 (+380493us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135481,"raft.queue_time_us":281938,"raft.run_cpu_time_us":5318,"raft.run_wall_time_us":200028,"spinlock_wait_cycles":48868736,"thread_start_us":341210,"threads_started":2,"wal-append.queue_time_us":176917}
I20260709 12:21:22.302146 15149 tablet_copy_client.cc:323] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:21:22.326421 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:22.329926 13718 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:22.326009 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:22.337085 14423 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:22.345019 14423 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:22.346167 15093 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:22.357877 13547 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:22.361249 15093 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:22.367154 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 85fee8a18ff4429cb870f59aa46b3892 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:21:22.367789 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 85fee8a18ff4429cb870f59aa46b3892 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-85fee8a18ff4429cb870f59aa46b3892
I20260709 12:21:22.368011 15093 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:22.376226 15144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9c30191f356495b9732659f925abc9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:22.410229 13549 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:22.434180 13797 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:22.482239 13811 tablet_copy_source_session.cc:215] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:21:22.484984 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46787.4us
I20260709 12:21:22.490499 15149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fee8a18ff4429cb870f59aa46b3892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:22.516062 15149 tablet_copy_client.cc:806] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:22.517091 15149 tablet_copy_client.cc:670] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:22.529171 15134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48736 (request call id 209) took 1057 ms. Trace:
I20260709 12:21:22.530470 15134 rpcz_store.cc:276] 0709 12:21:21.471711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:21.472510 (+   799us) service_pool.cc:224] Handling call
0709 12:21:21.776349 (+303839us) ts_tablet_manager.cc:934] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
0709 12:21:22.041133 (+264784us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:22.245552 (+204419us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:22.245678 (+   126us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:22.528153 (+282475us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:22.528567 (+   414us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:22.528995 (+   428us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:22.529079 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":329,"tablet-copy.queue_time_us":45109,"thread_start_us":219082,"threads_started":2}
I20260709 12:21:22.533160 15149 tablet_copy_client.cc:538] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:21:22.537709 13764 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88a3d1d336b2495ea9a3a7ae1d7d37a2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:21:22.537935 13651 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:22.539043 13722 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:21:22.542560 15134 tablet_copy_client.cc:806] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:22.540792 15027 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:22.543301 15134 tablet_copy_client.cc:670] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:22.543982 15147 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:21:22.557786 15134 tablet_copy_client.cc:538] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:22.601615 15149 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:22.601604 13643 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:22.599502 15144 tablet_copy_client.cc:806] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:22.614986 15144 tablet_copy_client.cc:670] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:22.621176 13797 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.634480 13722 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:22.590293 15141 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:22.627077 15090 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:21:22.643451 15144 tablet_copy_client.cc:538] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:22.655258 13722 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.664671 15090 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:22.693351 15134 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:22.698436 13722 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.702708 15141 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:22.714425 15144 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:22.715416 13643 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:22.717348 15138 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:22.723418 13727 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:22.736250 15138 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 12:21:22.802178 15171 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:21:22.807044 13796 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:22.815114 15075 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:22.841882 14425 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:22.853010 15027 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:22.878877 13718 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:22.880631 14985 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:21:23.021579 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76509.4us
I20260709 12:21:23.015892 14423 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:21:23.029546 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 88748.2us
I20260709 12:21:22.943425 13549 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 3, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:23.035271 13549 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:23.086683 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:23.099946 15149 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.101432 15149 tablet_bootstrap.cc:492] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:23.133920 15149 ts_tablet_manager.cc:1403] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.533s	user 0.036s	sys 0.016s
I20260709 12:21:23.176573 15149 raft_consensus.cc:359] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:23.179908 15149 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:23.180917 15149 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:23.204748 13718 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:21:23.231084 15134 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.232470 15134 tablet_bootstrap.cc:492] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:23.233384 15134 ts_tablet_manager.cc:1403] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.540s	user 0.030s	sys 0.019s
W20260709 12:21:23.321324 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67481.5us
W20260709 12:21:23.361056 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 114084us
W20260709 12:21:23.361093 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141389us
I20260709 12:21:23.388029 13545 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:23.391567 13545 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 4, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:21:23.452365 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59588.8us
I20260709 12:21:23.386133 15147 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:21:23.476655 15029 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:23.477339 15029 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:23.512295 15149 ts_tablet_manager.cc:1434] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.378s	user 0.011s	sys 0.000s
W20260709 12:21:23.536425 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 173582us
I20260709 12:21:23.544309 14425 raft_consensus.cc:1217] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:21:23.549764 13720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:23.550431 13720 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:23.565536 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:23.566421 13792 raft_consensus.cc:606] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:23.532855 15138 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:21:23.592979 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:23.630628 13651 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:23.594720 15029 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:23.635329 15134 raft_consensus.cc:359] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:23.637259 15134 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:23.646610 15144 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:23.648079 15144 tablet_bootstrap.cc:492] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:23.649132 15144 ts_tablet_manager.cc:1403] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.935s	user 0.030s	sys 0.019s
I20260709 12:21:23.600090 15099 raft_consensus.cc:993] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:23.661734 15099 raft_consensus.cc:1081] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:23.621685 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:23.664147 13727 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:23.666370 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:23.707975 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:23.709144 13643 raft_consensus.cc:606] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:23.721127 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68784.4us
I20260709 12:21:23.690964 15099 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:21:23.782523 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97451.5us
I20260709 12:21:23.792707 13651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:23.793641 13651 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:23.794170 13651 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:23.658643 15134 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:23.803493 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:23.819419 13643 raft_consensus.cc:606] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:23.817708 15094 raft_consensus.cc:993] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:23.782274 15099 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:23.842804 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57428.1us
I20260709 12:21:23.844076 15144 raft_consensus.cc:359] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:23.845460 15127 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:23.833184 15023 raft_consensus.cc:1081] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:23.840229 15099 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:23.864954 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86362.7us
I20260709 12:21:23.857753 13651 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:23.899616 13649 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:23.901096 13651 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:23.854324 15144 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:23.903482 15144 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:23.905512 15144 ts_tablet_manager.cc:1434] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.256s	user 0.004s	sys 0.008s
I20260709 12:21:23.909080 15207 ts_tablet_manager.cc:933] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:23.914525 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133432us
I20260709 12:21:23.857481 15127 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:23.919194 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:23.919800 13796 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:23.920152 13796 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:23.927829 13792 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:21:23.928634 13792 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:23.929003 13792 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:23.918632 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-85fee8a18ff4429cb870f59aa46b3892 received from {username='slave'} at 127.0.0.1:44068
I20260709 12:21:23.919878 13661 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-e9c30191f356495b9732659f925abc9f received from {username='slave'} at 127.0.0.1:60832
I20260709 12:21:23.944121 15134 ts_tablet_manager.cc:1434] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.710s	user 0.016s	sys 0.000s
W20260709 12:21:23.944725 15134 ts_tablet_manager.cc:1467] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Tablet startup took 710ms
I20260709 12:21:23.948145 13661 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-e9c30191f356495b9732659f925abc9f on tablet e9c30191f356495b9732659f925abc9f with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:23.960469 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-85fee8a18ff4429cb870f59aa46b3892 on tablet 85fee8a18ff4429cb870f59aa46b3892 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:23.961287 13792 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:23.964565 15127 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:23.945120 15134 ts_tablet_manager.cc:1469] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Trace:
0709 12:21:21.471711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:21.472510 (+   799us) service_pool.cc:224] Handling call
0709 12:21:21.776349 (+303839us) ts_tablet_manager.cc:934] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
0709 12:21:22.041133 (+264784us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:21:22.245552 (+204419us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:22.245678 (+   126us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:21:22.528153 (+282475us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:21:22.528567 (+   414us) tablet_replica.cc:206] Creating consensus instance
0709 12:21:22.528995 (+   428us) ts_tablet_manager.cc:989] Replica registered.
0709 12:21:22.529079 (+    84us) inbound_call.cc:177] Queueing success response
0709 12:21:22.692948 (+163869us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:21:23.233737 (+540789us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:21:23.257194 (+ 23457us) tablet_replica.cc:261] Starting instrumentation
0709 12:21:23.482851 (+225657us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":329,"mutex_wait_us":11661,"raft.queue_time_us":8760,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":14833,"tablet-copy.queue_time_us":45109,"tablet-download-pool.queue_time_us":156,"tablet-download-pool.run_cpu_time_us":1970,"tablet-download-pool.run_wall_time_us":8828,"thread_start_us":219082,"threads_started":2}
I20260709 12:21:23.984117 15099 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:23.996915 15099 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:23.999367 13792 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:24.009953 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 77954.9us
I20260709 12:21:24.020468 13796 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.027969 13796 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:24.032068 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100058us
I20260709 12:21:24.056173 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:21:24.058245 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:24.059279 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:24.061372 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:24.062192 13720 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:24.064354 14425 raft_consensus.cc:1217] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:21:24.066073 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:24.066847 13720 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.078804 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:24.088148 13643 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:24.099160 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:24.100663 13661 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-15d97b0b20fd480e8f728a8d38f52e03 received from {username='slave'} at 127.0.0.1:60832
I20260709 12:21:24.102191 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:24.108632 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:24.109400 13720 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.112128 13796 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:24.147446 13649 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:24.147960 13649 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:24.150697 13720 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:24.101243 13661 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-15d97b0b20fd480e8f728a8d38f52e03 on tablet 15d97b0b20fd480e8f728a8d38f52e03 with peer b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:24.157694 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 52221.3us
I20260709 12:21:24.161214 13649 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.166450 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:24.177947 15029 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:24.178676 15029 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:24.167521 15207 tablet_copy_client.cc:323] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:21:24.179595 15029 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.189913 13649 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.190119 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:21:24.190861 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-88a3d1d336b2495ea9a3a7ae1d7d37a2
I20260709 12:21:24.153286 15090 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:24.194326 15090 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:24.195529 15186 raft_consensus.cc:1064] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:24.171402 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:24.225251 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:24.226591 13643 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:21:24.234828 13649 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
W20260709 12:21:24.240445 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69233.8us
I20260709 12:21:24.236689 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:24.248040 13797 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:24.261914 15204 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:24.262603 15204 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:21:24.266953 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98953.4us
W20260709 12:21:24.275543 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50573.3us
I20260709 12:21:24.209218 15186 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:24.303463 13811 tablet_copy_source_session.cc:215] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:24.313594 15199 raft_consensus.cc:993] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:24.286157 15099 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:24.263485 15204 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.337385 15090 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:24.338603 15099 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:24.339697 15099 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.314168 15199 raft_consensus.cc:1081] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:24.359002 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:24.361099 14424 raft_consensus.cc:1217] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:21:24.365213 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64007us
I20260709 12:21:24.378890 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:24.381466 15206 raft_consensus.cc:993] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:24.382321 15106 raft_consensus.cc:1081] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:24.386081 13801 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:24.386842 13801 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.387281 13801 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.405125 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:24.407485 13723 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:24.408143 13723 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:24.408643 13723 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.421303 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:24.422646 15199 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:21:24.429208 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:24.431767 15207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:21:24.434698 13801 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.452083 13801 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:24.452878 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:24.453960 13643 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.458673 15040 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:24.469794 13643 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:24.472494 15207 tablet_copy_client.cc:806] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:21:24.472570 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:24.475749 15099 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:24.476657 15099 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:24.482324 15099 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.488462 13723 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.490785 13651 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:24.473189 15207 tablet_copy_client.cc:670] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:21:24.493389 13723 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:24.494057 14425 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:24.496026 15138 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:24.496815 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:24.497570 13649 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:24.497912 13649 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:24.510598 13649 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.510242 13550 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 5 to 6, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.514381 15199 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:24.515501 13550 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:24.516986 15193 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:24.520567 15193 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:24.521478 15193 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.522532 15193 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:24.525988 13649 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.519398 13545 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:24.533313 15209 raft_consensus.cc:1064] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:24.535472 13651 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:24.536082 15209 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:24.537250 15196 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:21:24.545063 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53902.7us
I20260709 12:21:24.550016 13722 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:21:24.553022 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:24.545096 15207 tablet_copy_client.cc:538] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:21:24.562700 13649 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:24.564224 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:24.565689 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:24.583736 13797 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:24.589951 13720 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:24.598445 15197 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:24.599076 15197 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:24.594826 15027 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:21:24.600000 15197 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:24.638731 13720 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.659341 13794 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.662145 15106 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.673266 13801 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:24.676980 15040 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:24.684576 13651 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:24.676167 15187 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:24.696832 13547 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:24.708870 13797 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:24.728313 14425 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:24.730587 13797 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:24.732131 15204 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:24.732573 15209 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:21:24.742728 15207 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:21:24.752693 13723 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:24.852716 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 52335.2us
I20260709 12:21:24.873225 13649 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:21:24.872501 14425 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:24.884725 15214 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:21:24.896530 13723 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:21:24.908674 15029 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:21:24.915113 15029 consensus_queue.cc:1243] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:21:24.919744 13723 pending_rounds.cc:85] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 2
I20260709 12:21:24.938006 13643 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:24.920200 13723 pending_rounds.cc:107] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:21:24.939754 13723 raft_consensus.cc:2889] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:24.967193 15207 tablet_bootstrap.cc:700] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent opening tablet: real 0.149s	user 0.001s	sys 0.002s
I20260709 12:21:24.888691 15206 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:25.006780 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137899us
I20260709 12:21:25.010174 13722 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:25.033488 15242 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:25.044512 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 178969us
I20260709 12:21:25.047508 13720 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:25.007608 15186 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:21:25.080248 15090 raft_consensus.cc:1064] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:21:25.103269 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73840.5us
I20260709 12:21:25.104149 14425 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:25.067035 15208 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:21:25.165576 15090 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:25.213279 13549 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:25.243294 15246 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:21:25.269910 13722 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:25.380414 13793 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:25.399982 13643 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:25.432559 13651 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:25.394003 13549 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:25.496742 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 197588us
I20260709 12:21:25.499459 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:25.504667 13793 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:25.411989 15099 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
I20260709 12:21:25.528029 15126 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:21:25.750654 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 204187us
W20260709 12:21:25.749872 15207 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.012s	sys 0.047s
W20260709 12:21:25.778007 15207 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.012s	sys 0.047s
W20260709 12:21:25.815533 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64364.6us
W20260709 12:21:25.869143 15090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.034s	sys 0.072s
W20260709 12:21:25.869849 15090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.034s	sys 0.072s
W20260709 12:21:25.879688 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56395.8us
W20260709 12:21:25.883965 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92460.8us
W20260709 12:21:25.884097 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99885.8us
I20260709 12:21:25.895673 13720 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:25.909557 15232 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:21:25.959095 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72641.9us
I20260709 12:21:25.966169 15268 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:25.975528 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55453.5us
W20260709 12:21:25.984423 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 207209us
I20260709 12:21:26.017746 15273 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 12:21:26.049911 15207 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:21:26.051205 15207 tablet_bootstrap.cc:492] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:21:26.052624 15207 ts_tablet_manager.cc:1403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 1.320s	user 0.040s	sys 0.057s
I20260709 12:21:26.056178 15207 raft_consensus.cc:359] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:26.056910 15207 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:21:26.058250 15207 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:26.061120 15207 ts_tablet_manager.cc:1434] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
W20260709 12:21:26.135551 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 118122us
W20260709 12:21:26.146168 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 142962us
W20260709 12:21:26.150413 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124019us
I20260709 12:21:26.199615 15257 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:26.206244 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-88a3d1d336b2495ea9a3a7ae1d7d37a2 received from {username='slave'} at 127.0.0.1:44068
I20260709 12:21:26.207151 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-88a3d1d336b2495ea9a3a7ae1d7d37a2 on tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:26.201380 15257 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:26.267824 15257 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:26.298758 13719 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:21:26.306397 13547 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:26.438416 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53904.6us
I20260709 12:21:26.453774 15288 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:26.455368 15288 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:26.468910 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:26.492137 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:26.485337 15288 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:26.521257 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:26.522229 13797 raft_consensus.cc:606] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:26.673276 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86606.5us
W20260709 12:21:26.676879 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 96133.2us
I20260709 12:21:26.693574 15260 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:26.703886 14423 raft_consensus.cc:1217] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:21:26.755555 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:26.771586 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106055us
W20260709 12:21:26.786327 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59385.8us
I20260709 12:21:26.871704 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:21:26.874922 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85692.6us
I20260709 12:21:26.872323 13794 raft_consensus.cc:606] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:26.882759 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54955.7us
I20260709 12:21:26.904551 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48612be3c56f482c81533c30abebe60b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:26.910564 13645 raft_consensus.cc:606] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:26.916414 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:26.934703 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:26.935469 13800 raft_consensus.cc:606] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:26.943924 13547 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 225) took 1655 ms. Trace:
W20260709 12:21:26.949692 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60935.9us
I20260709 12:21:26.949911 15280 raft_consensus.cc:993] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:26.950575 15260 raft_consensus.cc:1081] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:21:26.960203 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52892.5us
I20260709 12:21:26.989882 13550 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 221) took 1440 ms. Trace:
I20260709 12:21:26.989970 13652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:26.990500 13550 rpcz_store.cc:276] 0709 12:21:25.537170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:25.537380 (+   210us) service_pool.cc:224] Handling call
0709 12:21:25.670082 (+132702us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:21:25.702152 (+ 32070us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfe9c 00007f5390ad00f6 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:21:26.975854 (+1273702us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23553,"reported_tablets":4,"spinlock_wait_cycles":7326464}
I20260709 12:21:26.991158 13652 raft_consensus.cc:606] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:26.993726 13801 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:27.000924 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:26.944479 13547 rpcz_store.cc:276] 0709 12:21:25.288394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:25.288567 (+   173us) service_pool.cc:224] Handling call
0709 12:21:26.943820 (+1655253us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:26.491138 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:26.492978 (+  1840us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:26.493044 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:21:26.493082 (+    38us) tablet.cc:662] Decoding operations
0709 12:21:26.493577 (+   495us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:26.493713 (+   136us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:26.494925 (+  1212us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:26.496924 (+  1999us) tablet.cc:701] Row locks acquired
0709 12:21:26.497979 (+  1055us) write_op.cc:260] PREPARE: finished
0709 12:21:26.498624 (+   645us) write_op.cc:270] Start()
0709 12:21:26.499185 (+   561us) write_op.cc:276] Timestamp: P: 1783599686498397 usec, L: 0
0709 12:21:26.499282 (+    97us) op_driver.cc:348] REPLICATION: starting
0709 12:21:26.534198 (+ 34916us) log.cc:844] Serialized 538 byte log entry
0709 12:21:26.668446 (+134248us) op_driver.cc:464] REPLICATION: finished
0709 12:21:26.829621 (+161175us) write_op.cc:301] APPLY: starting
0709 12:21:26.829770 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:26.829935 (+   165us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:26.830123 (+   188us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:26.830587 (+   464us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:26.830996 (+   409us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:26.831095 (+    99us) write_op.cc:312] APPLY: finished
0709 12:21:26.831477 (+   382us) log.cc:844] Serialized 23 byte log entry
0709 12:21:26.831748 (+   271us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:26.831874 (+   126us) write_op.cc:454] Released schema lock
0709 12:21:26.831951 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:26.841227 (+  9276us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:26.841280 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":160819,"apply.run_cpu_time_us":2870,"apply.run_wall_time_us":11597,"num_ops":1,"prepare.queue_time_us":118978,"prepare.run_cpu_time_us":5668,"prepare.run_wall_time_us":78758,"replication_time_us":168984,"spinlock_wait_cycles":166272,"thread_start_us":183236,"threads_started":2}]]}
I20260709 12:21:27.006086 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:27.007855 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:26.988121 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:27.020444 15187 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:27.025992 13652 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:26.990985 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:27.038591 15288 raft_consensus.cc:993] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:27.039317 15288 raft_consensus.cc:1081] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
W20260709 12:21:27.040407 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49492.8us
I20260709 12:21:27.036180 15242 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.073436 15196 raft_consensus.cc:993] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:27.074579 15196 raft_consensus.cc:1081] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:27.077873 14423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48720
I20260709 12:21:27.078600 14423 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:27.078789 13651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:27.078975 14423 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:27.077768 13792 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.079387 13651 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:27.079782 13651 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.092538 15255 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:27.143616 15029 raft_consensus.cc:1064] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:21:27.144208 15029 raft_consensus.cc:1068] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Service unavailable: leader transfer in progress
I20260709 12:21:27.145861 15269 raft_consensus.cc:993] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:27.146363 15269 raft_consensus.cc:1081] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:27.149572 13650 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:27.153626 13650 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:27.154039 13650 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.164536 14424 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:27.166523 15258 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:27.167569 13795 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:27.169817 14424 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:27.173367 13651 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.176959 13720 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:27.179227 15209 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:21:27.181936 13720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48612be3c56f482c81533c30abebe60b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:27.183004 13720 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:27.183568 13720 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.204334 15027 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:27.208871 13720 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.215442 13651 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:27.219105 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:27.219803 13801 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:27.220148 13801 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:27.211712 14423 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:27.232494 13720 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:27.234225 13721 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:27.241473 15261 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:21:27.242879 13801 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.218952 13719 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.246800 15106 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.247972 13650 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:27.250968 13650 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:27.278280 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:27.279048 13793 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:27.279402 13793 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:27.279510 14425 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.282472 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:27.283104 13727 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:27.280479 13793 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:27.286232 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:27.289078 13792 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:27.289408 13792 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:27.293999 14420 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.297411 13727 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:27.297072 13792 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:27.298310 13792 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:27.322696 13793 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.308861 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:27.341250 15254 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:27.341873 15254 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:27.342890 15254 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:27.360308 13792 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
W20260709 12:21:27.361445 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 136742us
I20260709 12:21:27.366048 15084 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.358817 13795 raft_consensus.cc:2955] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.386368 13793 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:27.369642 13549 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.428471 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:27.429250 13794 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.430207 14423 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:27.428699 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:27.430229 15258 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:21:27.428552 15260 raft_consensus.cc:993] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:21:27.437515 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46642.4us
I20260709 12:21:27.446105 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:27.446853 13720 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.448355 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:27.448956 13727 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.423969 13718 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:27.462709 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:27.462808 13801 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.482668 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15d97b0b20fd480e8f728a8d38f52e03 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:27.487412 15087 raft_consensus.cc:1081] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:27.511742 14424 raft_consensus.cc:2955] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.517976 13720 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:27.519706 13727 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:27.537976 13649 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:27.538574 13649 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:27.539464 13649 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.540884 13649 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:27.549345 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
W20260709 12:21:27.583225 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111370us
W20260709 12:21:27.587819 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92384.1us
I20260709 12:21:27.583863 13649 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:27.591252 13545 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 2 to 3, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.600056 13801 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:27.595657 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:27.598857 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:27.597899 15242 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:21:27.620148 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 85fee8a18ff4429cb870f59aa46b3892 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:27.621534 15260 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:27.622138 15260 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:27.622737 15126 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:27.623275 15126 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:27.623184 15260 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:27.623983 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:27.624351 15126 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.628518 15241 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:27.644204 13722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:27.644945 13722 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:27.645375 13722 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:27.652019 15241 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:27.653332 15241 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.692812 13722 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.705192 13722 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:27.712570 13720 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:27.751467 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:27.752254 13651 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:27.752609 13651 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:27.753532 13651 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.757416 13651 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:27.763531 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:27.764380 13801 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:27.777064 13651 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:27.781342 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:27.782322 15126 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:27.782883 15126 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:27.783824 15126 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:27.785847 13801 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:27.799532 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 15d97b0b20fd480e8f728a8d38f52e03 with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:34444
I20260709 12:21:27.875604 15334 tablet_replica.cc:333] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:21:27.885547 13794 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:21:27.887594 15307 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:21:27.891678 13550 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 4 to 5, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.899430 13550 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:27.901005 15334 raft_consensus.cc:2243] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:27.896749 13652 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:27.902081 15334 raft_consensus.cc:2272] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:27.908740 15334 ts_tablet_manager.cc:1916] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:27.911154 13720 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:27.934466 15334 ts_tablet_manager.cc:1929] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:27.935459 15334 log.cc:1199] T 15d97b0b20fd480e8f728a8d38f52e03 P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/15d97b0b20fd480e8f728a8d38f52e03
I20260709 12:21:27.934551 15326 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:27.937753 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 15d97b0b20fd480e8f728a8d38f52e03 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:27.939931 13720 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:21:27.944450 15307 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:21:27.960143 14424 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:21:27.963596 15326 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:27.963627 15241 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.970742 13720 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.976948 14424 raft_consensus.cc:2955] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:27.991855 13549 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:27.994057 13549 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:28.009166 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e9c30191f356495b9732659f925abc9f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:21:28.012431 13637 tablet_service.cc:1558] Processing DeleteTablet for tablet 85fee8a18ff4429cb870f59aa46b3892 with delete_type TABLET_DATA_TOMBSTONED (TS 4dafe08a4f404502822a0732fe570de4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:54754
I20260709 12:21:28.025965 15337 tablet_replica.cc:333] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:21:28.032258 13545 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 4 to 5, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 5 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:28.053759 15337 raft_consensus.cc:2243] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:28.055408 15337 raft_consensus.cc:2272] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:28.063347 15337 ts_tablet_manager.cc:1916] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:28.069101 13545 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 3 to 4, VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) evicted. New cstate: current_term: 1 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:28.072203 13537 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet e9c30191f356495b9732659f925abc9f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 12:21:28.116835 13794 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:28.119169 15273 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:28.127964 13782 tablet_service.cc:1558] Processing DeleteTablet for tablet e9c30191f356495b9732659f925abc9f with delete_type TABLET_DATA_TOMBSTONED (TS a4188193a10c4d50a555d76ea85c60b7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60478
I20260709 12:21:28.135476 14424 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEARNER]: Advancing to term 3
I20260709 12:21:28.158156 14424 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:28.159914 15326 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.160008 13547 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:28.170054 13720 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:28.171687 15241 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:28.205371 13652 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:28.206386 15337 ts_tablet_manager.cc:1929] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:21:28.206972 15337 log.cc:1199] T 85fee8a18ff4429cb870f59aa46b3892 P 4dafe08a4f404502822a0732fe570de4: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/wals/85fee8a18ff4429cb870f59aa46b3892
I20260709 12:21:28.208133 15126 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:28.209208 13720 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:28.210211 15342 tablet_replica.cc:333] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:21:28.221315 13537 catalog_manager.cc:5028] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): tablet 85fee8a18ff4429cb870f59aa46b3892 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:28.230799 15260 raft_consensus.cc:1064] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:28.232733 15241 raft_consensus.cc:1064] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:21:28.243323 15260 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:28.229286 13801 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:21:28.258090 15342 raft_consensus.cc:2243] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:28.259539 14425 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:28.219472 15310 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:21:28.262944 15342 raft_consensus.cc:2272] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:28.263751 15310 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:28.266330 15342 ts_tablet_manager.cc:1916] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:21:28.268177 13801 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:28.287976 15342 ts_tablet_manager.cc:1929] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:21:28.290983 15347 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:28.300432 13794 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:21:28.293819 15241 raft_consensus.cc:1068] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
  Committed config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }.
  Pending config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:28.307420 15323 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:28.310971 15342 log.cc:1199] T e9c30191f356495b9732659f925abc9f P a4188193a10c4d50a555d76ea85c60b7: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/wals/e9c30191f356495b9732659f925abc9f
I20260709 12:21:28.365459 13722 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:28.374677 13793 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:28.392727 13537 catalog_manager.cc:5028] TS a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): tablet e9c30191f356495b9732659f925abc9f (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:28.416432 15325 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:21:28.419872 15349 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:21:28.432906 13652 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:28.655398 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 242800us
I20260709 12:21:28.701246 15273 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:28.426589 15326 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.278s
I20260709 12:21:28.719348 15323 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:29.849793 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080026f9a0 after lost signal to thread 12472
I20260709 12:21:29.936440 15354 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:29.937242 15354 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.939845 15354 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:29.940379 15354 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:29.940917 15354 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.944113 15323 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:29.944505 15354 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.945134 15354 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:29.944824 15323 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.945719 15354 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.947463 15323 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.947995 15323 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:29.948400 15354 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.949100 15354 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:29.949046 15323 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.949681 15354 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.951756 15354 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.952276 15354 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:29.952715 15354 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.954982 15354 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.955468 15354 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:29.955499 15355 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:29.956044 15354 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.956070 15355 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.961160 15355 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:29.951774 15323 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:29.962921 15357 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:29.963948 15354 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.963639 15357 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.965389 15355 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:29.965754 15358 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:29.962580 15356 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:29.976905 15355 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.978147 15358 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.980396 15355 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:29.981006 15358 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:29.977061 15356 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.985917 15356 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:29.987993 15362 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:29.988626 15362 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.983597 15361 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:29.990088 15361 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:29.992306 15361 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:29.994946 15362 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:29.995600 15362 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:29.995826 13547 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 245) took 1573 ms. Trace:
I20260709 12:21:29.996105 15362 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:29.996222 13547 rpcz_store.cc:276] 0709 12:21:28.422234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:28.422419 (+   185us) service_pool.cc:224] Handling call
0709 12:21:28.428212 (+  5793us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393bfcf3a 00007f53999b2e9c 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:29.995752 (+1567540us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":7069568}
I20260709 12:21:29.982748 15357 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:29.998459 15359 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:29.999926 15359 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.000397 15362 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.001688 15362 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.002266 15362 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:28.949028 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:30.004616 15362 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.005502 15359 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.006702 15359 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.007324 15359 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.001781 13545 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 225) took 1580 ms. Trace:
I20260709 12:21:30.010211 15362 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.010667 13545 rpcz_store.cc:276] 0709 12:21:28.421829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:28.422410 (+   581us) service_pool.cc:224] Handling call
0709 12:21:30.001710 (+1579300us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1293853,"reported_tablets":2,"spinlock_wait_cycles":1786752}
I20260709 12:21:30.011364 15359 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.013557 15362 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.019886 15362 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.021229 15359 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.021526 15365 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.021911 15359 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.022185 15365 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.021031 15362 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:30.039803 12474 debug-util.cc:398] Leaking SignalData structure 0x7b080000f860 after lost signal to thread 13767
I20260709 12:21:30.039754 15362 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:30.040400 12474 kernel_stack_watchdog.cc:198] Thread 13767 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 319ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:21:30.041167 15359 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.047045 15365 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.042104 15359 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.052831 15359 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.048113 15362 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.055923 15359 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.058027 15366 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.058629 15366 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.060833 15366 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.061605 15366 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.062101 15366 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.064803 15366 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.088398 15366 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.089076 15366 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.089460 15364 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.074075 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.090112 15364 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.068895 15323 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.091303 15323 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.093470 15323 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.064970 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 121) took 1351 ms. Trace:
I20260709 12:21:30.094305 13801 rpcz_store.cc:276] 0709 12:21:28.713410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.064059 (+1350649us) service_pool.cc:224] Handling call
0709 12:21:30.064425 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:30.064622 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:30.064652 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:30.064699 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:30.064731 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:30.064788 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:30.064895 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:30.067718 15354 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.095944 15354 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.102339 15364 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:30.110299 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126755us
I20260709 12:21:30.112804 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.098029 15354 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.119030 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.121955 15325 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.122627 15325 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.124091 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.124961 15325 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.125530 15325 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.126634 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.127303 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.127888 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.125993 15325 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.130791 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.133265 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.137092 13796 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:30.140870 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.132905 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.140578 15325 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.146469 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:30.147208 14420 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:30.147933 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:30.154428 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 1959) took 1437 ms. Trace:
I20260709 12:21:30.154904 13793 rpcz_store.cc:276] 0709 12:21:28.716641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.153608 (+1436967us) service_pool.cc:224] Handling call
0709 12:21:30.153887 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:30.154064 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:30.154088 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:30.154129 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:30.154160 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:30.154226 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:30.154352 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:30.157750 15365 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.112232 15364 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.158427 15365 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.177948 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:30.165155 15307 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.180053 15307 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.181627 15366 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.182235 15366 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.182803 15307 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.183369 15307 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.187498 15365 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.188071 15365 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.175489 15364 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.166100 15254 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.189419 15254 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.174922 15362 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.190722 15364 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.190944 15362 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.191298 15364 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.191522 15254 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.188526 15365 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.169552 15255 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.191805 15364 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.194125 15310 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.255990 15310 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.257828 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.258122 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.258563 13647 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:30.258903 13648 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:30.259879 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.260391 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.260511 13647 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:30.261055 13796 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:30.261315 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.261950 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.182828 15366 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.262969 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.263150 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.263768 13796 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:21:30.264137 15366 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.264597 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.240154 15254 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.266001 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.266835 15254 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.267554 13793 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:30.254559 15362 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.269731 15254 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.261963 13648 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:30.265255 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.258747 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.278419 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.175689 15260 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:30.209326 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79793.3us
I20260709 12:21:30.288893 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:30.291927 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.293883 13645 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:30.279570 15354 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.296002 15354 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.296828 15361 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.297333 15361 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.299311 15361 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.291661 15366 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.299894 13545 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 3 to 5, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:30.301023 15366 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.298152 15354 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.283782 15310 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.193869 15307 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.303130 15366 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.303560 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.304244 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.305307 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.305742 15368 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.306296 15368 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.306854 15362 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:30.307263 15362 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.307646 15362 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.305470 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.293494 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:30.292584 13800 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:30.322434 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.323452 13800 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:30.282428 15326 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.325537 15326 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.327095 15254 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:30.327540 15254 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.327919 15254 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.195122 15359 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.329654 15359 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.330417 15320 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.330998 15320 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.332664 15359 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.322486 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:30.320221 15260 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.336851 15355 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.340061 15355 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.344080 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 197) took 1652 ms. Trace:
I20260709 12:21:30.344734 13651 rpcz_store.cc:276] 0709 12:21:28.691482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.343341 (+1651859us) service_pool.cc:224] Handling call
0709 12:21:30.343621 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:30.343785 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:30.343808 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:30.343850 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:30.343880 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:30.343931 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:30.344022 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:30.337958 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.346599 15357 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.321312 15354 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.347366 15357 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.348310 15354 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.338501 15364 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.350180 15357 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.351725 15357 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.352236 15357 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.352722 15355 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.354451 15359 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:30.354884 15359 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.355300 15359 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.355391 15326 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.344871 15362 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.336915 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.357597 13796 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
W20260709 12:21:30.366782 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080026f280 after lost signal to thread 13584
I20260709 12:21:30.368770 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:30.308602 15368 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.371204 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.373236 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:30.378585 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:30.381623 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90761.2us
I20260709 12:21:30.280831 13801 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:30.346622 15320 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.291167 15255 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.393803 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.402487 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.381287 15357 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.409060 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.492635 15373 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.732183 15373 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.484556 15354 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.735543 15354 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.736279 15354 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.489419 15320 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.739488 15320 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.384850 15361 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:30.742579 15361 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.743117 15361 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:30.492488 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.745765 13721 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:30.759603 15352 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:30.762153 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 2.04185e+06us
I20260709 12:21:30.380220 15257 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.387192 15326 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:30.798441 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 395391us
I20260709 12:21:30.800127 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:30.805088 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.814817 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.816769 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.818725 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:30.400727 15307 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:30.822113 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.715313 15370 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:30.832792 15320 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.760294 15352 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.836810 15352 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.846020 13722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:30.848233 13722 raft_consensus.cc:606] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:30.870733 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:30.819842 15257 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.873926 15257 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:30.887930 15254 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.005s
I20260709 12:21:30.920020 15320 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.920465 15354 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.921404 15354 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:30.920933 15320 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.921864 15354 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:30.819748 15326 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.967618 15389 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:30.968523 15389 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:30.971325 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.973176 15378 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.974743 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:30.882310 15368 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:30.976132 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:30.894523 15379 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:30.918303 15385 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:30.955997 15354 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:30.988296 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163936us
I20260709 12:21:30.889309 15254 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:30.977669 15260 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:30.973754 15378 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.024405 15378 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.891930 15373 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.003988 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:31.015179 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 64920.4us
I20260709 12:21:30.981451 15255 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:30.987253 15310 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:31.027992 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 189959us
W20260709 12:21:31.032151 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 207481us
I20260709 12:21:31.047956 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.048981 14423 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:30.996136 15385 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.047381 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:30.987025 15368 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.052995 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.051107 15388 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.059199 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.061231 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.062470 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.062623 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.064023 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.065661 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.066727 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.067158 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.055068 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.068625 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.069306 13726 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:31.069506 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.025811 15379 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.054628 15385 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:31.268592 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 210453us
W20260709 12:21:31.269560 15359 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.914s	user 0.018s	sys 0.011s
I20260709 12:21:31.096542 15389 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.071079 15310 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:31.270927 12474 debug-util.cc:398] Leaking SignalData structure 0x7b080000f4e0 after lost signal to thread 13767
I20260709 12:21:31.272310 15310 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.171648 15255 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.273051 15255 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.172276 15366 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.274569 15366 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.275126 15366 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.275209 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 1968) took 1016 ms. Trace:
I20260709 12:21:31.275972 13792 rpcz_store.cc:276] 0709 12:21:30.259014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.268781 (+  9767us) service_pool.cc:224] Handling call
0709 12:21:31.274199 (+1005418us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:31.274446 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:31.274765 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:31.274797 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:31.274842 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:31.274886 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:31.274956 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:31.275104 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17047424}
I20260709 12:21:31.247942 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:31.278110 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 183738us
I20260709 12:21:31.279011 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:31.279412 15368 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.280138 15362 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.280737 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:31.047351 15347 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.282666 15347 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:31.283263 15347 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.280007 15368 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:31.284478 15368 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.284845 15368 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:31.287132 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 221504us
I20260709 12:21:31.287503 15255 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.288300 15255 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.251072 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:21:31.289117 15361 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.545s	user 0.018s	sys 0.001s
I20260709 12:21:31.289219 13801 raft_consensus.cc:606] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.289570 15361 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.252452 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.291347 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.292594 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.292807 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:31.253219 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.293583 15373 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.293823 13649 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:31.293979 15373 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.294423 15373 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.254166 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.298107 15255 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.298653 15255 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.293982 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.254426 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:31.112589 15372 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:31.038113 15378 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.113992 15390 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.255199 15326 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.255223 15254 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.256204 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.257397 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.257988 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.258453 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.259596 15354 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.259670 15388 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.113183 15377 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.262385 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:31.269480 15379 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.270028 15359 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.288491 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.248863 15257 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.293272 15320 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.302094 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.307189 15389 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.446286 13644 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:31.386092 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 200) took 1129 ms. Trace:
I20260709 12:21:31.317224 15260 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:31.447268 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 145262us
I20260709 12:21:31.325610 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:31.311251 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.448685 13643 raft_consensus.cc:606] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.395143 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:31.356403 15358 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 12:21:31.469730 15357 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:31.470086 15357 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:31.471082 15357 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.472436 15357 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.448405 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.480011 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:31.448966 13645 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:31.482244 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.482676 15358 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.483259 15358 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.484239 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.485154 15361 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.485365 15358 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.310441 15365 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.448355 13650 rpcz_store.cc:276] 0709 12:21:30.256401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:31.385245 (+1128844us) service_pool.cc:224] Handling call
0709 12:21:31.385586 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:31.385779 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:31.385805 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:31.385846 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:31.385879 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:31.385935 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:31.386025 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:31.485754 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:31.451272 15388 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.486344 15382 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.452291 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.452709 15404 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:31.486723 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.486994 15404 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.487505 15404 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.448820 15394 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.449126 15260 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.487982 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.452782 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.489738 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.491402 13652 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:31.493472 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.494186 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:31.303013 15387 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.274s	user 0.005s	sys 0.004s
I20260709 12:21:31.456296 15372 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.495123 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.457587 15377 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.497191 15372 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.498039 15377 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.405413 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.444080 15385 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.499692 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.456529 15359 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.501008 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.500896 15255 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.501710 13646 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:31.502110 15385 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.312203 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.502955 14419 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:31.503130 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.461180 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.504514 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.302153 15347 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.459230 15354 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.506640 13646 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.507892 15347 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.312551 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.509061 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.463150 15389 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.509744 15389 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:31.450736 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 188519us
I20260709 12:21:31.463371 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.511049 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:31.505911 15404 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.459913 15390 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.448966 14418 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:31.452065 15403 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:31.512997 15361 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.514642 15390 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.516645 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.309350 14424 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:21:31.436959 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 361652us
I20260709 12:21:31.393077 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.519829 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.520602 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 1977) took 1258 ms. Trace:
I20260709 12:21:31.520962 13793 rpcz_store.cc:276] 0709 12:21:30.261734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.268141 (+  6407us) service_pool.cc:224] Handling call
0709 12:21:31.392915 (+1124774us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:31.520515 (+127600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17128832}
I20260709 12:21:31.508262 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:31.386418 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080003bb20 after lost signal to thread 13676
I20260709 12:21:31.522446 15403 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.467427 15366 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.523643 13720 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:31.523890 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.524572 15366 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.450606 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.524906 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 1923) took 1470 ms. Trace:
I20260709 12:21:31.526691 13720 rpcz_store.cc:276] 0709 12:21:30.054132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:31.280983 (+1226851us) service_pool.cc:224] Handling call
0709 12:21:31.281326 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:31.524075 (+242749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:31.524714 (+   639us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400027bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:31.524851 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556749568}
W20260709 12:21:31.383960 15375 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.000s	user 0.004s	sys 0.000s
I20260709 12:21:31.528263 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.483568 15373 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.487071 15382 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:31.488467 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.530503 15382 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.530977 15382 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:31.532418 13647 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:31.523679 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.533802 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.448411 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.310838 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.490372 15260 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.536199 15255 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:31.451521 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 197177us
I20260709 12:21:31.490607 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.538507 13649 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:31.491509 13615 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.793s (SENT) [suppressed 3 similar messages]
I20260709 12:21:31.454394 15378 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:31.540854 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.541217 15378 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.542484 13724 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:31.542239 15358 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:31.454334 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:31.544037 13792 raft_consensus.cc:606] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:31.492446 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.545032 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.494053 15368 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.313695 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.546092 15389 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.332485 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.393704 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.547892 15368 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:31.548609 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 155017us
I20260709 12:21:31.548776 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:31.525764 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080011af60 after lost signal to thread 13751
I20260709 12:21:31.549649 15358 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:31.550009 15358 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.550387 15358 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:31.550530 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800102e80 after lost signal to thread 13825
W20260709 12:21:31.550886 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08002f55a0 after lost signal to thread 13826
I20260709 12:21:31.551694 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:31.552094 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800322040 after lost signal to thread 14449
I20260709 12:21:31.552273 13793 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:31.302654 15401 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.554355 14423 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.555357 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:21:31.527128 13648 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:31.489801 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.557641 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:31.558660 15403 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:31.559201 15403 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.534502 15357 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:31.488384 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.543156 15373 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.562582 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.562695 15357 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.563073 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.563373 15357 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:31.563607 15403 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.566066 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.566201 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.568848 15389 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.561272 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.570055 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.570230 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.570861 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.571019 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.571326 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.572065 13801 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 2: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:31.573333 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.575977 13800 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.576622 15376 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.576753 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:31.578805 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90509us
I20260709 12:21:31.582391 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.583505 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.489284 15257 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.696601 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.606772 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.664898 15261 raft_consensus.cc:993] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:31.706192 13796 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:21:31.573222 15361 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:31.707178 15369 raft_consensus.cc:1081] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:31.707278 15361 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.614317 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.674081 15357 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.706627 15347 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.709486 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.648100 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.623533 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.712684 13726 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.712956 13793 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:31.589056 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:31.587121 15254 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:31.590895 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.716037 15369 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.606611 15404 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.716619 15369 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.717041 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.717752 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.694135 15403 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:31.717782 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57259.8us
I20260709 12:21:31.718887 15254 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:31.718753 15403 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.719398 15254 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.646687 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.707808 15361 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:31.721474 15403 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.664674 13545 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 246) took 1569 ms. Trace:
I20260709 12:21:31.721787 15370 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.721946 15369 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.691692 13547 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 226) took 1574 ms. Trace:
I20260709 12:21:31.598977 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:31.723022 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76429.2us
I20260709 12:21:31.722908 13547 rpcz_store.cc:276] 0709 12:21:30.117494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.117680 (+   186us) service_pool.cc:224] Handling call
0709 12:21:31.691614 (+1573934us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":186202,"reported_tablets":2}
I20260709 12:21:31.631448 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.725745 13799 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:21:31.725962 15366 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.655344 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:31.647408 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:31.726646 15366 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:31.727022 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81287.2us
I20260709 12:21:31.728171 15405 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:31.714051 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.728698 15405 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:31.729204 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69711.5us
I20260709 12:21:31.699905 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.622032 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.716032 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.731309 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.732545 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.631186 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.729256 15405 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:31.749047 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.730201 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.624526 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.587370 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:31.604223 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.722039 13545 rpcz_store.cc:276] 0709 12:21:30.095576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:30.095700 (+   124us) service_pool.cc:224] Handling call
0709 12:21:31.664585 (+1568885us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:31.265185 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:31.265846 (+   661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:31.265887 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:21:31.265903 (+    16us) tablet.cc:662] Decoding operations
0709 12:21:31.266283 (+   380us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:31.266348 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:31.266375 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:31.266593 (+   218us) tablet.cc:701] Row locks acquired
0709 12:21:31.266610 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:21:31.266758 (+   148us) write_op.cc:270] Start()
0709 12:21:31.266900 (+   142us) write_op.cc:276] Timestamp: P: 1783599691266717 usec, L: 0
0709 12:21:31.266924 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:21:31.372075 (+105151us) log.cc:844] Serialized 595 byte log entry
0709 12:21:31.528274 (+156199us) op_driver.cc:464] REPLICATION: finished
0709 12:21:31.649257 (+120983us) write_op.cc:301] APPLY: starting
0709 12:21:31.649380 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:31.649532 (+   152us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:31.649683 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:31.650010 (+   327us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:31.650187 (+   177us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:31.650236 (+    49us) write_op.cc:312] APPLY: finished
0709 12:21:31.650484 (+   248us) log.cc:844] Serialized 23 byte log entry
0709 12:21:31.650685 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:31.650779 (+    94us) write_op.cc:454] Released schema lock
0709 12:21:31.650824 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:31.652592 (+  1768us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:31.652739 (+   147us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2128,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":120660,"apply.run_cpu_time_us":1935,"apply.run_wall_time_us":1942,"num_ops":1,"prepare.queue_time_us":961429,"prepare.run_cpu_time_us":27554,"prepare.run_wall_time_us":201803,"raft.queue_time_us":26831,"raft.run_cpu_time_us":8216,"raft.run_wall_time_us":156994,"replication_time_us":261186,"thread_start_us":710047,"threads_started":4,"wal-append.queue_time_us":126195}]]}
I20260709 12:21:31.718366 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.770898 14416 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.772498 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.710678 15357 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.577273 15358 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.774134 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.775398 13797 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.774632 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:31.776455 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.777122 15393 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:31.777522 15393 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.777691 15412 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.777918 15393 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.624423 15307 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:31.587174 15365 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.779537 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:31.643566 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.781401 15404 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.735024 15380 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:31.685927 13721 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:31.782397 15358 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.783496 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.784320 15365 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.786439 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.736382 15257 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.593595 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.787215 15241 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:31.738508 14423 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:31.787614 15241 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.788007 15241 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.677114 15376 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.738680 15382 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.789513 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.790339 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.790810 15376 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:31.791162 15382 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.621872 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.739831 15369 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.792023 15369 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.740835 13718 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:31.713686 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.728863 15366 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.632987 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:31.768395 15403 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:31.794237 15369 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.778650 15307 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.607528 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.800174 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.800648 13795 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.801990 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.808709 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:31.635121 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 143756us
I20260709 12:21:31.796762 14421 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.811373 15373 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.800906 15403 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.799180 15405 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.802142 15347 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.812893 15307 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:31.772161 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.813980 15393 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.853570 14417 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:31.801052 15380 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:31.854530 15380 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:31.854598 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.854776 15405 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.845947 15361 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.855548 15380 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.847234 13726 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:31.856523 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.856860 15380 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.847671 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.848384 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.846884 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.848753 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.850829 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.858999 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.859086 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:31.859338 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87260.4us
I20260709 12:21:31.855865 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.856990 13722 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.857538 13649 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.851091 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:31.815008 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:31.794672 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.805856 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.791323 15376 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:31.850701 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.805135 14424 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:31.864861 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.864878 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:31.880398 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.868881 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:31.862308 15257 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.866973 15392 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.862460 15357 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.878666 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:31.804203 15254 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.844813 15323 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:21:31.799754 15241 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.900923 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.882215 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.882372 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.903196 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:31.875733 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.805795 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.806205 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.867226 15403 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.884621 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.879582 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.912880 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.887084 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.887656 15393 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.883801 15385 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.914925 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.915612 15398 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:31.888273 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.918840 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.921996 15400 raft_consensus.cc:993] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:31.922703 15368 raft_consensus.cc:1081] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:21:31.904659 15405 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:21:31.924450 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.925354 14417 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:31.925868 15347 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:31.904845 15392 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.927196 15392 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:31.927627 15392 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.872710 15404 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.891578 15363 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.884455 15389 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:31.891960 13645 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.936431 15396 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:31.937144 15396 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.938414 15323 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:31.938565 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.892556 15358 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.892750 15361 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.893537 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.940737 15358 raft_consensus.cc:993] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:31.941241 15358 raft_consensus.cc:1081] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:31.895658 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:31.903853 13797 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.908120 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.891099 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:31.889959 15373 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:31.885759 15354 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:21:31.909812 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.904419 15382 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.929402 15257 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:31.941665 15363 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:31.928350 15347 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:31.952018 15365 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:31.952025 15348 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:31.952879 15326 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:31.953176 15348 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:31.953483 15326 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.934526 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.935985 15389 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:31.939009 15323 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.961400 15363 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:31.962532 15389 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.957628 15373 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:31.964076 15323 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.963843 15373 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.948873 15396 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:31.875698 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:31.943485 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47211us
I20260709 12:21:31.969805 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:31.956077 13644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
W20260709 12:21:31.944469 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53529.5us
I20260709 12:21:31.947489 13798 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:31.948802 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:31.951311 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.952034 15378 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:31.972514 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:31.974346 15379 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:31.948514 15358 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:31.965627 15257 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.974232 13644 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:31.975409 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:31.976671 13726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:21:31.983428 13726 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:31.983803 13726 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.979118 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:31.979312 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.980369 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:31.977357 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.962324 15254 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:31.986883 13652 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 2: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:31.986913 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.975703 15378 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:31.987946 15366 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:31.992497 15370 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.986636 13644 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:31.893155 15376 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:31.988263 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:31.990154 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:31.994441 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.995370 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:31.998418 13795 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:31.998680 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:32.003069 15395 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:31.993784 15378 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.002830 15388 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:31.987983 15412 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.007609 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:31.980940 15382 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.008186 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.004614 15388 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:32.006910 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.017954 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.018769 13798 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.019008 15241 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:32.019665 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:32.009202 15405 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:31.994109 15393 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:31.989833 15358 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:31.954155 15348 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.026414 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.004431 15378 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.028573 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:32.020241 15255 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:21:32.030231 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:32.033330 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.034468 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:32.035228 15359 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.008873 15390 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.042508 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.046239 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:32.046967 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:32.020509 15388 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:31.989064 15379 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.024803 15241 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.024830 15400 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:21:32.053882 15400 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.054246 15400 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.055112 15400 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.056381 15400 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.058511 13797 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:32.064065 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:32.050699 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:32.115345 13798 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
W20260709 12:21:32.082754 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76889.2us
I20260709 12:21:32.036325 15394 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:32.062909 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:32.083245 13726 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.086644 15379 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:32.089994 15382 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.012356 15347 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.087134 15257 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:32.087603 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:32.085960 15404 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:32.117537 15357 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:32.126497 15357 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.127895 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.128350 13795 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:32.128599 13801 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:32.128784 15357 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:32.120272 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57596.4us
I20260709 12:21:32.130085 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:32.131496 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.133366 13797 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:32.133523 13726 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:32.134965 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.136005 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:32.136525 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.138592 13690 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.138779 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:32.139792 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 119055us
I20260709 12:21:32.140539 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:32.140769 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.141793 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:32.121965 15241 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.142470 15390 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.143496 15323 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.142621 15241 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:32.143496 15390 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.145161 15241 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.146392 15358 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.150188 15354 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.152235 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.153421 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.154788 15354 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.121966 15348 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:32.156283 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.158598 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.159395 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.164510 15354 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.122663 15392 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.167340 15392 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:32.161948 15379 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:21:32.169378 15260 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.124949 15257 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.176381 13797 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:32.196805 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.197278 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.198585 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.199440 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.178531 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.178409 15255 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:32.174484 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.179595 15326 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:32.179641 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.180540 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.180814 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.181617 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.183883 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.186801 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.189010 13726 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:32.137560 15393 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:32.210041 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.172140 15414 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:32.242594 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.259141 15426 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.263434 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.263938 15427 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:21:32.270612 15426 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:32.270172 13644 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.206554 15354 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.215996 15347 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:32.296293 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45847.9us
I20260709 12:21:32.192052 15400 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:21:32.239766 15392 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.231546 15393 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:21:32.320648 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 110688us
I20260709 12:21:32.311239 15357 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.323036 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:32.326448 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 128097us
I20260709 12:21:32.357781 15373 raft_consensus.cc:993] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:32.322131 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.372237 15390 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.372947 15354 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.374240 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.372234 15326 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:32.374945 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:32.298662 15414 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.217234 15388 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.353080 15393 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.411530 14424 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:32.417102 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 95065.3us
I20260709 12:21:32.378741 15368 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.421816 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:32.421980 15424 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:32.422513 14424 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.422659 15424 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.424988 15424 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:32.401049 15359 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:21:32.427662 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 254669us
W20260709 12:21:32.410369 15307 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.545s	user 0.002s	sys 0.011s
W20260709 12:21:32.430754 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56617.3us
I20260709 12:21:32.433377 13801 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:32.436717 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.433770 15423 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.409773 15380 raft_consensus.cc:1081] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:32.407117 15390 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.440413 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.446409 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.415167 15326 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:32.448508 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.452232 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:32.454640 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:32.400341 15389 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.456120 15390 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.438169 15356 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.434629 15388 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.459092 13800 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:32.309487 15257 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.455541 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:32.466861 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.466089 15376 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.488467 13722 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:32.430770 15359 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:32.472267 14424 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:32.441840 15369 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.470680 15396 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.484613 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.428558 15368 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.499135 13652 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:32.441514 15307 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.501266 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:32.501521 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:32.501639 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:32.502641 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:32.505910 13724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:32.506192 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.506645 13724 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.507050 13724 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:32.507479 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.510638 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:32.510919 13723 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:32.489583 15355 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.523216 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.522490 13720 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:32.527864 15389 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:32.529073 15405 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.000s
I20260709 12:21:32.529495 15405 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.530632 14416 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:32.531780 15307 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:32.504650 15326 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.500371 15376 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.534394 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:32.534629 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.526729 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.536813 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:32.537138 15433 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:21:32.538735 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.541046 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.523398 13644 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:32.544018 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.544855 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:32.545475 15400 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:32.537649 15378 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:32.544590 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73569.1us
I20260709 12:21:32.547041 15368 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.551378 14419 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:32.552016 15359 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.544158 15423 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.557186 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:32.557538 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.564162 14424 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:32.631492 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73025.6us
I20260709 12:21:32.640039 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.543373 15392 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:32.632939 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76146.4us
I20260709 12:21:32.543426 15355 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.648881 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.646893 13800 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:32.647729 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.658429 13800 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:32.633054 15437 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:32.656488 13724 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.633634 15366 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.671056 13724 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:32.634074 15378 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:32.637669 15396 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:21:32.694340 15447 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:32.696403 15447 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:32.698580 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141140us
I20260709 12:21:32.700349 15442 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:32.701536 15400 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:32.702059 15400 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.645156 15426 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:21:32.662472 15362 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 12:21:32.748762 15362 consensus_queue.cc:1243] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:21:32.703377 15255 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.743554 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.752110 15255 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.669076 15401 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.669399 15368 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.697708 15447 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.767019 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.678602 15437 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.768486 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:32.770613 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.667047 15361 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 12:21:32.685268 15378 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:32.775990 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.006s	sys 0.023s
I20260709 12:21:32.677377 15403 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
W20260709 12:21:32.777078 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.006s	sys 0.024s
I20260709 12:21:32.662118 15430 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:32.779563 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 605120us
I20260709 12:21:32.779810 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.784446 13693 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.784847 13797 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60504
I20260709 12:21:32.785266 15448 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:21:32.776394 15389 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.791424 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.786705 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:32.792858 13793 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:32.793306 13793 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:32.787168 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:32.775848 15366 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:21:32.773646 15400 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.790746 13797 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:32.798063 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.785789 13799 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:32.772442 15392 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:21:32.798866 13797 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.786547 15437 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.794608 15255 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.801326 15400 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:32.803056 13723 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.783643 15442 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.807214 13547 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:32.803228 15425 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.812143 15432 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:32.787794 15443 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:21:32.827952 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.833029 15378 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.811524 15425 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.850075 15379 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.857625 13801 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:32.888844 14417 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:32.889114 15448 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:32.890993 14419 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:32.891332 15326 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:32.907877 15446 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:32.912792 15396 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:32.893872 15376 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:32.916922 15376 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:32.893211 15392 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:32.919047 15392 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:32.918355 15376 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.920079 15392 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.923411 14419 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:32.924242 15447 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:32.928082 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.929313 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:32.913105 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:32.930817 15347 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:32.932475 15427 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:32.932951 15427 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:32.934172 15260 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:32.935308 15449 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:32.935791 15449 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:32.936206 15449 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.939620 15373 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:32.946575 15431 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:32.947309 15431 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:32.953476 15444 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.261s	user 0.003s	sys 0.000s
I20260709 12:21:32.883620 13797 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.965159 15257 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.968941 13793 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.970075 13644 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:32.970829 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 458) took 1175 ms. Trace:
I20260709 12:21:32.971271 13800 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.972711 13793 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.973493 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:32.972741 13800 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.971309 14416 rpcz_store.cc:276] 0709 12:21:31.795008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:31.795139 (+   131us) service_pool.cc:224] Handling call
0709 12:21:31.795495 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:32.531258 (+735763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:32.970538 (+439280us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:32.970735 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690120064}
I20260709 12:21:32.976598 15362 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:32.977196 15362 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:32.978421 15362 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:32.933877 15427 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:32.985224 15392 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:32.986382 15359 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.988051 15427 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:32.989821 15376 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.994170 13794 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:32.996121 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 198099us
I20260709 12:21:32.996928 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:32.998238 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:32.997969 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68772.2us
W20260709 12:21:32.999646 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.933883 15361 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:21:32.936463 15390 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.021978 13726 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:33.024709 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:33.025849 13797 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:33.027871 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:33.033051 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:33.033730 13650 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.034037 13650 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:33.036146 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:33.037469 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:33.047951 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48297.1us
I20260709 12:21:33.051820 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:33.034797 13650 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.056216 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:33.056803 13650 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:32.900792 15379 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:33.003616 15414 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:33.060242 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:33.067502 15438 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:33.062129 13718 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:33.067848 15437 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:33.072213 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:33.086529 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:33.054077 15354 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:33.087277 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:33.059005 15448 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:33.089978 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:33.090795 13796 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.064832 15400 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:33.093827 13793 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:33.014830 15412 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 12:21:33.107224 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 98510.9us
I20260709 12:21:33.099813 13793 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:33.102578 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:33.112967 13722 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:33.061237 15260 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:21:33.073740 15446 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:33.117885 13650 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:33.034396 15355 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 12:21:33.110384 13646 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:33.122962 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:32.896992 15425 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:33.115141 15416 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:21:33.127916 13797 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:33.127187 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:33.157531 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:33.167599 13650 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:33.132012 13721 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:33.170210 15431 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:33.172235 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:33.150132 13644 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:33.175148 15380 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.175725 15380 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.176218 14417 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:33.087610 15434 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:21:33.159010 15446 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.181581 15437 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:21:33.182329 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55240.9us
W20260709 12:21:33.129936 15453 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:21:33.220927 15437 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:33.238487 15380 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:33.232854 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70639.1us
I20260709 12:21:33.243827 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:33.244711 13647 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:33.151237 15425 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.143587 15454 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:33.259492 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:33.260200 13651 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:33.261307 13800 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:33.265937 13722 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:33.312368 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:33.157866 15366 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:21:33.158452 15379 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.252852 15368 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:21:33.237498 15437 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.311825 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:33.330090 13796 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:33.318962 15260 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:33.315284 15378 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:33.333240 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:33.340893 15450 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:33.341384 15450 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:33.337316 15368 consensus_queue.cc:1243] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:21:33.325793 15462 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:21:33.338718 13721 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:33.343784 15260 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:33.358175 15260 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.354923 15449 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.355657 15462 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:33.361007 15462 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:33.376549 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:33.371941 15462 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:33.382066 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:33.381894 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:33.382622 13796 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:33.387801 13796 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:33.391571 15464 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:33.399631 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:33.390646 15326 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:33.426615 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:33.409049 15326 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.428378 13724 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:33.403324 15380 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:21:33.431217 13651 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 2: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:33.410566 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:33.402642 15464 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:33.438087 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:33.430346 15326 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:33.443051 15446 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:33.445341 15379 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:33.451884 14424 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:33.432509 14418 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:33.454907 15443 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:33.455364 15443 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:33.456269 15443 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:33.457427 15443 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:33.452677 14420 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:33.454933 13648 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:33.463690 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:33.466650 15376 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:33.467053 15376 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:33.467383 15376 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:33.486111 15464 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:33.488445 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:33.478389 13800 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:33.431059 15449 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:33.489573 15378 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:21:33.516065 15392 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.003s
I20260709 12:21:33.516305 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:33.517206 13727 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:33.519807 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:33.509917 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:33.523440 13721 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:33.516408 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:33.528297 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:33.506153 15396 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:33.587801 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45973.5us
I20260709 12:21:33.538506 15392 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:33.560688 15464 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:21:33.636519 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:33.580364 15396 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:33.664772 15396 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:33.632510 15464 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:33.679795 15464 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:33.154886 15388 consensus_queue.cc:1175] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:21:33.909262 15326 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.413968 15463 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:33.909471 13725 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:33.910238 13718 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:33.925468 15392 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:33.967697 15392 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:33.960614 15376 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:33.919796 15362 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.974025 15362 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:33.975039 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:33.976104 14416 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:33.976856 15362 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:33.930466 15438 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:21:33.980923 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:33.981835 13644 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:33.951323 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:33.931770 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 389354us
I20260709 12:21:33.916431 15464 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:33.999990 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 88254us
W20260709 12:21:34.000831 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:34.001448 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:33.926416 15424 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:34.007974 15424 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.006240 13796 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:34.013947 15424 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:33.910399 15361 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.029438 13650 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:33.923635 15254 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:21:34.026223 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:33.963528 15379 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:33.998123 15464 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.068266 13727 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:34.023406 15458 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:21:34.237300 15355 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:34.237416 13649 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:21:34.224939 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 263400us
I20260709 12:21:34.230626 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.269521 15380 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:21:34.312708 15260 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.033505 15254 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:34.144096 14419 raft_consensus.cc:2955] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:34.377262 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:34.097220 15464 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 12:21:34.382887 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:34.033144 15424 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:34.386729 15424 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.378794 15254 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.388393 15254 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.389184 15424 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.029055 15366 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:34.089526 15362 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.390064 15362 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.045790 15361 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.392649 15362 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.071195 15379 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.393078 15361 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.395340 15379 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.081996 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:34.163906 15390 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:34.396449 15390 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.399169 15390 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:34.131918 15468 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:21:34.196609 15490 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:21:34.370597 15257 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.144034 15307 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.400955 15257 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.401388 15307 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.403208 15257 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.220093 15482 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.033309 15484 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.090058 15376 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.360713 15452 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:34.224505 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 205478us
I20260709 12:21:34.200754 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1438) took 1113 ms. Trace:
I20260709 12:21:34.070055 15487 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.198166 15405 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:34.074576 15368 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:21:34.119582 15438 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.270628 15326 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.250886 13644 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:34.371680 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.373135 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:34.371778 15354 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.375692 15260 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.376240 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:34.380247 13727 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:34.384003 14424 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.448395 13727 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:34.395686 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:34.356750 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:34.415413 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 150171us
I20260709 12:21:34.418510 15390 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:34.418630 13652 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.422811 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.427464 15254 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.452764 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:34.431162 15405 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:34.421386 13724 rpcz_store.cc:276] 0709 12:21:33.086906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:33.087028 (+   122us) service_pool.cc:224] Handling call
0709 12:21:34.200500 (+1113472us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54002f5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:34.200648 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694784768}
W20260709 12:21:34.405005 15443 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.947s	user 0.016s	sys 0.003s
I20260709 12:21:34.406230 15307 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.409956 15369 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.448136 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.383948 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:34.385324 15396 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.003s
W20260709 12:21:34.477501 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 444747us
I20260709 12:21:34.351289 15450 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.480268 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:34.481062 13718 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:34.429186 15257 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.484580 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:34.440263 14416 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:34.424607 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.384865 15355 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.457123 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.475061 13726 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:34.476971 15369 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.477519 15496 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:34.477460 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:34.467549 14424 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:34.410495 15482 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.444523 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.483623 13724 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:34.488252 13650 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:34.419144 15484 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.492700 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.499483 15500 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:34.492013 15257 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.514410 13727 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:34.446502 15354 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.438485 15438 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.486833 15396 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.486725 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:34.552935 15479 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:21:34.553164 15396 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.439810 15487 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.472466 15254 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.428897 15452 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.456600 15390 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.472126 15386 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:21:34.510365 15500 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:34.561158 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81383.3us
I20260709 12:21:34.543803 13644 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:34.499727 15489 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.555841 15395 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:34.571933 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:34.573249 13643 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:34.557399 15491 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:34.578918 13797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.035s	sys 0.030s
W20260709 12:21:34.606570 13797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.035s	sys 0.030s
I20260709 12:21:34.608527 13726 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:34.608800 13727 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:34.548349 15496 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.576242 14420 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:34.548048 15405 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.606815 15369 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.574616 15379 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:34.633876 15379 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.608986 15454 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:34.634511 15348 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.624575 15452 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.607872 15448 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:34.642838 15452 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.634074 15260 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.630960 15366 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:34.632000 15347 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.643347 15452 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.628746 15484 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.642478 15405 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.641501 15426 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:34.663826 15501 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.656448 15257 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.648780 15454 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.642333 15348 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.655169 15366 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.621150 15489 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.691846 15363 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:34.692490 15363 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.653134 15347 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.694938 15254 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.695566 15487 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.695698 15347 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.696136 15354 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.698094 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:34.699584 15363 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:34.699862 15463 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.000s
I20260709 12:21:34.679652 15452 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.658413 15482 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.630116 15491 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.707209 15348 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.676438 15355 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.677228 13652 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:34.664537 15501 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.645745 15448 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.689522 15257 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.729437 15496 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.726825 15438 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.731040 15500 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.731878 15390 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.703781 15489 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.746024 15497 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.735585 15482 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.747910 15497 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:34.749754 15495 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:21:34.668826 15426 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.750131 15503 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:34.755825 13719 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:34.753602 13725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 1961) took 2253 ms (client timeout 1613 ms). Trace:
I20260709 12:21:34.763209 15448 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:34.763147 13725 rpcz_store.cc:269] 0709 12:21:32.500882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:32.501003 (+   121us) service_pool.cc:224] Handling call
0709 12:21:34.753351 (+2252348us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54000f6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:34.753502 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294408960}
I20260709 12:21:34.740716 15257 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.753358 15491 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.786803 15257 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.754020 15482 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.799575 15482 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.804901 15423 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.779405 15448 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.778834 15503 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:34.811820 13720 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:34.816380 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1430) took 1939 ms. Trace:
I20260709 12:21:34.821210 13720 rpcz_store.cc:276] 0709 12:21:32.876939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:32.877105 (+   166us) service_pool.cc:224] Handling call
0709 12:21:32.877430 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:34.811351 (+1933921us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54001eb520. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:21:34.812408 (+  1057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:34.816039 (+  3631us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54003fe3d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:34.816269 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414813824}
I20260709 12:21:34.819558 15257 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:34.825390 15426 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.826607 15370 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:34.833925 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 356532us
W20260709 12:21:34.834378 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 258558us
I20260709 12:21:34.805538 15423 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.839138 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:34.842324 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:34.831259 15370 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.845137 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.832943 15257 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.844568 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:34.846930 13800 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:34.851226 13797 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.847541 13801 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:34.845783 13647 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:34.845605 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.853729 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.854401 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:34.842003 15254 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:21:34.855196 15452 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.855798 15452 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.840181 15416 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:21:34.858122 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:34.862550 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:34.811587 15448 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.858691 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:34.861106 15510 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.861626 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.863245 13720 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:34.866959 13793 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:34.844107 15390 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.865509 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:34.867946 13651 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:34.865939 13799 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:34.857194 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:34.869298 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:34.834461 13721 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:34.861677 15362 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:34.870570 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:34.871877 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:34.859556 15501 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.871068 15358 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.876194 15484 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:34.876165 15456 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:34.884359 15484 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.884889 15484 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.880434 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:34.845906 15454 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.879036 15257 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:34.878814 15497 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.871619 15416 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:34.872941 15390 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.880098 14420 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:34.883381 15448 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.880306 15510 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.865620 15503 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:34.892799 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:34.892896 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:34.889645 15416 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.897222 13797 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:34.874383 15463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:54754 (request call id 78) took 2989 ms. Trace:
I20260709 12:21:34.880471 15362 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.909384 15423 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.884680 15456 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:34.890604 15491 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:34.897545 15254 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:34.887224 15358 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.932967 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.956143 13545 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 227) took 1902 ms. Trace:
I20260709 12:21:34.936275 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.959925 13545 rpcz_store.cc:276] 0709 12:21:33.053873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:33.054016 (+   143us) service_pool.cc:224] Handling call
0709 12:21:33.247422 (+193406us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:21:33.470654 (+223232us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393bfcf3a 00007f53999b2e9c 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:34.956047 (+1485393us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":113745536}
I20260709 12:21:34.960803 13798 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:34.937067 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.962199 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.928544 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:34.933204 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.949455 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.968174 13793 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:34.963156 13798 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:34.972093 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:34.973238 13798 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:34.924588 15450 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:34.927400 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 1972) took 1824 ms. Trace:
I20260709 12:21:34.976501 13719 rpcz_store.cc:276] 0709 12:21:33.103485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:33.103633 (+   148us) service_pool.cc:224] Handling call
0709 12:21:34.753965 (+1650332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:34.756232 (+  2267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:34.927114 (+170882us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54000f6c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:34.927316 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3794756352}
I20260709 12:21:34.976189 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:34.931661 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:34.981743 13797 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:34.965654 13727 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:34.985536 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.014071 13795 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:34.924818 15362 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:34.929576 15452 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.929379 15390 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:35.014061 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:34.945952 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.014365 15510 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:35.020159 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:35.022157 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:34.945668 15464 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:34.986460 15491 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:34.981463 15462 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:34.924679 15370 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:34.923220 15504 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:34.950198 15254 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:34.926983 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.026808 15423 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.032812 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.030700 15501 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:35.102171 15501 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.104599 15501 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:35.040050 15362 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:35.033188 15454 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.016993 15323 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.043001 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.025027 15347 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:35.095742 13650 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:35.112955 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:35.113744 13650 raft_consensus.cc:606] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:35.098147 15513 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.035321 15426 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.100309 15427 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.094723 15254 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:35.046679 15260 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.100554 15491 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.101619 13721 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:35.106541 15362 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.109741 13797 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:35.030728 15448 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:35.111500 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89413.1us
I20260709 12:21:35.116279 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.039906 15464 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.135025 15354 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.096581 15257 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.108104 15390 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.111106 15386 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.135614 15513 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.157660 15390 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.157791 15386 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.159709 15513 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:35.143132 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45957.5us
I20260709 12:21:35.135712 15358 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:35.125172 15452 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.160792 15509 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.160940 15452 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.149418 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:35.116962 15438 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.136147 15434 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:35.170738 15434 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.171168 15434 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:35.171298 15386 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.160007 15390 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.172966 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:35.106747 15454 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.162882 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:35.161468 15509 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.149322 15362 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:35.136924 15323 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.167667 15452 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.171924 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.140841 15427 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:34.922665 15463 rpcz_store.cc:276] 0709 12:21:31.881355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:31.881866 (+   511us) service_pool.cc:224] Handling call
0709 12:21:31.968141 (+ 86275us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b4c00092900. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5391ec0a20 00007f5391ed1a29 00007f5391e97644 00007f5391ebef2d 00007f5391ebeb93 00007f5393b4d1af 00007f5393b4cf3e 00007f5393bd2888 00007f5393bef75c 00007f5393bf076a 00007f5393bef91b
0709 12:21:32.029477 (+ 61336us) log.cc:844] Serialized 494 byte log entry
0709 12:21:32.356350 (+326873us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b1800149e00. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:34.864950 (+2508600us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40366,"raft.queue_time_us":59513,"raft.run_cpu_time_us":1106,"raft.run_wall_time_us":80079,"spinlock_wait_cycles":20093568,"thread_start_us":301652,"threads_started":2,"wal-append.queue_time_us":92707,"wal-append.run_cpu_time_us":1914,"wal-append.run_wall_time_us":1052919}
I20260709 12:21:35.174357 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.193150 13722 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:35.145548 15260 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.195358 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.196127 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:35.197696 15260 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:35.165699 15438 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.200574 15438 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.139859 15426 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.176499 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:35.184378 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:35.116096 15423 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.181782 15424 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:35.186369 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:35.195145 15368 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.156468 15354 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.204109 13720 raft_consensus.cc:2368] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
I20260709 12:21:35.156286 14416 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:35.153085 15448 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.209177 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:35.211885 15509 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.213654 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:35.187022 15347 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:35.162179 15257 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.224033 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.224746 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.204603 15452 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:35.205852 15424 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.207660 15260 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:35.226784 15483 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:35.237329 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.209815 15491 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:21:35.217675 15368 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.259238 13723 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:35.259411 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.261670 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 1976) took 2019 ms. Trace:
I20260709 12:21:35.233480 15424 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:35.247390 15427 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.227694 15347 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.251919 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.272442 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.253540 15257 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.254918 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.278884 13720 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:35.235466 15452 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:35.268003 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:35.269934 15497 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:35.273175 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:35.307626 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.262068 13723 rpcz_store.cc:276] 0709 12:21:33.242308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:33.242421 (+   113us) service_pool.cc:224] Handling call
0709 12:21:33.242845 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:35.259631 (+2016786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:35.261403 (+  1772us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54000f7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:35.261607 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4636392320}
I20260709 12:21:35.233525 15260 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.276763 13795 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:35.313563 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.305581 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:35.311659 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.314934 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
W20260709 12:21:35.316155 15485 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.840s	user 0.008s	sys 0.000s
I20260709 12:21:35.311055 15348 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:21:35.319515 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.321887 15348 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:35.322355 15348 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:35.322775 15348 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:35.323873 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.324482 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.325852 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.326448 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:35.277504 15347 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:35.306979 15452 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:35.332402 15354 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.333854 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.333973 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.378161 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.378468 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:35.312883 15497 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.335273 15423 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.338615 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.339778 15454 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.341197 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.342017 15464 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:35.395153 13720 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:35.343943 15504 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.343999 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.317471 15260 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:35.344875 15405 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:35.341246 15448 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:35.347256 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:35.349201 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.352927 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.404642 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.405759 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.354135 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.356822 15431 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.357499 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.359881 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.408524 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.362306 13547 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 248) took 2900 ms. Trace:
I20260709 12:21:35.363040 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.363834 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.364176 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.363876 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:35.365496 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.388823 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.390282 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.391527 13719 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:35.413858 13796 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:35.386511 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:35.358233 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.411523 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:35.420635 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:35.415340 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.410568 13795 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
W20260709 12:21:35.422983 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59190.9us
I20260709 12:21:35.389688 15438 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:35.411091 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:35.385989 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 311) took 1353 ms. Trace:
I20260709 12:21:35.430511 13795 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:35.386533 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.401489 15504 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:35.425304 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.409448 13547 rpcz_store.cc:276] 0709 12:21:32.441363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:32.441743 (+   380us) service_pool.cc:224] Handling call
0709 12:21:35.342141 (+2900398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:33.130095 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:33.130779 (+   684us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:33.130822 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:21:33.130842 (+    20us) tablet.cc:662] Decoding operations
0709 12:21:33.131205 (+   363us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:33.131282 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:33.131317 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:21:33.131526 (+   209us) tablet.cc:701] Row locks acquired
0709 12:21:33.131544 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:21:33.131693 (+   149us) write_op.cc:270] Start()
0709 12:21:33.131812 (+   119us) write_op.cc:276] Timestamp: P: 1783599693131650 usec, L: 0
0709 12:21:33.131837 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:21:33.132559 (+   722us) log.cc:844] Serialized 533 byte log entry
0709 12:21:33.441616 (+309057us) op_driver.cc:464] REPLICATION: finished
0709 12:21:33.921413 (+479797us) write_op.cc:301] APPLY: starting
0709 12:21:33.921504 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:33.921612 (+   108us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:33.997857 (+ 76245us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:34.026573 (+ 28716us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:34.070338 (+ 43765us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:21:34.079582 (+  9244us) write_op.cc:312] APPLY: finished
0709 12:21:34.130462 (+ 50880us) log.cc:844] Serialized 23 byte log entry
0709 12:21:34.151901 (+ 21439us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:34.161752 (+  9851us) write_op.cc:454] Released schema lock
0709 12:21:34.172927 (+ 11175us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:34.228677 (+ 55750us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:34.228941 (+   264us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":767893,"reported_tablets":1,"spinlock_wait_cycles":82816,"child_traces":[["op",{"apply.queue_time_us":479496,"apply.run_cpu_time_us":3097,"apply.run_wall_time_us":291239,"num_ops":1,"prepare.queue_time_us":299086,"prepare.run_cpu_time_us":8840,"prepare.run_wall_time_us":134336,"raft.queue_time_us":24050,"raft.run_cpu_time_us":49116,"raft.run_wall_time_us":462057,"replication_time_us":309559,"thread_start_us":526174,"threads_started":4,"wal-append.queue_time_us":188320,"wal-append.run_cpu_time_us":13412,"wal-append.run_wall_time_us":1858169}]]}
I20260709 12:21:35.425904 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.380275 15439 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:21:35.402364 15447 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.423038 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:35.453740 13792 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:35.409780 15426 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.422468 15431 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.588891 15520 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.603418 15520 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.604513 15431 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:35.432986 15484 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.005s
I20260709 12:21:35.582145 15386 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:35.606019 15386 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.606491 15386 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:35.423619 15509 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:35.426784 15405 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:35.619711 15379 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:35.620267 15405 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.427096 15378 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:35.622473 15378 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:35.622640 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.622924 15368 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.427361 15438 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.427788 15396 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:35.605723 15484 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.426884 15448 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.622965 15378 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:35.638028 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.421530 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:35.645893 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 224459us
I20260709 12:21:35.649148 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.614210 15366 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.593750 13721 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
W20260709 12:21:35.639865 15462 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.602s	user 0.007s	sys 0.006s
I20260709 12:21:35.630314 15438 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:35.592798 15447 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.689429 15513 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.689970 15388 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:35.690102 15513 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.693095 15513 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.693899 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:35.694211 15257 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.695358 15388 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:35.626403 15323 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.630607 15396 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.696112 15396 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:35.696030 15519 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:35.696826 15484 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:35.678015 15523 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:21:35.699106 15513 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.694957 15257 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:35.700093 15254 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.554s	user 0.007s	sys 0.003s
I20260709 12:21:35.699687 15513 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.702697 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:35.707871 14424 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:35.672705 15517 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.423s	user 0.001s	sys 0.000s
I20260709 12:21:35.709666 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:35.662483 15423 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.662246 15454 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:35.728991 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:35.663364 15483 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:35.738672 15497 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:35.708441 15395 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.739786 15395 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.740176 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.670414 13766 leader_election.cc:400] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
I20260709 12:21:35.643540 15448 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.761938 13766 leader_election.cc:403] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:21:35.701619 15257 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.707533 15388 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:35.700529 15254 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.683599 15260 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.701920 15513 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.915833 15366 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:35.766826 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.921578 15366 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.881551 15358 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.800872 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:35.888526 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:21:35.923621 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122827us
I20260709 12:21:35.767591 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.923170 15378 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.730870 15379 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:35.767297 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.766165 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.948625 13718 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:35.800276 15483 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:35.806818 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 160103us
I20260709 12:21:35.430855 14418 rpcz_store.cc:276] 0709 12:21:34.032718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:34.034069 (+  1351us) service_pool.cc:224] Handling call
0709 12:21:35.233965 (+1199896us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:35.249815 (+ 15850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:35.286961 (+ 37146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:35.305342 (+ 18381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:35.308776 (+  3434us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:35.318214 (+  9438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:35.328568 (+ 10354us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:35.380002 (+ 51434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10270080}
I20260709 12:21:35.765767 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.921977 13725 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:35.935590 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:35.957748 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:35.766556 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.834630 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.936905 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.930388 15513 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:35.938069 15528 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:21:35.938880 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.940249 15448 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.940762 15392 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.941565 15447 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:35.941601 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.942406 15366 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:35.802018 15446 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
W20260709 12:21:35.925362 15362 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.000s
W20260709 12:21:35.909798 15434 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.002s
I20260709 12:21:35.946849 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:36.002574 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77861.8us
I20260709 12:21:36.002561 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:36.005748 13649 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:36.007076 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.008558 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.959899 13652 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:35.758912 15454 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.886673 15491 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.959093 13720 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:35.970844 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.021804 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.998348 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.970023 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.029227 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.030064 13720 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:36.027554 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:35.908907 15520 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:36.033201 15424 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.755s	user 0.008s	sys 0.005s
I20260709 12:21:35.994150 15392 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.033028 15536 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:35.874197 15323 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:35.925796 15416 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.059307 15494 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:35.932344 15388 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:35.930941 15497 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:35.992017 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.759706 15423 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:35.950711 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:35.968781 15369 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:35.846942 15386 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.015407 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.015079 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:35.993947 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:35.950172 15257 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:21:36.022737 13724 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:35.797900 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:35.923166 15358 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:35.742311 14416 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:35.988025 15513 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.029274 13725 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:21:35.976432 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.037253 13646 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:36.037276 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.039264 15378 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:36.041033 15348 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.718s	user 0.013s	sys 0.000s
I20260709 12:21:36.041078 15260 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.042681 15443 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.043423 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:36.044524 15452 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.651s	user 0.003s	sys 0.007s
I20260709 12:21:35.999380 15447 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.047624 15456 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.206601 15358 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.047887 15254 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.013967 15373 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:21:35.741812 15368 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:36.014873 15362 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.014412 15434 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:36.100595 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47672.6us
I20260709 12:21:36.073251 15462 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:36.113217 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1540) took 1655 ms (client timeout 1903 ms). Trace:
I20260709 12:21:36.032845 15491 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.161134 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.162351 15405 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:36.165521 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.232041 13651 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:36.175735 13721 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:36.098119 15494 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.098392 15537 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.079686 15424 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.098599 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.178323 13726 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.174459 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
W20260709 12:21:36.190948 15532 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:21:36.191417 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.170320 13645 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:36.181555 15348 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.202549 15452 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.102625 15323 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.166011 15379 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:36.102646 15416 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:36.219026 13643 rpcz_store.cc:269] 0709 12:21:34.445880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:34.447802 (+  1922us) service_pool.cc:224] Handling call
0709 12:21:36.101281 (+1653479us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54002b39d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:36.101419 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226216704}
I20260709 12:21:36.113257 15497 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.232817 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1546) took 1372 ms. Trace:
I20260709 12:21:36.210568 15443 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.257360 13651 rpcz_store.cc:276] 0709 12:21:34.860233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:34.868587 (+  8354us) service_pool.cc:224] Handling call
0709 12:21:36.165355 (+1296768us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:36.232748 (+ 67393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3358080}
W20260709 12:21:36.169479 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113153us
I20260709 12:21:36.258230 13649 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:21:36.219667 15504 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.004s
W20260709 12:21:36.219450 15347 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.884s	user 0.006s	sys 0.007s
I20260709 12:21:36.214161 15368 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.239476 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.241078 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:36.276752 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.277799 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:36.241580 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226621us
I20260709 12:21:36.176878 15513 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.231834 15405 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.230165 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:36.273813 15497 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.259855 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.242064 15494 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:36.260805 13726 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:36.289443 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.289577 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:36.276600 15368 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.291869 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:36.292171 13726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48612be3c56f482c81533c30abebe60b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:43540
I20260709 12:21:36.245577 15520 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.292732 13726 raft_consensus.cc:606] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:36.293138 13726 raft_consensus.cc:614] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:36.292950 15520 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.277763 15369 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:36.294466 15254 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:36.294505 15369 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.295145 15254 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:36.295619 15369 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.295717 15513 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.290815 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.250614 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:36.301565 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:36.302095 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.303445 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.305619 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.267630 15443 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.267558 15416 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.275092 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.275722 15491 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.279088 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:36.323536 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44589.2us
I20260709 12:21:36.324506 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:36.233917 15462 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.337479 15462 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.258916 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:36.338716 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 79885.8us
I20260709 12:21:36.338783 15358 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.339326 15545 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:36.339504 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:36.339869 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.218696 15447 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.287108 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.341758 15510 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:36.342216 15510 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.343132 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.343330 15510 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.344061 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.344440 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.342320 14416 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:36.339533 15358 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.347023 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.347867 13793 raft_consensus.cc:2410] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:21:36.348349 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.342286 13645 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:36.342922 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.349476 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.349546 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.261888 15392 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.264415 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.244906 15537 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.303918 15520 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.287032 15454 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.314535 15434 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.280282 15504 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.286151 15405 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.315943 15452 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.328001 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:36.328389 15438 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.645s	user 0.007s	sys 0.006s
I20260709 12:21:36.297377 15254 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.334720 15513 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:36.349809 13801 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:36.350216 13793 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:36.341797 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.352146 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:36.404193 15431 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:36.404357 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52338.3us
I20260709 12:21:36.404969 15368 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.343236 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.406347 15513 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.287220 15379 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.402798 15438 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.407403 15368 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.408833 15395 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.410028 15438 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:36.412809 15396 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.716s	user 0.009s	sys 0.002s
I20260709 12:21:36.413290 15396 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.413628 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.414897 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:36.415465 15521 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.230s	user 0.005s	sys 0.006s
I20260709 12:21:36.416159 15504 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.416195 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.417524 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.303812 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:36.418694 14419 raft_consensus.cc:2368] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:21:36.419634 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.420478 13722 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:36.421545 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.422955 13767 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:36.423700 15544 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:21:36.424304 15497 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:36.294937 15386 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.418069 13727 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:36.432571 14419 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:36.323165 15380 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:36.324808 15547 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:36.433200 15527 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:21:36.476073 15531 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:36.475906 15547 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.476701 15531 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:36.477610 15549 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:21:36.433010 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:36.386988 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:36.437476 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.388468 15348 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.283404 15347 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.357712 15462 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:36.385217 14420 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 1.
I20260709 12:21:36.347487 15486 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:21:36.395403 15537 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.348531 15358 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.298574 14418 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:36.406769 15513 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.475062 15401 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:36.492437 15401 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.481840 15380 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.493008 15494 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:36.442220 15499 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:21:36.437889 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:36.321079 15446 raft_consensus.cc:2764] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:21:36.358294 15443 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:36.447054 15493 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.441645 15497 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:36.422957 15546 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:21:36.483804 15531 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.479907 15547 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.456969 15323 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.425469 15424 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.487475 15347 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.487882 15462 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.489018 15396 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.428649 15362 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.508875 15497 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.459870 15388 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.429934 15447 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.429265 15423 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.447353 15503 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.501066 15510 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.516865 15388 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.518841 15510 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:36.504835 15370 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.006s	sys 0.005s
I20260709 12:21:36.508955 15401 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.519863 15494 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:36.520529 15370 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.006s	sys 0.005s
I20260709 12:21:36.515286 15503 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.506152 15493 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.502043 15443 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.529989 15443 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.281551 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:36.312090 13644 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:36.643985 13646 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
I20260709 12:21:36.357883 13651 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:21:36.643733 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:36.329809 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:36.325847 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:36.647796 13720 raft_consensus.cc:2368] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:21:36.432385 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.431546 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.681077 15424 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:36.349902 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.674105 15347 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:36.683267 13645 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:36.683408 15347 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:36.468238 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:36.653582 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:36.658389 13801 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 3: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:36.661202 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 309) took 2662 ms. Trace:
I20260709 12:21:36.610462 13545 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 4, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 4 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:36.671077 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.672638 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:36.674046 15323 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:36.678853 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1543) took 1835 ms (client timeout 1922 ms). Trace:
W20260709 12:21:36.694411 13648 rpcz_store.cc:269] 0709 12:21:34.843397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:34.843841 (+   444us) service_pool.cc:224] Handling call
0709 12:21:35.213498 (+369657us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:36.678187 (+1464689us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400113354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:36.678527 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604981504}
I20260709 12:21:36.693089 14417 rpcz_store.cc:276] 0709 12:21:33.998836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:34.001938 (+  3102us) service_pool.cc:224] Handling call
0709 12:21:34.016833 (+ 14895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:34.017379 (+   546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:34.017413 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:34.065758 (+ 48345us) log.cc:844] Serialized 26 byte log entry
0709 12:21:34.394053 (+328295us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b
0709 12:21:34.455126 (+ 61073us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:34.478015 (+ 22889us) log.cc:844] Serialized 14 byte log entry
0709 12:21:34.540203 (+ 62188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:34.547917 (+  7714us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:34.798478 (+250561us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b18000d40e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:36.045114 (+1246636us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18000d40e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1
0709 12:21:36.317223 (+272109us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b18000d40e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:36.655646 (+338423us) raft_consensus.cc:1685] finished
0709 12:21:36.660357 (+  4711us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:36.660758 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7425,"prepare.queue_time_us":186,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":7194,"raft.queue_time_us":64307,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":16075,"spinlock_wait_cycles":25187584,"thread_start_us":346075,"threads_started":1,"wal-append.queue_time_us":397835}
I20260709 12:21:36.697340 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:36.699169 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 373288us
I20260709 12:21:36.699616 13648 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:36.699842 15379 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:36.700503 15379 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.700974 15379 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:36.701300 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.702142 15388 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:36.703159 13648 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
W20260709 12:21:36.707677 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 426015us
I20260709 12:21:36.707986 15493 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.709358 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:36.701479 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:36.710131 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:36.712605 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:36.713100 13793 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:36.714555 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65937us
I20260709 12:21:36.715181 15378 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:36.717514 15503 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.704795 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:36.721316 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49223.7us
W20260709 12:21:36.722191 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 253968us
I20260709 12:21:36.722431 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.723601 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:36.724346 15370 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:36.725734 15370 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:36.724367 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.728631 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:36.719378 15497 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.696862 13545 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:36.736671 15547 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:36.729902 14417 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:36.742219 15401 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:36.743072 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.745030 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:36.746061 15323 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.746801 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:36.747269 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:36.747756 13724 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:36.747885 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:36.746801 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.748920 15323 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.749127 13643 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:36.749454 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:36.750306 13797 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:36.750898 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:36.751870 13799 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.746203 13725 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:36.750321 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:36.748922 13796 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:36.730332 15370 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.759904 15370 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:36.768514 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.734679 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.787317 13793 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:36.761844 15462 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:36.800688 15347 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:36.803509 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.805747 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.803977 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:36.807273 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 106064us
I20260709 12:21:36.811494 15357 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:36.812117 15357 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.815033 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.816788 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:36.818140 13643 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.818540 15388 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:36.818720 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:36.819031 15388 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:36.819434 15388 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:36.788856 15497 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.822142 15503 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:36.822649 15503 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:36.823173 15503 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:36.822729 15497 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.825815 15497 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.826716 15510 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:36.827219 15510 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:36.827593 15510 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:36.837286 15369 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.541s	user 0.018s	sys 0.010s
I20260709 12:21:36.837898 15369 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:36.838817 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.839737 13650 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:36.840601 15369 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.840824 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.842597 14424 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:36.837330 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:36.843125 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.847710 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:36.848806 14418 raft_consensus.cc:2368] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3.
I20260709 12:21:36.850212 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.851511 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.895823 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:36.840865 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.039064 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:36.901981 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 192935us
I20260709 12:21:37.046217 13799 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
W20260709 12:21:37.047224 15557 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.157s	user 0.010s	sys 0.004s
I20260709 12:21:37.029211 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:37.027887 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53714.3us
I20260709 12:21:37.054984 13643 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:36.896462 15357 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:36.834919 15347 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:37.025310 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:36.909993 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:37.062666 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 258826us
I20260709 12:21:37.065953 13615 leader_election.cc:400] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:21:37.066493 13615 leader_election.cc:403] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:21:37.067210 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 155341us
I20260709 12:21:37.068699 15348 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:21:37.068797 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:37.071250 15547 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:37.071820 15547 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:37.072273 15547 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.073356 15524 raft_consensus.cc:993] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:37.074021 15524 raft_consensus.cc:1081] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:21:37.027434 15531 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:37.074790 15526 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.075462 15526 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.075922 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:37.077790 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:37.078778 13767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.848s (SENT) [suppressed 5 similar messages]
I20260709 12:21:37.079250 15446 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:37.082499 13795 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:21:37.090939 15446 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.093724 15446 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.094590 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.095132 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.095582 13649 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:37.096922 13649 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:37.097544 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1425) took 1122 ms. Trace:
W20260709 12:21:37.101122 15443 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.005s
I20260709 12:21:37.104504 15370 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:37.112720 15513 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.019s
I20260709 12:21:37.113381 15513 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.112048 13795 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:37.118048 15464 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:21:37.082346 15405 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.010s
I20260709 12:21:37.097855 13649 rpcz_store.cc:276] 0709 12:21:35.975044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.096295 (+1121251us) service_pool.cc:224] Handling call
0709 12:21:37.096707 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.097284 (+   577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.097454 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.126677 13724 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:37.128036 15464 consensus_queue.cc:1243] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:21:37.132025 13615 leader_election.cc:400] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3.
I20260709 12:21:37.132762 13615 leader_election.cc:403] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:21:37.078817 15369 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:37.071345 15537 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.101758 15443 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:37.147742 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70043.3us
I20260709 12:21:37.088512 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.152529 13647 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:37.153712 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:37.134140 15378 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:37.139892 15464 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:37.130033 15446 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:37.156172 15370 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:37.158030 15513 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.159822 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:37.160354 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:37.160983 15561 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:21:37.168430 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1430) took 1149 ms. Trace:
I20260709 12:21:37.178674 13795 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:37.178953 15424 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.181983 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:37.183399 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:37.142618 15405 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.184674 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:37.184754 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:37.185467 13652 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:37.186563 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:37.187808 15405 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.187985 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2004) took 1419 ms. Trace:
I20260709 12:21:37.188884 15395 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:37.194757 13720 rpcz_store.cc:276] 0709 12:21:35.768423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:36.036919 (+268496us) service_pool.cc:224] Handling call
0709 12:21:37.187775 (+1150856us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54001f0754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:37.187917 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091996032}
I20260709 12:21:37.195716 15395 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:37.196209 15395 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:37.196667 15395 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.186914 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 1995) took 1421 ms. Trace:
I20260709 12:21:37.172549 15537 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.204212 13722 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:37.176767 13647 rpcz_store.cc:276] 0709 12:21:36.019000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:36.909629 (+890629us) service_pool.cc:224] Handling call
0709 12:21:37.168309 (+258680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.126458 15347 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:37.210871 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 370067us
I20260709 12:21:37.145346 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:37.199944 13719 rpcz_store.cc:276] 0709 12:21:35.765400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:35.765532 (+   132us) service_pool.cc:224] Handling call
0709 12:21:37.186543 (+1421011us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:37.186714 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498719104}
I20260709 12:21:37.219375 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:37.169603 15464 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:37.225931 13799 raft_consensus.cc:606] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.234949 13799 raft_consensus.cc:614] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:37.238745 15526 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.226131 13648 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.252372 15369 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:37.252956 15369 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:37.253602 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:37.255085 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 190160us
I20260709 12:21:37.256910 15369 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:37.264355 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50040.4us
I20260709 12:21:37.267175 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1431) took 1218 ms. Trace:
I20260709 12:21:37.267699 13649 rpcz_store.cc:276] 0709 12:21:36.044990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.167237 (+1122247us) service_pool.cc:224] Handling call
0709 12:21:37.229906 (+ 62669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.262691 (+ 32785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.262720 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.262768 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:37.262801 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.262860 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.263004 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.271692 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1549) took 1186 ms. Trace:
I20260709 12:21:37.243292 15380 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:37.266995 15379 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.002s
I20260709 12:21:37.284103 13727 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:37.244072 15510 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.272187 13644 rpcz_store.cc:276] 0709 12:21:36.085481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.252945 (+1167464us) service_pool.cc:224] Handling call
0709 12:21:37.270886 (+ 17941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.271164 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.271197 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.271263 (+    66us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:37.271305 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.271368 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.271491 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28672}
W20260709 12:21:37.375744 15558 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.309s	user 0.002s	sys 0.004s
I20260709 12:21:37.411990 13720 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:37.337937 15464 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.267998 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1437) took 1216 ms. Trace:
W20260709 12:21:37.415849 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 337102us
I20260709 12:21:37.415254 13647 rpcz_store.cc:276] 0709 12:21:36.047011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.211960 (+1164949us) service_pool.cc:224] Handling call
0709 12:21:37.247549 (+ 35589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.262585 (+ 15036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.262626 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.262695 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:37.262758 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.262847 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.262995 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:37.425527 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:37.430609 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:37.441802 15537 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.443092 15526 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:37.443276 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.441577 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:37.443748 15526 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.327205 15379 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:37.454649 13797 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.704s	user 0.020s	sys 0.005s
I20260709 12:21:37.458619 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:37.459084 13797 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:37.453850 13727 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:37.457693 15443 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.484154 13795 raft_consensus.cc:606] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:37.458184 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:37.487394 13647 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:37.456560 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1554) took 1353 ms. Trace:
I20260709 12:21:37.488715 13649 rpcz_store.cc:276] 0709 12:21:36.087018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.274800 (+1187782us) service_pool.cc:224] Handling call
0709 12:21:37.323140 (+ 48340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.420051 (+ 96911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.420086 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.431246 (+ 11160us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:37.439949 (+  8703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.440066 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.440256 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:37.483152 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67440us
I20260709 12:21:37.446791 15464 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:37.495909 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 229140us
I20260709 12:21:37.499414 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:37.499936 15510 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:37.446228 15526 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.504798 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:37.484920 15357 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:37.509510 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 389674us
W20260709 12:21:37.511096 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 320444us
I20260709 12:21:37.535321 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:37.535753 15379 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.537256 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.543831 15555 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:37.537940 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:37.544912 15555 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:37.546651 15379 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:37.547195 15379 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:37.544514 13643 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:21:37.512722 15497 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:37.539901 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1558) took 1445 ms. Trace:
I20260709 12:21:37.549448 13650 rpcz_store.cc:276] 0709 12:21:36.094291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.538766 (+1444475us) service_pool.cc:224] Handling call
0709 12:21:37.539206 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.539473 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.539504 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.539576 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:37.539640 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.539702 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.539827 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:37.557008 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71683.3us
I20260709 12:21:37.548247 15379 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:37.564479 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:37.563431 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1556) took 1475 ms. Trace:
I20260709 12:21:37.566803 13649 rpcz_store.cc:276] 0709 12:21:36.087592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.512327 (+1424735us) service_pool.cc:224] Handling call
0709 12:21:37.551439 (+ 39112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.562982 (+ 11543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.563041 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.563092 (+    51us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:37.563224 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.563281 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.563376 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.570685 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1557) took 1476 ms. Trace:
I20260709 12:21:37.571177 13647 rpcz_store.cc:276] 0709 12:21:36.093919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.526760 (+1432841us) service_pool.cc:224] Handling call
0709 12:21:37.568335 (+ 41575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.570250 (+  1915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.570308 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.570360 (+    52us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:37.570400 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.570465 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.570597 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.556195 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1555) took 1468 ms. Trace:
I20260709 12:21:37.577678 13718 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:37.578379 15424 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3.
I20260709 12:21:37.579144 15424 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:37.579734 15424 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.580350 15378 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:37.581058 15378 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:37.581473 15378 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:37.581845 15424 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.564993 15357 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:37.586766 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.581871 15369 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.592589 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.594071 15357 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:37.592839 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1561) took 1494 ms. Trace:
I20260709 12:21:37.595679 15369 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:37.595433 15357 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:37.601140 15348 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
I20260709 12:21:37.601632 15348 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:21:37.574229 13644 rpcz_store.cc:276] 0709 12:21:36.087311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.379298 (+1291987us) service_pool.cc:224] Handling call
0709 12:21:37.503507 (+124209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.512626 (+  9119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.514436 (+  1810us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.521815 (+  7379us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:37.526691 (+  4876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.530043 (+  3352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.553557 (+ 23514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.602164 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.603031 14424 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:37.597368 14419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:21:37.603744 13644 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:37.604228 14419 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:37.604552 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1562) took 1506 ms. Trace:
I20260709 12:21:37.604776 14419 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:37.605058 13644 rpcz_store.cc:276] 0709 12:21:36.098242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.602892 (+1504650us) service_pool.cc:224] Handling call
0709 12:21:37.603344 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.604224 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.604485 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.598402 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:37.607021 14417 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:37.599478 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:37.611635 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1552) took 1525 ms (client timeout 1848 ms). Trace:
I20260709 12:21:37.612329 15531 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:37.613363 15531 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.595355 13649 rpcz_store.cc:276] 0709 12:21:36.097976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.575088 (+1477112us) service_pool.cc:224] Handling call
0709 12:21:37.584419 (+  9331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.592473 (+  8054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.592504 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.592554 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:37.592596 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.592659 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.592778 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":938,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":149}
I20260709 12:21:37.624388 15564 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:37.624508 15566 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:37.628137 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1559) took 1533 ms. Trace:
I20260709 12:21:37.628559 13650 rpcz_store.cc:276] 0709 12:21:36.094605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.563196 (+1468591us) service_pool.cc:224] Handling call
0709 12:21:37.615881 (+ 52685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.622805 (+  6924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:37.622849 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:37.622913 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:37.627933 (+  5020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:37.627984 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.628086 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:37.632926 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:37.632791 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.566336 15555 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.643775 14425 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:21:37.634033 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74877.6us
W20260709 12:21:37.612215 13646 rpcz_store.cc:269] 0709 12:21:36.086424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:36.276409 (+189985us) service_pool.cc:224] Handling call
0709 12:21:37.611393 (+1334984us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400096854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:37.611558 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":840491904}
I20260709 12:21:37.641283 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.634241 15566 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.657440 13643 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:37.655405 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:37.629357 13648 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:37.660318 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.639150 15369 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:37.668289 15456 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:21:37.669617 15566 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.667328 15369 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.678845 15369 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:37.679041 15503 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.855s	user 0.014s	sys 0.009s
I20260709 12:21:37.679114 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:37.679504 15503 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.680537 14420 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:37.692728 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:37.684973 13652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1442) took 1455 ms (client timeout 1769 ms). Trace:
W20260709 12:21:37.696254 13652 rpcz_store.cc:269] 0709 12:21:36.229906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:36.277541 (+ 47635us) service_pool.cc:224] Handling call
0709 12:21:37.684736 (+1407195us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:37.684905 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2085065472}
I20260709 12:21:37.702075 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.640261 15395 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.667158 15348 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:37.719726 15562 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:37.720630 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:37.724242 15547 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.651s	user 0.014s	sys 0.000s
I20260709 12:21:37.724751 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2014) took 1911 ms. Trace:
I20260709 12:21:37.725132 13644 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:37.708060 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.681869 15503 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:37.706398 15370 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:21:37.734338 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 212374us
I20260709 12:21:37.740741 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:37.749759 13643 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:21:37.728983 15348 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:37.732896 13723 rpcz_store.cc:276] 0709 12:21:35.799253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:36.447473 (+648220us) service_pool.cc:224] Handling call
0709 12:21:36.481095 (+ 33622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:36.481530 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:36.481562 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:36.652918 (+171356us) log.cc:844] Serialized 26 byte log entry
0709 12:21:36.758712 (+105794us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:36.846166 (+ 87454us) log.cc:844] Serialized 14 byte log entry
0709 12:21:36.868490 (+ 22324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:36.915879 (+ 47389us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:36.915983 (+   104us) raft_consensus.cc:1685] finished
0709 12:21:36.916002 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:37.710149 (+794147us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b54000f6c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:37.710309 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19166,"prepare.queue_time_us":63954,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":44703,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":161850,"spinlock_wait_cycles":76146944,"thread_start_us":78272,"threads_started":1,"wal-append.queue_time_us":184248}
I20260709 12:21:37.739053 15510 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:37.739080 15405 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:37.774685 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:37.764659 15388 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.945s	user 0.020s	sys 0.008s
I20260709 12:21:37.772344 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:37.781350 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49892.2us
W20260709 12:21:37.770557 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100695us
I20260709 12:21:37.763513 14418 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:37.759097 15348 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:37.793347 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144160us
I20260709 12:21:37.741420 15547 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.749014 15378 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.796782 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.767185 15531 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.775178 15464 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:37.804607 13647 raft_consensus.cc:2410] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:37.803242 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.803880 15565 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:37.806702 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:37.805855 15497 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:21:37.819999 15493 raft_consensus.cc:993] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:21:37.820588 15493 raft_consensus.cc:1081] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
W20260709 12:21:37.825899 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0225710999998228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:21:37.827929 15378 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.827844 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:37.790795 15388 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.830000 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:37.830714 15388 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.830930 15443 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:37.831355 15443 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:37.831367 15537 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:37.831918 15443 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:37.834214 15379 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.827812 14419 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:37.814549 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:37.838532 13651 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:37.843564 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.847023 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:37.847146 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:37.847849 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:37.814043 15572 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:37.837625 15379 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.837680 14419 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.863646 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:37.855545 13797 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
W20260709 12:21:37.855701 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52966us
I20260709 12:21:37.856418 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:37.855669 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:37.858605 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.880801 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.881513 13644 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:37.881863 13644 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:37.883122 15395 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:37.846374 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:37.892248 13644 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.893725 13644 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:37.859982 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48672.9us
I20260709 12:21:37.864176 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:37.850656 15565 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:37.859364 14420 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:37.867386 15347 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.869354 15572 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.870946 13799 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:37.850567 15510 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:37.875629 13792 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:37.844108 15464 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.862284 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:37.878103 15369 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.862376 15378 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:37.911839 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57677.1us
I20260709 12:21:37.805998 15405 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.895478 13647 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:37.899163 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:37.908815 13648 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:37.875489 15547 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.839882 15370 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:37.914390 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71940.2us
I20260709 12:21:37.916081 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:37.916766 15347 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:37.917408 15405 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.922528 13795 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:37.925364 15497 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:37.965694 15497 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:37.966818 15497 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.968452 15571 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:37.969530 15347 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:37.913070 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:37.970713 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57791.2us
I20260709 12:21:37.934230 15464 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.970625 15347 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.969120 15571 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:37.973289 15370 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:37.974653 15510 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:37.975317 15510 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:37.975003 15571 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.975457 15503 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:37.976323 15510 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.937871 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:37.946689 15369 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:37.930639 15565 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.976639 15357 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:21:37.962531 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46567.3us
I20260709 12:21:37.992224 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:37.993242 15347 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:37.993682 15347 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:37.986569 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106657us
I20260709 12:21:37.993993 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:37.995504 15379 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:37.998414 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60666.1us
I20260709 12:21:37.999284 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:38.000983 15443 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:37.998457 15379 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.003822 15503 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:38.004181 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:38.004592 15379 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:38.004958 14419 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.005273 15503 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:38.005808 15503 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:37.947981 15572 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:37.990815 15357 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:38.006659 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:38.007573 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:38.007555 15357 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.008186 15578 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:38.002015 15569 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:38.009063 15357 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.017529 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:38.022918 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:38.024081 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:38.026463 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:38.027420 13649 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:38.028441 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:38.017545 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:38.045017 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51083.7us
I20260709 12:21:38.046557 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.047565 13647 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:38.159871 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.053185 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:38.089393 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:38.072584 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.159509 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:37.994047 15347 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.067137 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.072021 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.066711 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.073308 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.184047 13800 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
W20260709 12:21:38.181293 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 179116us
I20260709 12:21:38.205406 13643 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:38.209187 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.071610 13794 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:38.172883 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.189496 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.121783 15369 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:38.055517 15464 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:38.211438 13796 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.206043 15569 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:38.220386 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:38.184566 15379 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:38.225168 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:38.222697 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:38.222702 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:38.229902 13793 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:38.231716 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:38.222501 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:38.235659 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 238556us
I20260709 12:21:38.232362 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:38.294636 13726 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
I20260709 12:21:38.297456 13725 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:38.342624 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:38.344424 13720 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.233748 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.355751 13643 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:38.184672 15503 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.287103 15464 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:38.359968 13722 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:21:38.353308 15348 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.008s
I20260709 12:21:38.362776 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.299234 15568 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:38.298520 15446 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:38.341260 15555 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:38.358906 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:38.366853 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:38.344067 15493 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:38.340492 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:38.374037 13643 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:38.225123 15569 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:38.362035 15585 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.077s	user 0.001s	sys 0.001s
I20260709 12:21:38.342101 13644 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:38.409581 13645 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:38.409924 15493 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.414673 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.417161 14419 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
W20260709 12:21:38.425642 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 127672us
I20260709 12:21:38.437218 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.439453 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:38.375651 15443 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:38.446324 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.447187 13643 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:38.462800 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.463582 13800 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 1.
I20260709 12:21:38.404986 15348 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:38.466604 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 107794us
I20260709 12:21:38.472087 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:38.467355 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:38.456765 13721 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:38.418444 15555 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.418503 15446 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.470239 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:38.426111 15568 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.496419 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:38.497186 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47814.4us
I20260709 12:21:38.498008 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:38.465898 15370 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:38.500099 13650 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4.
I20260709 12:21:38.507078 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.509469 15592 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:38.493760 15577 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:38.515270 15577 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:38.515776 15577 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:38.494530 15589 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:21:38.519467 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.520187 13645 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:38.490049 15443 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:38.521732 15443 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:38.522128 15443 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:38.524762 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 291717us
I20260709 12:21:38.482376 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:38.531733 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:38.538884 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1553) took 2446 ms. Trace:
I20260709 12:21:38.510080 15592 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:38.539460 13651 rpcz_store.cc:276] 0709 12:21:36.086727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.252945 (+1166218us) service_pool.cc:224] Handling call
0709 12:21:37.259323 (+  6378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.842539 (+583216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:38.529809 (+687270us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54001e5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:38.533023 (+  3214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271051520}
I20260709 12:21:38.541963 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.502456 15370 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:38.546639 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:38.547789 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60943.3us
I20260709 12:21:38.549685 13796 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:38.556002 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:38.535715 15531 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:38.562188 13720 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:38.565021 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:38.549248 15592 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:38.551524 15370 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:38.573724 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.574517 13794 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:38.550220 15562 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:38.573807 15597 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.578514 15597 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.578936 15446 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:38.573567 15379 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:38.581651 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:38.582599 15587 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
W20260709 12:21:38.583034 15357 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 12:21:38.583997 13792 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:21:38.580706 15493 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:38.563985 15531 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:38.597004 13796 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:38.586191 13648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:38.589718 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:38.523298 15464 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.596154 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.571695 15497 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:38.598232 15531 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:38.580722 15597 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:38.619349 15568 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:38.622896 15348 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:38.623080 13645 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:38.627007 15555 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:38.634853 13644 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:21:38.636740 13801 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 5. Ops: [3.3-3.3]
I20260709 12:21:38.637486 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.712914 13797 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:38.637765 13800 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:38.650871 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.654760 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.660233 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:38.676158 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 98619.1us
I20260709 12:21:38.677673 13648 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:38.658130 15588 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.292s	user 0.001s	sys 0.001s
I20260709 12:21:38.699509 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.692777 15405 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:21:38.692988 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:38.693444 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:38.729720 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:38.711449 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 139326us
I20260709 12:21:38.722493 13644 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:38.708385 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:38.734130 13799 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:21:38.740875 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:38.737836 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:38.740228 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:38.683257 15378 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:38.742579 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:38.746338 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:38.747124 13797 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:38.752117 13799 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:38.754866 15357 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:38.755375 15357 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:38.755759 15357 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.704629 15443 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.764020 15395 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:38.766635 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:38.741393 13644 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.771584 15577 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.774446 15579 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.748157 15568 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:38.780054 15592 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.742609 15565 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:38.745535 15348 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:38.746845 15378 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:38.798594 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 324) took 1209 ms. Trace:
W20260709 12:21:38.799285 15347 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.001s
I20260709 12:21:38.801157 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:38.800941 15370 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.803344 15443 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:38.803797 15569 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:38.806531 15579 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:38.812408 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:38.811663 15456 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:38.813539 15378 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:38.813674 15555 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:38.813854 15600 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:38.853422 14422 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:38.793629 15568 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:38.809612 15565 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:38.803838 15395 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.859664 15456 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.230891 15577 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:38.857440 15568 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:38.860455 15565 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:39.248850 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 151317us
I20260709 12:21:38.870571 13545 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 149) took 3497 ms. Trace:
I20260709 12:21:39.231952 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:38.830705 15347 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.234462 15571 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:39.112810 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:38.858599 15600 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.833143 15379 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:39.235877 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:38.889698 15531 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.148620 13644 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.869364 15579 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:38.990839 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 282627us
I20260709 12:21:39.150293 15357 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.856338 15564 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:38.864938 15555 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:38.853422 13547 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:38.927079 14419 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:38.987370 15599 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
W20260709 12:21:39.031240 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 200337us
I20260709 12:21:39.180389 15378 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:38.865687 15427 raft_consensus.cc:993] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:39.240813 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:39.229714 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 463128us
I20260709 12:21:39.231382 15611 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:39.265516 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:38.814013 14418 rpcz_store.cc:276] 0709 12:21:37.589631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.589831 (+   200us) service_pool.cc:224] Handling call
0709 12:21:37.590332 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.799440 (+209108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:38.798286 (+998846us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54006e0fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:38.798491 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370923392}
I20260709 12:21:39.278496 15456 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.279007 15600 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.279093 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:39.369601 13547 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:39.280849 15531 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.259258 13545 rpcz_store.cc:276] 0709 12:21:35.358820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:35.359182 (+   362us) service_pool.cc:224] Handling call
0709 12:21:38.855944 (+3496762us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:37.237868 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:37.238681 (+   813us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:37.238746 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:21:37.238783 (+    37us) tablet.cc:662] Decoding operations
0709 12:21:37.239354 (+   571us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:37.239464 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:37.239493 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:21:37.264005 (+ 24512us) tablet.cc:701] Row locks acquired
0709 12:21:37.264067 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:21:37.276833 (+ 12766us) write_op.cc:270] Start()
0709 12:21:37.322619 (+ 45786us) write_op.cc:276] Timestamp: P: 1783599697269927 usec, L: 0
0709 12:21:37.367074 (+ 44455us) op_driver.cc:348] REPLICATION: starting
0709 12:21:37.497798 (+130724us) log.cc:844] Serialized 913 byte log entry
0709 12:21:37.814829 (+317031us) op_driver.cc:464] REPLICATION: finished
0709 12:21:37.962149 (+147320us) write_op.cc:301] APPLY: starting
0709 12:21:37.962281 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:37.962446 (+   165us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:37.964334 (+  1888us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:37.964886 (+   552us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:37.965237 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:21:37.965297 (+    60us) write_op.cc:312] APPLY: finished
0709 12:21:37.965611 (+   314us) log.cc:844] Serialized 29 byte log entry
0709 12:21:37.965815 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:37.965915 (+   100us) write_op.cc:454] Released schema lock
0709 12:21:37.965984 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:37.966474 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:37.966509 (+    35us) write_op.cc:454] Released schema lock
0709 12:21:38.776369 (+809860us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf132a 00007f5393b76f14 00007f5393b76ec3 00007f5393b76e38 00007f5393b76df3 00007f5393b760dc 00007f5393b9fe16 00007f5393b87cbb 00007f5393b7239d 00007f5393b75d75
Metrics: {"mutex_wait_us":2895,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":146932,"apply.run_cpu_time_us":2765,"apply.run_wall_time_us":4318,"num_ops":2,"prepare.queue_time_us":415643,"prepare.run_cpu_time_us":47959,"prepare.run_wall_time_us":331793,"raft.queue_time_us":86794,"raft.run_cpu_time_us":19766,"raft.run_wall_time_us":984900,"replication_time_us":436897,"spinlock_wait_cycles":3092352,"thread_start_us":415798,"threads_started":4,"wal-append.queue_time_us":131633,"wal-append.run_cpu_time_us":12680,"wal-append.run_wall_time_us":1337345}]]}
I20260709 12:21:39.276432 15348 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:39.364243 15592 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:39.269167 15571 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.276799 15395 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.371289 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2106) took 1324 ms. Trace:
I20260709 12:21:39.371544 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:39.271610 15577 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.377295 13800 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:39.281169 14424 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:39.271385 15379 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.358966 15357 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:39.274466 15610 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.384481 13794 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:39.368523 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56479.3us
I20260709 12:21:39.368007 13766 leader_election.cc:400] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
I20260709 12:21:39.390296 13792 rpcz_store.cc:276] 0709 12:21:38.046725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.046856 (+   131us) service_pool.cc:224] Handling call
0709 12:21:39.369622 (+1322766us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54001ca7d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:39.371221 (+  1599us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919555968}
I20260709 12:21:39.375025 15370 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.271314 15405 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.399398 13766 leader_election.cc:403] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:21:39.383113 13547 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:21:39.406327 13795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 169) took 1559 ms (client timeout 1907 ms). Trace:
I20260709 12:21:39.407393 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:39.409457 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:39.410586 15616 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:39.410243 13547 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:21:39.411175 15616 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.405879 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:39.411998 13547 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 5 to 6, VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) evicted. New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:39.412726 13800 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:39.414023 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:39.414566 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:39.416069 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:39.417243 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:39.418530 15547 raft_consensus.cc:1081] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:21:39.419301 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:39.419539 15547 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.419958 13798 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:39.420110 15547 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.398339 15577 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.424239 15555 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:39.424897 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62453.5us
I20260709 12:21:39.426328 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:39.428683 15347 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:39.430351 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 331) took 1653 ms. Trace:
I20260709 12:21:39.430910 15622 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:39.432680 15464 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:39.433063 15464 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:39.433207 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1494) took 1089 ms. Trace:
I20260709 12:21:39.369129 15611 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.422484 14416 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:39.430786 14420 rpcz_store.cc:276] 0709 12:21:37.753912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.764301 (+ 10389us) service_pool.cc:224] Handling call
0709 12:21:37.804050 (+ 39749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:37.900096 (+ 96046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:39.396132 (+1496036us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:39.407627 (+ 11495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79417728}
I20260709 12:21:39.410404 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:39.424343 15577 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:39.406780 13795 rpcz_store.cc:269] 0709 12:21:37.847067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:37.847205 (+   138us) service_pool.cc:224] Handling call
0709 12:21:39.406110 (+1558905us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b54000e56d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:39.406246 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132542848}
I20260709 12:21:39.444051 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:39.417280 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:39.439249 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:39.434789 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:39.446822 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:39.396525 15379 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.467337 15562 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.469930 15594 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:39.468355 15503 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:39.493234 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:39.403815 15607 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:21:39.401026 15443 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:39.435937 15569 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4.
I20260709 12:21:39.433645 13793 rpcz_store.cc:276] 0709 12:21:38.344058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.344262 (+   204us) service_pool.cc:224] Handling call
0709 12:21:39.407398 (+1063136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:39.431890 (+ 24492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:39.431920 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:39.431967 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:39.432003 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:39.432068 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:39.433021 (+   953us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000e56d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:39.433127 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2405708672}
I20260709 12:21:39.424259 15378 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.485322 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:39.500403 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 114178us
I20260709 12:21:39.492148 15564 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:39.437173 15357 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:39.398624 15568 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.435619 15571 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.433842 13644 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.538692 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:39.539361 14425 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:39.439249 15617 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:21:39.539911 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 140788us
I20260709 12:21:39.540282 15617 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.540733 15617 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:21:39.537345 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 98200.4us
I20260709 12:21:39.539311 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54768 (request call id 1587) took 1558 ms. Trace:
W20260709 12:21:39.513685 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124366us
I20260709 12:21:39.547070 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:39.546747 13644 rpcz_store.cc:276] 0709 12:21:37.982176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.634682 (+652506us) service_pool.cc:224] Handling call
0709 12:21:39.539208 (+904526us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":36619}
I20260709 12:21:39.548774 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:39.550141 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64941.4us
W20260709 12:21:39.552066 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 82177.4us
I20260709 12:21:39.553200 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:39.481866 15547 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.567734 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:39.569175 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:39.434006 15464 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.570284 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:39.464886 15405 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.404768 15610 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.534693 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 424) took 1343 ms. Trace:
I20260709 12:21:39.581681 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:39.587159 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46553.5us
I20260709 12:21:39.595336 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:39.595950 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:39.566754 15578 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:39.595392 13797 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:39.569226 15443 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.613785 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:39.614858 13725 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:39.615301 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.622627 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.582432 15579 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:39.522593 15562 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.633328 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.634326 13724 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:39.588608 15607 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:39.640971 13719 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:21:39.515101 15594 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:39.638712 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85554.5us
I20260709 12:21:39.612914 15592 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:39.587303 15631 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:39.656643 13725 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:39.644836 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:39.589061 15378 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:21:39.624826 15632 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:39.628165 15578 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:39.645558 15569 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:21:39.646813 15464 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.678681 13793 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.661511 13724 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:39.678340 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.679085 15632 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.695730 15630 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:39.626916 15443 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.699420 15568 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.700865 13798 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:39.680996 15579 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:39.729805 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 176) took 1576 ms (client timeout 1588 ms). Trace:
I20260709 12:21:39.704550 15578 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.681995 15611 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:39.683431 15545 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:39.626148 15564 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.660405 15623 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 12:21:39.667126 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119584us
I20260709 12:21:39.662575 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:39.716442 13800 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:39.717777 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 498) took 1521 ms. Trace:
W20260709 12:21:39.720697 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 169591us
W20260709 12:21:39.721592 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 184020us
I20260709 12:21:39.722182 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:39.655938 15631 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:39.724272 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:39.728279 15630 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.728906 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:39.677800 14391 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.624s (SENT)
I20260709 12:21:39.770021 13795 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:39.778291 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:39.772553 13801 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:39.776499 13796 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
W20260709 12:21:39.811455 13798 rpcz_store.cc:269] 0709 12:21:38.153243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:39.418910 (+1265667us) service_pool.cc:224] Handling call
0709 12:21:39.729729 (+310819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:39.706393 15571 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.824972 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.705777 15547 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:39.815114 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 185576us
I20260709 12:21:39.773712 15592 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:39.810245 14424 rpcz_store.cc:276] 0709 12:21:38.196467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.196666 (+   199us) service_pool.cc:224] Handling call
0709 12:21:38.645157 (+448491us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:38.645562 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:39.392101 (+746539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:39.717507 (+325406us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:39.717705 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486309504}
I20260709 12:21:39.758405 15632 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.820312 15611 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:39.770797 15628 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:21:39.751955 15572 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:39.744204 15565 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.745361 15594 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:39.822813 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:39.815555 15631 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.749550 15616 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:39.827592 15596 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:39.829180 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:39.825678 13718 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:39.832629 13793 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:39.856976 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:39.833804 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.857767 13725 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:39.589529 14421 rpcz_store.cc:276] 0709 12:21:38.187286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.189432 (+  2146us) service_pool.cc:224] Handling call
0709 12:21:39.441668 (+1252236us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:39.441889 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:39.442158 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:39.442188 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:39.442241 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:39.471779 (+ 29538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:39.489702 (+ 17923us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:39.530646 (+ 40944us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3122944}
W20260709 12:21:39.830093 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 283098us
I20260709 12:21:39.735550 13644 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:39.841068 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:39.842110 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 164354us
I20260709 12:21:39.732129 15610 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:39.843437 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.737087 15405 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.763952 13721 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:39.863775 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:39.857858 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:39.870955 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:39.871210 13718 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.860468 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:39.872189 13719 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:39.861807 13652 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:39.868523 13726 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:39.866748 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:39.873370 13722 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:39.874171 13725 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:39.729979 15579 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:39.879559 14391 leader_election.cc:336] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.627s (SENT)
I20260709 12:21:39.852267 15572 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:39.866865 15545 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.835268 15592 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:39.902468 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 86166.1us
W20260709 12:21:39.904417 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44040.7us
W20260709 12:21:39.905397 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76289.5us
I20260709 12:21:39.832654 15571 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.880218 15616 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:39.851329 15611 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.850503 15547 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.954404 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:39.955822 15579 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.956192 15547 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.882040 15594 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:39.956969 15572 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:39.957168 15594 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.958648 15572 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:39.960227 15594 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:39.922766 15630 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.960574 13792 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:39.960793 15592 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:39.961032 13792 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:39.962615 13792 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:39.963898 13792 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:39.928660 13723 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:39.938297 15634 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:21:39.939744 15627 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:21:39.948051 15565 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:39.889979 15395 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:39.891027 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:39.975495 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55209.3us
I20260709 12:21:39.978893 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:39.871060 15370 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:21:39.901512 15378 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:39.980150 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:40.003770 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125204us
I20260709 12:21:40.003945 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:40.004848 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.730s (SENT)
I20260709 12:21:40.004972 13798 raft_consensus.cc:2376] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:21:40.005396 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:40.007633 13652 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:40.007774 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:39.920841 15616 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:40.008481 13800 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:39.862699 15635 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.138s
I20260709 12:21:40.011842 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:39.955891 15611 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:40.011224 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 127749us
I20260709 12:21:39.962234 15637 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:40.015646 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 150799us
I20260709 12:21:39.973404 15636 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.024741 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:40.032051 13798 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:39.982038 15395 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:40.037494 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 204404us
I20260709 12:21:40.039635 13799 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:40.045773 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.042847 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:40.048329 13793 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.060027 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.061151 13722 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:40.069022 13725 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.002497 15613 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:40.015188 15630 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:39.919417 15545 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.070000 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:40.014694 15562 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:40.042582 15379 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.005s
I20260709 12:21:40.078958 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:40.079216 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:40.028745 15531 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:40.030514 15555 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:40.081046 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69357.3us
W20260709 12:21:40.083180 14391 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.761s (SENT)
I20260709 12:21:39.975147 15591 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:21:40.044978 15395 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.083774 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.085758 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:40.086086 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:40.077174 15617 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.004s
W20260709 12:21:40.093482 14391 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.906s (SENT)
I20260709 12:21:40.099179 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:40.103353 13617 connection.cc:256] RPC call timeout handler was delayed by -1.8698660909999489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 12:21:40.105897 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2794525770000291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 12:21:40.106577 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1273525769997832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 12:21:40.107128 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1093525769997541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
I20260709 12:21:40.077898 15630 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:40.107702 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0238525770000706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 12:21:40.108495 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0202525770000648s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 12:21:40.109586 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:40.082504 15577 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:40.113094 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.114279 13649 raft_consensus.cc:2410] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:40.115267 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:40.121241 15616 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.122916 15616 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:40.125406 15642 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:40.126525 15630 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.126974 15631 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.129940 15631 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.130784 15646 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:40.132172 15646 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:40.132937 15646 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:40.133484 15646 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.097141 15626 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:40.082777 15379 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.135560 13727 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:40.136641 15619 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:40.136981 15379 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.134613 15626 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.137764 15571 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.138288 13798 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:40.139633 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.141213 13723 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:40.143365 15545 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.144122 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.144711 13643 raft_consensus.cc:2376] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:21:40.145665 13793 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:40.146102 15503 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.146580 15503 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.147687 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.148382 13645 raft_consensus.cc:2376] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5. Re-sending same reply.
I20260709 12:21:40.149318 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.150027 13651 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4.
W20260709 12:21:40.150666 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51565.9us
I20260709 12:21:40.152420 15555 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:40.153026 15555 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:40.150851 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.098124 15617 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.153568 13643 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:40.158130 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.161505 13652 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:21:40.161932 13652 pending_rounds.cc:85] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 4
I20260709 12:21:40.162307 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.162983 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:40.165311 13792 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:40.167200 13800 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:40.162177 13652 pending_rounds.cc:107] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:21:40.168790 13652 raft_consensus.cc:2889] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:21:40.171339 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 192531us
I20260709 12:21:40.172345 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:40.125963 15642 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:40.174589 15642 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.174404 15613 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.176864 15613 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:40.177553 15631 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:40.178001 15631 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:40.178388 15631 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.183054 15619 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.185472 13649 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:21:40.186708 15626 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.187767 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.188423 13651 raft_consensus.cc:2376] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:21:40.188577 15626 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.189304 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.190632 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.191262 13645 raft_consensus.cc:2376] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:21:40.192288 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.193035 13645 raft_consensus.cc:2376] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3. Re-sending same reply.
I20260709 12:21:40.194151 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.194587 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:40.187050 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.195663 13649 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:40.187037 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:40.193994 15619 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:40.199107 15562 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.205379 15641 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:40.206434 15611 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:40.206847 15611 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:40.206969 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.208231 15611 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.209741 15611 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:40.213735 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104903us
I20260709 12:21:40.214958 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:40.217442 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.044946 15636 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.162539 15577 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:40.224924 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.220328 15569 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:40.105625 15443 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:40.221880 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.222219 13643 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:40.153628 15555 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.222566 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.155854 15617 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.233170 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:40.227429 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:40.215677 13721 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:21:40.232121 13693 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.224462 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.227456 15577 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.228219 15569 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:40.228407 15641 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:40.230702 15379 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:40.343549 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111528us
I20260709 12:21:40.344725 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.246376 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:40.252460 15649 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:21:40.235728 15547 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:21:40.309489 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51101us
W20260709 12:21:40.350368 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 104032us
I20260709 12:21:40.229024 15613 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:21:40.358033 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:40.230425 15594 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:21:40.259719 15569 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:40.362552 15621 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:40.350457 15443 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:40.363631 15631 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.251847 15564 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:21:40.366994 15640 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:40.367374 15613 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:40.371686 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:40.253253 15643 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.269s
I20260709 12:21:40.183059 15642 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:40.375876 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.358505 15579 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:40.357649 15497 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:40.457485 15497 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.460253 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.461120 13646 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:21:40.423056 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 186705us
I20260709 12:21:40.463587 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 179) took 1628 ms. Trace:
I20260709 12:21:40.463960 13799 rpcz_store.cc:276] 0709 12:21:38.834837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.834957 (+   120us) service_pool.cc:224] Handling call
0709 12:21:39.718467 (+883510us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acff8c 00007f5390acfef6 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:40.039428 (+320961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:40.040114 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:40.463382 (+423268us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b54000d2fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:40.463527 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614944640}
I20260709 12:21:40.464362 15569 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.461957 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.465631 13646 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:40.466708 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.468257 13646 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:40.382970 15613 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:40.469790 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.378763 15594 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:40.470860 15594 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:40.472270 15594 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.473840 15594 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.442685 15646 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.476037 15646 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.373256 15630 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.478032 15630 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:40.478340 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.420069 15562 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:40.482667 13719 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:40.449160 15623 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:40.450426 15631 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.376345 15395 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.451915 15636 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.483585 15623 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.384014 15621 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:40.452533 15632 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.008s
I20260709 12:21:40.484637 15623 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.485481 15395 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.485275 15632 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.485173 15621 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.487696 15632 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.454092 14423 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:21:40.426043 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54428.4us
I20260709 12:21:40.448516 15649 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:40.489977 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:40.490325 15649 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:40.491472 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.491259 15649 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.492133 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:40.492260 15636 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:40.466939 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.492938 15636 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.495045 15531 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:40.495234 15590 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:40.459806 15497 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.470359 15613 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.496454 15590 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:40.496841 15613 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.492959 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:40.439906 15642 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.467664 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.432590 15568 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:40.498930 15568 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:40.500170 15568 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.242799 15578 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.123s
I20260709 12:21:40.496850 15590 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:40.503821 15590 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.505074 15590 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.492412 15649 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:40.495566 15531 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.507447 15531 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:40.510151 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.506136 15625 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:40.511711 15625 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.511829 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.512854 15625 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.453802 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:40.470160 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.485945 15623 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:40.528720 13792 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:40.524507 13795 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:40.538873 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85139.6us
I20260709 12:21:40.482589 15579 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:40.454524 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 224693us
W20260709 12:21:40.543725 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:40.489739 15572 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.531s	user 0.006s	sys 0.004s
I20260709 12:21:40.544437 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:40.549948 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.553701 13725 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:40.554066 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:40.555142 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:40.466627 15569 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.552078 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.514078 15625 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.558725 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.558650 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.477908 15577 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.583873 13727 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.578984 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:40.583431 13798 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:40.563802 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.587695 13725 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:21:40.589272 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:40.592168 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:40.595278 13793 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:40.541637 15578 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.600047 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:40.602041 13647 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:40.551999 15631 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:40.582933 15642 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:40.612126 13792 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:40.571256 14421 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:40.613983 14425 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:40.571902 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:40.619344 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.619652 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.579339 14420 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:40.580829 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.571305 15636 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:40.572190 15624 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:21:40.583335 15347 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:21:40.562618 15622 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:21:40.623948 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.646588 13722 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:40.647105 13719 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.624809 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.625057 15607 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:40.625751 15644 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:40.626905 15464 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:21:40.629593 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.644155 14416 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:40.630641 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.599669 15626 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:40.595276 15348 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:40.655628 13718 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:40.660353 15555 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.658056 13723 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:40.662024 15579 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.664007 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.665277 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.621554 15610 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
W20260709 12:21:40.632648 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.641893 13792 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:40.644308 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.650333 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.635463 15631 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:40.599495 15592 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:21:40.635878 15497 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:21:40.656538 13727 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:40.657230 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.673413 13726 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.674453 13718 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:40.683271 13719 raft_consensus.cc:2368] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3.
I20260709 12:21:40.611820 15578 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.683804 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.684829 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.621874 15572 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:21:40.700414 13722 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.646456 15642 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:40.642165 15569 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:21:40.669308 15643 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:21:40.699862 15626 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.674989 15644 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.704577 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.707901 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.710078 13718 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:40.711172 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.683239 15613 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.713147 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.716280 13726 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:40.711829 13719 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:40.678623 15634 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:21:40.702818 15596 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:21:40.742906 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 197092us
I20260709 12:21:40.743304 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:40.743709 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48204.9us
I20260709 12:21:40.744681 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:40.749112 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100516us
I20260709 12:21:40.750394 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.752961 13718 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:40.726306 15348 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.770118 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:40.712781 15531 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.783888 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.786720 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:40.786738 13719 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:40.720964 15613 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.757732 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.820578 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:40.768224 15545 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:40.746459 15565 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:21:40.815161 15644 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.836127 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.839474 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.846369 15634 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:40.846711 15348 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.887048 15645 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:40.887570 15645 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.888537 15645 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.889613 15531 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.891637 14417 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:40.764830 15631 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.892364 15578 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:40.789186 15626 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:40.893738 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73182.7us
W20260709 12:21:40.855296 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:40.879256 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.833611 15577 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.898641 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:40.881677 15634 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.883759 13644 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:40.884954 15661 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:21:40.891599 15591 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:40.824240 15395 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:40.709987 15642 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:40.897624 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.881413 15624 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:21:40.861011 15664 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:21:40.858685 15497 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:40.945832 15624 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:40.861392 15674 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:40.947294 15674 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:40.948050 15674 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:40.911342 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:40.863875 15672 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:21:40.863570 15677 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:40.914140 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:40.838055 15370 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:21:40.919236 15613 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:40.921617 15621 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:40.916414 15395 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:40.924912 15634 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:40.910637 15645 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:40.945703 15636 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:21:40.955601 13725 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:21:40.958630 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44611.2us
I20260709 12:21:40.925052 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:40.954715 15677 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:40.967337 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.972018 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.972909 13726 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.973914 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.974748 13726 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.975803 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.976549 13726 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.977484 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.961782 15653 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:40.963485 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 180446us
I20260709 12:21:40.968473 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:40.982926 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:40.983348 13725 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:40.980983 13719 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:40.978621 15651 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:21:40.987924 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:40.947844 15624 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:40.992818 15624 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:40.992691 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.965194 15395 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:40.995256 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.967962 15636 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:40.967793 15621 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:41.001588 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:41.009047 15621 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:41.004347 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.011337 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.006198 15646 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:41.020473 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.007452 15636 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.022536 15636 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:41.024163 15674 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.026925 15674 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:41.004725 15643 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:41.027452 15643 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:41.028538 15503 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:41.029913 15497 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:41.030650 15497 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:41.031756 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.031605 15497 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.032613 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:41.033417 15497 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:41.033670 14423 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:41.034689 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:41.035779 13722 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:41.035964 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:41.036944 14423 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:41.038548 15555 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:41.038604 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.039307 13723 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4. Re-sending same reply.
I20260709 12:21:41.040282 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.041373 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:40.972236 15677 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:41.026584 15562 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:40.971880 15591 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:21:41.028357 15643 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.034914 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.043330 15562 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.044385 13648 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:21:41.009378 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.046042 13719 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:41.045894 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.012305 13725 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:41.046710 13726 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:41.022028 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.044765 15562 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:41.051154 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:41.055389 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:41.043243 15639 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.056574 13723 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:41.065850 13722 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:41.106153 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:41.107692 14424 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 2: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:41.049862 13721 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:41.060559 15639 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:41.131673 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 188) took 1151 ms. Trace:
I20260709 12:21:41.063800 15674 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.149335 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:41.167958 13795 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:41.131812 15562 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.162999 13801 rpcz_store.cc:276] 0709 12:21:39.980055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:40.097432 (+117377us) service_pool.cc:224] Handling call
0709 12:21:40.168901 (+ 71469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:40.182638 (+ 13737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:40.182681 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:40.182734 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:40.182784 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:41.131307 (+948523us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5400287dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:41.131374 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:41.131571 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31013248}
I20260709 12:21:41.225980 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:41.228300 15643 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:41.241928 13725 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:41.232749 15644 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:41.269940 13722 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:41.280536 13725 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:41.283154 13801 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:41.268747 15562 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.274628 15639 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.270153 15644 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.269100 13721 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:41.321555 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.338889 13722 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:41.335841 13725 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:41.342967 15564 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:41.343703 15564 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:21:41.347434 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122613us
I20260709 12:21:41.345155 15564 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:41.352821 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.331812 15644 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:41.370162 13723 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:41.362936 13725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34450
I20260709 12:21:41.375115 13725 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:21:41.375761 13725 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:41.330196 15427 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
W20260709 12:21:41.360888 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.329628 15626 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.365022 15607 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.383533 15631 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:41.398020 15693 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
W20260709 12:21:41.425798 15690 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:21:41.402550 15637 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.403913 15607 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:41.377835 15395 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:41.439534 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.404066 15626 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.497483 13725 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:41.498880 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110015us
I20260709 12:21:41.503028 13723 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:41.528746 13725 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:41.422130 15631 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.529335 13723 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:41.439960 15637 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.494570 15564 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:41.535780 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:41.525936 15693 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:21:41.546808 15688 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:21:41.547680 15696 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:21:41.547910 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.550230 13801 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:41.530910 15642 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:41.551472 15674 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:41.553911 15637 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:41.530187 15694 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:21:41.568394 13722 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:21:41.568944 13722 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 2
I20260709 12:21:41.569916 13722 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:21:41.570281 13722 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:41.571826 13725 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:41.589206 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 502) took 1097 ms. Trace:
I20260709 12:21:41.589527 15631 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:41.583606 15636 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.003s
I20260709 12:21:41.585378 13723 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.617744 13723 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:41.609218 15624 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.002s
I20260709 12:21:41.590576 15395 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:41.627285 15677 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.001s
I20260709 12:21:41.566421 15642 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.629855 13792 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:41.571298 15348 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:21:41.559860 15607 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.642570 15649 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:41.589972 14417 rpcz_store.cc:276] 0709 12:21:40.492044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:40.547640 (+ 55596us) service_pool.cc:224] Handling call
0709 12:21:41.588940 (+1041300us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400600254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:41.589107 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495426944}
W20260709 12:21:41.649983 15653 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.653s	user 0.021s	sys 0.000s
I20260709 12:21:41.653947 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:41.640741 15497 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.607s	user 0.014s	sys 0.005s
I20260709 12:21:41.656432 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:41.657208 13723 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:21:41.657814 13723 pending_rounds.cc:85] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:21:41.643054 15649 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:41.658102 13723 pending_rounds.cc:107] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:21:41.658782 13723 raft_consensus.cc:2889] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:41.665521 15626 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:41.665932 15621 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:41.666059 13720 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:41.665563 15649 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.667447 15649 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:41.650557 15653 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:41.644842 15607 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.635854 15677 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.668529 15579 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:41.679831 15636 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:41.690122 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.686779 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:41.651412 15624 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:21:41.693205 15395 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:41.702456 15395 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:41.703696 15395 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.662381 15634 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:21:41.715801 15677 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:41.718941 15624 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:41.719574 15624 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:41.720616 15624 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.724388 15642 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:41.694427 15636 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.722115 15624 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.693996 15579 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:41.739434 15562 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:41.769604 15707 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:21:41.775243 15636 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:41.790843 15579 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.752390 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:41.795506 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.746840 15704 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:41.796394 15704 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.799427 15704 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:41.803906 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 295) took 1246 ms. Trace:
I20260709 12:21:41.804534 13648 rpcz_store.cc:276] 0709 12:21:40.557915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:40.558382 (+   467us) service_pool.cc:224] Handling call
0709 12:21:40.570826 (+ 12444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:41.044830 (+474004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:41.803486 (+758656us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400114254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:41.803814 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011147008}
I20260709 12:21:41.807909 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 435) took 1316 ms. Trace:
W20260709 12:21:41.811168 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 118977us
I20260709 12:21:41.808526 14425 rpcz_store.cc:276] 0709 12:21:40.491215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:40.491373 (+   158us) service_pool.cc:224] Handling call
0709 12:21:40.491675 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:40.621554 (+129879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:41.807538 (+1185984us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:41.807818 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275786624}
I20260709 12:21:41.819954 15621 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:41.781006 15644 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.810876 15704 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.763461 15562 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:41.839365 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:41.848773 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.850656 15625 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:41.851173 15562 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:41.851320 15625 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.853564 15625 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:41.853855 15639 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.855094 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.858667 13718 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:41.857438 15703 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.370s	user 0.002s	sys 0.000s
I20260709 12:21:41.860642 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:41.860906 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:41.864567 13649 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:41.861070 15659 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.870908 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:41.871536 13652 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:41.871625 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:41.874508 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55869.3us
I20260709 12:21:41.874789 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:41.875713 13643 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:41.876521 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:41.877174 13727 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:21:41.878710 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:41.880337 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:41.880666 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:41.880971 13651 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:41.883410 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.849309 15634 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:41.833757 15621 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:21:41.833042 15704 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:41.882558 15636 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:41.895219 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:41.945127 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50066.3us
W20260709 12:21:41.945901 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.945075 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2146) took 1422 ms. Trace:
I20260709 12:21:41.953051 13793 rpcz_store.cc:276] 0709 12:21:40.523063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:40.525855 (+  2792us) service_pool.cc:224] Handling call
0709 12:21:40.526163 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:40.595790 (+ 69627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:41.944794 (+1349004us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:41.945004 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135722368}
I20260709 12:21:41.959074 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:41.961220 15706 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:21:41.961455 13644 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:41.967499 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:41.967569 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 127190us
I20260709 12:21:41.969782 15677 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:41.970293 15677 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:41.970643 15677 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:41.951588 15636 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:41.971186 15636 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:41.867430 15659 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.976598 15659 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:41.982043 13726 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:41.982707 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1538) took 1352 ms. Trace:
I20260709 12:21:41.889675 15634 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:41.984872 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:41.982908 13726 rpcz_store.cc:276] 0709 12:21:40.630722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:41.050285 (+419563us) service_pool.cc:224] Handling call
0709 12:21:41.981644 (+931359us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:41.981862 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:41.982501 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:41.982660 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12789632}
W20260709 12:21:41.986577 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:41.992908 13649 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:41.987874 14425 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:41.966516 15687 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:21:41.966302 15642 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:42.006047 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:41.994448 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:42.001255 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:42.002187 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:42.021322 13796 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:42.023238 13797 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:42.003484 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:42.026916 14424 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:41.985891 15497 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:42.027298 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.028012 13647 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:21:42.028080 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.041461 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:41.966876 15623 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:21:41.994748 15395 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.005021 15642 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:42.046128 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.024436 15503 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.004421 15637 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:42.004648 15643 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:42.044157 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.044811 15596 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:42.056569 15642 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.072279 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1541) took 1426 ms. Trace:
I20260709 12:21:42.057942 14417 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:42.049360 15714 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:42.056175 15395 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:42.081315 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 73356.1us
I20260709 12:21:42.082906 13796 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:42.045408 15497 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.064373 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.078413 13719 rpcz_store.cc:276] 0709 12:21:40.645867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:41.051076 (+405209us) service_pool.cc:224] Handling call
0709 12:21:41.827370 (+776294us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:41.827753 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:41.834224 (+  6471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:41.834293 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:41.846820 (+ 12527us) log.cc:844] Serialized 26 byte log entry
0709 12:21:41.980405 (+133585us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:41.980453 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:41.980527 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:42.071924 (+ 91397us) raft_consensus.cc:1685] finished
0709 12:21:42.071952 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:42.072203 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6764,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":18197,"spinlock_wait_cycles":15454976,"thread_start_us":93889,"threads_started":1,"wal-append.queue_time_us":181686}
I20260709 12:21:42.096037 13792 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:21:42.096783 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.078676 15596 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:42.062189 15643 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.062371 15637 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.090544 15659 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:42.067821 15607 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:21:42.122895 13800 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:42.133322 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.133615 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.122651 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:42.123767 15720 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:42.098587 15644 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:42.134708 13794 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:42.136466 15607 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:42.138887 13652 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:42.150286 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.090875 15714 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.161140 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:42.162293 13792 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
W20260709 12:21:42.153439 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59956us
I20260709 12:21:42.127283 15659 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:42.143139 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.164654 13718 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:42.168277 15659 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:42.171469 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2049) took 1342 ms. Trace:
I20260709 12:21:42.138749 15427 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:21:42.175386 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:42.154069 15704 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:42.186735 15497 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.174115 13722 rpcz_store.cc:276] 0709 12:21:40.829051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:41.345857 (+516806us) service_pool.cc:224] Handling call
0709 12:21:41.380152 (+ 34295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:41.581051 (+200899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:42.171216 (+590165us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b5400157cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:42.171406 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":418289536}
I20260709 12:21:42.188802 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:42.189486 13718 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4. Re-sending same reply.
I20260709 12:21:42.191012 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:42.159010 15607 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:42.200243 13651 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:42.169049 15720 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:42.203292 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.202432 15720 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:42.201133 15649 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.003s
W20260709 12:21:42.212628 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.213738 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.219012 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:42.219537 15395 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:42.211158 15720 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.220147 15708 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:21:42.222091 15643 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:42.227875 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:42.232122 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:42.234046 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.238931 13792 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:42.233901 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.241124 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.242677 13800 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:21:42.245237 13800 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:21:42.245440 14420 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:42.249601 15714 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:42.206203 15634 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:42.206663 15607 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.202165 15677 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.200950 15497 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 12:21:42.304067 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.258210 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.277315 13724 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:42.256798 13718 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:42.271504 13547 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 235) took 5526 ms. Trace:
I20260709 12:21:42.232772 15626 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.148s
I20260709 12:21:42.261803 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.260587 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.505402 13719 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:42.419462 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 181403us
W20260709 12:21:42.264561 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.504s (SENT) [suppressed 35 similar messages]
W20260709 12:21:42.506577 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.260123 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:42.432880 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.474030 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:42.507630 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.507736 13651 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:21:42.486581 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 492269us
I20260709 12:21:42.508728 13722 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:42.260957 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.509738 13727 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:42.266034 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.443357 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:42.486994 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 180974us
I20260709 12:21:42.511220 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:42.471459 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.439749 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.512508 13652 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.259730 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:42.487635 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.375924 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1539) took 1744 ms. Trace:
I20260709 12:21:42.254153 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:42.516770 13725 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:42.514071 13724 rpcz_store.cc:276] 0709 12:21:40.631001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:41.052013 (+421012us) service_pool.cc:224] Handling call
0709 12:21:42.276924 (+1224911us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:42.277139 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:42.375532 (+ 98393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:42.375840 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5760768}
W20260709 12:21:42.524983 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.510695 13650 raft_consensus.cc:2376] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3. Re-sending same reply.
I20260709 12:21:42.528630 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.261377 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.235221 15725 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:42.239780 15564 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:21:42.547152 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.272751 13711 tablet_service.cc:1558] Processing DeleteTablet for tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 with delete_type TABLET_DATA_TOMBSTONED (TS b4f6bfcda63148e4965ca02c639336f1 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:34444
W20260709 12:21:42.549608 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.260646 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.544493 13652 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:42.442824 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.279732 15726 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:42.448388 15639 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.001s
I20260709 12:21:42.242108 15644 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:42.537832 13648 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:42.257339 15637 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.276826 14420 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:42.538760 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:42.561893 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 297247us
I20260709 12:21:42.516559 15607 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.440223 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.568828 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.422361 15503 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:42.569608 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:42.550432 15610 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:42.496281 15596 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.238448 15649 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.309s
I20260709 12:21:42.571311 13722 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:42.570990 13693 connection.cc:256] RPC call timeout handler was delayed by -3.063540808000198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
I20260709 12:21:42.573199 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:42.572608 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61451.9us
W20260709 12:21:42.600713 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.573810 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.574055 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:42.601756 15737 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.503854 13547 rpcz_store.cc:276] 0709 12:21:36.745544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:36.796996 (+ 51452us) service_pool.cc:224] Handling call
0709 12:21:42.271400 (+5474404us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:39.881759 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:39.897589 (+ 15830us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:39.897647 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:21:39.897675 (+    28us) tablet.cc:662] Decoding operations
0709 12:21:39.898802 (+  1127us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:39.898888 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:39.898932 (+    44us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:21:39.899680 (+   748us) tablet.cc:701] Row locks acquired
0709 12:21:39.899711 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:21:39.899937 (+   226us) write_op.cc:270] Start()
0709 12:21:39.900106 (+   169us) write_op.cc:276] Timestamp: P: 1783599699899864 usec, L: 0
0709 12:21:39.900136 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:21:40.440569 (+540433us) spinlock_profiling.cc:243] Waited 50.1 ms on lock 0x7b5400328820. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53988da5c4 00007f5393b6b5ab 00007f5393b6b2fa 00007f5393bef965 00007f5393befe09 00007f539762a581 00007f5397629d58 00007f539762e9f1 00007f539762e9a9 00007f539762e939 00007f539762e901
0709 12:21:40.441236 (+   667us) log.cc:844] Serialized 2074 byte log entry
0709 12:21:40.888141 (+446905us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393befe28 00007f539762a581 00007f5397629d58 00007f539762e9f1 00007f539762e9a9 00007f539762e939 00007f539762e901 00007f539762db2d 0000000000388634 0000000000388479
0709 12:21:41.404675 (+516534us) op_driver.cc:464] REPLICATION: finished
0709 12:21:41.730932 (+326257us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539762aef0
0709 12:21:41.818853 (+ 87921us) write_op.cc:301] APPLY: starting
0709 12:21:41.818972 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:41.819224 (+   252us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:41.819257 (+    33us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:41.820245 (+   988us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:41.820541 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:21:41.820620 (+    79us) write_op.cc:312] APPLY: finished
0709 12:21:41.820987 (+   367us) log.cc:844] Serialized 47 byte log entry
0709 12:21:41.821216 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:41.821384 (+   168us) write_op.cc:454] Released schema lock
0709 12:21:41.821447 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:41.823119 (+  1672us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:41.823165 (+    46us) write_op.cc:454] Released schema lock
0709 12:21:42.247691 (+424526us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf132a 00007f5393b76f14 00007f5393b76ec3 00007f5393b76e38 00007f5393b76df3 00007f5393b760dc 00007f5393b9fe16 00007f5393b87cbb 00007f5393b7239d 00007f5393b75d75
Metrics: {"mutex_wait_us":143658,"reported_tablets":7,"spinlock_wait_cycles":2432,"child_traces":[["op",{"apply.queue_time_us":404607,"apply.run_cpu_time_us":3091,"apply.run_wall_time_us":4240,"num_ops":5,"prepare.queue_time_us":279022,"prepare.run_cpu_time_us":22889,"prepare.run_wall_time_us":1019290,"raft.queue_time_us":384369,"raft.run_cpu_time_us":36030,"raft.run_wall_time_us":890674,"replication_time_us":1478695,"spinlock_wait_cycles":144828672,"thread_start_us":761813,"threads_started":4,"wal-append.queue_time_us":194175}]]}
I20260709 12:21:42.582572 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.575317 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.267925 15636 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.550534 15564 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:42.264514 15642 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:42.449191 15497 consensus_queue.cc:1175] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260709 12:21:42.568194 15427 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.588454 13652 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:42.590924 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:42.567206 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.602331 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:42.630743 13720 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:42.571096 15464 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:42.603242 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.605145 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:42.634984 13722 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:42.605969 13726 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:42.603343 15610 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.636458 13726 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:42.637269 15610 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:42.608887 13693 connection.cc:256] RPC call timeout handler was delayed by -3.0473408080001718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:21:42.638188 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8403408079998371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 12:21:42.638540 15740 tablet_replica.cc:333] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
W20260709 12:21:42.638792 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8331408079998255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 12:21:42.639083 15621 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:42.607941 13650 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:42.639883 15740 raft_consensus.cc:2243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:21:42.640301 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.640671 15740 pending_rounds.cc:70] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Trying to abort 1 pending ops.
I20260709 12:21:42.615674 13648 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.575675 15730 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:42.620191 13549 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:42.607304 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.622684 15677 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.598340 15725 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:42.599602 15395 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:21:42.586858 15729 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.587260 15644 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:42.692734 15644 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.693192 13722 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:42.693713 13722 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:42.688647 15725 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:21:42.620193 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 366101us
I20260709 12:21:42.695540 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:42.599335 15637 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:42.576829 15726 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.628922 13798 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:42.600980 15738 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:42.639248 13693 connection.cc:256] RPC call timeout handler was delayed by -2.7611408080001638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
I20260709 12:21:42.663504 15631 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.597123 15579 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:42.640934 13650 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.652828 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:42.712460 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59777.7us
I20260709 12:21:42.668249 13799 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:21:42.663978 13549 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:42.668444 13652 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:21:42.668602 13726 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:42.623750 15564 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.643710 15625 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.640933 15427 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.673135 15596 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.625532 15737 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.675282 15642 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:42.606334 15659 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.677572 13648 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:42.680409 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:42.624945 15503 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.687552 13720 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:42.690779 15636 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.691056 15729 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.664096 13644 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.698011 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:42.699676 13693 connection.cc:256] RPC call timeout handler was delayed by -2.6873408080000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2157s for a response
W20260709 12:21:42.764493 13693 connection.cc:256] RPC call timeout handler was delayed by -2.6522408079999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 12:21:42.764979 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5712408079998568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22860000000000003s for a response
W20260709 12:21:42.765645 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4974408080001922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2368s for a response
W20260709 12:21:42.766088 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4632408080001369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24060000000000004s for a response
I20260709 12:21:42.607573 15634 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.694706 15725 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.662868 15730 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:42.783581 15730 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.703290 13725 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:21:42.705220 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:42.785451 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.718232 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.786087 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:42.733831 15395 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:21:42.719425 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.723618 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.787325 15737 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.787978 13647 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:21:42.740012 15621 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:21:42.788945 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.789577 13647 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:42.640990 15740 pending_rounds.cc:77] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Aborting op as it isn't in flight: id { term: 3 index: 5 } timestamp: 7305624323038130176 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } } new_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } } }
I20260709 12:21:42.763098 15736 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:42.790496 15740 raft_consensus.cc:2917] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted
I20260709 12:21:42.790798 15736 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:42.791103 15740 raft_consensus.cc:2889] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:21:42.791275 15736 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.791560 15740 raft_consensus.cc:2272] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:21:42.663933 15464 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.794943 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.796241 15427 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:42.724493 13549 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:21:42.715150 15738 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.734897 13722 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.803903 13722 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.714504 15631 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.754508 15642 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:42.702354 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:42.798652 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100717us
I20260709 12:21:42.762912 15732 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:42.799552 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:42.787391 13652 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.796955 15464 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:42.754295 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.805047 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.851269 13647 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:42.851725 13647 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:42.780923 15634 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.805050 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:42.808295 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.856807 13792 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:42.814752 13719 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:42.760421 15503 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.815315 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.819507 15636 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:42.862000 14423 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:21:42.825160 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.763211 15708 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:21:42.740201 15625 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.886974 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:42.888056 14425 raft_consensus.cc:2410] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:42.830998 13723 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:42.830829 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:42.799230 13643 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:21:42.831866 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.510s (SENT)
I20260709 12:21:42.889263 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:42.833755 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.890331 13795 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:42.833752 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:42.891433 13798 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:42.840131 13800 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:42.844928 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.840190 15732 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:42.854748 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.819360 15596 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:42.818423 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.901503 13648 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:21:42.862747 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.846006 13645 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:42.825688 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78532.8us
W20260709 12:21:42.917572 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 85721.5us
I20260709 12:21:42.918927 13652 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:42.828888 15637 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:42.771714 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:42.919425 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.626s (SENT)
I20260709 12:21:42.921869 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.922748 13648 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.924927 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.879745 13722 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:21:42.923170 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.679s (SENT)
I20260709 12:21:42.898792 13647 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:42.932789 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:42.936815 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:42.938230 13648 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:42.939956 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.867882 15464 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:21:42.944401 13647 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:42.941579 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111998us
I20260709 12:21:42.868130 13721 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:42.839592 15642 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:42.887405 15726 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:42.944307 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.912091 15732 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:42.956068 13652 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:42.912449 15741 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:42.959304 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:42.963963 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:42.939072 15626 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:42.968704 13648 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:42.965067 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:42.973322 13651 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:42.938746 15621 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:42.946622 15727 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:21:42.958913 13646 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:42.915179 15746 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 12:21:43.004211 13647 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:42.946282 15427 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:21:43.006388 15740 ts_tablet_manager.cc:1916] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:21:43.012041 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 207095us
I20260709 12:21:43.045135 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:43.016105 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:43.046231 13652 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:43.018373 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.033944 13648 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:43.050391 13651 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:42.954650 15579 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:21:43.058171 13648 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:42.991669 15624 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:43.073362 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 242647us
I20260709 12:21:43.007778 15610 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:43.053738 15745 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:21:43.068122 15728 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:21:43.088163 13650 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:43.007951 15704 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 12:21:43.094631 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.096894 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:43.106459 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152135us
I20260709 12:21:43.064107 15565 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:43.093418 13652 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:43.114099 15464 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.123539 13652 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:42.987855 15621 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.070392 15659 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:43.095330 15726 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:43.101522 15745 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:43.120039 15636 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:43.163506 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.164781 15621 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:43.163970 15636 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.129285 15565 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:43.148397 15729 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.065352 15637 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.099550 15624 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.148717 15762 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:43.185786 15769 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:21:43.177150 15743 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 12:21:43.148360 15464 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.189898 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:43.194989 13793 raft_consensus.cc:2410] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:43.172281 15631 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.168916 15745 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:43.192798 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.196309 13646 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:21:43.178792 15738 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.210553 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.173832 15726 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.183712 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
W20260709 12:21:43.197947 15744 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:21:43.188711 15762 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:43.228744 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.231813 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:43.250265 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 327088us
I20260709 12:21:43.197315 15769 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:43.198184 15634 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.215444 15631 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:43.258474 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.699s (SENT)
W20260709 12:21:43.263039 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 166281us
I20260709 12:21:43.200433 15503 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:43.273614 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 228534us
I20260709 12:21:43.275198 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:43.264214 13617 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.775s (SENT)
I20260709 12:21:43.282720 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:43.239859 15642 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.244258 15760 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:43.248952 15762 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:43.245304 14421 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:43.291368 15709 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:43.302354 15756 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:43.302865 15709 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.305212 15464 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:43.306145 15770 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:43.306494 15503 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.306658 15772 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:43.307135 15503 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.307458 15769 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.302954 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:43.309721 15503 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:43.310170 15782 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:43.308048 15642 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.307206 15772 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.219776 15565 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.282511 15767 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.305290 15709 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:43.320361 15754 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:21:43.320026 15767 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.343906 15762 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.280889 15631 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:43.449455 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:43.442564 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 178362us
I20260709 12:21:43.434954 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:43.356220 13796 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.001s	sys 0.023s
W20260709 12:21:43.455693 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:43.455895 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.794s (SENT)
W20260709 12:21:43.390672 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:43.457005 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.798s (SENT)
W20260709 12:21:43.457697 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:43.458797 13796 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.003s	sys 0.038s
I20260709 12:21:43.461851 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.459084 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:43.461776 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 306577us
I20260709 12:21:43.457203 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.467252 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.468189 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:43.465392 13652 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:43.324376 15763 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.470908 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.471688 13652 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:43.472891 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.473583 13652 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
W20260709 12:21:43.481712 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 214583us
I20260709 12:21:43.431428 15634 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.357556 15464 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:21:43.433929 15768 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.442003 15503 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.496942 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:43.321920 15642 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.443436 15741 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:43.506096 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:43.463032 15767 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:43.510097 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.505496 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.506434 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:43.460886 15776 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:43.470698 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.499446 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:43.497447 15624 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.531014 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:43.502998 15758 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:43.511445 15503 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:43.535470 15725 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.004s
I20260709 12:21:43.533628 15731 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:43.553642 15731 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:43.554795 15731 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.555980 15731 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:43.502182 15763 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.536228 15610 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:43.547555 15785 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:43.516319 15756 raft_consensus.cc:2764] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:21:43.552233 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.520181 15776 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.503002 15634 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.574296 15785 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:43.574690 15787 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:43.531256 15760 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:43.538148 13646 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:43.589265 15634 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:43.589200 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:43.548667 15788 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:43.584640 15776 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:43.591025 15788 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.573747 15636 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:21:43.590732 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.591544 15788 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:43.559904 15767 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:21:43.505908 15768 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.598403 15741 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.612658 15785 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.592216 15787 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.563041 15725 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.632014 15768 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.633603 15737 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:43.634130 15737 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.634570 15737 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:43.637548 13645 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
I20260709 12:21:43.639264 15746 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:43.639796 15746 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:43.640250 15746 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:43.643450 15607 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.050s	user 0.008s	sys 0.006s
W20260709 12:21:43.620354 15564 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.880s	user 0.008s	sys 0.004s
I20260709 12:21:43.644399 15725 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.644668 15564 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.645438 15725 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:43.645628 15777 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:21:43.645927 13649 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 2: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:43.646993 15631 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:43.590782 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.638000 13645 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:43.626067 15787 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:43.659595 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:43.670516 15503 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:43.676687 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1476) took 1060 ms. Trace:
I20260709 12:21:43.677309 13651 rpcz_store.cc:276] 0709 12:21:42.615857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.995162 (+379305us) service_pool.cc:224] Handling call
0709 12:21:43.050110 (+ 54948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:43.050921 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:43.676393 (+625472us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b54002e45d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:43.676589 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124983168}
I20260709 12:21:43.645913 15725 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.657469 13646 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:43.694963 15564 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:43.702847 13648 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:21:43.705459 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.710091 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:43.710279 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.711422 14417 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:43.721354 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 296) took 1432 ms. Trace:
I20260709 12:21:43.721884 13721 rpcz_store.cc:276] 0709 12:21:42.288887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.627485 (+338598us) service_pool.cc:224] Handling call
0709 12:21:43.721114 (+1093629us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54003fe8d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:43.721274 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238680960}
I20260709 12:21:43.725132 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:43.726356 15579 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:43.657292 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 304) took 1017 ms. Trace:
I20260709 12:21:43.712510 15725 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:43.726768 15579 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.740433 15579 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:43.745478 15736 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.954s	user 0.011s	sys 0.005s
I20260709 12:21:43.751196 15736 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.751864 15565 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:43.753674 15736 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:43.732380 15642 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:43.732700 13645 rpcz_store.cc:276] 0709 12:21:42.644258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.669940 (+ 25682us) service_pool.cc:224] Handling call
0709 12:21:43.651767 (+981827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:43.760355 15726 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:43.764726 15745 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.539s	user 0.004s	sys 0.008s
W20260709 12:21:43.766341 15732 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.767s	user 0.010s	sys 0.003s
I20260709 12:21:43.766558 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.773985 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.774894 14423 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:43.776168 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.767241 15745 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.780768 15772 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.771095 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:43.783269 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:43.786788 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:43.797518 15731 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:43.797891 15749 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:43.798079 15731 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.812342 15789 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:43.812976 15657 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:43.813098 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:43.817339 13644 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:43.828047 15757 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:43.828620 15757 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:43.829108 15757 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:43.820348 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.821564 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.823896 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1464) took 1336 ms. Trace:
I20260709 12:21:43.832497 13649 rpcz_store.cc:276] 0709 12:21:42.487181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.487356 (+   175us) service_pool.cc:224] Handling call
0709 12:21:43.823632 (+1336276us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:43.823819 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2602928896}
I20260709 12:21:43.834056 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.821257 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.835420 13649 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:43.831715 13648 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:21:43.839711 15790 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:21:43.841320 15790 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:43.841784 15790 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:43.788235 15564 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:43.845120 13644 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:43.847124 15625 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.848585 13645 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:43.849970 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.850168 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:43.859091 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1467) took 1353 ms. Trace:
I20260709 12:21:43.863042 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:43.859688 13643 rpcz_store.cc:276] 0709 12:21:42.506038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.506166 (+   128us) service_pool.cc:224] Handling call
0709 12:21:43.858836 (+1352670us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:43.859015 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510973696}
I20260709 12:21:43.831519 15731 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:43.813937 15789 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:43.916492 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:43.917289 13798 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:43.916656 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:43.917749 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.918587 14423 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
W20260709 12:21:43.925415 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 105197us
I20260709 12:21:43.926481 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.927757 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:43.794983 15732 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.908180 15564 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.919258 15730 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:43.912441 15789 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:43.930912 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 473920us
I20260709 12:21:43.845145 15737 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:43.945729 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.862s (SENT)
I20260709 12:21:43.951733 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:43.923972 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.956153 15787 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:43.957247 15763 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:43.935545 15767 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:43.936731 15677 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:21:43.909632 15776 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.938439 15726 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:43.957628 15765 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:43.943379 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:43.938912 15564 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.961373 14425 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:43.971163 15745 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:43.972918 15762 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:43.974411 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.022142 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.022799 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:43.976238 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.024338 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:43.975943 15789 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:44.027536 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:43.967115 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.887s (SENT)
I20260709 12:21:44.012903 15765 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:43.965898 15767 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.968314 15677 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:43.968755 15348 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:44.026504 13649 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:44.029677 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:44.059378 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.028715 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:44.006502 15726 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:44.084439 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.088174 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.098687 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70055.5us
W20260709 12:21:44.103762 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 136622us
W20260709 12:21:44.104552 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.891s (SENT)
W20260709 12:21:44.105079 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.018s	sys 0.023s
I20260709 12:21:44.105006 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:44.105742 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.019s	sys 0.023s
I20260709 12:21:44.106941 15737 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.107156 15776 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:44.095043 15726 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:44.107656 15765 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:44.109952 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 603905us
I20260709 12:21:44.111689 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:44.089994 15677 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:21:44.086571 15674 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:44.096246 15789 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:44.091773 15767 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:44.142438 15348 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:44.108529 15770 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:44.074725 15746 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:44.144296 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.144847 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.084s (SENT)
I20260709 12:21:43.950695 15790 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:44.116418 15565 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:21:44.164755 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52992.1us
I20260709 12:21:44.170378 15763 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:44.171187 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:44.171121 15763 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.136559 15674 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:44.173831 15674 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:44.155493 15740 ts_tablet_manager.cc:1929] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
W20260709 12:21:44.169606 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.015s	sys 0.067s
W20260709 12:21:44.274729 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.016s	sys 0.067s
W20260709 12:21:44.275099 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 810300us
W20260709 12:21:44.237046 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.260058 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.278378 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.260700 13719 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:44.260825 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.280298 13723 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:44.260681 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.262816 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.281801 13725 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:44.259338 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.113054 15725 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:44.166397 15788 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.003s
I20260709 12:21:44.273368 13798 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
W20260709 12:21:44.274075 13617 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.094s (SENT)
W20260709 12:21:44.287385 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.318162 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.310668 13724 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:44.166742 15720 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:44.166368 15745 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:44.167588 15738 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:44.254652 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:44.164433 15770 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:44.329150 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.329998 15732 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:44.271215 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:44.324095 13727 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:44.339857 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:44.343673 15745 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:21:44.344422 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70406.7us
W20260709 12:21:44.354876 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.120s (SENT)
I20260709 12:21:44.344756 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:44.355906 13649 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:44.348811 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.309726 15785 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:44.358546 15785 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.327523 15790 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:44.335278 15740 log.cc:1199] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b4f6bfcda63148e4965ca02c639336f1: Deleting WAL directory at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/wals/88a3d1d336b2495ea9a3a7ae1d7d37a2
I20260709 12:21:44.364243 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:44.329460 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:44.156689 15776 consensus_queue.cc:1175] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dafe08a4f404502822a0732fe570de4" responder_term: 4 status { last_received { term: 3 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:21:44.375000 15746 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:44.381247 15757 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.552s	user 0.006s	sys 0.006s
I20260709 12:21:44.355023 13644 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:44.362648 15785 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.361704 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.352066 15738 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:44.382946 13645 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:44.327046 15788 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:44.403378 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48551.9us
W20260709 12:21:44.406066 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.157s (SENT)
I20260709 12:21:44.402200 13649 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:44.406538 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:44.392117 15787 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.411038 15740 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34444 (request call id 77) took 2139 ms. Trace:
I20260709 12:21:44.433349 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:44.434340 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.434145 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:44.434315 13649 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:44.409073 15348 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:44.408407 15564 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:44.452497 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46464.2us
I20260709 12:21:44.390965 15770 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.416923 15745 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:44.386821 15757 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:44.491592 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.447670 15623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.003s	sys 0.033s
W20260709 12:21:44.450525 15730 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.506s	user 0.003s	sys 0.010s
W20260709 12:21:44.510620 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.450181 15785 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:21:44.447633 15779 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.047s	sys 0.045s
W20260709 12:21:44.514919 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55841.3us
I20260709 12:21:44.517608 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:44.519908 15757 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:44.521514 15779 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.048s	sys 0.045s
I20260709 12:21:44.520987 15677 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.525362 15348 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:44.526412 15785 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:44.527453 15785 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:44.502564 15731 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.003s
I20260709 12:21:44.498953 15787 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:44.500926 15623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.987s	user 0.003s	sys 0.034s
W20260709 12:21:44.542590 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:44.547401 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.549291 13649 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:44.550983 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:44.554314 15677 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.559195 15788 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.447419 15740 rpcz_store.cc:276] 0709 12:21:42.272114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.272377 (+   263us) service_pool.cc:224] Handling call
0709 12:21:42.638089 (+365712us) ts_tablet_manager.cc:1151] Deleting tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2
0709 12:21:42.638194 (+   105us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:21:43.632771 (+994577us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:44.141942 (+509171us) tablet_metadata.cc:665] Metadata flushed
0709 12:21:44.410946 (+269004us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":66448,"thread_start_us":29761,"threads_started":1}
I20260709 12:21:44.561651 13536 catalog_manager.cc:5028] TS b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): tablet 88a3d1d336b2495ea9a3a7ae1d7d37a2 (table test-workload [id=5df2a3b9258a41f6b50be2d77e637387]) successfully deleted
I20260709 12:21:44.561527 15726 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:44.562394 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.563917 15726 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:44.500774 15745 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:44.565199 15579 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.798s	user 0.000s	sys 0.009s
I20260709 12:21:44.575225 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:44.545522 15767 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.602042 13644 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:44.609333 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:44.627739 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:44.628774 13649 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:21:44.603188 15738 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:44.631922 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:44.624212 15623 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:44.568455 15730 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:44.640323 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:44.643249 13718 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.118s	user 0.064s	sys 0.078s
I20260709 12:21:44.535856 15731 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:44.646277 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.098s	user 0.021s	sys 0.015s
W20260709 12:21:44.651034 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.021s	sys 0.015s
I20260709 12:21:44.546855 15787 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:44.651721 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.14526e+06us
I20260709 12:21:44.565759 15579 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:44.643646 13718 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.064s	sys 0.078s
W20260709 12:21:44.642170 15621 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.061s	user 0.015s	sys 0.020s
I20260709 12:21:44.649814 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:44.661489 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:44.643460 15726 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:21:44.663932 13800 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.022s	sys 0.051s
I20260709 12:21:44.653431 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:44.669369 13652 raft_consensus.cc:2368] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5.
I20260709 12:21:44.643139 15725 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:44.671166 13800 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.022s	sys 0.051s
W20260709 12:21:44.653524 15636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.012s	sys 0.024s
W20260709 12:21:44.657986 15659 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.144s	user 0.024s	sys 0.047s
I20260709 12:21:44.645969 15730 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:44.653412 15763 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:44.657860 15621 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.015s	sys 0.021s
W20260709 12:21:44.665325 15789 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.005s
I20260709 12:21:44.706929 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 211) took 1842 ms. Trace:
I20260709 12:21:44.663978 15348 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:44.730316 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:44.705905 15636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.012s	sys 0.025s
W20260709 12:21:44.706245 15659 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.192s	user 0.024s	sys 0.048s
I20260709 12:21:44.705427 15794 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:44.723451 15626 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.725016 13800 rpcz_store.cc:276] 0709 12:21:42.896367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.896522 (+   155us) service_pool.cc:224] Handling call
0709 12:21:42.912430 (+ 15908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:43.055061 (+142631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:43.055099 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:43.128005 (+ 72906us) log.cc:844] Serialized 26 byte log entry
0709 12:21:43.293514 (+165509us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:43.550940 (+257426us) log.cc:844] Serialized 14 byte log entry
0709 12:21:43.644865 (+ 93925us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:21:44.681848 (+1036983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:44.681955 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:44.682018 (+    63us) raft_consensus.cc:1685] finished
0709 12:21:44.706626 (+ 24608us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:44.706845 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48470,"prepare.queue_time_us":257226,"prepare.run_cpu_time_us":1659,"prepare.run_wall_time_us":13707,"raft.queue_time_us":311144,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":8766,"spinlock_wait_cycles":9495808,"thread_start_us":1434797,"threads_started":4,"wal-append.queue_time_us":277385}
I20260709 12:21:44.754407 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.756050 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.755977 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.791582 13727 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:44.725723 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:44.788996 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.793511 13723 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:44.758535 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.744228 15636 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:44.798563 13724 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:44.799528 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1653) took 1335 ms. Trace:
I20260709 12:21:44.799330 15636 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.799953 13652 rpcz_store.cc:276] 0709 12:21:43.463890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.496662 (+ 32772us) service_pool.cc:224] Handling call
0709 12:21:43.862672 (+366010us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:44.799239 (+936567us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:44.799428 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859617920}
I20260709 12:21:44.802424 13767 leader_election.cc:400] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5.
I20260709 12:21:44.802840 13767 leader_election.cc:403] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:21:44.803169 13726 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:44.803929 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:44.755020 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.759177 15731 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:44.761036 14421 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:44.794709 13719 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:44.795691 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.795229 15626 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.804277 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.698753 15726 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:44.830039 15726 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:44.807087 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.831902 13725 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:44.807092 13652 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:44.765726 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:44.816036 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.833386 13722 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:44.729183 15763 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:44.814922 15623 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:44.817250 13721 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:44.819056 15626 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:44.839645 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.819342 15795 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:44.723706 15565 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 12:21:44.820240 15730 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:44.822063 15579 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:44.823112 15767 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.824522 15787 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:44.749209 15773 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 12:21:44.825564 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:44.825730 15745 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:44.826282 13724 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:44.828578 15659 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:44.748847 15788 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:44.743291 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:44.837139 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:44.883659 13800 raft_consensus.cc:2368] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3.
I20260709 12:21:44.885329 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:44.837886 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112251us
I20260709 12:21:44.840545 13725 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:44.841619 15763 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:44.842182 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:44.843492 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.887038 13798 raft_consensus.cc:2368] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
W20260709 12:21:44.815877 15674 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.641s	user 0.004s	sys 0.007s
I20260709 12:21:44.847723 13797 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:44.850708 13795 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:44.853765 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:44.857563 13652 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:44.859906 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:44.862694 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:44.892328 13799 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4.
I20260709 12:21:44.892590 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:44.893177 13652 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:44.866084 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.868249 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:44.853022 15596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.880s	user 0.041s	sys 0.049s
I20260709 12:21:44.839560 15725 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:21:44.878944 13765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.321s	user 0.008s	sys 0.039s
W20260709 12:21:44.880483 13650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.198s	user 0.024s	sys 0.110s
I20260709 12:21:44.888258 13800 raft_consensus.cc:2368] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
W20260709 12:21:44.868139 15637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.221s	user 0.010s	sys 0.030s
I20260709 12:21:44.891193 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:44.894634 13723 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:21:44.894851 13727 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:44.896222 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:44.900964 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:44.901386 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:44.851493 15795 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:44.945930 13652 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:45.223531 13793 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 4. Ops: []
I20260709 12:21:45.358060 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.358603 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:44.980665 14424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.182s	user 0.000s	sys 0.086s
I20260709 12:21:45.359442 13652 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:45.360158 14424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.562s	user 0.001s	sys 0.091s
W20260709 12:21:44.947816 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61005.5us
I20260709 12:21:44.902840 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:45.193030 14389 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.284s	user 0.008s	sys 0.035s
W20260709 12:21:45.362074 14389 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.454s	user 0.010s	sys 0.037s
W20260709 12:21:45.050717 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.142s	user 0.007s	sys 0.039s
W20260709 12:21:45.363363 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.455s	user 0.009s	sys 0.051s
I20260709 12:21:45.363467 13797 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3.
I20260709 12:21:45.364724 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:45.050123 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.128s	user 0.020s	sys 0.029s
W20260709 12:21:45.368057 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.446s	user 0.028s	sys 0.036s
W20260709 12:21:45.368456 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.44682e+06us
W20260709 12:21:44.980883 13720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.417s	user 0.013s	sys 0.028s
W20260709 12:21:45.371160 13720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.808s	user 0.014s	sys 0.033s
I20260709 12:21:44.947022 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:45.041466 13648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.189s	user 0.024s	sys 0.026s
W20260709 12:21:45.372727 13648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.520s	user 0.025s	sys 0.040s
W20260709 12:21:45.192970 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.002s	sys 0.009s
I20260709 12:21:44.938036 13649 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:21:45.375795 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:45.376194 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:45.376834 13649 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.377223 13723 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:21:45.193110 13801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.238s	user 0.013s	sys 0.028s
W20260709 12:21:45.380692 13801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.425s	user 0.016s	sys 0.029s
I20260709 12:21:45.362685 13724 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:44.929008 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:45.384066 13798 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
W20260709 12:21:45.362687 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:45.393703 13719 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:45.402545 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.404729 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.371235 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.358973 13792 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
W20260709 12:21:45.426633 13650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.688s	user 0.036s	sys 0.122s
W20260709 12:21:45.427105 13765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.820s	user 0.012s	sys 0.060s
W20260709 12:21:45.427263 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.005s	sys 0.011s
I20260709 12:21:45.383919 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:44.884682 15649 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.261s	user 0.015s	sys 0.035s
I20260709 12:21:45.436561 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:45.446053 13724 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:45.458431 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:45.458644 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:45.459578 14425 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:45.447172 13643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.073s	user 0.002s	sys 0.015s
W20260709 12:21:45.461479 13643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.002s	sys 0.015s
W20260709 12:21:45.049857 13646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.338s	user 0.011s	sys 0.038s
W20260709 12:21:45.468660 13651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.721s	user 0.009s	sys 0.018s
W20260709 12:21:45.470196 13651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.729s	user 0.009s	sys 0.018s
W20260709 12:21:45.152701 13645 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.747s	user 0.014s	sys 0.000s
W20260709 12:21:45.192873 15741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.217s	user 0.014s	sys 0.015s
I20260709 12:21:45.457027 13793 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:45.451788 13800 raft_consensus.cc:2410] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:21:45.452913 13719 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:45.452877 13727 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:45.166791 15720 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.838s	user 0.008s	sys 0.006s
I20260709 12:21:45.445998 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:45.210887 15644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.462s	user 0.014s	sys 0.031s
I20260709 12:21:44.878136 15745 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:45.372368 15596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.407s	user 0.043s	sys 0.060s
I20260709 12:21:45.447721 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:45.493844 15741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.510s	user 0.018s	sys 0.016s
I20260709 12:21:45.482333 13652 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:21:45.504369 13718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.013s	sys 0.036s
W20260709 12:21:45.592538 13718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.014s	sys 0.041s
I20260709 12:21:45.498202 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:45.448387 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:45.493717 13726 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.008s	sys 0.014s
I20260709 12:21:45.487248 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:45.613466 13726 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.012s	sys 0.016s
I20260709 12:21:45.613782 13719 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:45.614950 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:45.545849 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.616165 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.902726 15626 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:21:45.427009 15637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.780s	user 0.013s	sys 0.049s
W20260709 12:21:45.164844 15779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.005s	sys 0.001s
I20260709 12:21:44.965243 15427 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:45.058074 15709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.014s	sys 0.029s
I20260709 12:21:45.548797 13724 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:45.646790 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:44.890012 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:45.549868 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:45.649334 13792 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:45.138468 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:45.355875 15791 raft_consensus.cc:476] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Not starting pre-election -- already a leader
W20260709 12:21:45.665755 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:45.614044 15596 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:45.136595 15767 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:44.928433 15787 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:45.486529 13646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.775s	user 0.013s	sys 0.040s
I20260709 12:21:45.425194 15738 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:45.192613 15759 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.580s	user 0.006s	sys 0.022s
W20260709 12:21:45.703902 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 807758us
I20260709 12:21:45.705475 13797 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
W20260709 12:21:45.487497 15649 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.849s	user 0.016s	sys 0.040s
I20260709 12:21:45.593665 13795 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:45.616050 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1671) took 1264 ms. Trace:
I20260709 12:21:45.618129 13719 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:45.620769 13799 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:45.621495 13723 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:21:45.590765 15644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.860s	user 0.014s	sys 0.038s
I20260709 12:21:45.623961 13649 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:45.710572 13652 rpcz_store.cc:276] 0709 12:21:44.351321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.594023 (+1242702us) service_pool.cc:224] Handling call
0709 12:21:45.614991 (+ 20968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:45.615599 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:45.615625 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:45.615687 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:45.615739 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.615816 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:45.615977 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:45.718240 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:45.719588 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1674) took 1320 ms. Trace:
I20260709 12:21:44.880962 15659 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:45.638443 13794 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:45.634436 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:45.751783 15741 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:44.928087 15674 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:45.639739 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.647469 13724 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:45.582441 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:45.654366 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:45.658108 13645 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:21:45.658461 14388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.408s	user 0.028s	sys 0.061s
I20260709 12:21:45.659550 14421 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:45.575254 15798 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:45.210803 15769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.230s	user 0.012s	sys 0.019s
I20260709 12:21:45.671336 14425 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:21:45.639724 15779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.010s	sys 0.016s
I20260709 12:21:45.549794 15623 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
W20260709 12:21:45.647712 15709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.813s	user 0.018s	sys 0.033s
I20260709 12:21:45.630107 15626 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:45.693670 13793 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:45.955742 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1535) took 1083 ms. Trace:
I20260709 12:21:45.956143 13793 rpcz_store.cc:276] 0709 12:21:44.872627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.435933 (+563306us) service_pool.cc:224] Handling call
0709 12:21:45.955662 (+519729us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:45.956707 13792 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:45.957286 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1530) took 1094 ms. Trace:
I20260709 12:21:45.651295 15565 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:45.689116 15767 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:45.776748 13647 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:45.673841 13721 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:45.955861 15808 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.530s	user 0.000s	sys 0.001s
I20260709 12:21:45.778998 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.715345 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:45.709359 15759 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.097s	user 0.007s	sys 0.025s
I20260709 12:21:45.787204 13719 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:45.979524 13795 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:45.720124 13652 rpcz_store.cc:276] 0709 12:21:44.399312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.718519 (+1319207us) service_pool.cc:224] Handling call
0709 12:21:45.718908 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:45.719157 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:45.719196 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:45.719262 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:45.719307 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.719380 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:45.719516 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:45.980437 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.796694 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.980655 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.663722 15427 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:45.804486 14422 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:45.759450 15822 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.805970 13800 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:45.989328 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1522) took 1128 ms. Trace:
I20260709 12:21:45.989789 13800 rpcz_store.cc:276] 0709 12:21:44.860826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.639397 (+778571us) service_pool.cc:224] Handling call
0709 12:21:45.989245 (+349848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:45.991014 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:45.992965 13800 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:45.817394 13724 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:45.820343 14388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.909s	user 0.028s	sys 0.061s
W20260709 12:21:45.872216 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.359s	user 0.011s	sys 0.076s
W20260709 12:21:45.806699 13691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.045s	sys 0.048s
W20260709 12:21:45.993884 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.481s	user 0.011s	sys 0.094s
W20260709 12:21:45.993968 13691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.064s	sys 0.059s
I20260709 12:21:45.994122 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:45.994552 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:45.994781 13724 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:45.850355 15726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.017s	user 0.028s	sys 0.059s
I20260709 12:21:45.793874 15625 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:45.917397 13798 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4.
I20260709 12:21:45.703279 15787 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:46.016216 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:46.017027 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:46.017724 15787 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:45.820703 15607 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:45.803371 15803 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:21:45.768412 15763 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:45.716611 15770 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:45.844204 15785 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.043s	sys 0.103s
W20260709 12:21:45.887531 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:45.894865 15621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.135s	user 0.082s	sys 0.134s
I20260709 12:21:45.819423 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1656) took 2229 ms. Trace:
I20260709 12:21:45.863042 15644 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:45.957320 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:46.054200 13793 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:45.701792 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:45.957758 13792 rpcz_store.cc:276] 0709 12:21:44.862733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.549497 (+686764us) service_pool.cc:224] Handling call
0709 12:21:45.957228 (+407731us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:45.930238 15636 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.115s	user 0.012s	sys 0.091s
I20260709 12:21:45.849465 14420 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:45.944515 15769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.982s	user 0.012s	sys 0.028s
I20260709 12:21:45.919981 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:45.614459 15745 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:45.978173 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:45.933513 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1664) took 1781 ms. Trace:
I20260709 12:21:45.929543 15741 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:45.987288 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 201) took 3760 ms (client timeout 1616 ms). Trace:
W20260709 12:21:46.068773 15636 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.250s	user 0.013s	sys 0.091s
I20260709 12:21:46.007283 13724 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:46.013342 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1529) took 1150 ms. Trace:
W20260709 12:21:46.009073 15726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.175s	user 0.032s	sys 0.063s
I20260709 12:21:46.019089 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:46.064174 15644 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:46.090962 13799 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:21:46.091393 15801 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:21:46.092882 15644 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:46.075143 15807 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.412s	user 0.000s	sys 0.002s
W20260709 12:21:46.093642 15805 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
W20260709 12:21:46.093976 15813 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:21:46.092945 15826 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:21:46.096225 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1670) took 1753 ms. Trace:
I20260709 12:21:46.097012 15659 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.018945 15822 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.100206 15822 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:46.101106 15818 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.332s	user 0.000s	sys 0.002s
I20260709 12:21:46.103181 15827 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:46.103839 15827 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.057121 13646 rpcz_store.cc:276] 0709 12:21:43.590139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.689621 (+ 99482us) service_pool.cc:224] Handling call
0709 12:21:43.689932 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:43.690155 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:21:43.711064 (+ 20909us) log.cc:844] Serialized 14 byte log entry
0709 12:21:45.751281 (+2040217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:45.761950 (+ 10669us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:45.769649 (+  7699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.786633 (+ 16984us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:45.808003 (+ 21370us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19125,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":76360,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":12578,"thread_start_us":1987024,"threads_started":1}
I20260709 12:21:46.107617 15427 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:46.096719 13643 rpcz_store.cc:276] 0709 12:21:44.342613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.362102 (+ 19489us) service_pool.cc:224] Handling call
0709 12:21:44.364752 (+  2650us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:44.365068 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:44.365109 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:44.365735 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.461965 (+1096230us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:45.462030 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.462149 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.095936 (+633787us) raft_consensus.cc:1685] finished
0709 12:21:46.095959 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.096153 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1087823,"threads_started":1,"wal-append.queue_time_us":1387652}
W20260709 12:21:46.027667 15815 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:21:45.848958 15596 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:46.110872 15833 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:21:46.111829 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 217) took 2195 ms. Trace:
I20260709 12:21:46.056437 15770 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:45.973181 15825 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:46.112263 13801 rpcz_store.cc:276] 0709 12:21:43.916605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.916856 (+   251us) service_pool.cc:224] Handling call
0709 12:21:43.920359 (+  3503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:43.928741 (+  8382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:43.928786 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:43.938993 (+ 10207us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.381386 (+1442393us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:45.381454 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.381576 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.111590 (+730014us) raft_consensus.cc:1685] finished
0709 12:21:46.111612 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.111760 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7926,"raft.queue_time_us":12144,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":7513,"thread_start_us":1425851,"threads_started":1,"wal-append.queue_time_us":1848477}
I20260709 12:21:46.113875 15770 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:45.985201 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1568) took 1127 ms. Trace:
I20260709 12:21:46.114494 13721 rpcz_store.cc:276] 0709 12:21:44.854376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.854992 (+   616us) service_pool.cc:224] Handling call
0709 12:21:45.981979 (+1126987us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:46.115633 15810 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:21:46.060513 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:46.116531 15674 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:46.117048 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1649) took 2654 ms. Trace:
I20260709 12:21:46.018738 15607 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.022720 15637 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:46.118084 15637 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:46.117489 13651 rpcz_store.cc:276] 0709 12:21:43.462576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.680540 (+217964us) service_pool.cc:224] Handling call
0709 12:21:43.680975 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:43.681606 (+   631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:43.681661 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:43.711974 (+ 30313us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.472148 (+1760174us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:45.472216 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.472326 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.116678 (+644352us) raft_consensus.cc:1685] finished
0709 12:21:46.116706 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.116971 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3270,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":147,"thread_start_us":1730805,"threads_started":1,"wal-append.queue_time_us":2223005}
I20260709 12:21:46.118687 15637 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:46.059818 15767 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:46.038823 15785 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.044s	sys 0.105s
I20260709 12:21:45.973654 15565 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.124833 15565 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:45.960853 15798 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.035501 15626 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:46.117347 13801 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:46.070662 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1479) took 3340 ms. Trace:
I20260709 12:21:46.068868 13723 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:46.047935 15621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.288s	user 0.083s	sys 0.134s
I20260709 12:21:46.035843 15836 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:46.035151 15820 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:46.035490 15579 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:45.998597 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:46.113606 15825 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.055375 15791 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:46.079313 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:46.085140 13794 rpcz_store.cc:269] 0709 12:21:42.227016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.227539 (+   523us) service_pool.cc:224] Handling call
0709 12:21:45.985694 (+3758155us) spinlock_profiling.cc:243] Waited 3.31 s on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:45.987203 (+  1509us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7611678976}
I20260709 12:21:46.109766 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1659) took 2288 ms. Trace:
W20260709 12:21:46.034657 14391 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.028s	sys 0.040s
I20260709 12:21:46.118619 15745 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:46.128254 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 310) took 2627 ms. Trace:
I20260709 12:21:46.134528 13718 rpcz_store.cc:276] 0709 12:21:43.500965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.501121 (+   156us) service_pool.cc:224] Handling call
0709 12:21:43.520083 (+ 18962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:44.654031 (+1133948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:44.654079 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:44.663399 (+  9320us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.592945 (+929546us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:45.592990 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.593078 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.128042 (+534964us) raft_consensus.cc:1685] finished
0709 12:21:46.128062 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.128195 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":61079,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"thread_start_us":1980638,"threads_started":2,"wal-append.queue_time_us":1124258}
I20260709 12:21:46.013325 15795 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.140537 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:46.141561 13801 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:46.148558 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1577) took 1124 ms. Trace:
I20260709 12:21:46.149250 13718 rpcz_store.cc:276] 0709 12:21:45.023839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.137074 (+1113235us) service_pool.cc:224] Handling call
0709 12:21:46.147527 (+ 10453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.148017 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:46.148076 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:46.148139 (+    63us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:46.148193 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.148276 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.148467 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:46.117092 14425 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:46.126933 15626 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:46.290591 15607 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.127182 15798 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.116358 13721 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:46.237082 15745 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:46.291472 15788 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.403s	user 0.012s	sys 0.003s
I20260709 12:21:46.291946 15788 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:46.291795 15745 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.128553 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:46.291514 13721 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:46.083359 14416 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:46.294548 13721 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:46.295759 13801 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:46.295862 13721 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:46.129242 15579 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.301937 15745 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.302445 15579 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:46.129314 15836 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:46.298132 15741 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:46.305076 15836 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:46.129603 13719 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:21:46.131470 14391 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.028s	sys 0.040s
I20260709 12:21:46.131681 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:46.309397 13794 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:46.305996 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 877714us
I20260709 12:21:46.313346 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:46.130540 15791 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.317058 15791 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.127339 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 311) took 2625 ms. Trace:
I20260709 12:21:46.384322 13720 rpcz_store.cc:276] 0709 12:21:43.501775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.501894 (+   119us) service_pool.cc:224] Handling call
0709 12:21:43.502241 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:43.510883 (+  8642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:43.510914 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:43.519517 (+  8603us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.399237 (+1879720us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:45.399297 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.399402 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.127048 (+727646us) raft_consensus.cc:1685] finished
0709 12:21:46.127076 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.127226 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2739,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":10623,"thread_start_us":1835337,"threads_started":1,"wal-append.queue_time_us":2231104}
I20260709 12:21:46.284098 15825 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:46.106159 15827 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:46.126183 15725 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.672s	user 0.004s	sys 0.006s
I20260709 12:21:46.323880 15621 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:21:46.129117 15820 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.475092 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 319) took 1037 ms. Trace:
I20260709 12:21:46.475651 13720 rpcz_store.cc:276] 0709 12:21:45.437307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.473543 (+1036236us) service_pool.cc:224] Handling call
0709 12:21:46.474306 (+   763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.474601 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:46.474636 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:46.474699 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:46.474753 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.474828 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.475010 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:46.478513 15725 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5.
I20260709 12:21:46.477851 15820 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:46.379747 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.004s	sys 0.000s
W20260709 12:21:46.480523 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.004s	sys 0.000s
I20260709 12:21:46.270046 15739 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:46.482867 15739 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:46.484191 15739 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:46.413724 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1583) took 1105 ms. Trace:
I20260709 12:21:46.062436 13792 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 3. Ops: []
I20260709 12:21:46.801110 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2194) took 1101 ms. Trace:
I20260709 12:21:46.801542 13792 rpcz_store.cc:276] 0709 12:21:45.699319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.061569 (+362250us) service_pool.cc:224] Handling call
0709 12:21:46.062101 (+   532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.800752 (+738651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.801035 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:46.803921 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.357s	user 0.032s	sys 0.036s
W20260709 12:21:46.804301 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.359s	user 0.032s	sys 0.037s
I20260709 12:21:46.128302 13800 rpcz_store.cc:276] 0709 12:21:44.862496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.992331 (+1129835us) service_pool.cc:224] Handling call
0709 12:21:45.992710 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.012659 (+ 19949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.013246 (+   587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:46.464771 15637 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:46.469718 14424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.001s	sys 0.012s
W20260709 12:21:46.813208 14424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.328s	user 0.003s	sys 0.012s
I20260709 12:21:46.466779 14425 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:46.701776 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:21:46.756112 13615 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.090s	user 0.000s	sys 0.022s
W20260709 12:21:47.033928 13615 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.368s	user 0.001s	sys 0.022s
W20260709 12:21:47.034708 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 1.3693e+06us
I20260709 12:21:47.035125 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:46.717919 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.038062 15764 raft_consensus.cc:2764] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:21:47.039112 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.042848 15850 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.043915 15850 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:46.757560 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.367s	user 0.002s	sys 0.015s
W20260709 12:21:46.819166 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.072651 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:46.675190 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.039s	sys 0.054s
W20260709 12:21:46.710902 13644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.004s	sys 0.004s
W20260709 12:21:47.074828 13644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.004s	sys 0.005s
I20260709 12:21:46.077737 13645 rpcz_store.cc:276] 0709 12:21:44.152138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.154791 (+  2653us) service_pool.cc:224] Handling call
0709 12:21:45.863561 (+1708770us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:46.497550 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:46.723665 13719 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:47.079722 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1575) took 2101 ms. Trace:
I20260709 12:21:47.080250 13719 rpcz_store.cc:276] 0709 12:21:44.977738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.998199 (+1020461us) service_pool.cc:224] Handling call
0709 12:21:47.079301 (+1081102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:46.478062 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 320) took 1040 ms. Trace:
I20260709 12:21:46.713663 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1584) took 1350 ms. Trace:
W20260709 12:21:46.772193 13725 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.287s	user 0.016s	sys 0.010s
I20260709 12:21:46.487371 13727 rpcz_store.cc:276] 0709 12:21:45.308086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.497932 (+189846us) service_pool.cc:224] Handling call
0709 12:21:46.413429 (+915497us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54002f7954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:46.413622 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1230554368}
W20260709 12:21:47.087553 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.040s	sys 0.054s
I20260709 12:21:47.089371 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:46.128163 13647 rpcz_store.cc:276] 0709 12:21:42.730591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.077957 (+347366us) service_pool.cc:224] Handling call
0709 12:21:43.084446 (+  6489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:45.964728 (+2880282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.070210 (+105482us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b5400095954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:46.070565 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6190974848}
I20260709 12:21:47.092770 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
W20260709 12:21:47.095224 13722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.256s	user 0.021s	sys 0.000s
W20260709 12:21:47.095695 13722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.259s	user 0.022s	sys 0.000s
W20260709 12:21:46.660116 13726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.027s	user 0.001s	sys 0.000s
I20260709 12:21:46.745445 13797 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
W20260709 12:21:47.097966 13726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.484s	user 0.003s	sys 0.001s
I20260709 12:21:46.819259 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:46.576040 15779 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.005s	sys 0.016s
W20260709 12:21:46.752795 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.011s	sys 0.016s
W20260709 12:21:46.576697 15709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.003s	sys 0.033s
W20260709 12:21:47.103783 15779 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.005s	sys 0.016s
W20260709 12:21:47.104501 15709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.003s	sys 0.034s
W20260709 12:21:47.104635 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.105361 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.105803 15709 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:47.106422 15709 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:47.108217 14417 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.392s	user 0.009s	sys 0.005s
W20260709 12:21:47.108801 14417 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.406s	user 0.009s	sys 0.005s
I20260709 12:21:47.108112 15709 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:46.564953 15730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.014s	user 0.004s	sys 0.012s
W20260709 12:21:47.110631 13725 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.625s	user 0.018s	sys 0.011s
W20260709 12:21:47.110951 15730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.617s	user 0.004s	sys 0.012s
I20260709 12:21:46.806702 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.113576 14422 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:46.817169 15637 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:46.814060 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 357) took 3047 ms. Trace:
I20260709 12:21:46.485963 15739 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:47.162160 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:46.673182 15846 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:21:47.168006 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.728s	user 0.009s	sys 0.017s
W20260709 12:21:46.819937 13650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.349s	user 0.007s	sys 0.021s
W20260709 12:21:47.168794 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.80626e+06us
I20260709 12:21:46.819388 15827 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:46.819939 15825 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:46.826082 15823 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.891s	user 0.000s	sys 0.009s
I20260709 12:21:46.131351 13648 rpcz_store.cc:276] 0709 12:21:43.821421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.851101 (+ 29680us) service_pool.cc:224] Handling call
0709 12:21:43.851446 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:43.851821 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:43.851849 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:43.852341 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.373293 (+1520952us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:45.373361 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:45.373469 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.109509 (+736040us) raft_consensus.cc:1685] finished
0709 12:21:46.109538 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.109701 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6646,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3166,"thread_start_us":1520487,"threads_started":1,"wal-append.queue_time_us":1796928}
I20260709 12:21:46.928117 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 321) took 1489 ms. Trace:
W20260709 12:21:46.950234 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.010s	sys 0.008s
W20260709 12:21:46.983851 13799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.011s	sys 0.009s
W20260709 12:21:46.984395 13796 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.692s	user 0.004s	sys 0.013s
W20260709 12:21:47.005622 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.017895 14416 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.723s	user 0.014s	sys 0.003s
W20260709 12:21:47.018895 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.024209 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:47.028537 13649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.142s	user 0.012s	sys 0.022s
I20260709 12:21:47.030428 14421 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:47.030682 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.174s	user 0.005s	sys 0.030s
I20260709 12:21:47.022006 15767 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:21:47.047745 13615 connection.cc:256] RPC call timeout handler was delayed by -1.2839823680001246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 12:21:47.067849 15850 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.083683 13720 rpcz_store.cc:276] 0709 12:21:45.437716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.476968 (+1039252us) service_pool.cc:224] Handling call
0709 12:21:46.477402 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.477628 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:46.477665 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:46.477722 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:46.477767 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.477840 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.477977 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:47.099901 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1565) took 2262 ms. Trace:
I20260709 12:21:47.100507 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2190) took 1432 ms. Trace:
W20260709 12:21:46.781291 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1523) took 1920 ms (client timeout 1963 ms). Trace:
I20260709 12:21:47.090585 13724 rpcz_store.cc:276] 0709 12:21:45.363233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.993874 (+630641us) service_pool.cc:224] Handling call
0709 12:21:46.713565 (+719691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:47.109936 15709 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:47.090837 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.119594 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:47.117259 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:47.081940 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:47.121261 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:47.139704 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.415s	user 0.013s	sys 0.017s
I20260709 12:21:47.105242 15779 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:47.145009 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 455) took 1655 ms. Trace:
I20260709 12:21:47.140324 14424 rpcz_store.cc:276] 0709 12:21:43.766835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.769262 (+  2427us) service_pool.cc:224] Handling call
0709 12:21:43.769707 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:43.770238 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:43.770293 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:43.797864 (+ 27571us) log.cc:844] Serialized 26 byte log entry
0709 12:21:45.360724 (+1562860us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:45.484516 (+123792us) log.cc:844] Serialized 14 byte log entry
0709 12:21:46.813673 (+1329157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.813765 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:46.813811 (+    46us) raft_consensus.cc:1685] finished
0709 12:21:46.813828 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.813991 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63468,"prepare.queue_time_us":130500,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":278760,"raft.run_cpu_time_us":560,"raft.run_wall_time_us":19620,"thread_start_us":2891143,"threads_started":2,"wal-append.queue_time_us":2118359,"wal-append.run_cpu_time_us":4728,"wal-append.run_wall_time_us":1158624}
I20260709 12:21:47.148257 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.156409 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1676) took 2706 ms. Trace:
I20260709 12:21:47.216559 13719 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:47.216915 13644 rpcz_store.cc:276] 0709 12:21:44.435215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.778432 (+1343217us) service_pool.cc:224] Handling call
0709 12:21:45.896259 (+117827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:46.102673 (+206414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:46.102712 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:46.103269 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:21:47.075771 (+972502us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:47.075886 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.121058 (+ 45172us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400114c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:47.121119 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:47.121218 (+    99us) raft_consensus.cc:1685] finished
0709 12:21:47.121232 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.138904 (+ 17672us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388663040,"thread_start_us":971755,"threads_started":1,"wal-append.queue_time_us":344359}
I20260709 12:21:47.218475 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1569) took 2363 ms. Trace:
I20260709 12:21:47.219223 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 322) took 1781 ms. Trace:
I20260709 12:21:47.159024 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.218997 13726 rpcz_store.cc:276] 0709 12:21:44.855412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.855535 (+   123us) service_pool.cc:224] Handling call
0709 12:21:44.871360 (+ 15825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:44.871656 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:21:45.613974 (+742318us) log.cc:844] Serialized 14 byte log entry
0709 12:21:47.176155 (+1562181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.198965 (+ 22810us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:47.203767 (+  4802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.210293 (+  6526us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.218414 (+  8121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":913399,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"thread_start_us":2283361,"threads_started":3,"wal-append.queue_time_us":787021}
I20260709 12:21:47.160559 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.187178 13718 rpcz_store.cc:276] 0709 12:21:45.438011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.923745 (+1485734us) service_pool.cc:224] Handling call
0709 12:21:46.926793 (+  3048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.927305 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:46.927361 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:46.927464 (+   103us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:21:46.927521 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:46.927627 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:46.927845 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:47.194303 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.195560 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.159608 15844 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.193922 13797 rpcz_store.cc:276] 0709 12:21:45.667710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.980316 (+312606us) service_pool.cc:224] Handling call
0709 12:21:47.100024 (+1119708us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54003c0354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:47.100240 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756832000}
W20260709 12:21:47.198908 13649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.313s	user 0.013s	sys 0.022s
I20260709 12:21:47.192757 13722 rpcz_store.cc:276] 0709 12:21:44.834835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.835095 (+   260us) service_pool.cc:224] Handling call
0709 12:21:44.835579 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:44.835993 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:44.836031 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:44.836627 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:21:47.096330 (+2259703us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:47.096403 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.096525 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:47.096589 (+    64us) raft_consensus.cc:1685] finished
0709 12:21:47.096599 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.096771 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":522044,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":120523,"thread_start_us":2259043,"threads_started":1,"wal-append.queue_time_us":1742049}
I20260709 12:21:47.173288 15827 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.219722 13719 rpcz_store.cc:276] 0709 12:21:45.438321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.217795 (+1779474us) service_pool.cc:224] Handling call
0709 12:21:47.218480 (+   685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.218851 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:47.218875 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.218945 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:47.218977 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.219041 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.219150 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:47.216737 13650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.760s	user 0.008s	sys 0.022s
I20260709 12:21:47.197160 13796 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:47.240049 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:47.222185 13721 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.918s	user 0.006s	sys 0.023s
I20260709 12:21:47.240701 15827 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.195060 14416 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:47.222888 13792 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:47.223978 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 325) took 1784 ms. Trace:
I20260709 12:21:47.245923 13718 rpcz_store.cc:276] 0709 12:21:45.439193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.222750 (+1783557us) service_pool.cc:224] Handling call
0709 12:21:47.223207 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.223602 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:47.223629 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.223679 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:47.223726 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.223779 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.223912 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:47.235204 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61725.2us
I20260709 12:21:47.222015 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 323) took 1783 ms. Trace:
I20260709 12:21:47.231251 13727 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 1.
I20260709 12:21:47.232718 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1675) took 2799 ms. Trace:
I20260709 12:21:47.249050 13726 rpcz_store.cc:276] 0709 12:21:45.438610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.220885 (+1782275us) service_pool.cc:224] Handling call
0709 12:21:47.221369 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.221653 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:47.221683 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.221755 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:47.221787 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.221845 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.221953 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:47.227201 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.214620 14417 rpcz_store.cc:276] 0709 12:21:45.483420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.494340 (+ 10920us) service_pool.cc:224] Handling call
0709 12:21:45.658369 (+164029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.109540 (+1451171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:47.109615 (+    75us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.109697 (+    82us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:47.109805 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.110213 (+   408us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.137402 (+ 27189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14563,"raft.queue_time_us":124360,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":139,"thread_start_us":1411252,"threads_started":1}
W20260709 12:21:47.259711 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1512) took 2413 ms (client timeout 1801 ms). Trace:
W20260709 12:21:47.260265 13796 rpcz_store.cc:269] 0709 12:21:44.846489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.846675 (+   186us) service_pool.cc:224] Handling call
0709 12:21:47.259295 (+2412620us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:47.259540 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3220268416}
W20260709 12:21:47.261341 13798 rpcz_store.cc:269] 0709 12:21:44.861051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.385118 (+524067us) service_pool.cc:224] Handling call
0709 12:21:46.780992 (+1395874us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:46.781201 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003991296}
I20260709 12:21:47.262535 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 457) took 1686 ms. Trace:
I20260709 12:21:47.229489 15844 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.262959 14416 rpcz_store.cc:276] 0709 12:21:45.575914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.613252 (+ 37338us) service_pool.cc:224] Handling call
0709 12:21:45.850524 (+237272us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:47.262407 (+1411883us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14791936}
I20260709 12:21:47.265209 15844 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.267807 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.268507 15867 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:47.290488 15825 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.291014 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.292290 15865 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:47.293210 15825 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.293879 15825 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.269064 15623 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:47.416646 15623 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.249898 13649 rpcz_store.cc:276] 0709 12:21:44.433535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.719632 (+1286097us) service_pool.cc:224] Handling call
0709 12:21:45.720020 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:45.720313 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:45.720351 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:45.766621 (+ 46270us) log.cc:844] Serialized 26 byte log entry
0709 12:21:47.232273 (+1465652us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:47.232328 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.232448 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:47.232499 (+    51us) raft_consensus.cc:1685] finished
0709 12:21:47.232512 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.232657 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1345624,"threads_started":1,"wal-append.queue_time_us":801565}
I20260709 12:21:47.280416 13795 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:47.280537 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53467.9us
I20260709 12:21:47.283023 13721 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:21:47.420563 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1576) took 2401 ms. Trace:
I20260709 12:21:47.284737 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1648) took 3822 ms. Trace:
I20260709 12:21:47.421298 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.421625 13650 rpcz_store.cc:276] 0709 12:21:43.462234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:43.666787 (+204553us) service_pool.cc:224] Handling call
0709 12:21:43.667201 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:43.667459 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:21:43.679552 (+ 12093us) log.cc:844] Serialized 14 byte log entry
0709 12:21:47.284069 (+3604517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.284198 (+   129us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:47.284302 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.284396 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.284640 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16854,"prepare.queue_time_us":16742,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":228782,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":10346,"thread_start_us":3555971,"threads_started":2,"wal-append.queue_time_us":1553305,"wal-append.run_cpu_time_us":3580,"wal-append.run_wall_time_us":1768433}
W20260709 12:21:47.423408 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.421003 13721 rpcz_store.cc:276] 0709 12:21:45.018989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.115803 (+1096814us) service_pool.cc:224] Handling call
0709 12:21:46.116190 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.420039 (+1303849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.420395 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:47.294245 15865 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.424990 15865 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.428210 15865 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.428959 15865 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.269744 15827 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.446460 15827 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.449782 15827 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.244632 15850 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.450842 15850 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.301190 13723 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.169s	user 0.016s	sys 0.003s
I20260709 12:21:47.454612 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
W20260709 12:21:47.307916 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.602s	user 0.015s	sys 0.008s
W20260709 12:21:47.311476 13799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.012s	sys 0.010s
W20260709 12:21:47.308663 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.452s	user 0.006s	sys 0.030s
I20260709 12:21:47.289853 15867 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.460807 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.80641e+06us
I20260709 12:21:47.461606 15867 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.324108 15863 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:47.462672 15863 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.326956 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1574) took 2403 ms. Trace:
I20260709 12:21:47.464056 13725 rpcz_store.cc:276] 0709 12:21:44.923727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:44.923967 (+   240us) service_pool.cc:224] Handling call
0709 12:21:45.374041 (+450074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:45.374372 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:21:47.111440 (+1737068us) log.cc:844] Serialized 14 byte log entry
0709 12:21:47.309673 (+198233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.311438 (+  1765us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:47.311535 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.316923 (+  5388us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.326857 (+  9934us) inbound_call.cc:177] Queueing success response
0709 12:21:47.379374 (+ 52517us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b1800131ec0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
Metrics: {"mutex_wait_us":5451,"prepare.queue_time_us":542582,"prepare.run_cpu_time_us":1412,"prepare.run_wall_time_us":5337,"raft.queue_time_us":125720,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":3719,"spinlock_wait_cycles":3913600,"thread_start_us":1818477,"threads_started":3,"wal-append.queue_time_us":208809}
W20260709 12:21:47.327764 13690 connection.cc:256] RPC call timeout handler was delayed by -1.2814816380000593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
I20260709 12:21:47.455395 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 326) took 2016 ms. Trace:
I20260709 12:21:47.466529 13723 rpcz_store.cc:276] 0709 12:21:45.439462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:45.977864 (+538402us) service_pool.cc:224] Handling call
0709 12:21:47.455289 (+1477425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:47.467430 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 327) took 1678 ms. Trace:
I20260709 12:21:47.468102 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:47.331564 15868 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.470731 15868 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.342679 15869 raft_consensus.cc:2764] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
W20260709 12:21:47.371626 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75295.7us
I20260709 12:21:47.472266 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:47.358740 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 145930us
I20260709 12:21:47.473951 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:47.388811 15739 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:47.389626 15872 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:21:47.408371 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 122595us
I20260709 12:21:47.284924 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.410452 15874 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:21:47.433243 15876 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:47.433233 13550 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:47.443887 15741 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:47.444468 15767 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.471114 15637 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:47.385010 15825 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.495982 15623 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.496498 15825 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.467943 13724 rpcz_store.cc:276] 0709 12:21:45.788931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.231712 (+1442781us) service_pool.cc:224] Handling call
0709 12:21:47.232229 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.460265 (+228036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:47.460299 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.460374 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:47.460429 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.467155 (+  6726us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:47.467203 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.467349 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523775232}
I20260709 12:21:47.454138 15850 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.477833 15876 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.482440 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.498700 15876 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:47.498824 15850 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.497223 15623 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:47.500644 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:47.500615 15623 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.501919 13796 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 3: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:47.484133 15874 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:47.502669 15874 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:47.502779 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:47.485322 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 200570us
I20260709 12:21:47.503223 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.503793 13798 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:47.504382 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.504743 15649 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:47.504938 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:47.484663 15741 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.474819 15739 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:47.486138 15869 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:47.508468 15730 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:47.473412 15868 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.446465 15871 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:47.509020 15739 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:47.488933 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.511317 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:47.511282 15825 raft_consensus.cc:2764] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:21:47.477799 15872 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.513005 15872 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:47.513185 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:47.513886 13800 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:47.501897 13792 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:21:47.515995 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.518663 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:47.481407 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:47.519578 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:47.520095 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49872.3us
I20260709 12:21:47.520287 13792 raft_consensus.cc:2376] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:21:47.491217 15637 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:47.503183 15874 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:47.521207 15730 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:21:47.523494 15730 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:47.508596 15869 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:47.524724 15869 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:47.525487 13801 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:47.524573 15637 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.528591 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:47.486663 15804 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:47.529352 13800 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:21:47.518788 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:47.530730 13798 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:47.524607 15730 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.534600 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2192) took 1866 ms. Trace:
I20260709 12:21:47.535120 13795 rpcz_store.cc:276] 0709 12:21:45.668243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.011138 (+342895us) service_pool.cc:224] Handling call
0709 12:21:47.279963 (+1268825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:47.419439 (+139476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.534307 (+114868us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001ca2d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:47.534502 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901281664}
I20260709 12:21:47.483130 15867 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:47.429760 15865 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.507879 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.538141 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:47.543785 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.544792 15868 raft_consensus.cc:2764] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:21:47.545696 15868 raft_consensus.cc:2764] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:21:47.546248 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:47.547163 15741 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.547749 15804 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:47.548594 15804 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:47.552750 15869 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.226609 15779 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.579169 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76175.2us
I20260709 12:21:47.538287 15867 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:47.588810 13550 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:47.554966 15767 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.591882 15867 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:21:47.610458 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 178814us
I20260709 12:21:47.549984 15804 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.613318 13797 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:47.614351 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61921.2us
I20260709 12:21:47.615384 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.623428 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.613863 15649 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:47.627316 15623 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.628422 13549 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 249) took 8730 ms. Trace:
I20260709 12:21:47.628021 15649 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.617272 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.619553 15865 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.640914 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:47.642016 15844 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.644907 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.645826 15844 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:47.651341 15867 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.653024 15867 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:47.653476 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.654455 15881 raft_consensus.cc:2764] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:21:47.625208 15863 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.651420 15869 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.656183 15779 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.655829 15739 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.654675 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 180855us
W20260709 12:21:47.664211 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52144.3us
I20260709 12:21:47.649576 15844 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:47.680872 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2197) took 1925 ms. Trace:
W20260709 12:21:47.691286 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75891.2us
I20260709 12:21:47.694769 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.690424 13550 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 2 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:21:47.698884 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.698446 13550 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 3 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:47.701442 15882 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:47.705236 15844 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.705545 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.702044 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.706521 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.707815 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:47.711369 15767 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.712076 15881 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:47.713491 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:47.717172 15876 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.717793 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.718533 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.681391 15825 raft_consensus.cc:2764] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:21:47.693075 15874 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.700466 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:47.742544 15890 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.062s	user 0.003s	sys 0.001s
I20260709 12:21:47.743464 15820 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:21:47.731709 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50520.3us
I20260709 12:21:47.691684 13799 rpcz_store.cc:276] 0709 12:21:45.755466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:46.091973 (+336507us) service_pool.cc:224] Handling call
0709 12:21:46.092432 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:46.092758 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:21:47.458815 (+1366057us) log.cc:844] Serialized 14 byte log entry
0709 12:21:47.666868 (+208053us) log.cc:844] Serialized 14 byte log entry
0709 12:21:47.676060 (+  9192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:47.680421 (+  4361us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:47.680515 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:47.680630 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:47.680804 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":577248,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":104,"raft.queue_time_us":61972,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":1538868,"threads_started":3,"wal-append.queue_time_us":165840}
I20260709 12:21:47.743983 15820 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.628860 13549 rpcz_store.cc:276] 0709 12:21:38.869173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:38.869460 (+   287us) service_pool.cc:224] Handling call
0709 12:21:39.722742 (+853282us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:21:47.597133 (+7874391us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:43.252775 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:43.496898 (+244123us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:43.500245 (+  3347us) write_op.cc:435] Acquired schema lock
0709 12:21:43.500266 (+    21us) tablet.cc:662] Decoding operations
0709 12:21:43.575403 (+ 75137us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:43.575502 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:43.575538 (+    36us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:21:43.575969 (+   431us) tablet.cc:701] Row locks acquired
0709 12:21:43.575984 (+    15us) write_op.cc:260] PREPARE: finished
0709 12:21:43.576181 (+   197us) write_op.cc:270] Start()
0709 12:21:43.576326 (+   145us) write_op.cc:276] Timestamp: P: 1783599703576120 usec, L: 0
0709 12:21:43.576348 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:21:43.612132 (+ 35784us) log.cc:844] Serialized 1308 byte log entry
0709 12:21:46.377106 (+2764974us) op_driver.cc:464] REPLICATION: finished
0709 12:21:46.624622 (+247516us) write_op.cc:301] APPLY: starting
0709 12:21:46.625246 (+   624us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:46.627430 (+  2184us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:46.629869 (+  2439us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:46.633025 (+  3156us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:46.633359 (+   334us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:21:46.633441 (+    82us) write_op.cc:312] APPLY: finished
0709 12:21:46.633772 (+   331us) log.cc:844] Serialized 35 byte log entry
0709 12:21:46.635010 (+  1238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:46.642001 (+  6991us) write_op.cc:454] Released schema lock
0709 12:21:46.646367 (+  4366us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:46.646870 (+   503us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:46.646903 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20194,"reported_tablets":11,"spinlock_wait_cycles":129095424,"child_traces":[["op",{"apply.queue_time_us":244265,"apply.run_cpu_time_us":3296,"apply.run_wall_time_us":24016,"num_ops":3,"prepare.queue_time_us":351819,"prepare.run_cpu_time_us":40260,"prepare.run_wall_time_us":2787932,"raft.queue_time_us":281939,"raft.run_cpu_time_us":175223,"raft.run_wall_time_us":222382,"replication_time_us":2800540,"thread_start_us":3021972,"threads_started":4,"wal-append.queue_time_us":2028784,"wal-append.run_cpu_time_us":7201,"wal-append.run_wall_time_us":1994061}]]}
W20260709 12:21:47.748503 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57562.8us
I20260709 12:21:47.730912 13797 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:47.749850 15820 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:47.753077 15779 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:47.753523 14416 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:47.751608 15863 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:47.754875 15863 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:47.756141 15863 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.749855 15865 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:21:47.739744 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51057.5us
I20260709 12:21:47.761301 15874 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:47.762280 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49058.6us
I20260709 12:21:47.763659 15877 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:47.764284 15877 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:47.764330 15779 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:47.764818 15877 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:47.765304 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.739159 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57481.9us
I20260709 12:21:47.760330 15876 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.761382 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:47.783857 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 91091.1us
W20260709 12:21:47.784380 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 160037us
I20260709 12:21:47.786604 15779 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.790095 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82552.7us
I20260709 12:21:47.793315 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.752211 15637 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:47.800217 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.806561 15853 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:47.820132 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:47.819904 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.820706 13725 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:47.822956 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:47.807567 15869 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:47.802290 15739 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:47.826905 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.829165 15623 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:47.829630 15623 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:47.833077 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:47.814793 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.848475 15825 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:47.849538 15825 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:47.850893 15623 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.853224 15623 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:47.854671 15869 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:47.856034 15869 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.856536 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.857656 15883 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:47.827370 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:47.858289 15883 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:47.859397 15883 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.860775 14420 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:21:47.861654 15885 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:47.864610 15820 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:47.866379 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:47.866845 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.868202 15872 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:47.871083 15871 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:47.872181 15871 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.875018 15871 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:47.874462 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.821723 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:47.882440 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46263.1us
I20260709 12:21:47.884119 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:21:47.890954 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.894191 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:47.894903 13799 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:47.901829 15825 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:47.914361 13725 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:47.893267 15853 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:47.927492 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:47.898522 15739 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:47.930063 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:47.930603 15820 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:47.950109 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83832.4us
I20260709 12:21:47.933698 14417 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:47.953307 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:47.939297 15739 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:47.970441 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:47.973863 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 208808us
I20260709 12:21:47.984223 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:47.984858 13719 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:21:47.990655 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 164575us
I20260709 12:21:47.983800 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:47.994354 13726 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:48.016978 13720 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:48.017530 14418 pending_rounds.cc:85] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 4
I20260709 12:21:47.959326 15863 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:47.967370 15709 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:48.026340 14418 pending_rounds.cc:107] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 12:21:48.026160 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55834.3us
I20260709 12:21:47.985984 15739 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:47.951903 15871 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:48.055891 14418 raft_consensus.cc:2889] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:48.059612 15709 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:48.062743 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:48.064651 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134704us
I20260709 12:21:48.058598 13719 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
W20260709 12:21:48.069527 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.073637 15898 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:48.076103 14417 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:48.078404 15871 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:48.078701 15872 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:48.079952 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:48.080655 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98607.6us
I20260709 12:21:48.061409 15844 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:21:48.103127 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:48.067809 15709 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:48.107795 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:48.108649 13649 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:48.111515 13648 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:48.111112 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:48.112861 15730 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:48.113595 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.112996 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.127957 13720 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
W20260709 12:21:48.151248 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85716us
I20260709 12:21:48.154129 13799 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:48.157012 13650 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 4. Ops: []
I20260709 12:21:48.112855 15871 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:48.160297 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80449.6us
I20260709 12:21:48.125293 15867 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.163093 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:48.163364 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:48.163997 13799 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:48.164136 13798 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:48.130656 15649 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:48.159881 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.159793 15623 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:21:48.160646 15863 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:21:48.169165 15863 consensus_queue.cc:1243] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:21:48.171022 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.169727 15863 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:48.192024 13648 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:48.145970 15877 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:48.214228 15877 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:48.171806 15767 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:48.215512 15767 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:48.215377 15898 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:48.216931 15767 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:48.217763 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.218497 15905 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:48.218807 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
W20260709 12:21:48.220124 15897 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:21:48.232237 13643 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:48.234009 13645 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:48.236431 13646 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:21:48.236749 13645 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:48.239079 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:48.174541 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.178560 15867 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:48.192822 15863 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:48.301826 13799 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:48.260484 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:48.262614 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 151894us
W20260709 12:21:48.265081 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:48.302043 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.192029 15649 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:48.304713 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.306771 13648 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:48.395845 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 201454us
I20260709 12:21:48.397606 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:48.395030 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:48.425051 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:48.299650 15876 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:48.278478 15730 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:48.249642 13644 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:48.251356 15871 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.184s
W20260709 12:21:48.420657 14391 leader_election.cc:336] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.616s (SENT)
W20260709 12:21:48.490536 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.450788 15876 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:48.450951 15869 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:48.519631 13693 connection.cc:256] RPC call timeout handler was delayed by -3.7838735959999212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
W20260709 12:21:48.577179 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 181278us
W20260709 12:21:48.578153 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 174668us
I20260709 12:21:48.580247 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.552588 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.502691 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:48.583309 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:48.647338 13693 connection.cc:256] RPC call timeout handler was delayed by -3.7424735959998543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22719999999999999s for a response
W20260709 12:21:48.669780 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:48.666671 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.602s (SENT) [suppressed 19 similar messages]
I20260709 12:21:48.727828 13647 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:48.729722 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.730720 15730 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:48.687888 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:48.734205 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.663s (SENT)
I20260709 12:21:48.734769 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:48.734863 13650 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
W20260709 12:21:48.741559 13649 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.007s
I20260709 12:21:48.741773 13648 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:48.667363 15764 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:48.760483 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.759846 15900 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:48.147553 15853 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:48.749861 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:48.786031 15879 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:48.804041 15909 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.355s	user 0.000s	sys 0.000s
W20260709 12:21:48.805471 15908 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
W20260709 12:21:48.806572 15911 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
W20260709 12:21:48.811480 15912 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.568s	user 0.000s	sys 0.001s
I20260709 12:21:48.812603 15876 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:48.837329 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:48.872213 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82937.4us
W20260709 12:21:48.760601 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6060 after lost signal to thread 12472
I20260709 12:21:48.571200 15863 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:48.807916 15920 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
W20260709 12:21:48.916514 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 445882us
I20260709 12:21:48.897890 13649 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:48.946607 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6740 after lost signal to thread 13584
I20260709 12:21:48.918038 13650 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:48.917801 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:48.929687 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:48.931857 13798 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.008s
I20260709 12:21:48.742338 15649 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:48.938289 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173335us
I20260709 12:21:48.792532 15869 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:48.971863 15739 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.899s	user 0.015s	sys 0.000s
W20260709 12:21:48.947342 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6980 after lost signal to thread 13618
I20260709 12:21:48.972060 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:48.972765 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 238633us
W20260709 12:21:48.949908 15926 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:21:48.938334 15879 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:48.975404 15881 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:48.976186 15928 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:48.976837 15764 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:48.977245 15936 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:48.977372 15900 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:48.979064 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:48.968590 15916 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:48.979333 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:48.979813 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:48.980007 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.983758 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:48.987232 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:48.914942 15867 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:48.991477 13798 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
W20260709 12:21:49.010352 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08003b26c0 after lost signal to thread 13676
W20260709 12:21:49.010742 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2380 after lost signal to thread 13677
I20260709 12:21:48.994040 13648 raft_consensus.cc:2410] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:49.002893 15881 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:48.996227 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.011045 15764 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:49.012168 13652 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:21:49.011960 15881 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:48.999706 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 333070us
I20260709 12:21:49.009943 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28452187c3924de9940235e9b965477d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:49.013820 15881 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:49.013861 13650 raft_consensus.cc:606] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:49.017454 13617 connection.cc:256] RPC call timeout handler was delayed by -4.0335600969997358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 12:21:49.019418 13617 connection.cc:256] RPC call timeout handler was delayed by -3.5957625830001234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:21:49.031819 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52888.2us
I20260709 12:21:49.034881 13723 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:49.038863 13643 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.009549 15863 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:49.015404 15916 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:49.001566 15739 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:49.046052 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.049467 15916 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.052624 15917 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:49.053220 15900 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:49.055430 15637 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:49.010206 15885 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:21:49.058188 15879 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.059548 15939 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:49.090639 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.092154 13649 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:49.093276 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:49.014799 15936 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:49.115789 15936 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:49.064254 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47008.9us
W20260709 12:21:49.070838 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:49.071677 14446 debug-util.cc:398] Leaking SignalData structure 0x7b0800197e20 after lost signal to thread 13751
W20260709 12:21:49.117321 14446 debug-util.cc:398] Leaking SignalData structure 0x7b080028ad40 after lost signal to thread 13752
W20260709 12:21:49.118702 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6e20 after lost signal to thread 13825
W20260709 12:21:49.120141 14446 debug-util.cc:398] Leaking SignalData structure 0x7b080039f9e0 after lost signal to thread 14449
I20260709 12:21:49.073374 15739 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:49.052644 15928 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:21:49.047549 15741 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 12:21:49.061689 13693 connection.cc:256] RPC call timeout handler was delayed by -3.0143735960000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30810000000000004s for a response
W20260709 12:21:49.124387 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8865735959998347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32230000000000003s for a response
W20260709 12:21:49.124733 13693 connection.cc:256] RPC call timeout handler was delayed by -2.7659735960000944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33570000000000005s for a response
I20260709 12:21:49.124788 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:49.105167 15865 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:21:49.120273 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:49.127334 15623 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.130627 15623 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.131284 15869 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.132383 13645 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:21:49.133134 15623 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.133744 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.135376 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.137321 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:21:49.142473 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1695371880000494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 12:21:49.143095 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0813371879999067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 12:21:49.143482 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0471371879998514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
I20260709 12:21:49.142550 13643 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:49.125450 13651 raft_consensus.cc:2376] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:21:49.148489 13649 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5.
W20260709 12:21:49.090538 15913 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.829s	user 0.003s	sys 0.015s
I20260709 12:21:49.151678 13646 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:49.152505 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.153447 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.154157 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.154521 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.147734 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.171797 13651 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:21:49.164204 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:49.173295 13725 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:49.120820 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:49.166640 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:49.184181 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.504s (SENT)
I20260709 12:21:49.167555 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:49.120464 14446 debug-util.cc:398] Leaking SignalData structure 0x7b080039f8c0 after lost signal to thread 14450
I20260709 12:21:49.179685 14418 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:21:49.127065 15850 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:21:49.186926 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.128611 15865 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:49.191864 13798 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:49.194103 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:49.194423 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74210.9us
I20260709 12:21:49.157433 15869 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:49.196148 13720 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:49.200217 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.202880 13727 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:21:49.202831 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 147842us
I20260709 12:21:49.203688 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:49.204540 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:49.205761 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173119us
W20260709 12:21:49.208374 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.563s (SENT)
I20260709 12:21:49.208817 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:49.190207 15850 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:49.200327 13721 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4.
I20260709 12:21:49.211280 13644 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5.
I20260709 12:21:49.200189 15739 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:49.218408 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.220641 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.222142 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:49.230010 13722 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:49.230365 13798 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:21:49.219290 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08002cd0a0 after lost signal to thread 15806
I20260709 12:21:49.214810 15850 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:49.241969 15649 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:49.240007 15739 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:49.242573 15649 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:49.243070 15649 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.243614 15739 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.245262 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.216456 15885 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:21:49.246748 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.247460 13797 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:49.246901 15885 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:21:49.250726 13598 auto_rebalancer.cc:629] Failed to schedule move for tablet 749228d3459a444a9382bc38bbfcbbee: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:21:49.251550 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:49.252146 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.254012 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.247303 15865 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.255033 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45dc2f34593649778ec4b80d29638572"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:21:49.238988 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6440 after lost signal to thread 15807
I20260709 12:21:49.257433 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:49.257205 15938 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:21:49.258175 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.259053 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.261171 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:49.261739 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.275532 13798 raft_consensus.cc:606] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:49.287856 13798 raft_consensus.cc:614] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:49.256489 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:49.282836 14391 leader_election.cc:336] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.853s (SENT)
I20260709 12:21:49.284721 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:49.289827 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:49.283838 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:49.212872 15869 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.319386 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:49.321851 13796 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:49.274293 15623 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:49.293550 14446 debug-util.cc:398] Leaking SignalData structure 0x7b0800381f80 after lost signal to thread 15814
I20260709 12:21:49.343686 13724 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:21:49.335315 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.266558 15885 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:49.294651 15779 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.345148 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:49.345752 14446 debug-util.cc:398] Leaking SignalData structure 0x7b0800382120 after lost signal to thread 15818
I20260709 12:21:49.399348 15941 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.399906 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 470) took 1156 ms. Trace:
I20260709 12:21:49.400121 15941 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.402381 15942 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:49.345078 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.403498 15948 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.403913 13721 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:49.404229 15948 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.342413 15804 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.356895 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:49.373651 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124268us
I20260709 12:21:49.395769 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:49.407073 13648 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:49.326645 15877 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.403018 15941 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.405001 15871 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:49.420425 13795 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:21:49.422849 13796 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:49.359735 13794 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:49.330713 15881 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:49.282788 15767 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 12:21:49.405786 15804 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.425622 15881 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:49.426211 15949 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:21:49.426324 15918 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:49.427562 15804 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:49.415447 15948 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:49.412281 14446 debug-util.cc:398] Leaking SignalData structure 0x7b0800354380 after lost signal to thread 15819
I20260709 12:21:49.411916 15865 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:21:49.426136 15881 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:49.482621 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.084s (SENT)
I20260709 12:21:49.357239 15779 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:49.358088 15623 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:49.502800 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:49.520622 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.535s (SENT)
I20260709 12:21:49.522604 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:49.530424 13648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
W20260709 12:21:49.477133 14446 debug-util.cc:398] Leaking SignalData structure 0x7b0800354300 after lost signal to thread 15834
I20260709 12:21:49.531360 13648 raft_consensus.cc:606] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:49.535385 13648 raft_consensus.cc:614] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:49.358369 15916 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.473269 14420 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:21:49.425329 15877 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.549084 13726 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:49.544894 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.558104 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.558919 13725 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:49.438835 15871 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:49.537770 14446 debug-util.cc:398] Leaking SignalData structure 0x7b080023b9c0 after lost signal to thread 15835
W20260709 12:21:49.569535 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 285808us
W20260709 12:21:49.581575 13598 auto_rebalancer.cc:629] Failed to schedule move for tablet 4613215991134c66b270c6024bec9d73: Illegal state: Replica b4f6bfcda63148e4965ca02c639336f1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:21:49.401032 14417 rpcz_store.cc:276] 0709 12:21:48.243334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:48.243512 (+   178us) service_pool.cc:224] Handling call
0709 12:21:48.444444 (+200932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:48.470145 (+ 25701us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:21:49.193622 (+723477us) log.cc:844] Serialized 14 byte log entry
0709 12:21:49.292523 (+ 98901us) log.cc:844] Serialized 14 byte log entry
0709 12:21:49.350418 (+ 57895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:49.356299 (+  5881us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:49.359043 (+  2744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:49.397719 (+ 38676us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:49.399750 (+  2031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18977,"prepare.queue_time_us":766356,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":36434,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3485,"spinlock_wait_cycles":21629312,"thread_start_us":265932,"threads_started":2,"wal-append.queue_time_us":59817}
W20260709 12:21:49.588038 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:49.590773 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 152066us
I20260709 12:21:49.568892 15941 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:49.605927 15954 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:21:49.606431 15874 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:49.608043 15956 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:21:49.611830 15876 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:49.556370 15825 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:49.601274 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.642023 15869 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.584452 13721 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 1. Current term is 2. Ops: []
W20260709 12:21:49.650823 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48085.9us
I20260709 12:21:49.647225 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.664855 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:49.666514 13726 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:49.668733 15941 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.674629 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.674927 15871 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:49.560611 15885 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:49.675454 13719 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:49.675485 15941 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.684469 15779 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:49.690771 14420 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:49.694041 15623 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:49.647372 15739 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 12:21:49.714614 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:49.704838 15863 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.656s	user 0.006s	sys 0.006s
I20260709 12:21:49.757347 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:49.758246 13721 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:49.659020 15825 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:49.762632 15850 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:49.764292 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.767071 15964 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:49.768296 15964 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:49.768707 15964 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:49.810422 13652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:49.811323 13652 raft_consensus.cc:606] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:49.811805 13652 raft_consensus.cc:614] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:21:49.817754 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.820143 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:49.822503 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:49.773932 14418 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.055s	sys 0.100s
W20260709 12:21:49.883193 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:49.904083 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:50.377360 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:50.376564 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 617423us
W20260709 12:21:50.244588 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:49.909236 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 429602us
I20260709 12:21:50.108214 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.390795 13720 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:50.391677 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:50.282223 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:50.377401 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 557385us
I20260709 12:21:50.392637 13725 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
W20260709 12:21:49.963497 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:49.953811 14422 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:49.973483 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:21:50.394556 13798 raft_consensus.cc:606] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:50.397400 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.398515 13727 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:21:50.086969 13550 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 228) took 7513 ms. Trace:
W20260709 12:21:50.399500 13767 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.018s	sys 0.050s
W20260709 12:21:49.886957 14418 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.055s	sys 0.100s
W20260709 12:21:50.400000 13767 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.018s	sys 0.050s
I20260709 12:21:49.938616 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:50.408830 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 752740us
I20260709 12:21:50.411342 13722 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:49.962754 15951 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.537s	user 0.003s	sys 0.000s
W20260709 12:21:49.919893 15947 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.494s	user 0.003s	sys 0.000s
I20260709 12:21:50.407954 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:50.086364 15881 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.000s
I20260709 12:21:50.400019 15885 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:50.389137 15853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.010s
W20260709 12:21:49.972856 15649 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.001s
I20260709 12:21:49.886663 15623 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:50.413988 13643 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:21:49.969959 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.096823 13545 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.405397 15894 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.432644 13651 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:21:50.435390 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.399680 15850 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:50.439265 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.399791 15877 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:50.397143 15825 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:50.440847 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:49.908659 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:50.441020 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.473376 13727 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:50.438163 13724 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:50.474802 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:50.437780 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:50.409557 15928 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.431577 15881 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.445822 15623 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.446488 13693 leader_election.cc:400] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:21:50.481959 13693 leader_election.cc:403] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:21:50.447456 15916 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:50.433004 15853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.000s	sys 0.013s
I20260709 12:21:50.450701 13645 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:50.451108 15779 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:50.452075 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 355) took 1302 ms. Trace:
I20260709 12:21:50.452250 13646 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:21:50.453588 13545 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:50.434710 15891 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:21:50.456647 13720 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:50.493503 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.456437 13647 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:21:50.458119 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:21:50.417212 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:50.496827 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50562.3us
I20260709 12:21:50.441128 15649 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.466810 15894 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.496445 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:50.472165 13725 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:50.499380 13652 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:21:49.888298 15964 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.439965 13718 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:21:50.484745 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:50.486428 13723 rpcz_store.cc:276] 0709 12:21:49.149843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.149987 (+   144us) service_pool.cc:224] Handling call
0709 12:21:49.154967 (+  4980us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:49.155419 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:49.155452 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:49.156006 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 12:21:49.201241 (+ 45235us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:49.201293 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:49.201379 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:49.961615 (+760236us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:21:50.451778 (+490163us) raft_consensus.cc:1685] finished
0709 12:21:50.451801 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:50.452013 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8225,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":19393,"spinlock_wait_cycles":11942272,"thread_start_us":44818,"threads_started":1,"wal-append.queue_time_us":168473}
W20260709 12:21:50.490839 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:50.490660 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:50.399531 13550 rpcz_store.cc:276] 0709 12:21:42.573103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:42.578613 (+  5510us) service_pool.cc:224] Handling call
0709 12:21:50.086851 (+7508238us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:47.906553 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:47.948742 (+ 42189us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:47.955353 (+  6611us) write_op.cc:435] Acquired schema lock
0709 12:21:47.955373 (+    20us) tablet.cc:662] Decoding operations
0709 12:21:48.250569 (+295196us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:48.393577 (+143008us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:48.403927 (+ 10350us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:21:48.734427 (+330500us) tablet.cc:701] Row locks acquired
0709 12:21:48.734494 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:21:48.734749 (+   255us) write_op.cc:270] Start()
0709 12:21:48.734971 (+   222us) write_op.cc:276] Timestamp: P: 1783599708734646 usec, L: 0
0709 12:21:48.735001 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:21:48.756077 (+ 21076us) log.cc:844] Serialized 1684 byte log entry
0709 12:21:48.982988 (+226911us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b
0709 12:21:49.325935 (+342947us) op_driver.cc:464] REPLICATION: finished
0709 12:21:49.614437 (+288502us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b18002be400. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:49.614654 (+   217us) write_op.cc:301] APPLY: starting
0709 12:21:49.614764 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:49.614941 (+   177us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:49.614975 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:49.634877 (+ 19902us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:49.704820 (+ 69943us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:21:49.704869 (+    49us) write_op.cc:312] APPLY: finished
0709 12:21:49.756679 (+ 51810us) log.cc:844] Serialized 41 byte log entry
0709 12:21:49.756917 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:49.757044 (+   127us) write_op.cc:454] Released schema lock
0709 12:21:49.757113 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:49.757638 (+   525us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:49.757669 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":202501,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":194317,"apply.run_cpu_time_us":4117,"apply.run_wall_time_us":142955,"num_ops":4,"prepare.queue_time_us":193287,"prepare.run_cpu_time_us":47265,"prepare.run_wall_time_us":1284331,"raft.queue_time_us":70359,"raft.run_cpu_time_us":20393,"raft.run_wall_time_us":153436,"replication_time_us":590727,"spinlock_wait_cycles":51690112,"thread_start_us":516455,"threads_started":4,"wal-append.queue_time_us":422711}]]}
I20260709 12:21:50.490051 13545 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 5, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 5 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.502475 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.502936 15649 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:50.500522 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:50.503784 13793 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:50.502224 13545 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:50.503378 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.506665 13721 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 3: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:50.498741 15825 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:21:50.507365 15894 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:50.508138 13718 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:21:50.508824 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:50.483286 15900 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:50.510556 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.487911 15739 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.521250 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.481591 15623 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.515220 14421 raft_consensus.cc:2410] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:50.514801 15891 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.497632 15779 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:50.531422 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:50.534327 13720 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:50.534734 13720 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:50.515093 15649 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:50.484135 15928 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.539538 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:50.538796 13727 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:21:50.540676 15649 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.531244 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:50.498445 15885 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:50.541121 13724 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:21:50.542068 14419 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:50.516532 15900 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:50.551374 15900 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:50.551868 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:50.552208 15928 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:50.552752 15894 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:50.553263 15894 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.553781 15649 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:50.553890 15894 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:50.520372 15881 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:50.504019 14420 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:50.544373 13725 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:50.544589 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 473) took 1290 ms. Trace:
I20260709 12:21:50.556341 13652 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 1.
I20260709 12:21:50.533397 15844 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:50.534662 15877 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:50.546754 13721 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:50.561779 13720 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.546767 13646 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:21:50.567458 13652 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:50.569896 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:50.572291 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:50.538582 15891 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.575855 13725 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:50.542833 15739 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.576736 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:50.582085 15974 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:21:50.582868 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:50.585868 15941 raft_consensus.cc:2764] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:21:50.587303 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:50.588722 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:50.589658 15863 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:21:50.590076 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:50.561290 15885 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.578086 15881 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:50.596588 15881 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.595273 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:50.597646 15779 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:50.603711 15969 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:21:50.604458 15971 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:21:50.569317 15877 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.604888 13720 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:21:50.614202 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.607951 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:50.634881 13722 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:50.695290 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2116) took 1006 ms. Trace:
I20260709 12:21:50.695760 13722 rpcz_store.cc:276] 0709 12:21:49.689258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.938139 (+248881us) service_pool.cc:224] Handling call
0709 12:21:50.695197 (+757058us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:50.696395 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:50.697499 13727 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:50.700639 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 283890us
I20260709 12:21:50.702488 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.608765 15739 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:50.709504 13724 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:21:50.715416 13727 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:50.731340 13718 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:50.716472 15877 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:50.738471 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 267358us
I20260709 12:21:50.634552 15881 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:50.725771 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:50.737538 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.675s (SENT)
I20260709 12:21:50.606534 15891 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:50.607230 15623 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:50.755250 13720 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:21:50.796500 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 381016us
W20260709 12:21:50.799337 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 331235us
I20260709 12:21:50.693274 15850 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.792104 15863 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:50.805791 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:50.830135 15930 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:50.831490 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.832554 13798 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4.
W20260709 12:21:50.835088 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 132121us
W20260709 12:21:50.837692 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:50.842686 15779 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:21:50.846343 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.697s (SENT)
I20260709 12:21:50.829890 15850 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.851125 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 375) took 1228 ms. Trace:
I20260709 12:21:50.841378 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.864945 13801 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:50.805702 15968 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:50.806299 15739 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:50.874614 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.875311 13801 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:50.876791 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:50.855815 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50142.5us
I20260709 12:21:50.863360 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:50.869905 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:50.870383 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.871104 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.868939 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:50.880652 13718 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
W20260709 12:21:50.882519 15990 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:21:50.882831 13796 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:50.885586 15850 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:50.886718 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:50.882776 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 292796us
I20260709 12:21:50.891120 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.846276 15917 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:50.856837 15939 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:50.916736 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:50.919283 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.903398 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:50.922353 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:50.917248 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:50.925499 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.923230 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:50.933980 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.941864 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.942909 13797 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:50.918838 15917 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.922888 13721 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:50.855119 15980 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 12:21:50.954493 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129433us
W20260709 12:21:50.962308 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62612.7us
I20260709 12:21:50.950675 15941 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:50.968324 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:50.939766 15779 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:50.893847 15739 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.955610 15649 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:50.950361 15850 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:50.915059 15804 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:21:50.969228 13801 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:21:50.971335 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135503us
W20260709 12:21:50.984439 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:50.950206 15825 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:50.954800 15985 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:50.558112 14418 rpcz_store.cc:276] 0709 12:21:49.252398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.252643 (+   245us) service_pool.cc:224] Handling call
0709 12:21:49.253173 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:49.253424 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:21:50.422804 (+1169380us) log.cc:844] Serialized 14 byte log entry
0709 12:21:50.510816 (+ 88012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:50.519624 (+  8808us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:50.524776 (+  5152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:50.532135 (+  7359us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:50.542511 (+ 10376us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9854,"prepare.queue_time_us":10152,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":4772,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8050,"spinlock_wait_cycles":29984512,"thread_start_us":1203576,"threads_started":2,"wal-append.queue_time_us":111615}
I20260709 12:21:50.958281 15917 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:50.967607 15994 raft_consensus.cc:2764] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:21:50.941135 15939 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:50.971697 15885 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:50.888466 15968 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.971251 15980 consensus_queue.cc:1243] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:21:51.008594 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.752s (SENT)
I20260709 12:21:51.010228 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:50.957708 15900 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:50.974275 15863 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:51.038431 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87716.8us
I20260709 12:21:51.004619 15985 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:50.983680 15850 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.104519 13651 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:51.144773 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 175075us
W20260709 12:21:51.145215 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 190233us
I20260709 12:21:50.864300 14417 rpcz_store.cc:276] 0709 12:21:49.617419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.666209 (+ 48790us) service_pool.cc:224] Handling call
0709 12:21:49.677734 (+ 11525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:49.678279 (+   545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:49.678313 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:49.678931 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:21:49.865639 (+186708us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:49.889943 (+ 24304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:49.924685 (+ 34742us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:50.825372 (+900687us) raft_consensus.cc:1685] finished
0709 12:21:50.825397 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:50.846167 (+ 20770us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":830,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":143278,"threads_started":1,"wal-append.queue_time_us":752816}
I20260709 12:21:51.133042 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.022634 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.166172 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:51.182154 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.184362 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:50.999964 15891 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.188858 13652 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:51.005648 15649 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:51.190155 13647 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:21:51.194496 13643 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:21:51.198388 13643 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 2
I20260709 12:21:51.198768 13643 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:21:51.199152 13643 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:51.166985 15985 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.183943 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.192745 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.009114 15877 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.182s
I20260709 12:21:51.184958 15850 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.184095 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.184865 15739 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:51.202862 13651 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:21:51.207454 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.203872 15980 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:51.208410 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.165582 15999 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.241590 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:51.167606 15983 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:21:51.169978 15916 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
I20260709 12:21:51.184274 15804 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:21:51.246596 13652 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
W20260709 12:21:51.249821 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 64384.3us
I20260709 12:21:51.258464 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:51.252641 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57122.9us
I20260709 12:21:51.253782 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:51.260943 13651 raft_consensus.cc:2376] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4. Re-sending same reply.
I20260709 12:21:51.254107 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.263299 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.264721 13801 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:51.261981 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.266829 13651 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:51.267676 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:51.268493 15894 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.714s	user 0.015s	sys 0.000s
I20260709 12:21:51.268826 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.268952 15894 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.270701 13801 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:51.271842 13798 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 4. Ops: []
I20260709 12:21:51.267710 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2110) took 1580 ms. Trace:
I20260709 12:21:51.249958 14422 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:51.272682 13719 rpcz_store.cc:276] 0709 12:21:49.687360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.687506 (+   146us) service_pool.cc:224] Handling call
0709 12:21:50.409356 (+721850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:50.437439 (+ 28083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:50.437480 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:50.469546 (+ 32066us) log.cc:844] Serialized 26 byte log entry
0709 12:21:50.754233 (+284687us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:50.827923 (+ 73690us) log.cc:844] Serialized 14 byte log entry
0709 12:21:50.924816 (+ 96893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:50.924899 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:51.267479 (+342580us) raft_consensus.cc:1685] finished
0709 12:21:51.267500 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:51.267654 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32451,"prepare.queue_time_us":71945,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":6157,"raft.queue_time_us":162256,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":43880,"thread_start_us":356932,"threads_started":2,"wal-append.queue_time_us":573548}
I20260709 12:21:51.274173 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:51.284273 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 299018us
W20260709 12:21:51.287405 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4820720810002967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 12:21:51.271955 15951 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:21:51.271505 15894 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:51.293918 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.812s (SENT)
I20260709 12:21:51.294421 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.204882 15918 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 12:21:51.291747 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:51.289386 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4685720810002749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
I20260709 12:21:51.248629 13646 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:51.200840 16003 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 12:21:51.200002 15891 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.300172 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:21:51.300501 16006 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:21:51.301054 15987 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:21:51.301385 15984 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.333767 15941 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:51.333568 15984 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.334559 15941 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:51.335819 15976 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.791s	user 0.007s	sys 0.000s
I20260709 12:21:51.336800 15984 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:51.343490 15941 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:51.343935 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.344228 16013 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.344751 16013 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:51.298110 15998 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
W20260709 12:21:51.299468 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2786720810004226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 12:21:51.308054 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.313764 15980 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:51.314217 16001 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:21:51.314121 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.317428 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:51.318732 16016 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:21:51.319759 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.320911 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:51.367091 13650 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:51.367743 13792 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:51.367921 15804 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:51.368332 15804 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.368685 15804 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:51.322806 15995 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
W20260709 12:21:51.323359 15997 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:21:51.324388 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.324923 15968 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:51.358093 15844 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:21:51.257602 15999 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:51.325615 14416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 465) took 3533 ms (client timeout 3356 ms). Trace:
W20260709 12:21:51.324946 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.326519 15939 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:21:51.330017 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.330052 14422 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:51.354707 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2786720810004226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
I20260709 12:21:51.357908 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:51.363055 13801 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:21:51.364831 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:51.361562 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44236.3us
I20260709 12:21:51.411098 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:51.375957 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90286.4us
I20260709 12:21:51.376298 13792 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:51.378887 13799 raft_consensus.cc:2368] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4.
I20260709 12:21:51.391600 15964 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.360852 16019 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:51.418623 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:51.408042 15881 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.655s	user 0.014s	sys 0.001s
I20260709 12:21:51.406968 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2221) took 2141 ms. Trace:
W20260709 12:21:51.407469 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2273720810003397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 12:21:51.408150 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50402.4us
I20260709 12:21:51.405764 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2112) took 1717 ms. Trace:
W20260709 12:21:51.427831 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2210720810003295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
W20260709 12:21:51.428386 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2138720810003178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:21:51.428807 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2138720810003178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:21:51.429246 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1733720810002524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
W20260709 12:21:51.430243 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1733720810002524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
W20260709 12:21:51.430655 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1373720810001942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
W20260709 12:21:51.431135 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1373720810001942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
W20260709 12:21:51.431608 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1319720810001854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 12:21:51.432068 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1319720810001854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 12:21:51.432349 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0446720810000443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
I20260709 12:21:51.329514 15649 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.422327 15983 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:21:51.427749 13793 rpcz_store.cc:276] 0709 12:21:49.259754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.259873 (+   119us) service_pool.cc:224] Handling call
0709 12:21:49.260180 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:50.510411 (+1250231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:51.400704 (+890293us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003d54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:51.400939 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859532672}
I20260709 12:21:51.422936 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:51.421090 15894 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:51.412006 15984 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:51.430990 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:51.428299 13726 rpcz_store.cc:276] 0709 12:21:49.688044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:49.770667 (+ 82623us) service_pool.cc:224] Handling call
0709 12:21:50.376966 (+606299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:50.432108 (+ 55142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:50.432208 (+   100us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:50.444537 (+ 12329us) log.cc:844] Serialized 26 byte log entry
0709 12:21:50.732287 (+287750us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b18003d2f40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:21:50.906578 (+174291us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:50.907029 (+   451us) log.cc:844] Serialized 14 byte log entry
0709 12:21:50.948118 (+ 41089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:50.948247 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:51.401674 (+453427us) raft_consensus.cc:1685] finished
0709 12:21:51.405489 (+  3815us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:51.405679 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17426,"prepare.queue_time_us":63965,"prepare.run_wall_time_us":8308,"raft.queue_time_us":95055,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":28223,"spinlock_wait_cycles":47711360,"thread_start_us":197014,"threads_started":2,"wal-append.queue_time_us":506941}
I20260709 12:21:51.426779 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:51.403348 14416 rpcz_store.cc:269] 0709 12:21:47.792143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:47.795795 (+  3652us) service_pool.cc:224] Handling call
0709 12:21:51.325368 (+3529573us) spinlock_profiling.cc:243] Waited 2.62 s on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:51.325518 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6037050624}
I20260709 12:21:51.446148 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.416970 15779 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.446975 16013 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:51.446769 15871 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:51.426117 15900 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:51.448285 16014 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:21:51.414635 15968 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:51.470288 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47449us
W20260709 12:21:51.470475 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138391us
I20260709 12:21:51.524984 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.417220 16019 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:51.435659 15881 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.430689 16016 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.424289 15989 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:21:51.469872 15877 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:51.518567 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.921s (SENT)
I20260709 12:21:51.525054 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:51.423406 15916 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 12:21:51.456297 15984 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.467442 15996 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:51.472450 13799 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:51.533710 13718 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:21:51.537135 15881 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:51.537850 16015 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:51.553967 13644 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:21:51.620004 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.628477 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:51.573776 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:51.540470 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108147us
I20260709 12:21:51.636929 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.563261 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.639624 13793 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:21:51.639249 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 114302us
I20260709 12:21:51.641302 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.642392 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:51.603375 16014 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.067s	user 0.004s	sys 0.000s
I20260709 12:21:51.518045 15779 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.574642 15989 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:21:51.630151 15984 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.574991 15980 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.629492 15996 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.657205 15850 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.566972 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:51.652776 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.500136 15917 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.651048 15989 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:51.657560 15996 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:51.649554 16013 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:51.552915 15968 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.659225 16026 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:21:51.664274 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.644174 15983 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.638720 15877 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.540066 16016 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.666307 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.648506 16015 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:51.690629 16011 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:51.691166 16011 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:51.691296 16016 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.691934 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.694384 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.698354 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.699766 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:51.672197 13796 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:51.646175 15891 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:51.673702 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:51.673974 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1610) took 1059 ms. Trace:
I20260709 12:21:51.573956 15804 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.566375 15986 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:21:51.677865 16013 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.682799 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:51.683902 15917 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:51.684136 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.684514 15983 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.684994 13795 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4.
I20260709 12:21:51.685657 15985 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.686816 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.688318 15877 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:51.688926 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.690129 16015 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.671814 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:51.703440 13722 rpcz_store.cc:276] 0709 12:21:50.614523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.696535 (+ 82012us) service_pool.cc:224] Handling call
0709 12:21:50.696912 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:50.872794 (+175882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:50.872844 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:50.891824 (+ 18980us) log.cc:844] Serialized 26 byte log entry
0709 12:21:51.161927 (+270103us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:51.192608 (+ 30681us) log.cc:844] Serialized 14 byte log entry
0709 12:21:51.192880 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:51.192959 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:51.670968 (+478009us) raft_consensus.cc:1685] finished
0709 12:21:51.670993 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:51.673888 (+  2895us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31505,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":1627,"raft.queue_time_us":263254,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":8259,"spinlock_wait_cycles":4487424,"thread_start_us":354289,"threads_started":2,"wal-append.queue_time_us":288595}
W20260709 12:21:51.745313 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49716.4us
W20260709 12:21:51.773974 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74310.4us
I20260709 12:21:51.774461 16018 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.777575 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:51.779173 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:51.793224 14424 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:51.778681 15941 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:51.677685 15993 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.675185 15999 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:51.680810 15881 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:51.796161 13792 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:21:51.674755 15871 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.121s
I20260709 12:21:51.692219 16011 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.664415 15980 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:51.774044 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 303646us
I20260709 12:21:51.837627 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.922497 13799 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:51.808938 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.726985 15984 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.767913 16013 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.769016 15939 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:51.837845 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.839527 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.927435 15939 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.928053 15939 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.928402 15941 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:51.928857 15941 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:51.781684 15983 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.838738 15989 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:51.935519 15877 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:51.935808 15989 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:51.936627 15993 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.937222 15993 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.797334 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.938694 15939 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:51.939565 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 142347us
I20260709 12:21:51.941593 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:51.778690 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:51.948179 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.949128 15999 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.949717 15999 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.867888 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:51.792491 16018 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.841575 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:51.923452 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:51.945572 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:51.946137 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 167347us
I20260709 12:21:51.949028 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:51.963902 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:51.964360 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:51.888195 15804 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:51.949839 15863 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:51.942634 13798 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:21:51.955384 13615 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56f25c6fe9f245d99ae467bdd3948582. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:21:51.972980 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50388.1us
W20260709 12:21:51.974778 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:51.974963 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.211s (SENT)
I20260709 12:21:51.960325 16008 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.999954 16048 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:52.000834 15985 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:52.000396 16008 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:51.957015 15779 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:51.961495 16011 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:51.961872 15999 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.941622 15939 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:52.013558 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:51.966704 16018 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:51.976248 15863 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:51.957803 15877 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:52.054574 15844 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:52.055300 15844 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:52.061275 16042 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:21:52.064983 14422 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.002s
W20260709 12:21:52.068770 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 104371us
I20260709 12:21:52.065665 14422 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:52.069730 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:52.067297 14424 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:52.071549 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:52.042439 15968 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:52.063668 15863 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:52.079733 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 118045us
I20260709 12:21:52.066298 15877 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:52.082298 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:52.087275 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:52.087476 13646 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:52.070079 16026 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:52.094725 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 118225us
I20260709 12:21:52.095470 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:52.096596 13647 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:52.080096 15844 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:52.099736 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:52.083904 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:52.111512 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:52.075728 16016 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:52.100858 15980 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:52.138743 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1710) took 1727 ms. Trace:
W20260709 12:21:52.120694 15964 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.000s
I20260709 12:21:52.139207 13648 rpcz_store.cc:276] 0709 12:21:50.411379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.411546 (+   167us) service_pool.cc:224] Handling call
0709 12:21:50.431384 (+ 19838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:21:50.821136 (+389752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:50.821177 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:21:50.829520 (+  8343us) log.cc:844] Serialized 52 byte log entry
0709 12:21:50.900308 (+ 70788us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:50.950884 (+ 50576us) log.cc:844] Serialized 14 byte log entry
0709 12:21:52.129934 (+1179050us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b180039e980. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:21:52.130230 (+   296us) log.cc:844] Serialized 14 byte log entry
0709 12:21:52.130562 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:52.130685 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:52.138453 (+  7768us) raft_consensus.cc:1685] finished
0709 12:21:52.138485 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:52.138685 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1156889,"prepare.queue_time_us":54117,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":62895,"raft.run_cpu_time_us":998,"raft.run_wall_time_us":29729,"spinlock_wait_cycles":5600256,"thread_start_us":343760,"threads_started":2,"wal-append.queue_time_us":311778}
I20260709 12:21:52.152451 13797 raft_consensus.cc:2368] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3.
W20260709 12:21:52.155906 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44466.2us
I20260709 12:21:52.157887 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.158156 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.158648 13718 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:52.158779 13725 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:52.160320 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:52.161088 13718 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:52.161675 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:52.163496 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:52.164600 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:52.164911 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.165205 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.165409 13726 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 2: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:52.165719 13720 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:52.166395 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.166630 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.166769 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.167416 13719 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:52.167958 13651 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:52.167742 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.168749 13721 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:52.166422 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.171633 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:52.173120 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.171660 13726 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:52.176494 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.182914 13722 raft_consensus.cc:2410] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:52.181553 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:52.160312 14420 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:52.185644 15968 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:52.186043 15968 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:52.186445 15968 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:52.187619 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:52.188655 16044 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:21:52.189071 16044 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:52.189430 16044 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:21:52.190411 16012 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:52.191319 16050 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 12:21:52.192705 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.193912 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:52.195026 13718 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:21:52.152009 15871 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.103946 16026 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:52.205622 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:52.206609 13725 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:52.219762 13648 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 5. Ops: []
I20260709 12:21:52.221143 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:52.173709 15779 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:52.230877 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 131246us
I20260709 12:21:52.174501 15964 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:21:52.233368 16053 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:21:52.233163 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:52.192849 15980 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.237879 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:52.250860 16069 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:52.221903 15871 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:52.269212 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:52.273105 16051 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:52.216346 15649 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:52.272575 16062 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:21:52.448171 15996 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.014s
I20260709 12:21:52.322314 16060 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.241058 15980 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.236922 15964 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:52.318431 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 81902us
I20260709 12:21:52.230491 15877 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:21:52.272076 16040 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:21:52.251897 16043 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:52.376617 15941 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:52.391304 16047 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:52.322929 15902 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:52.270706 16057 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:52.469269 16008 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:52.474941 16049 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:21:52.809198 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.811267 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:52.816031 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:52.806185 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 573062us
I20260709 12:21:52.393222 13645 raft_consensus.cc:606] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:52.862110 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50946.9us
I20260709 12:21:52.862460 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:52.512705 16062 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.485428 16026 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:52.811504 15983 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.879s	user 0.014s	sys 0.000s
W20260709 12:21:52.869128 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:52.880826 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:52.869808 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:52.884896 16020 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:52.835412 16060 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:52.886620 16060 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.839221 15964 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:52.838757 15980 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:52.812472 15877 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:52.930768 16026 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:52.861738 16057 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:52.882030 16008 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:52.873721 16069 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:52.958775 13643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.018s	sys 0.049s
I20260709 12:21:52.860067 16047 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:52.962810 13693 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.047s	sys 0.068s
I20260709 12:21:52.880115 16062 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:21:52.897038 16043 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:21:52.968017 13643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.019s	sys 0.050s
W20260709 12:21:52.935470 15804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.039s	sys 0.077s
W20260709 12:21:52.971742 13693 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.047s	sys 0.068s
I20260709 12:21:52.880909 16031 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:52.956630 15894 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.048s	sys 0.077s
W20260709 12:21:52.980904 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 806634us
I20260709 12:21:52.854822 15996 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:52.981307 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1726) took 1680 ms. Trace:
I20260709 12:21:52.423142 16051 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:52.947402 15877 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:52.958559 16057 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.007072 13643 rpcz_store.cc:276] 0709 12:21:51.301019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.301186 (+   167us) service_pool.cc:224] Handling call
0709 12:21:51.301560 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:51.301804 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:21:52.320363 (+1018559us) log.cc:844] Serialized 14 byte log entry
0709 12:21:52.980877 (+660514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:52.980949 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:21:52.981006 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:52.981081 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:52.981235 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":830228,"prepare.queue_time_us":1673177,"raft.queue_time_us":10624,"thread_start_us":831330,"threads_started":2}
I20260709 12:21:53.010811 16038 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:52.924863 15983 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:53.003350 15894 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.049s	sys 0.077s
I20260709 12:21:53.013120 16031 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.016451 15996 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:53.020421 16004 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.972s	user 0.014s	sys 0.009s
I20260709 12:21:52.988942 16035 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:52.990924 15804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.040s	sys 0.077s
W20260709 12:21:53.025276 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163109us
I20260709 12:21:53.024853 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:52.964283 16008 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:52.875682 15989 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:52.864382 15902 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.030505 16038 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:52.976534 16069 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.059191 13793 raft_consensus.cc:2368] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3.
I20260709 12:21:53.059839 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2236) took 2173 ms. Trace:
I20260709 12:21:53.060837 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1733) took 1757 ms. Trace:
I20260709 12:21:53.097432 13796 rpcz_store.cc:276] 0709 12:21:50.886276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.887494 (+  1218us) service_pool.cc:224] Handling call
0709 12:21:53.054028 (+2166534us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:53.059761 (+  5733us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704952192}
I20260709 12:21:52.880115 16049 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.082559 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:53.083788 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:53.106092 13651 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:53.106932 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:53.107580 13649 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:53.079162 16016 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:53.059469 15994 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.088819 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:53.092191 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1555) took 1681 ms (client timeout 1801 ms). Trace:
I20260709 12:21:53.097512 13644 rpcz_store.cc:276] 0709 12:21:51.303495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.355712 (+ 52217us) service_pool.cc:224] Handling call
0709 12:21:51.356269 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:51.640232 (+283963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.052460 (+1412228us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400112954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:53.060768 (+  8308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423206784}
I20260709 12:21:53.071444 13795 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:21:53.122125 13792 rpcz_store.cc:269] 0709 12:21:51.412547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.418210 (+  5663us) service_pool.cc:224] Handling call
0709 12:21:53.089575 (+1671365us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:53.092114 (+  2539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":833537792}
I20260709 12:21:53.124279 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:53.126847 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:53.128922 13796 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:21:53.131619 15902 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.088392 15996 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:53.158601 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 340) took 1050 ms. Trace:
I20260709 12:21:53.159662 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:53.159338 13650 rpcz_store.cc:276] 0709 12:21:52.086373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.086511 (+   138us) service_pool.cc:224] Handling call
0709 12:21:53.088642 (+1002131us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:53.137120 (+ 48478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3276160}
I20260709 12:21:53.160490 13651 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:21:53.162786 14389 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.036s	sys 0.082s
W20260709 12:21:53.182232 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 315517us
W20260709 12:21:53.095644 16013 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.169s	user 0.017s	sys 0.003s
W20260709 12:21:53.182303 14389 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.046s	sys 0.091s
I20260709 12:21:53.112119 15989 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:53.185179 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:21:53.207794 14424 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:53.255457 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:53.157503 16007 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:53.274909 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:53.193775 15902 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:53.162724 15996 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.304721 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:53.271934 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:53.136515 15994 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:53.320909 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:53.269809 15941 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.108443 16043 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.280699 15804 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:53.199813 16013 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.209062 16055 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:21:53.332379 13649 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:53.337577 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:53.348966 13644 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:21:53.316699 15863 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.228s	user 0.014s	sys 0.002s
W20260709 12:21:53.329999 16036 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.327s	user 0.017s	sys 0.034s
W20260709 12:21:53.372427 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 189139us
W20260709 12:21:53.378082 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 295626us
I20260709 12:21:53.378437 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:53.379314 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:53.202960 16038 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.283404 16007 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.350545 16016 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:53.291092 15902 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.282860 16065 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:21:53.263124 16008 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:53.374734 16036 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.372s	user 0.017s	sys 0.035s
I20260709 12:21:53.379367 13649 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:53.390690 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:53.366058 15863 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.367908 15989 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.481992 13644 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:53.651187 13649 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:53.676210 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 285577us
I20260709 12:21:53.407514 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:53.490097 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81508.5us
I20260709 12:21:53.385525 16047 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:53.513085 16060 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.626s	user 0.012s	sys 0.003s
I20260709 12:21:53.682657 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:53.683328 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:53.408857 15993 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.468s	user 0.010s	sys 0.006s
I20260709 12:21:53.692598 13644 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
W20260709 12:21:53.677323 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.507s (SENT) [suppressed 45 similar messages]
I20260709 12:21:53.383955 15941 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:53.407202 16026 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:21:53.464998 16044 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.275s	user 0.007s	sys 0.008s
W20260709 12:21:53.465067 16018 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.399s	user 0.012s	sys 0.005s
W20260709 12:21:53.505823 16075 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.369s	user 0.000s	sys 0.001s
I20260709 12:21:53.389452 16078 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.318s
I20260709 12:21:53.678817 15825 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:21:53.787493 16078 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:53.787572 15825 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:53.788013 16078 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:53.788102 15825 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:53.789129 15825 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.789341 16078 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.790447 15825 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:21:53.790848 16078 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:53.631170 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 385) took 1854 ms. Trace:
I20260709 12:21:53.699810 16085 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:21:53.722290 15877 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.000s
I20260709 12:21:53.802835 15877 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.805486 15877 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.722870 16013 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:53.724275 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:53.806147 15871 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:53.806664 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2252) took 1464 ms (client timeout 1825 ms). Trace:
I20260709 12:21:53.806885 16085 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:53.807400 16085 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:53.807073 13800 rpcz_store.cc:269] 0709 12:21:52.342361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.408790 (+ 66429us) service_pool.cc:224] Handling call
0709 12:21:53.724093 (+1315303us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:53.806596 (+ 82503us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13257600}
I20260709 12:21:53.796547 14421 rpcz_store.cc:276] 0709 12:21:51.776837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.823294 (+ 46457us) service_pool.cc:224] Handling call
0709 12:21:53.630035 (+1806741us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:53.630351 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.630668 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:53.630713 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:53.630789 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:53.630839 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.630925 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:53.631086 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136870656}
I20260709 12:21:53.815594 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2249) took 1482 ms. Trace:
I20260709 12:21:53.816169 13801 rpcz_store.cc:276] 0709 12:21:52.333452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.334166 (+   714us) service_pool.cc:224] Handling call
0709 12:21:53.814685 (+1480519us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:53.814931 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.815177 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:53.815213 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:53.815269 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:53.815314 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.815384 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:53.815516 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5430016}
I20260709 12:21:53.725306 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2253) took 1382 ms. Trace:
W20260709 12:21:53.726056 16011 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.685s	user 0.010s	sys 0.005s
I20260709 12:21:53.820619 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:53.821429 13727 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.821338 15994 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:53.796590 16057 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.734s	user 0.014s	sys 0.000s
I20260709 12:21:53.823664 16057 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.818171 13792 rpcz_store.cc:276] 0709 12:21:52.342682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.153083 (+810401us) service_pool.cc:224] Handling call
0709 12:21:53.153831 (+   748us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:53.154327 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:53.154374 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:53.195551 (+ 41177us) log.cc:844] Serialized 499 byte log entry
0709 12:21:53.369324 (+173773us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:53.369421 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.369672 (+   251us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:53.725050 (+355378us) raft_consensus.cc:1685] finished
0709 12:21:53.725083 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:53.725237 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":88905,"threads_started":1,"wal-append.queue_time_us":285431}
I20260709 12:21:53.386289 16051 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:53.475370 16038 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.826061 16057 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.699939 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:53.703234 16093 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 12:21:53.827301 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 127453us
I20260709 12:21:53.673267 16031 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.465977 16049 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.636232 16019 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:53.704741 16090 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 12:21:53.704047 16086 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:21:53.768633 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1557) took 2297 ms (client timeout 1921 ms). Trace:
W20260709 12:21:53.849037 13797 rpcz_store.cc:269] 0709 12:21:51.471306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.471453 (+   147us) service_pool.cc:224] Handling call
0709 12:21:53.768358 (+2296905us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:53.768538 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435917056}
W20260709 12:21:53.768824 15999 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.706s	user 0.018s	sys 0.002s
W20260709 12:21:53.773291 15980 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.879s	user 0.007s	sys 0.005s
W20260709 12:21:53.774029 16074 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.456s	user 0.000s	sys 0.001s
I20260709 12:21:53.465976 16008 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:53.628612 15984 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.754683 15941 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.618095 16088 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.235s
I20260709 12:21:53.770589 15983 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:53.407079 16016 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:53.775558 15968 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.589s	user 0.020s	sys 0.001s
W20260709 12:21:53.776292 15964 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.886s	user 0.014s	sys 0.002s
I20260709 12:21:53.776015 16044 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.702059 16060 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.778167 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:53.778975 16018 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.878041 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:53.780755 16062 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.777s	user 0.012s	sys 0.000s
I20260709 12:21:53.781317 15863 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:53.782709 16069 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.785202 13649 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:53.466158 16087 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.817603 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:53.886328 16048 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.885s	user 0.030s	sys 0.021s
I20260709 12:21:53.821655 15623 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:53.743232 15902 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:53.827358 13793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1562) took 2158 ms (client timeout 2120 ms). Trace:
I20260709 12:21:53.828186 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:53.464077 15996 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:53.513075 15939 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:53.840672 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:21:53.841684 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:53.868683 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:53.744987 15993 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:53.878405 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100340us
I20260709 12:21:53.847048 16086 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:53.819825 13651 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:53.899224 15982 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:53.885372 16088 consensus_queue.cc:1243] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:21:53.901098 15941 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.903676 16044 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.905498 16060 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.905717 16063 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:53.906160 13766 connection.cc:256] RPC call timeout handler was delayed by -1.0865953719999197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 12:21:53.997080 13766 connection.cc:256] RPC call timeout handler was delayed by -1.0424953719998484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 12:21:53.997550 13766 connection.cc:256] RPC call timeout handler was delayed by -1.021795371999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
I20260709 12:21:53.907613 16018 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:53.885738 15968 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.002767 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:38410 (request call id 26) took 2217 ms. Trace:
I20260709 12:21:54.003005 15968 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.888301 15964 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.003281 13645 rpcz_store.cc:276] 0709 12:21:51.785276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.818585 (+ 33309us) service_pool.cc:224] Handling call
0709 12:21:54.002539 (+2183954us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bee853 00007f5398bbc60a 00007f53921427d4 00007f5392142762 00007f5392142691 00007f539214260c 00007f53921418c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:54.002647 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954805632}
I20260709 12:21:54.005088 15964 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.868598 15980 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.006525 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2247) took 1675 ms. Trace:
I20260709 12:21:54.006976 13795 rpcz_store.cc:276] 0709 12:21:52.331038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.331198 (+   160us) service_pool.cc:224] Handling call
0709 12:21:52.331641 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.124990 (+793349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.006221 (+881231us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54002891d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:54.006453 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684632320}
I20260709 12:21:54.007490 15980 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:54.008255 13647 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:53.888561 16051 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:53.865782 15999 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:54.009759 13724 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.010857 15999 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:53.894938 16062 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.911217 13650 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:21:53.911669 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:53.913519 16007 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:53.877048 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:53.891191 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:53.906190 16016 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:53.926800 13793 rpcz_store.cc:269] 0709 12:21:51.668846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.671062 (+  2216us) service_pool.cc:224] Handling call
0709 12:21:53.827144 (+2156082us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400129654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:53.827279 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2927766272}
W20260709 12:21:53.926831 16048 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.919s	user 0.031s	sys 0.021s
W20260709 12:21:53.960745 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 283432us
I20260709 12:21:54.025146 16048 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:21:54.026144 13693 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.602s (SENT)
I20260709 12:21:54.026806 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.028122 16018 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:53.976694 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:53.856021 16049 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 12:21:53.962504 16015 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.016s	sys 0.016s
I20260709 12:21:53.981472 13726 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 2.
I20260709 12:21:53.930505 15623 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:53.983997 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:53.992687 16096 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:21:53.966905 16087 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:53.997115 16063 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:54.000859 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:54.015619 13720 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:54.015514 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:53.990475 15982 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:54.026309 16048 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:53.930891 15939 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:54.030978 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 202867us
I20260709 12:21:54.028527 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:54.031417 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2246) took 1701 ms. Trace:
I20260709 12:21:54.108863 13719 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:54.033839 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:54.043752 15993 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:54.058400 16072 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.851s	user 0.000s	sys 0.003s
I20260709 12:21:54.063133 13727 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:54.035182 13723 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:21:54.013986 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:54.031796 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.643s (SENT)
I20260709 12:21:54.066661 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:53.991217 16086 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:54.089643 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:54.037338 15649 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:21:54.103049 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 215523us
I20260709 12:21:54.101845 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.104564 13725 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:54.103224 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:54.103642 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 119780us
I20260709 12:21:54.107304 14422 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:54.119239 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:54.070429 16015 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.085s	user 0.017s	sys 0.016s
I20260709 12:21:54.121348 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:54.122926 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.124249 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:54.068848 15980 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:54.124857 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124095us
I20260709 12:21:54.125784 13648 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:54.118633 13721 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:54.129205 13651 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:54.137902 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.135206 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:54.133406 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:21:54.135679 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103909us
I20260709 12:21:54.138839 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:54.139356 13801 raft_consensus.cc:606] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:54.139500 13693 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.703s (SENT)
I20260709 12:21:54.139775 13801 raft_consensus.cc:614] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:54.140211 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:54.140709 13693 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.729s (SENT)
I20260709 12:21:54.141387 13726 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:21:54.141445 13693 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.773s (SENT)
I20260709 12:21:54.143046 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:54.142045 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.111900 15982 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:54.108892 13798 rpcz_store.cc:276] 0709 12:21:52.330458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.330656 (+   198us) service_pool.cc:224] Handling call
0709 12:21:53.716172 (+1385516us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:53.716399 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.716655 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:21:53.717132 (+   477us) log.cc:844] Serialized 14 byte log entry
0709 12:21:54.031027 (+313895us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:54.031094 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:21:54.031137 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.031204 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:54.031342 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3033,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":99521,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":108,"spinlock_wait_cycles":10975616,"thread_start_us":249728,"threads_started":1}
I20260709 12:21:54.092670 16051 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:54.155330 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:54.156194 13719 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:54.111689 15739 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:21:54.151671 13693 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.799s (SENT)
I20260709 12:21:54.125283 16062 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:54.128296 15850 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:54.121039 16009 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.128367 15649 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:54.170446 13693 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.801s (SENT)
I20260709 12:21:54.181725 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.171556 13725 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:54.170934 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:54.151541 16087 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.154760 15993 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.272466 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1741) took 1207 ms. Trace:
I20260709 12:21:54.278319 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 386) took 2290 ms. Trace:
I20260709 12:21:54.258977 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:54.189003 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1749) took 1122 ms. Trace:
I20260709 12:21:54.280386 13650 rpcz_store.cc:276] 0709 12:21:53.066263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.407845 (+341582us) service_pool.cc:224] Handling call
0709 12:21:53.635311 (+227466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.014585 (+379274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.188626 (+174041us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:54.188913 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634048896}
I20260709 12:21:54.281559 13719 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:54.277148 13648 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:54.282341 13725 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:54.279343 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 170975us
I20260709 12:21:54.284235 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.170432 14420 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:54.314733 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:54.316197 13719 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:21:54.320410 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 181408us
I20260709 12:21:54.280202 14423 rpcz_store.cc:276] 0709 12:21:51.888902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:51.889203 (+   301us) service_pool.cc:224] Handling call
0709 12:21:53.823347 (+1934144us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:53.823541 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.823755 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:21:53.886416 (+ 62661us) log.cc:844] Serialized 14 byte log entry
0709 12:21:54.178210 (+291794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:54.178309 (+    99us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:21:54.178387 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.178507 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:54.178677 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4059,"prepare.queue_time_us":52263,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"spinlock_wait_cycles":2922240,"thread_start_us":341356,"threads_started":2}
I20260709 12:21:54.335811 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:54.184964 15983 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.146299 15980 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.146615 15891 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
W20260709 12:21:54.331204 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 160799us
I20260709 12:21:54.180634 16038 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:54.404881 13693 leader_election.cc:336] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.871s (SENT)
I20260709 12:21:54.405902 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.279256 13652 rpcz_store.cc:276] 0709 12:21:53.064638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.064759 (+   121us) service_pool.cc:224] Handling call
0709 12:21:53.078296 (+ 13537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:53.316603 (+238307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:53.316645 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:53.317165 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:21:53.404255 (+ 87090us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:53.404312 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:53.404446 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:54.272197 (+867751us) raft_consensus.cc:1685] finished
0709 12:21:54.272229 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:54.272389 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118730,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":54398,"thread_start_us":255817,"threads_started":2,"wal-append.queue_time_us":487524}
W20260709 12:21:54.339534 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55437.9us
I20260709 12:21:54.271044 15739 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:54.378624 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:54.383018 16020 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.380914 13721 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:21:54.389637 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:21:54.371806 16077 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.923s	user 0.001s	sys 0.001s
I20260709 12:21:54.167024 15939 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:54.281279 15968 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.402097 13725 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:21:54.402742 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 262883us
I20260709 12:21:54.378626 13719 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
W20260709 12:21:54.392565 16008 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.002s
I20260709 12:21:54.408113 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.432382 13726 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:54.413180 13801 raft_consensus.cc:606] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:54.433729 16076 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.425313 13721 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:54.427132 13725 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:54.407217 15877 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:21:54.433822 13801 raft_consensus.cc:614] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:54.408200 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 490) took 1218 ms. Trace:
W20260709 12:21:54.425750 16107 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.366s	user 0.002s	sys 0.000s
W20260709 12:21:54.424454 16085 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.615s	user 0.004s	sys 0.013s
W20260709 12:21:54.429425 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.921s (SENT)
I20260709 12:21:54.471805 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.475490 13648 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:54.412375 15993 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.478485 13652 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
W20260709 12:21:54.441759 16116 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:21:54.429507 16062 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:54.438095 16098 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:21:54.505777 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.511179 13650 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:21:54.513991 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84576.8us
W20260709 12:21:54.514946 13693 leader_election.cc:336] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.936s (SENT)
I20260709 12:21:54.515456 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:54.517023 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:54.508471 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3857908530003442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 12:21:54.424019 16020 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:54.411648 16038 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:54.434419 16076 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.438113 15983 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:21:54.471897 15877 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:54.408432 16051 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.435336 15968 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:54.489209 16063 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:54.413883 15649 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:54.525923 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.528323 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.467706 16036 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:21:54.509958 16098 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.568437 16098 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:54.570746 15980 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:54.570899 16120 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:21:54.478471 16095 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:21:54.519289 16088 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:21:54.524874 16113 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.358s	user 0.000s	sys 0.001s
I20260709 12:21:54.480619 16085 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.546037 16008 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:54.563795 16105 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 12:21:54.564981 16121 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:21:54.566407 16118 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:21:54.495337 15739 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.567793 16038 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:54.574488 13693 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.963s (SENT)
W20260709 12:21:54.593447 13693 leader_election.cc:336] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.984s (SENT)
I20260709 12:21:54.580334 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.593952 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.594653 13643 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:54.583827 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.584085 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:54.596491 13649 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:54.587092 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 3.204s (SENT)
I20260709 12:21:54.597760 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.611353 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:54.613631 13693 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.076s (SENT)
W20260709 12:21:54.614662 13693 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.120s (SENT)
I20260709 12:21:54.615099 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.616071 16018 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:21:54.616284 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:54.617681 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:54.620421 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.811s (SENT)
I20260709 12:21:54.620658 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.620929 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.621320 13644 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:54.621773 15964 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.624670 16019 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:54.625401 16125 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:54.626919 15877 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.629667 15649 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:54.630559 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:54.631942 14419 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:54.639407 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1740) took 1575 ms. Trace:
I20260709 12:21:54.640166 16020 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:54.596122 16063 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:21:54.644055 15968 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:54.639945 13647 rpcz_store.cc:276] 0709 12:21:53.064248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.064361 (+   113us) service_pool.cc:224] Handling call
0709 12:21:53.080829 (+ 16468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.008748 (+927919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.639075 (+630327us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54001e6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:54.639325 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132373376}
W20260709 12:21:54.651427 13618 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.041s	sys 0.086s
I20260709 12:21:54.650297 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:54.653988 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60407.7us
W20260709 12:21:54.653074 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44562.7us
I20260709 12:21:54.662727 14423 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:54.670017 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:54.670455 13647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:54.670985 13727 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:54.671249 13647 raft_consensus.cc:606] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:54.671751 13647 raft_consensus.cc:614] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:21:54.616379 13693 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.157s (SENT)
I20260709 12:21:54.675988 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:54.673651 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:54.649572 16057 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:54.664621 15894 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:54.610186 16008 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:54.650378 16085 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:54.699121 15989 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:54.442701 14418 rpcz_store.cc:276] 0709 12:21:53.187453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.192156 (+  4703us) service_pool.cc:224] Handling call
0709 12:21:53.313649 (+121493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:53.347836 (+ 34187us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:21:53.864482 (+516646us) log.cc:844] Serialized 14 byte log entry
0709 12:21:54.340979 (+476497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:54.382711 (+ 41732us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:54.386520 (+  3809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.393896 (+  7376us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:54.405372 (+ 11476us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3441,"prepare.queue_time_us":480706,"raft.queue_time_us":33682,"spinlock_wait_cycles":22813952,"thread_start_us":900418,"threads_started":3,"wal-append.queue_time_us":305657}
I20260709 12:21:54.649842 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:54.700805 15989 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:54.691236 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:54.701282 15989 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:54.703291 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:54.592566 16076 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:54.601893 16126 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 12:21:54.682236 16057 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:54.687718 16020 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:54.687597 15894 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:54.666627 16045 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:21:54.678617 16009 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:21:54.719270 15982 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
W20260709 12:21:54.724180 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:54.734560 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.019s	sys 0.022s
W20260709 12:21:54.709649 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93277.8us
I20260709 12:21:54.665199 16123 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:21:54.743790 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.019s	sys 0.022s
I20260709 12:21:54.666424 15902 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:21:54.726210 13649 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:54.759138 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.766803 13651 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
W20260709 12:21:54.764912 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 627056us
I20260709 12:21:54.722832 15928 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:54.722229 16020 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:54.665623 16124 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:21:54.773928 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:54.733434 16008 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:21:54.776849 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.011s	sys 0.037s
W20260709 12:21:54.778697 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.011s	sys 0.037s
I20260709 12:21:54.736958 13646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9c30191f356495b9732659f925abc9f"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:21:54.764142 16076 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:54.736459 16009 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:54.781368 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:54.788187 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1968550970000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:21:54.808466 13649 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:54.808676 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1968550970000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 12:21:54.809418 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1149550969998927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 12:21:54.810146 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:54.810546 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.791129 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:54.814473 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:54.815430 13651 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:54.789343 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.796168 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:54.796684 16122 ts_tablet_manager.cc:933] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
I20260709 12:21:54.763866 15982 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.797588 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:54.806581 16050 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.778474 16020 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:54.702200 15989 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.774906 15928 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:21:54.826211 13618 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.165s	user 0.042s	sys 0.087s
W20260709 12:21:54.826736 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.16597e+06us
I20260709 12:21:54.807299 16126 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:21:54.813849 16078 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.022s	user 0.009s	sys 0.008s
I20260709 12:21:54.808605 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:54.722591 16019 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:54.808471 13646 raft_consensus.cc:606] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:54.860656 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:54.861351 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:54.847790 13618 connection.cc:256] RPC call timeout handler was delayed by -1.116987151000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
I20260709 12:21:54.820194 16125 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:54.855880 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:54.874555 13618 connection.cc:256] RPC call timeout handler was delayed by -1.0980592979999528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3219s for a response
I20260709 12:21:54.879070 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:54.882601 13618 connection.cc:256] RPC call timeout handler was delayed by -1.030389987000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 12:21:54.826093 16018 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:54.825829 15993 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:54.885664 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:54.834611 15928 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:54.783893 16057 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.907948 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.908776 13643 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:54.909660 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:54.889345 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:54.887312 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.840425 16126 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:54.897066 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:54.811549 16076 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:54.938115 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:54.834872 16080 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:21:54.860285 15739 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:21:54.855840 16019 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:54.956130 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:54.956414 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:54.941526 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:54.882447 13646 raft_consensus.cc:614] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:21:54.825376 16050 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:54.939932 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52745.9us
I20260709 12:21:55.005537 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:55.007170 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:54.946796 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:54.880564 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 522) took 1016 ms. Trace:
I20260709 12:21:54.808287 15894 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.959050 13643 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:54.907246 16125 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:55.018389 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:55.020978 16130 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.025609 16126 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:55.026113 16130 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:55.022994 13799 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:55.027189 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:54.942570 15989 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.027014 15964 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:55.022027 14416 rpcz_store.cc:276] 0709 12:21:53.843653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.846746 (+  3093us) service_pool.cc:224] Handling call
0709 12:21:54.795969 (+949223us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:54.860308 (+ 64339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8888704}
W20260709 12:21:55.041311 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.041302 15964 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:21:55.042461 15964 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.027354 15968 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:54.906679 16018 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.906560 15993 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:54.893813 16009 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:55.042842 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:55.467506 13650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.010s	sys 0.005s
W20260709 12:21:55.830229 13650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.014s	sys 0.011s
W20260709 12:21:55.263972 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2136) took 3097 ms (client timeout 1811 ms). Trace:
I20260709 12:21:55.215816 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:55.831013 13720 rpcz_store.cc:269] 0709 12:21:52.166224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:52.166396 (+   172us) service_pool.cc:224] Handling call
0709 12:21:55.263583 (+3097187us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:55.263825 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4250329856}
W20260709 12:21:55.638952 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.001s	sys 0.016s
W20260709 12:21:55.158766 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.152899 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.833818 13721 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:21:55.508008 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.103130 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:55.264514 13722 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:55.836351 13649 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:55.054320 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.528384 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:55.824899 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 817833us
I20260709 12:21:55.837266 13727 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
W20260709 12:21:55.116372 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 327159us
I20260709 12:21:54.954177 13645 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:54.948307 16085 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:54.959594 16019 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:55.103389 13725 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:21:55.827123 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.831570 13643 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:55.273643 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 387) took 1458 ms (client timeout 1750 ms). Trace:
I20260709 12:21:55.215728 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:55.102389 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:55.047282 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:55.846690 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.905030 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:55.847640 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:55.049213 16096 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:21:55.813423 16020 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.978s	user 0.012s	sys 0.001s
W20260709 12:21:55.350926 15649 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.721s	user 0.005s	sys 0.008s
I20260709 12:21:55.909449 13727 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:21:55.905455 13718 rpcz_store.cc:269] 0709 12:21:53.815090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.817328 (+  2238us) service_pool.cc:224] Handling call
0709 12:21:55.246221 (+1428893us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400156dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:55.273525 (+ 27304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2171249024}
I20260709 12:21:55.906322 13719 raft_consensus.cc:2368] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:21:55.910463 13644 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:21:55.256886 16119 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.786s	user 0.003s	sys 0.000s
I20260709 12:21:55.760444 16050 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:55.237003 16086 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.089s	user 0.013s	sys 0.000s
I20260709 12:21:55.249179 16128 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:21:55.907732 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 379472us
I20260709 12:21:55.113493 16126 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:55.913151 13723 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:55.935652 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:55.919334 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:55.774729 16133 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:21:55.778705 16130 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:55.175267 16124 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:55.831332 16122 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.000s	sys 0.011s
I20260709 12:21:55.932164 16009 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.925555 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:55.341491 16095 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:55.924681 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:55.926167 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.927800 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.964663 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
W20260709 12:21:55.930682 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:55.967089 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:55.216976 15900 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:55.485168 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:55.972678 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 254) took 1181 ms. Trace:
I20260709 12:21:55.973332 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1767) took 1280 ms. Trace:
I20260709 12:21:55.973112 13794 rpcz_store.cc:276] 0709 12:21:54.790954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.791116 (+   162us) service_pool.cc:224] Handling call
0709 12:21:54.818074 (+ 26958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:54.818403 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:54.818446 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:54.821482 (+  3036us) log.cc:844] Serialized 26 byte log entry
0709 12:21:55.338862 (+517380us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:55.339082 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.339197 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:55.972430 (+633233us) raft_consensus.cc:1685] finished
0709 12:21:55.972452 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:55.972612 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":460273,"threads_started":1,"wal-append.queue_time_us":1080253}
I20260709 12:21:55.936918 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:55.936964 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:55.267984 15928 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:55.259984 15964 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:55.941723 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.977511 13719 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:21:55.961975 16122 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.006s	sys 0.012s
I20260709 12:21:55.978569 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:55.973753 13647 rpcz_store.cc:276] 0709 12:21:54.693150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.694478 (+  1328us) service_pool.cc:224] Handling call
0709 12:21:54.723103 (+ 28625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:54.723641 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:54.723682 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:54.733454 (+  9772us) log.cc:844] Serialized 26 byte log entry
0709 12:21:54.794126 (+ 60672us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b1800352180. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:21:55.054158 (+260032us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:55.172814 (+118656us) log.cc:844] Serialized 14 byte log entry
0709 12:21:55.173242 (+   428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:55.173392 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:55.973041 (+799649us) raft_consensus.cc:1685] finished
0709 12:21:55.973146 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:55.973277 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":116159,"raft.queue_time_us":2291,"raft.run_cpu_time_us":552,"raft.run_wall_time_us":15017,"spinlock_wait_cycles":34855168,"thread_start_us":259653,"threads_started":1,"wal-append.queue_time_us":1174448}
I20260709 12:21:55.939860 13725 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:55.946949 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:55.979607 13719 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:21:55.951481 13651 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.132s	user 0.014s	sys 0.004s
I20260709 12:21:55.927496 15918 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:55.910997 13720 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:55.956509 13649 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:55.956918 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:55.936425 16050 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:55.943562 16098 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.375s	user 0.019s	sys 0.004s
I20260709 12:21:55.921100 16020 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:55.983209 15918 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:55.988359 16020 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:55.988958 15918 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:55.989383 16117 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.601s	user 0.000s	sys 0.003s
I20260709 12:21:55.979012 15900 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:55.989940 16050 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:55.990928 15983 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:21:55.991372 15983 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:55.991417 15900 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:55.992267 16050 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:55.992754 15983 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:55.964625 13727 raft_consensus.cc:2376] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7. Re-sending same reply.
I20260709 12:21:55.215513 15982 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:55.967664 13643 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:55.967909 16130 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:55.937304 16128 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:55.965526 13721 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.996817 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 395) took 1658 ms. Trace:
I20260709 12:21:55.999214 13726 rpcz_store.cc:276] 0709 12:21:54.338384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.433226 (+ 94842us) service_pool.cc:224] Handling call
0709 12:21:54.475843 (+ 42617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:54.488754 (+ 12911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:54.488804 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:54.508502 (+ 19698us) log.cc:844] Serialized 26 byte log entry
0709 12:21:54.821359 (+312857us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:54.821413 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:54.821481 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:55.996563 (+1175082us) raft_consensus.cc:1685] finished
0709 12:21:55.996587 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:55.996741 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18519,"raft.queue_time_us":22592,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":9374,"thread_start_us":247396,"threads_started":1,"wal-append.queue_time_us":857722}
I20260709 12:21:55.995987 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.002080 13727 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:55.205457 16076 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:55.975293 13723 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.003279 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.014801 13719 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.975919 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:55.979985 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.023860 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.025041 13725 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:55.964223 16145 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:55.980335 13651 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:55.982808 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.034991 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:56.035240 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1514) took 1222 ms (client timeout 1598 ms). Trace:
I20260709 12:21:56.035391 13720 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:56.035864 13727 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:56.036490 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.037060 13720 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.039290 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.040351 13725 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:55.217464 16125 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.041326 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:55.939038 16086 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:56.035914 13651 rpcz_store.cc:269] 0709 12:21:54.813299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.814057 (+   758us) service_pool.cc:224] Handling call
0709 12:21:56.035157 (+1221100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:56.029027 13718 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:56.036753 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.063275 13727 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:55.994176 15983 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:21:55.961022 16124 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.080345 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.080683 13718 pending_rounds.cc:85] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 2
I20260709 12:21:56.081131 13718 pending_rounds.cc:107] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:21:56.081440 13718 raft_consensus.cc:2889] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:56.002192 16008 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:21:56.086895 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 150040us
I20260709 12:21:56.083420 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2149) took 1001 ms. Trace:
I20260709 12:21:56.084501 13720 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:56.088135 13726 rpcz_store.cc:276] 0709 12:21:55.081833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.002642 (+920809us) service_pool.cc:224] Handling call
0709 12:21:56.083349 (+ 80707us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:56.088805 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.089394 13720 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.002655 16050 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:56.085445 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1766) took 1572 ms. Trace:
I20260709 12:21:56.087656 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:56.095191 13650 rpcz_store.cc:276] 0709 12:21:54.512701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.512883 (+   182us) service_pool.cc:224] Handling call
0709 12:21:54.513314 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.513617 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:21:54.907423 (+393806us) log.cc:844] Serialized 14 byte log entry
0709 12:21:56.085093 (+1177670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:56.085153 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:56.085199 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.085266 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:56.085388 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16046,"prepare.queue_time_us":454295,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":148435,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":10963,"thread_start_us":1543553,"threads_started":3,"wal-append.queue_time_us":1026494}
W20260709 12:21:56.105825 13795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.004s	sys 0.014s
W20260709 12:21:56.106304 13795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.005s	sys 0.014s
I20260709 12:21:56.108438 13723 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:56.109340 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2155) took 1004 ms. Trace:
I20260709 12:21:56.109766 13723 rpcz_store.cc:276] 0709 12:21:55.104787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.935866 (+831079us) service_pool.cc:224] Handling call
0709 12:21:56.109233 (+173367us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:21:56.113274 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 275455us
W20260709 12:21:56.115989 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148995us
W20260709 12:21:56.116413 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.118129 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:21:56.115383 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 131699us
I20260709 12:21:55.998394 16098 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.119838 13721 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:56.121026 16128 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:56.122910 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104181us
I20260709 12:21:56.121486 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.121877 13720 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:56.126940 16019 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.128037 13720 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:56.129211 16019 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.058373 16122 tablet_copy_client.cc:323] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.65:33075
I20260709 12:21:56.101074 16008 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:56.137557 16008 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.138511 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.112681 15982 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.140012 15982 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.142833 16161 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.143363 16161 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.144405 13720 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:56.102681 16130 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:56.134981 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:56.132203 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 402) took 1003 ms. Trace:
I20260709 12:21:56.148064 13726 rpcz_store.cc:276] 0709 12:21:55.128685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.088851 (+960166us) service_pool.cc:224] Handling call
0709 12:21:56.124120 (+ 35269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.124346 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.124371 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:56.124417 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:56.124441 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.124480 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:56.132138 (+  7658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:56.131819 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:56.146178 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:56.129617 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 136843us
I20260709 12:21:56.113732 16050 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:56.154569 13648 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:21:56.119271 16145 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:56.157588 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:56.118721 16018 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.131313 15993 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.155871 16020 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:56.166816 16018 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.172600 16086 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.168985 15993 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:56.173214 16086 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:56.174022 16161 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.170740 15918 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 12:21:56.177378 16142 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:21:56.177562 16159 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:56.177179 16050 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.163158 16128 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.159327 16145 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:56.172329 16018 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.185961 16098 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.186120 15993 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:56.186702 16098 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:21:56.187321 15993 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.188573 16098 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:56.189131 16098 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.189958 16145 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.186733 16018 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.186124 15928 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.173717 16086 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.159562 16158 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.192520 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.193277 14420 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:56.193720 16086 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.161491 16019 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:21:56.196010 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:56.166216 16155 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:56.198020 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:56.219805 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:56.245343 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49463.3us
W20260709 12:21:56.228631 15989 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.120s	user 0.010s	sys 0.001s
W20260709 12:21:56.247694 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94114.7us
I20260709 12:21:56.236171 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.235561 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:56.256906 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110816us
I20260709 12:21:56.224395 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:56.227411 16129 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:56.182461 16020 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:56.259385 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.178292 16159 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.249819 16144 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:21:56.224021 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.267577 15964 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.269032 14420 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:56.269979 16159 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.264523 16086 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:56.275496 16020 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.276201 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:56.278018 16169 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:56.278483 16169 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:56.278821 16169 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.279060 16008 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.280270 13719 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:56.225069 16124 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.282014 13797 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:56.284408 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:56.253412 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.263590 16129 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:56.295945 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2258) took 1465 ms. Trace:
I20260709 12:21:56.267936 16144 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:56.299860 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:56.308503 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.876s (SENT)
I20260709 12:21:56.317164 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:56.320674 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:21:56.338126 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.296587 13795 rpcz_store.cc:276] 0709 12:21:54.831051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.831705 (+   654us) service_pool.cc:224] Handling call
0709 12:21:54.832250 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:54.832526 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:21:55.274305 (+441779us) log.cc:844] Serialized 14 byte log entry
0709 12:21:56.263783 (+989478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:56.295531 (+ 31748us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:56.295621 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.295716 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:56.295865 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11682,"prepare.queue_time_us":1100434,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":67911,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":1398619,"threads_started":3,"wal-append.queue_time_us":892677}
I20260709 12:21:56.292205 16157 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.356639 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:56.374339 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:56.377050 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 252537us
I20260709 12:21:56.380002 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:56.380890 13793 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:56.383708 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:56.249657 16155 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.300806 16159 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:56.394035 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:56.394801 13643 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:56.398737 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:56.408978 13650 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:56.410012 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.412932 13795 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:56.414275 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:56.415215 13651 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:56.354801 16144 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:56.423543 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:56.370226 16076 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:56.370177 16158 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:56.438071 13647 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:56.438992 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:56.448289 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
W20260709 12:21:56.437749 15968 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.003s
I20260709 12:21:56.412281 16050 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.457849 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:21:56.411957 15993 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:56.411993 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6178281459997379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 12:21:56.546774 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6043281459997161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1545s for a response
W20260709 12:21:56.548488 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4144281459998638s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 12:21:56.548853 13693 connection.cc:256] RPC call timeout handler was delayed by -1.349628145999759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:21:56.549165 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3091281459996935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
W20260709 12:21:56.549532 13693 connection.cc:256] RPC call timeout handler was delayed by -1.290228145999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 12:21:56.549916 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2731281459996353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 12:21:56.546774 15993 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.449712 16076 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:56.432600 16159 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:56.436132 16157 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.434778 15989 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:21:56.551695 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2677281459996266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260709 12:21:56.566886 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0400281459997132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
I20260709 12:21:56.553256 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:56.447240 16085 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:56.571713 13643 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
W20260709 12:21:56.571797 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 263414us
I20260709 12:21:56.438367 16130 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 12:21:56.583701 15983 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:56.584309 15983 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:56.585246 15983 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.586855 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.587795 14416 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:56.456578 16158 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:56.464864 16098 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.542672 15968 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:56.438455 16168 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.162s
I20260709 12:21:56.574837 16182 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:56.451236 16165 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.176s
W20260709 12:21:56.610572 13693 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.507s (SENT)
W20260709 12:21:56.611544 13693 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.525s (SENT)
I20260709 12:21:56.612075 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:56.612329 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:56.613250 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2270) took 1594 ms (client timeout 2089 ms). Trace:
W20260709 12:21:56.613387 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:56.613677 13799 rpcz_store.cc:269] 0709 12:21:55.018747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.612025 (+1593278us) service_pool.cc:224] Handling call
0709 12:21:56.613173 (+  1148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:21:56.430506 16162 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.159s
I20260709 12:21:56.628482 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.631320 13720 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:56.599689 13793 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:56.575037 13646 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:21:56.555087 16050 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.601490 16170 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.601200 16159 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.604858 16018 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:56.605772 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.642287 13726 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:56.606551 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.606638 16179 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.644455 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:56.645359 13723 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:56.607921 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:56.591013 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.943s (SENT)
W20260709 12:21:56.620988 14391 connection.cc:256] RPC call timeout handler was delayed by -1.5485505189999458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
I20260709 12:21:56.576129 16174 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:56.637890 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:56.646746 14391 connection.cc:256] RPC call timeout handler was delayed by -1.5377505189999283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 12:21:56.646653 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:56.650475 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.646644 13724 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:56.650348 13720 raft_consensus.cc:2376] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6. Re-sending same reply.
W20260709 12:21:56.651006 14391 connection.cc:256] RPC call timeout handler was delayed by -1.5038958270001785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 12:21:56.613503 16085 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.646869 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:56.659360 14391 connection.cc:256] RPC call timeout handler was delayed by -1.3010505190000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 12:21:56.659816 14391 connection.cc:256] RPC call timeout handler was delayed by -1.2668505189999451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19030000000000002s for a response
W20260709 12:21:56.660148 14391 connection.cc:256] RPC call timeout handler was delayed by -1.237150518999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
I20260709 12:21:56.660319 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.661226 13720 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:56.446803 16129 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.655400 16174 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:56.656267 16177 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:21:56.657294 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66307.1us
W20260709 12:21:56.669530 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.961s (SENT)
I20260709 12:21:56.657498 16155 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.627315 16182 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.651983 16158 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.601549 16008 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.653728 16145 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:56.662827 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 205053us
I20260709 12:21:56.652230 16169 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.664989 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:56.667943 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:56.639284 15993 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.746294 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1760) took 2244 ms. Trace:
W20260709 12:21:56.732031 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 120173us
I20260709 12:21:56.779959 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:56.780813 13693 leader_election.cc:336] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.530s (SENT)
I20260709 12:21:56.780043 13648 rpcz_store.cc:276] 0709 12:21:54.501763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.503000 (+  1237us) service_pool.cc:224] Handling call
0709 12:21:54.503320 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.155009 (+1651689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.746018 (+591009us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:56.746208 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3797079936}
I20260709 12:21:56.682772 13720 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:21:56.781488 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.782334 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.694482 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:56.731127 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:56.788232 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57211.2us
I20260709 12:21:56.790069 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:56.705924 14450 heartbeater.cc:646] Failed to heartbeat to 127.12.45.126:44051 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.45.126:44051 timed out after 15.000s (SENT)
W20260709 12:21:56.807277 13693 leader_election.cc:336] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.545s (SENT)
W20260709 12:21:56.693324 16181 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:21:56.664541 16183 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.811143 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:56.805023 15891 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:21:56.704159 16008 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.809371 16191 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.813436 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:56.791178 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 96785.8us
I20260709 12:21:56.745877 16085 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:56.805401 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:56.808382 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.805485 13724 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:56.664687 16170 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.836040 16174 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.836205 16170 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.838294 16127 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:56.838771 16127 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:56.839675 16127 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.843621 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.665428 16179 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.817431 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:56.804667 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:56.849216 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44711.8us
I20260709 12:21:56.810473 15993 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:21:56.849740 13693 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.572s (SENT)
I20260709 12:21:56.666450 16126 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.148s
I20260709 12:21:56.817119 16157 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:56.823493 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:56.826617 16023 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:56.662355 16018 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.791123 16162 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:56.693797 16076 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.785853 15902 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:56.856317 16182 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.666560 16185 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:56.791807 16050 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.805282 15968 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:21:56.873521 16185 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.842901 16008 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.822830 16078 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:56.732023 16155 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
W20260709 12:21:56.868664 16009 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.906s	user 0.009s	sys 0.004s
W20260709 12:21:56.870988 16086 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.599s	user 0.007s	sys 0.006s
I20260709 12:21:56.867686 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1763) took 2362 ms. Trace:
I20260709 12:21:56.894618 13645 rpcz_store.cc:276] 0709 12:21:54.504856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.508438 (+  3582us) service_pool.cc:224] Handling call
0709 12:21:54.535259 (+ 26821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:55.846581 (+1311322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.867382 (+1020801us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400198a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:56.867613 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730397440}
I20260709 12:21:56.894629 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:56.896361 16185 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.898458 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 409) took 1052 ms. Trace:
I20260709 12:21:56.899001 16057 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.898967 13719 rpcz_store.cc:276] 0709 12:21:55.845789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.147795 (+302006us) service_pool.cc:224] Handling call
0709 12:21:56.148071 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.280670 (+132599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.898183 (+617513us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001a5254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:56.898383 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303572352}
W20260709 12:21:56.900063 16128 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.004s
I20260709 12:21:56.851667 15891 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.859844 16047 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.864812 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:56.848174 15964 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:56.850023 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:21:56.855706 13693 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.610s (SENT)
I20260709 12:21:56.848358 16127 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:56.886761 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.887476 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.875311 16162 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:56.866469 16018 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.888445 14416 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:56.859207 16023 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:56.888609 15894 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:56.905288 14391 leader_election.cc:336] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.590s (SENT)
I20260709 12:21:56.858623 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:56.908934 13798 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:56.939049 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:56.939905 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81392.1us
I20260709 12:21:56.869460 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:56.941251 13795 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:56.859895 16183 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:56.858830 16157 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:56.870954 16145 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.914332 13799 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
W20260709 12:21:56.916332 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60657.1us
I20260709 12:21:56.934334 16162 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:56.886077 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:56.927881 13726 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:56.869655 16144 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.881695 16078 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:56.936657 16179 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:56.922431 16185 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:56.944397 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:56.916410 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66473.3us
I20260709 12:21:56.973459 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:56.974753 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:56.949616 16190 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:21:56.949513 16023 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:56.947250 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:56.940793 15902 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:21:56.968488 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2265) took 2013 ms (client timeout 1932 ms). Trace:
W20260709 12:21:56.986207 13797 rpcz_store.cc:269] 0709 12:21:54.955575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.955757 (+   182us) service_pool.cc:224] Handling call
0709 12:21:56.968197 (+2012440us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400102554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:56.968377 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2966074752}
I20260709 12:21:56.977099 15891 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:56.933848 14420 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:56.979862 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:56.942973 15968 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:21:56.946511 16157 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:56.988335 13797 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:57.001504 15982 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:57.004644 16200 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:21:57.007836 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 255) took 2197 ms. Trace:
I20260709 12:21:56.952389 16169 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:56.978075 16086 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:60824 (request call id 9) took 6549 ms. Trace:
I20260709 12:21:57.008921 16086 rpcz_store.cc:276] 0709 12:21:50.409260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:50.416988 (+  7728us) service_pool.cc:224] Handling call
0709 12:21:50.541747 (+124759us) log.cc:844] Serialized 499 byte log entry
0709 12:21:51.338865 (+797118us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b180038e060. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6d142 00007f5393b4deaa 00007f5393b54d53
0709 12:21:51.340515 (+  1650us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b540031c7a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6d142 00007f5393b4deaa 00007f5393b54d53 00007f5393b54cd9 00007f5393b54c69 00007f5393b54c31 00007f5393b53dfd 0000000000388634 0000000000388479 00007f5390b07475
0709 12:21:51.514523 (+174008us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b180038e060. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:51.570669 (+ 56146us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b540031c7a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6d142 00007f5393b4deaa 00007f5393b54d53 00007f5393b54cd9 00007f5393b54c69 00007f5393b54c31 00007f5393b53dfd 0000000000388634 0000000000388479 00007f5390b07475
0709 12:21:53.013231 (+1442562us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b4c00092900. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5391ec0a20 00007f5391ed1a29 00007f5391ed213f 00007f5391e962ab 00007f5391ec3f3c 00007f5391ec43fe 00007f539215150b 00007f5393b50381 00007f5393b4e26b 00007f5393b54d53 00007f5393b54cd9
0709 12:21:56.945213 (+3931982us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":3216,"raft.queue_time_us":89962,"raft.run_cpu_time_us":39127,"raft.run_wall_time_us":2802904,"spinlock_wait_cycles":309666560,"thread_start_us":204868,"threads_started":1,"wal-append.queue_time_us":312974,"wal-append.run_cpu_time_us":2661,"wal-append.run_wall_time_us":1022998}
I20260709 12:21:57.012990 16157 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:56.980577 16141 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:57.006124 16152 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:56.996681 15844 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:56.960328 15939 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:57.016904 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 76731.8us
I20260709 12:21:56.991676 15623 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:57.018759 16141 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:57.019630 15891 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:56.992899 16023 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:57.021585 15844 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:56.977495 16078 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.029501 16078 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:57.029201 15844 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.030349 15844 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:57.027726 16141 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:57.034492 16157 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:57.022624 16152 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.038486 16129 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.045776 16162 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:57.049140 16198 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
W20260709 12:21:57.049187 16196 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 12:21:57.053668 16152 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:57.055155 16162 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:57.057396 16141 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:56.947576 13693 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.707s (SENT)
I20260709 12:21:56.940618 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:57.098757 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:57.102908 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 155341us
W20260709 12:21:57.104012 13693 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.756s (SENT)
I20260709 12:21:57.104679 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:57.101426 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 160641us
I20260709 12:21:57.106923 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:56.975966 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:56.983410 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:56.970206 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 261) took 1841 ms (client timeout 1590 ms). Trace:
I20260709 12:21:57.109993 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2150) took 2027 ms. Trace:
I20260709 12:21:57.132354 13718 rpcz_store.cc:276] 0709 12:21:55.082131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.080023 (+997892us) service_pool.cc:224] Handling call
0709 12:21:56.080466 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:56.100295 (+ 19829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:56.100341 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:56.100814 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:21:56.293859 (+193045us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:21:56.370770 (+ 76911us) log.cc:844] Serialized 14 byte log entry
0709 12:21:56.371113 (+   343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:56.371220 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:56.705728 (+334508us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b180007be80. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:57.030044 (+324316us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180007be80. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:21:57.109105 (+ 79061us) raft_consensus.cc:1685] finished
0709 12:21:57.109195 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:57.109644 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":77605,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":15697,"raft.queue_time_us":141,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15104,"spinlock_wait_cycles":105405952,"thread_start_us":147113,"threads_started":1,"wal-append.queue_time_us":280166}
I20260709 12:21:57.110311 13725 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:21:57.138294 13725 pending_rounds.cc:85] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 3
I20260709 12:21:57.138631 13725 pending_rounds.cc:107] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:21:57.138999 13725 raft_consensus.cc:2889] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:57.008394 13801 rpcz_store.cc:276] 0709 12:21:54.810222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.811279 (+  1057us) service_pool.cc:224] Handling call
0709 12:21:54.811693 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:55.079476 (+267783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:55.079514 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:55.828649 (+749135us) log.cc:844] Serialized 26 byte log entry
0709 12:21:55.970208 (+141559us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:55.970818 (+   610us) log.cc:844] Serialized 14 byte log entry
0709 12:21:57.007032 (+1036214us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b180035f4a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:21:57.007213 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.007322 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:57.007387 (+    65us) raft_consensus.cc:1685] finished
0709 12:21:57.007541 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:57.007744 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8058,"prepare.queue_time_us":139,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":245403,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":84631,"spinlock_wait_cycles":51283200,"thread_start_us":429382,"threads_started":3,"wal-append.queue_time_us":225055}
I20260709 12:21:57.139274 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:57.140487 13652 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:57.140533 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 260) took 2119 ms (client timeout 1936 ms). Trace:
I20260709 12:21:57.120261 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 383) took 3301 ms. Trace:
W20260709 12:21:57.141422 13800 rpcz_store.cc:269] 0709 12:21:55.021433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.140010 (+2118577us) service_pool.cc:211] Skipping call since client already timed out
0709 12:21:57.140242 (+   232us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:21:57.188504 13652 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:57.199004 15649 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:57.153376 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:57.123243 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 147185us
W20260709 12:21:57.108306 13693 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.828s (SENT)
W20260709 12:21:57.155311 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.177753 15623 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:57.143227 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2142) took 2063 ms. Trace:
I20260709 12:21:57.182744 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:57.187254 13801 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:57.208513 13720 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:21:57.227033 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.163880 13724 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:57.141695 13722 rpcz_store.cc:276] 0709 12:21:53.810693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:53.812491 (+  1798us) service_pool.cc:224] Handling call
0709 12:21:55.264222 (+1451731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:55.836721 (+572499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.111351 (+1274630us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400174b50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:57.111788 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3337782656}
W20260709 12:21:57.172513 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:57.148595 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2264) took 2233 ms (client timeout 1678 ms). Trace:
I20260709 12:21:57.150584 13726 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:57.159669 13649 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:57.220685 15900 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:57.143131 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:57.232002 13645 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:21:57.131151 13792 rpcz_store.cc:269] 0709 12:21:55.129331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.969895 (+1840564us) service_pool.cc:211] Skipping call since client already timed out
0709 12:21:56.970092 (+   197us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:21:57.174866 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.216580 15891 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.235193 13725 rpcz_store.cc:276] 0709 12:21:55.079717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.060579 (+980862us) service_pool.cc:224] Handling call
0709 12:21:56.061008 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:57.139804 (+1078796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.142200 (+  2396us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000f6754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:57.143125 (+   925us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2409978624}
I20260709 12:21:57.127410 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:57.235917 13801 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.242203 13724 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:21:57.238361 13798 rpcz_store.cc:269] 0709 12:21:54.939733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.948526 (+  8793us) service_pool.cc:224] Handling call
0709 12:21:57.148051 (+2199525us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54003e17d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:57.148312 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4276211584}
I20260709 12:21:57.232227 16048 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:21:57.254894 16179 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:21:57.257133 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 114459us
I20260709 12:21:57.257499 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:57.266942 13545 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 150) took 16626 ms (client timeout 15000 ms). Trace:
I20260709 12:21:57.267344 16155 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:21:57.268302 16155 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:57.271029 16144 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:57.271960 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163779us
W20260709 12:21:57.273000 13693 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.873s (SENT)
I20260709 12:21:57.273540 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:57.286749 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 133887us
W20260709 12:21:57.292749 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 165532us
I20260709 12:21:57.294008 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:21:57.296578 16152 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:57.296734 15623 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:21:57.298420 13795 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:21:57.298413 15623 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:57.298913 13795 pending_rounds.cc:85] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 4
I20260709 12:21:57.300719 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:57.305480 13795 pending_rounds.cc:107] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:21:57.305804 13795 raft_consensus.cc:2889] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:57.309298 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2275) took 1020 ms. Trace:
I20260709 12:21:57.310035 13722 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:57.310703 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:57.294018 15968 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:57.309794 13797 rpcz_store.cc:276] 0709 12:21:56.288965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:56.987450 (+698485us) service_pool.cc:224] Handling call
0709 12:21:56.987895 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:56.988179 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:21:57.198136 (+209957us) log.cc:844] Serialized 14 byte log entry
0709 12:21:57.305000 (+106864us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:57.305107 (+   107us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:21:57.305192 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.307105 (+  1913us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:57.309065 (+  1960us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43930,"thread_start_us":59802,"threads_started":1}
I20260709 12:21:57.317517 13652 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:57.307955 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:57.299425 16157 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:57.322553 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:57.303596 15649 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.323012 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:57.270138 16155 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:57.329632 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46088.9us
I20260709 12:21:57.300508 15623 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.333104 13725 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:21:57.335953 13720 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:57.341643 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68789us
I20260709 12:21:57.335973 16157 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:57.343940 16141 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:57.344347 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:57.339164 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:21:57.337754 13724 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.347774 16130 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:57.333135 16144 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:21:57.348515 13724 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.329666 16145 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:57.351676 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:57.348915 16015 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:57.340037 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:57.356185 14417 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:57.352270 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:57.358531 13799 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:57.334038 15968 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.359436 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:57.364562 16130 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:57.364073 13801 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.349596 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:57.369904 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:57.349514 16152 consensus_queue.cc:1175] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 5 status { last_received { term: 5 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:21:57.373728 14420 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:21:57.376457 15968 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:57.378057 16057 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:57.378795 16057 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:57.379511 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:57.380039 16057 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.344360 16157 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.384562 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:57.385346 13693 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.913s (SENT)
I20260709 12:21:57.385950 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:21:57.387724 13693 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.919s (SENT)
I20260709 12:21:57.387895 16114 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:57.388270 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:57.389410 16008 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:57.389930 13693 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.172s (SENT)
I20260709 12:21:57.390465 16008 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:57.367146 16130 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.391148 13801 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.391476 13800 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:21:57.391927 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:57.398267 13798 raft_consensus.cc:2376] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6. Re-sending same reply.
I20260709 12:21:57.388409 16114 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:57.402697 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74241us
I20260709 12:21:57.401213 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:57.405892 14420 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:57.407292 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:57.408015 13648 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:57.409168 13726 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:21:57.407389 13598 auto_rebalancer.cc:629] Failed to schedule move for tablet 48612be3c56f482c81533c30abebe60b: Illegal state: Replica b4f6bfcda63148e4965ca02c639336f1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:21:57.414796 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 176296us
I20260709 12:21:57.414368 13795 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6. Re-sending same reply.
I20260709 12:21:57.422592 13718 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:57.430800 15939 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:57.409739 16114 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:57.267751 13545 rpcz_store.cc:269] 0709 12:21:40.626530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:40.627406 (+   876us) service_pool.cc:224] Handling call
0709 12:21:57.250753 (+16623347us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:21:50.967636 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:21:51.138570 (+170934us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:21:51.140862 (+  2292us) write_op.cc:435] Acquired schema lock
0709 12:21:51.143177 (+  2315us) tablet.cc:662] Decoding operations
0709 12:21:51.144108 (+   931us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:21:51.144194 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:21:51.144246 (+    52us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:21:51.193831 (+ 49585us) tablet.cc:701] Row locks acquired
0709 12:21:51.193851 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:21:51.209466 (+ 15615us) write_op.cc:270] Start()
0709 12:21:51.245800 (+ 36334us) write_op.cc:276] Timestamp: P: 1783599711206184 usec, L: 0
0709 12:21:51.245835 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:21:51.347922 (+102087us) log.cc:844] Serialized 1674 byte log entry
0709 12:21:51.743953 (+396031us) op_driver.cc:464] REPLICATION: finished
0709 12:21:52.062128 (+318175us) write_op.cc:301] APPLY: starting
0709 12:21:52.062247 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 12:21:52.062465 (+   218us) tablet.cc:1370] finished BulkCheckPresence
0709 12:21:52.082387 (+ 19922us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:21:52.173825 (+ 91438us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:21:52.229640 (+ 55815us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:21:52.240036 (+ 10396us) write_op.cc:312] APPLY: finished
0709 12:21:52.329690 (+ 89654us) log.cc:844] Serialized 41 byte log entry
0709 12:21:52.407934 (+ 78244us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:52.433724 (+ 25790us) write_op.cc:454] Released schema lock
0709 12:21:52.496498 (+ 62774us) write_op.cc:341] FINISH: Updating metrics
0709 12:21:52.866457 (+369959us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:21:52.874999 (+  8542us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":264205,"apply.run_cpu_time_us":6328,"apply.run_wall_time_us":768553,"num_ops":4,"prepare.queue_time_us":392800,"prepare.run_cpu_time_us":25253,"prepare.run_wall_time_us":509063,"raft.queue_time_us":49328,"raft.run_cpu_time_us":18658,"raft.run_wall_time_us":198543,"replication_time_us":494960,"thread_start_us":607443,"threads_started":4,"wal-append.queue_time_us":344218,"wal-append.run_cpu_time_us":10342,"wal-append.run_wall_time_us":1701312}]]}
I20260709 12:21:57.442711 16078 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:57.443267 16078 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.443683 16078 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:57.448091 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:57.448859 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:57.453598 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63723.7us
W20260709 12:21:57.454546 13693 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.228s (SENT)
I20260709 12:21:57.454654 16176 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.455163 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:57.455659 16176 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.456858 16152 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:57.457928 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:57.458703 16144 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:57.459949 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:57.459627 16144 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.460860 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:57.462723 16144 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:57.463363 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:57.459628 16152 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:57.464624 16152 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.423472 16008 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:21:57.449894 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:57.465211 16152 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.418776 15968 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:57.469624 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:57.446622 16141 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:21:57.470134 16114 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.486660 14419 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:57.482962 16144 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:21:57.487739 16048 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:21:57.490233 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69131.7us
I20260709 12:21:57.489362 16048 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:57.491564 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:57.487999 16144 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:57.481225 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:57.497843 16125 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:57.497936 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:57.499017 13649 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:57.498768 16130 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.481300 15939 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.491466 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:57.493512 16048 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.484894 16141 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.504487 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:57.516639 14417 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.516973 13727 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:21:57.519755 13727 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:21:57.520303 13722 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260709 12:21:57.528003 16176 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:57.536612 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:57.537811 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.539319 16129 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:21:57.540086 13797 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:57.540609 16129 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:57.541015 16129 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.496006 15968 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.544198 15968 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:57.451529 15649 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.503145 16125 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.496809 16144 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.563987 13720 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:21:57.567512 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72265us
W20260709 12:21:57.575883 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78157.7us
I20260709 12:21:57.574410 13801 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:21:57.580461 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86762.6us
I20260709 12:21:57.584293 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:57.580726 13801 pending_rounds.cc:85] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 5
I20260709 12:21:57.585774 13801 pending_rounds.cc:107] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:21:57.586125 13801 raft_consensus.cc:2889] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:57.559482 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:57.588567 13718 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:21:57.575069 16125 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:57.590963 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:57.599635 13724 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:21:57.550084 16057 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.569377 14418 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:57.600087 13724 pending_rounds.cc:85] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:21:57.606516 13724 pending_rounds.cc:107] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:21:57.606829 13724 raft_consensus.cc:2889] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:21:57.602651 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:57.609548 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:57.608163 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:57.608809 13649 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:57.601426 16145 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:57.630963 13644 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:57.542182 16129 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.613509 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:57.628690 16130 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.628506 16145 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:57.650553 13724 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:57.649562 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:57.667361 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.643067 15968 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:21:57.672631 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60190.2us
W20260709 12:21:57.683736 16204 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
W20260709 12:21:57.685887 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99045us
I20260709 12:21:57.688105 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:57.641547 15894 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.551s	user 0.016s	sys 0.004s
I20260709 12:21:57.689440 15982 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:21:57.689944 16127 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:57.695344 16169 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:21:57.696336 16169 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:21:57.696911 16169 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:21:57.697229 16169 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:57.689496 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:57.655542 16145 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:57.718195 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 128245us
I20260709 12:21:57.640231 16078 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.669497 16008 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:21:57.720139 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137092us
I20260709 12:21:57.725191 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:57.654191 13645 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
W20260709 12:21:57.731086 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.735635 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:57.717724 13661 tablet_copy_service.cc:140] P 4dafe08a4f404502822a0732fe570de4: Received BeginTabletCopySession request for tablet 56f25c6fe9f245d99ae467bdd3948582 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:60832)
I20260709 12:21:57.743042 16176 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:57.751448 16176 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:57.752678 16176 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.754447 15844 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:21:57.756058 16036 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:21:57.758824 16161 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.759575 16161 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.768445 13796 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:57.776881 15894 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:21:57.778085 15894 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:57.778509 15894 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:57.780299 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53741.2us
I20260709 12:21:57.782163 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:57.788354 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66282.7us
I20260709 12:21:57.795775 16086 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:57.796715 16086 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:57.802628 15989 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:57.803306 15989 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.761967 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 263) took 1780 ms. Trace:
I20260709 12:21:57.805663 13794 rpcz_store.cc:276] 0709 12:21:55.938073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:55.974954 (+ 36881us) service_pool.cc:224] Handling call
0709 12:21:55.984396 (+  9442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:57.135234 (+1150838us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b180035f320. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393becbfb 00007f5393bf764b
0709 12:21:57.135700 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:57.135752 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:57.137215 (+  1463us) log.cc:844] Serialized 26 byte log entry
0709 12:21:57.266556 (+129341us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:21:57.291585 (+ 25029us) log.cc:844] Serialized 14 byte log entry
0709 12:21:57.327104 (+ 35519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:57.330407 (+  3303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:21:57.675517 (+345110us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:21:57.698485 (+ 22968us) raft_consensus.cc:1685] finished
0709 12:21:57.698515 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:21:57.717863 (+ 19348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44378,"prepare.queue_time_us":8981,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":70,"raft.queue_time_us":161070,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":8050,"spinlock_wait_cycles":107843200,"thread_start_us":290204,"threads_started":2,"wal-append.queue_time_us":266734}
I20260709 12:21:57.758951 13646 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:57.755914 15844 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:57.813064 15844 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:21:57.815706 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 222888us
I20260709 12:21:57.726174 16008 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:21:57.801632 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66112.9us
I20260709 12:21:57.755898 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:57.843462 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54368.9us
I20260709 12:21:57.821467 16008 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:57.825733 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:57.833042 15891 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:57.843550 16129 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:57.761986 14420 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:21:57.881860 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:21:57.869634 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113824us
I20260709 12:21:57.882767 13643 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:57.724128 16169 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.779673 15894 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.825712 16179 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:21:57.897030 16067 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:21:57.898087 14416 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:21:57.898414 13651 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:21:57.898851 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65435.7us
I20260709 12:21:57.899567 16212 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:57.904145 16127 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:21:57.727021 16141 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:57.922434 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:57.819805 16144 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.855168 16152 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:57.933027 16078 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:57.938649 16156 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:21:57.930339 15891 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:21:57.946897 15891 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:57.947618 16161 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:57.948014 15891 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.952910 15989 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:57.826997 15844 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:57.952828 14417 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:57.959376 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:21:57.958864 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86581us
I20260709 12:21:57.938475 15894 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:21:57.974622 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 152354us
I20260709 12:21:57.931180 16057 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:57.999068 16209 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:21:57.999742 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:58.000339 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:58.007273 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 123058us
I20260709 12:21:58.009328 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:58.011567 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:58.012539 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.551s (SENT)
W20260709 12:21:58.014613 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 131455us
I20260709 12:21:57.865464 16086 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.020417 13724 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.027040 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:58.016423 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:58.010514 16141 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:57.992847 16078 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:21:58.040542 16078 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:21:58.041985 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.038503 13649 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:58.046617 13722 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:58.027477 13652 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:21:58.034794 16020 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:21:58.017177 16067 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.051532 13726 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:58.051905 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:58.013962 16212 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:58.052848 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:58.054057 13723 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.054477 16212 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:58.062733 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60562.5us
W20260709 12:21:58.063627 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.570s (SENT)
I20260709 12:21:58.064031 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:58.065311 13723 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:21:58.067838 15968 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:58.031509 15623 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:21:58.053097 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:58.028308 15844 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:21:58.066140 13693 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.230s (SENT)
I20260709 12:21:58.051656 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.075413 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:58.080288 16125 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.087119 13647 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:58.090386 16057 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:58.115628 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 233900us
W20260709 12:21:58.091001 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2750984750000498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 12:21:58.087497 15968 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:58.151005 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.091127 16152 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:58.165761 16020 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:58.174207 15982 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:58.173009 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.188059 16238 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:21:58.165812 16223 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.189967 16224 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:21:58.194295 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 158037us
W20260709 12:21:58.203915 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.204766 13793 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:21:58.205538 13793 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:58.206868 13793 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.208228 13793 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.117429 16067 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.236701 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.191617 16144 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.182123 16020 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:58.185554 16057 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.190438 16078 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:21:58.232806 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 169237us
I20260709 12:21:58.192870 16224 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:58.255687 13794 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:58.255133 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.260679 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:21:58.267870 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.275807 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.281479 13646 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:58.191027 16145 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.254088 16238 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:58.190326 16223 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:58.298170 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:58.300735 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1517984749998504s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 12:21:58.303673 13801 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 5. Ops: []
I20260709 12:21:58.304565 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:58.309082 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:58.311254 13794 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.314236 13799 raft_consensus.cc:2368] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:21:58.294145 16157 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.321655 16161 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:21:58.322192 16161 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.322602 16161 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:21:58.322732 13650 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.327160 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.331881 13801 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
W20260709 12:21:58.319235 13693 leader_election.cc:336] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.857s (SENT)
I20260709 12:21:58.334702 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:21:58.336288 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 78331us
I20260709 12:21:58.339398 13793 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:21:58.336608 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 309022us
I20260709 12:21:58.315785 15891 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:58.345809 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.317528 15649 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.348388 13792 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:21:58.351544 16157 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.356117 13799 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:58.308295 16057 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:21:58.355985 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 339676us
W20260709 12:21:58.347419 16130 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.006s
I20260709 12:21:58.364156 16145 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:58.365054 16230 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:21:58.358687 16020 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.369017 16245 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:21:58.314417 16008 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.369313 16020 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.344238 16067 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.345181 15891 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.379222 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.380364 16227 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.380585 16048 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.807s	user 0.012s	sys 0.003s
I20260709 12:21:58.380702 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.390676 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:58.381001 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61804.6us
I20260709 12:21:58.383800 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.387573 13801 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4.
I20260709 12:21:58.395596 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:58.400321 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91547.7us
I20260709 12:21:58.376314 16125 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:58.404567 13693 leader_election.cc:336] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.956s (SENT)
I20260709 12:21:58.404889 13643 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:21:58.358940 16224 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.402170 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.384675 13645 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:58.407881 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:58.410724 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:58.408440 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.406395 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.406044 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.427059 13800 raft_consensus.cc:2410] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:21:58.402825 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.427381 13799 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:21:58.390187 15891 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.411221 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.361950 15649 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.431474 13792 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:21:58.416692 16114 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.926s	user 0.010s	sys 0.005s
W20260709 12:21:58.411067 16225 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:21:58.439647 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.440344 16169 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:58.439682 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:58.455883 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:58.437542 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.678s (SENT)
I20260709 12:21:58.464915 13801 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:21:58.444872 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.408010 16230 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.466794 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.467084 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.445575 13724 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6. Re-sending same reply.
I20260709 12:21:58.475857 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.489699 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:58.496125 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44439us
I20260709 12:21:58.499634 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:58.499939 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.379760 16238 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.500654 13726 raft_consensus.cc:2376] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5. Re-sending same reply.
I20260709 12:21:58.398948 16048 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.463811 16125 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:58.508589 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.680s (SENT)
I20260709 12:21:58.492928 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:58.499637 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.504098 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:21:58.522455 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.524551 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:58.524174 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:58.531117 13725 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.426102 16227 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.531268 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.502949 16223 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.532855 13727 raft_consensus.cc:2368] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6.
W20260709 12:21:58.537735 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:58.539497 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47048.4us
I20260709 12:21:58.500402 16114 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:21:58.543845 13647 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:58.544665 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:21:58.547178 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.547259 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.503335 16067 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:21:58.561033 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.561913 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.559948 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:58.563640 13719 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.565068 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.565229 16114 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:58.565505 16251 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:21:58.565696 16114 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:58.565820 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:58.566179 16251 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:21:58.566380 16240 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.566956 16240 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.567667 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:58.568353 13801 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.571590 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.572185 16231 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.147s
I20260709 12:21:58.574450 16227 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.575886 16224 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.579530 16048 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:58.579805 15939 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:58.581323 16237 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:58.583215 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.584738 14417 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.591777 15649 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.496775 16230 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.529085 16141 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:21:58.599685 16230 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:58.601122 16008 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:58.601359 13646 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
W20260709 12:21:58.602283 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97435.8us
I20260709 12:21:58.602591 13722 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:58.553360 16157 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:58.604032 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:58.607242 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:58.608224 16245 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.608891 13727 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:21:58.609779 13724 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:21:58.610298 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.610641 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.611016 13720 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:21:58.611406 13726 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:58.583217 13644 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:58.610409 16023 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:58.624927 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:58.608153 13661 tablet_copy_service.cc:161] P 4dafe08a4f404502822a0732fe570de4: Beginning new tablet copy session on tablet 56f25c6fe9f245d99ae467bdd3948582 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:60832: session id = b708d8f5f6ac4aabb1eb5cb821ace195-56f25c6fe9f245d99ae467bdd3948582
I20260709 12:21:58.567278 16251 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.522359 16125 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.608884 16245 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.632405 16141 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:58.611349 16114 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.663378 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:58.650879 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:58.651162 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:58.650701 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46758.8us
I20260709 12:21:58.669994 13644 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:21:58.658872 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 99412.1us
I20260709 12:21:58.645560 13795 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:58.662708 16008 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:58.679006 16008 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.688983 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:58.629433 16023 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:58.668892 16237 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:21:58.698737 16258 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:58.716571 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.717094 16258 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.648144 16245 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.719642 16258 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:58.720095 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.721280 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.698613 15891 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.658391 16127 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 12:21:58.715421 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.603s (SENT) [suppressed 34 similar messages]
I20260709 12:21:58.658475 16176 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:58.715972 16020 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:21:58.717408 16015 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:58.723336 13800 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:21:58.614707 16240 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:58.708276 16114 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:58.726148 13801 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:21:58.716547 16237 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:58.721341 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.643388 16155 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:21:58.732363 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.733522 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:58.653082 16141 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.659780 16057 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:58.707012 16129 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.820s	user 0.014s	sys 0.009s
I20260709 12:21:58.740382 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:58.741333 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 105111us
I20260709 12:21:58.744521 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.724174 16023 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:58.779386 13647 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:58.706624 16008 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:58.753504 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.756016 13799 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:58.757637 13797 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:21:58.758483 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.746611 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:58.768817 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.769320 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:58.740955 16245 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:58.765442 13646 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:58.742842 16015 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:21:58.778648 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90789.5us
I20260709 12:21:58.784605 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.789821 13799 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:58.786636 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:58.790571 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.748436 16223 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:21:58.791839 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68581.5us
I20260709 12:21:58.793059 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.764221 16161 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.762243 16141 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.762311 16176 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.756410 16237 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:58.797626 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:58.798609 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.799748 13800 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:21:58.800284 16223 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.787173 16217 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 12:21:58.806767 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:58.808252 16223 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:58.809217 15844 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.727s	user 0.009s	sys 0.005s
I20260709 12:21:58.810611 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.811834 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.811962 14420 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:58.812966 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.812842 16008 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:58.813783 16008 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:58.814476 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:58.815191 14423 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:21:58.816500 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:58.816826 16008 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:58.817229 13647 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:58.787002 16258 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.814044 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:21:58.818198 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:58.823010 13647 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:58.824563 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:58.818121 16258 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.828419 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:58.816500 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:58.819963 16221 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 12:21:58.831063 14420 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:21:58.831852 16161 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.832121 16067 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:58.832572 16067 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:58.832947 16067 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:58.835182 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:58.837153 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.838352 16125 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.839666 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:58.840248 15891 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:58.841251 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:58.841439 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2260) took 3995 ms (client timeout 1980 ms). Trace:
W20260709 12:21:58.841661 16256 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:21:58.841871 13796 rpcz_store.cc:269] 0709 12:21:54.846362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.846545 (+   183us) service_pool.cc:224] Handling call
0709 12:21:58.841209 (+3994664us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:58.841362 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6461721344}
I20260709 12:21:58.843222 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.843654 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:58.844410 13798 raft_consensus.cc:2368] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:21:58.845168 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.838071 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:58.840510 13644 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:58.785082 13722 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:21:58.846220 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.872699 13798 raft_consensus.cc:2368] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:21:58.860410 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:58.861865 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.864660 13799 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:21:58.885871 13792 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
I20260709 12:21:59.135080 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:58.916685 13644 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:21:58.906414 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:58.854748 13794 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:58.789320 16238 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:21:58.913718 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:58.927372 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:59.020354 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 182444us
I20260709 12:21:59.143843 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:59.138238 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:59.145190 13651 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:21:58.862823 13726 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:59.135181 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.138523 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.146853 13800 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:21:58.893392 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.163018 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.148499 13801 raft_consensus.cc:2368] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:21:58.925107 16086 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:59.164592 13644 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:21:59.161580 16238 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:21:58.861824 16152 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.019305 16231 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.127218 16224 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:59.187476 13644 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:21:59.147859 16223 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.058035 16008 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.861999 16057 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.285s
I20260709 12:21:58.998611 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:59.058493 15982 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.214633 15982 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.020690 16048 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.216015 13723 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:21:59.215807 16048 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.217967 16048 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.233551 16048 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:59.183220 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.234233 16048 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.234731 16048 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:59.234685 16238 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:58.848606 16156 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.339s
I20260709 12:21:59.238427 13646 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:59.238335 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.241838 13800 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:59.017228 16233 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.074674 15989 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.992409 16129 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.135478 16244 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:58.988435 16230 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:21:59.148205 15623 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:21:58.938622 16221 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:58.851941 15844 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.319s
I20260709 12:21:59.241334 16008 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.183871 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.198637 13644 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:21:58.895303 15891 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.288s
I20260709 12:21:59.202884 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:59.205435 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61637.5us
I20260709 12:21:59.274096 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:59.194011 16152 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.199735 16224 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.277000 13726 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:21:59.278920 16224 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.284595 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2296) took 1085 ms. Trace:
I20260709 12:21:59.285096 13795 rpcz_store.cc:276] 0709 12:21:58.199378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.199572 (+   194us) service_pool.cc:224] Handling call
0709 12:21:59.284348 (+1084776us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:21:59.284495 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883446656}
I20260709 12:21:59.289435 16008 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:59.290980 16023 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 12:21:59.291498 16023 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.292503 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1638) took 1773 ms. Trace:
I20260709 12:21:59.292950 13725 rpcz_store.cc:276] 0709 12:21:57.519091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.521889 (+  2798us) service_pool.cc:224] Handling call
0709 12:21:57.522200 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:58.531617 (+1009417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.292183 (+760566us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000f7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:21:59.292430 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2319834496}
I20260709 12:21:59.275807 16176 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.295027 16211 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:21:59.295547 16211 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:21:59.296576 16211 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.297785 16211 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:21:59.299050 15891 consensus_queue.cc:1243] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:21:59.300220 13727 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:21:59.301363 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.303072 14418 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:59.212745 13661 tablet_copy_source_session.cc:215] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:21:59.215276 16258 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.182175 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.218135 16057 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:21:59.224300 13652 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:21:59.224972 13723 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:21:59.232090 16067 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.182420 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:59.310020 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.310493 13801 raft_consensus.cc:2368] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:21:59.233742 13647 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:21:59.160591 13550 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.205113 16223 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.308696 16057 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.316102 16258 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:21:59.318961 16258 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.319458 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.320006 13550 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.321070 16067 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.321344 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1796) took 1053 ms. Trace:
I20260709 12:21:59.294114 16023 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.161901 15649 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.289708 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:59.321750 13652 rpcz_store.cc:276] 0709 12:21:58.262813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.262926 (+   113us) service_pool.cc:224] Handling call
0709 12:21:58.688364 (+425438us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:59.223841 (+535477us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acff8c 00007f5390acfef6 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:59.224031 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:59.315631 (+ 91600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.315865 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42001408}
I20260709 12:21:59.330511 13648 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:59.305497 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:59.331560 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.321007 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.255785 16141 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.345623 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:21:59.345978 16266 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.125s	user 0.000s	sys 0.003s
I20260709 12:21:59.322542 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.346400 13723 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:59.346544 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.322064 13796 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:21:59.347738 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.347865 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.348667 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.348762 13795 raft_consensus.cc:2368] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:21:59.349411 13799 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:21:59.349866 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75881.8us
I20260709 12:21:59.243500 16231 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.347759 13798 raft_consensus.cc:2368] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:21:59.322165 13550 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 5, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.261034 16129 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.259747 15989 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.321102 16258 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.340889 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:59.341774 16161 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:21:59.341858 15891 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:21:59.377583 16161 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:21:59.379987 16161 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:21:59.380385 16161 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:59.381208 16161 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.350214 16251 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.343696 15649 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.344921 13661 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:60832 (request call id 362) took 1701 ms. Trace:
I20260709 12:21:59.347301 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.306089 16221 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:59.363956 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.363911 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.364632 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.406682 13723 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:21:59.365443 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.366058 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:59.361689 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:59.408066 13721 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:21:59.408351 13644 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:59.311141 16008 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.267037 16230 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.367898 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:21:59.313143 16224 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:21:59.369750 13550 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 4 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.259955 16233 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.397516 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.404467 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:21:59.416616 13726 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:59.415498 13550 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 6, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:21:59.262219 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.435505 13794 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:21:59.439405 14391 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.001s	sys 0.009s
W20260709 12:21:59.439888 14391 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.001s	sys 0.009s
W20260709 12:21:59.440248 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 533751us
I20260709 12:21:59.446657 13720 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:21:59.447731 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:21:59.449004 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 542681us
I20260709 12:21:59.449251 13795 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:21:59.411415 16179 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.415143 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:59.455670 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.449779 13795 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:59.383951 16251 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.430785 16152 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.435174 13550 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:21:59.478773 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:59.480618 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
W20260709 12:21:59.454134 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:59.489943 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 148390us
W20260709 12:21:59.495141 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 136553us
I20260709 12:21:59.449764 16161 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:59.452443 16224 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:59.505504 13724 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:21:59.505285 13550 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:21:59.495914 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:59.494926 13795 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.490487 16023 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.492079 15844 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.516561 13795 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:59.513109 13719 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:59.432426 16008 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.483206 16141 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.404018 13661 rpcz_store.cc:276] 0709 12:21:57.694570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.717370 (+ 22800us) service_pool.cc:224] Handling call
0709 12:21:58.607903 (+890533us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398b883d8 00007f5397c3c4b4 00007f5397c3c442 00007f5397c3c371
0709 12:21:59.344786 (+736883us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116834688}
I20260709 12:21:59.526410 13799 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:21:59.527465 13793 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:21:59.528404 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49696.1us
I20260709 12:21:59.533187 16152 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:21:59.533609 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:21:59.534463 16228 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:21:59.535874 16228 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:21:59.536223 16228 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.536535 16228 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:21:59.537096 16237 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.704s	user 0.006s	sys 0.005s
I20260709 12:21:59.537555 16237 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.538275 16283 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.538431 16271 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:21:59.538995 16283 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.541559 16283 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.542410 16008 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.524029 13550 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:21:59.495195 16270 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:21:59.554703 13550 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.557124 13550 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 3, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 3 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:59.482442 16179 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.530982 15844 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.575042 16272 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 12:21:59.584280 16114 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.001s
I20260709 12:21:59.584909 15649 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.574710 16048 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:21:59.586313 16233 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.587083 16048 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:59.587478 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91665.4us
I20260709 12:21:59.588405 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:59.589856 13617 leader_election.cc:400] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6.
I20260709 12:21:59.525864 16023 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.590633 16223 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.592370 16023 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.594834 16231 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.595638 16231 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:21:59.595772 16221 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.601029 16179 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.601217 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:21:59.603181 16251 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:21:59.604673 16274 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:21:59.590157 13617 leader_election.cc:403] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:21:59.582803 15989 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.559360 16270 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.619879 13726 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:59.616469 13648 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
W20260709 12:21:59.630774 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 363) took 1935 ms (client timeout 1682 ms). Trace:
I20260709 12:21:59.632020 16224 raft_consensus.cc:2764] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:21:59.631145 13648 rpcz_store.cc:269] 0709 12:21:57.695292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:57.695449 (+   157us) service_pool.cc:224] Handling call
0709 12:21:58.678151 (+982702us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:59.289513 (+611362us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfe9c 00007f5390acfa5b 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:21:59.630668 (+341155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73618432}
W20260709 12:21:59.634023 16268 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.427s	user 0.002s	sys 0.002s
W20260709 12:21:59.634815 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45182.5us
I20260709 12:21:59.616787 13550 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.636041 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:21:59.636981 15982 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.636413 13550 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 4, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 4 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:21:59.614243 13721 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:21:59.639009 13646 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:21:59.639673 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.640825 13651 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:21:59.641117 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:21:59.637908 13550 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.643260 13550 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 6, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.644824 13550 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 4, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.646548 16176 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:59.647018 16176 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.647377 16176 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:21:59.651041 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:21:59.582238 15844 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.587132 16057 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.626667 16270 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:59.667915 16015 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.677737 13799 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:59.649849 16086 raft_consensus.cc:2764] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:21:59.699851 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:21:59.687111 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 145422us
I20260709 12:21:59.672627 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.638187 15982 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6.
I20260709 12:21:59.700672 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:21:59.710273 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:21:59.715955 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 221487us
I20260709 12:21:59.716491 13651 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:21:59.695457 13795 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:21:59.679309 16129 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:21:59.628427 16231 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:21:59.701905 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:59.710680 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:59.718777 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:21:59.722971 13644 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:59.722975 13648 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:21:59.719431 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.725675 14423 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:59.684110 13550 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:21:59.729312 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.735088 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.736042 14421 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:21:59.715895 16156 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.736953 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:59.737181 16156 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.706919 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.738399 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.739171 14419 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:21:59.739624 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:21:59.740132 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.740307 13646 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:59.741400 15623 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:21:59.741838 15623 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:21:59.742202 15623 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:21:59.751952 16156 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.754745 16015 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:21:59.755296 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:21:59.756247 13725 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:21:59.716369 16238 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.762781 16238 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:21:59.765223 16238 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:21:59.706348 16218 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.766137 16218 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:21:59.769594 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65320.1us
I20260709 12:21:59.769928 13644 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:21:59.772114 13643 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
W20260709 12:21:59.779258 13800 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.000s
I20260709 12:21:59.780025 13800 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:21:59.700279 16217 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:21:59.791237 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67889.2us
I20260709 12:21:59.800357 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:59.804863 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:59.808506 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:21:59.810353 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:21:59.815769 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:21:59.826905 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54588.3us
I20260709 12:21:59.829022 13651 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:21:59.831506 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.832223 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:21:59.823689 16156 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:21:59.829948 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:21:59.838495 14424 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:21:59.840111 13550 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 3, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:21:59.726106 16230 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.768605 16218 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:21:59.939946 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:21:59.813254 16129 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.152s
I20260709 12:21:59.840740 15989 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 12:21:59.819217 16217 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.088924 16218 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.089574 16218 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.090837 16217 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.092137 16218 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:21:59.870131 16141 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:00.093282 16141 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:21:59.813216 16242 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:21:59.888685 16281 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.048830 16238 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:00.044188 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.070797 13550 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:00.073249 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:00.077513 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 247636us
I20260709 12:22:00.081759 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 507) took 1354 ms. Trace:
I20260709 12:22:00.082185 16049 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:00.051069 16156 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:00.083366 16286 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.083948 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:00.167682 14424 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:22:00.086400 15649 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:00.088001 13795 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:00.088101 14421 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:22:00.069403 16228 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.001s
W20260709 12:22:00.045791 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 208040us
W20260709 12:22:00.111564 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1592) took 1402 ms (client timeout 1507 ms). Trace:
W20260709 12:22:00.122628 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 316370us
I20260709 12:22:00.123206 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:00.182016 13792 rpcz_store.cc:269] 0709 12:21:58.709314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.731673 (+ 22359us) service_pool.cc:224] Handling call
0709 12:22:00.111289 (+1379616us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x7b54003d4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:00.111488 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190634880}
W20260709 12:22:00.080940 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.038s	sys 0.027s
I20260709 12:22:00.182688 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:00.183041 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.038s	sys 0.027s
W20260709 12:22:00.183420 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 649845us
I20260709 12:22:00.155058 16049 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:00.192710 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.070070 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:00.167029 13800 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:00.198750 13792 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:22:00.173781 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.198940 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:00.165884 13550 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 4 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:00.182430 13799 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:00.199595 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:00.161651 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 117313us
I20260709 12:22:00.200358 13800 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:00.155539 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:00.201303 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:00.202104 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54380.8us
I20260709 12:22:00.154842 16238 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.172307 16156 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:00.178040 16078 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.228158 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:00.135620 16242 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.229209 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.212160 13550 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 5, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:00.213977 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 384614us
I20260709 12:22:00.215929 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.220108 13644 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:00.080432 15623 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.161564 16237 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.080962 16176 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.232884 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:00.233315 16287 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:22:00.226331 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.267010 13652 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:00.238476 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.179803 16286 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.242447 16078 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.201179 16245 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:22:00.226089 16238 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.243955 16277 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:00.189864 16230 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:00.227123 14416 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
W20260709 12:22:00.290402 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:00.179487 16141 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.211356 16280 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:22:00.296463 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:00.297236 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2310) took 1476 ms. Trace:
I20260709 12:22:00.297609 13801 rpcz_store.cc:276] 0709 12:21:58.820813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.502195 (+681382us) service_pool.cc:224] Handling call
0709 12:21:59.516946 (+ 14751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:59.561942 (+ 44996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.561976 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:59.613678 (+ 51702us) log.cc:844] Serialized 26 byte log entry
0709 12:21:59.722495 (+108817us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:21:59.722593 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.722668 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:00.297013 (+574345us) raft_consensus.cc:1685] finished
0709 12:22:00.297033 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.297174 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36769,"raft.queue_time_us":32772,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":52022,"thread_start_us":55815,"threads_started":1,"wal-append.queue_time_us":490716}
I20260709 12:22:00.299913 15989 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:00.300400 15989 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.300761 15989 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:00.154807 14418 rpcz_store.cc:276] 0709 12:21:58.721902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:58.736374 (+ 14472us) service_pool.cc:224] Handling call
0709 12:21:58.855323 (+118949us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:21:58.855491 (+   168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:59.309002 (+453511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.054549 (+745547us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:00.076271 (+ 21722us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051930752}
I20260709 12:22:00.251961 13646 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:00.225847 16129 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:00.190131 16228 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.257843 13797 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:22:00.308542 13797 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 4
I20260709 12:22:00.308935 13797 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:22:00.309262 13797 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:00.187459 15649 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:00.154166 16281 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.262436 13645 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.268383 13643 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:22:00.274911 16291 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:22:00.276130 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 285) took 1454 ms. Trace:
I20260709 12:22:00.275866 13550 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.280529 16288 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:22:00.258004 16258 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.283761 16176 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.341147 13550 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.248517 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.189494 16231 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:00.346369 16078 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.346647 16231 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.347059 16231 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:00.349007 16281 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.350538 16288 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:00.350939 16288 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:00.350871 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.351524 16288 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.351837 13646 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:22:00.352890 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.353732 13646 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 2.
I20260709 12:22:00.354342 16258 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.356794 16258 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.304720 13650 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:00.310653 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55268.7us
I20260709 12:22:00.290877 16254 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.293419 16284 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.335265 14424 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:00.313628 13798 rpcz_store.cc:276] 0709 12:21:58.821459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.526097 (+704638us) service_pool.cc:224] Handling call
0709 12:21:59.562709 (+ 36612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:59.609659 (+ 46950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.609692 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:21:59.610207 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:21:59.718085 (+107878us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:21:59.718126 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:21:59.718199 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:00.275856 (+557657us) raft_consensus.cc:1685] finished
0709 12:22:00.275879 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.276046 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46541,"raft.queue_time_us":47255,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"thread_start_us":107437,"threads_started":1,"wal-append.queue_time_us":258280}
I20260709 12:22:00.344340 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:00.345377 13550 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.360282 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:00.313805 16129 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:00.365223 16254 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.294219 16230 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.379182 13550 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 4, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.377642 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.380982 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:00.367534 16270 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.676s	user 0.014s	sys 0.001s
I20260709 12:22:00.313470 16023 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.411345 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:00.383596 16277 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:00.424424 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1820) took 1092 ms. Trace:
I20260709 12:22:00.377833 15939 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:22:00.350129 16156 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.414546 13650 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:00.408215 13643 raft_consensus.cc:2376] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6. Re-sending same reply.
W20260709 12:22:00.411659 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51497.7us
W20260709 12:22:00.417459 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 246644us
I20260709 12:22:00.386616 15891 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:00.424826 13647 rpcz_store.cc:276] 0709 12:21:59.331534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.331820 (+   286us) service_pool.cc:224] Handling call
0709 12:21:59.332129 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:21:59.649221 (+317092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.649252 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:21:59.649296 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:21:59.649324 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.424149 (+774825us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54002e45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:00.424202 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.424359 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728699264}
I20260709 12:22:00.408052 16254 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.450011 13550 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:00.379673 16223 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:22:00.413520 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.455160 16127 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:22:00.459245 16129 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.379733 16284 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:00.426431 16277 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.462564 16277 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:00.462682 16298 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:00.463483 16284 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:00.463975 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:00.463785 16251 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:00.447490 15891 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.466594 15891 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:00.468453 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:00.469151 16282 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:00.469585 16282 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:00.474969 16296 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:22:00.476220 16294 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:22:00.478607 13550 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.480309 13550 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:00.468441 16223 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:00.483289 16223 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:00.484393 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2314) took 1459 ms. Trace:
I20260709 12:22:00.484234 16223 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.484776 13795 rpcz_store.cc:276] 0709 12:21:59.024706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.718125 (+693419us) service_pool.cc:224] Handling call
0709 12:21:59.719970 (+  1845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:00.174687 (+454717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.484141 (+309454us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:00.484329 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845344000}
I20260709 12:22:00.485502 16223 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.487748 16242 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.424165 16023 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.488417 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.488875 16282 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.489460 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.490103 14417 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:00.490237 16282 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.490391 16086 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.490940 16086 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.491027 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:00.491325 16086 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.491698 14424 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:00.494572 16242 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:00.494930 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:00.495218 16242 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:00.496754 16242 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.497896 16286 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:00.498905 15623 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.499188 16286 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:00.500336 16228 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.459407 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.494593 16156 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.502565 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.464099 16284 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.497416 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.462607 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:00.504516 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 299048us
I20260709 12:22:00.481762 13550 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 3 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:00.506893 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 308059us
W20260709 12:22:00.511760 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49251.2us
I20260709 12:22:00.512305 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:00.512558 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:00.509852 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.513784 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.519883 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:00.521577 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.417172 16270 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.524755 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:00.528940 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.521658 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.533514 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:00.529098 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.535035 14424 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:00.536837 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:00.490540 16023 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.468451 16122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56f25c6fe9f245d99ae467bdd3948582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:00.539855 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:00.520910 16228 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:00.520407 16280 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:00.540839 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 270656us
I20260709 12:22:00.524022 16286 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:00.507620 13646 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:00.520272 16297 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.572971 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:00.577172 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8550663400001213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2525s for a response
I20260709 12:22:00.577472 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.535636 16251 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:00.538513 16244 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.538893 16176 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:00.582230 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.450004 15649 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.586833 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.521023 16284 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.520767 16156 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:22:00.550724 16127 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:00.556033 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:00.610201 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:00.589043 13693 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.549s (SENT)
I20260709 12:22:00.551829 16245 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:00.526511 16221 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 12:22:00.568365 15623 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:00.553076 15844 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.606513 13647 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:00.616012 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:00.569269 16280 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.553176 16298 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:00.621978 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:00.626698 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:00.601307 16141 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:00.580781 16297 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:00.551744 15939 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:22:00.591182 16299 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:00.585163 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:00.627801 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88029.9us
I20260709 12:22:00.720626 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:00.629294 13652 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:00.584905 16244 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:00.555831 16161 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 12:22:00.630645 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72844.8us
I20260709 12:22:00.645565 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.605468 16015 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 12:22:00.634423 16280 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.623908 16286 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:00.623059 16251 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:00.584164 16231 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.736687 13651 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 12:22:00.737802 16231 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.602397 16258 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:00.746160 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:00.746449 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.747010 14424 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:00.720232 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.748402 13719 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:00.624701 16304 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.626897 16303 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:00.645126 16141 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.729809 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:00.731545 16279 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:00.725016 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:00.759616 13723 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:00.741676 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.707742 16297 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.633503 15844 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:22:00.728590 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 139782us
I20260709 12:22:00.728166 16244 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.724817 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.631891 16242 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 12:22:00.747884 16023 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:00.762101 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:00.723454 16288 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.796999 13720 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:00.798916 16288 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.799772 16288 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.800338 16288 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.801052 14424 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:00.793586 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.749601 16303 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.758860 16141 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:00.823113 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:00.783385 15844 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.827665 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.786448 16307 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:00.777981 16279 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.785964 16156 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:00.780889 16231 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:00.769233 13645 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:00.790341 16286 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.828351 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.832831 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:00.835538 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44570.6us
I20260709 12:22:00.844880 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:00.867410 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 302979us
I20260709 12:22:00.868508 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:00.874410 16156 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:00.755373 16304 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.876528 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:00.830787 16299 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.874502 13652 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 12:22:00.818964 16303 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:00.828395 16258 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:00.826874 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 410) took 1139 ms. Trace:
I20260709 12:22:00.845719 16228 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:00.830659 16319 raft_consensus.cc:2764] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:00.826706 16245 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:00.832820 16322 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:00.870142 15989 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
W20260709 12:22:00.891237 13693 leader_election.cc:336] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.866s (SENT)
I20260709 12:22:00.846477 16279 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:00.901471 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.892638 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:00.910985 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.912277 13721 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:00.913972 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.914657 13723 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:00.912406 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.871435 16023 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:00.837145 16254 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.915868 13727 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:22:00.922192 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.889505 16299 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.847505 16277 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.924839 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.921727 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:00.927366 13724 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:00.869776 16286 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.872776 16231 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.937500 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.938642 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:00.939188 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.940567 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.941988 13798 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:00.941574 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.942157 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.943447 13794 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:00.943539 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.944501 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.944753 13797 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:00.945900 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:00.946583 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.948796 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.949517 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.949841 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:00.950803 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:00.952503 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:00.947551 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:00.948218 13727 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:00.954818 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:00.955809 13727 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:00.905373 16270 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.959156 13719 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:00.930842 16284 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:00.827768 16298 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:00.906325 16258 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.989773 16313 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:00.990314 16311 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:22:00.990561 16313 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:00.991050 16313 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:00.993556 16049 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.995431 16015 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:00.995540 16049 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:00.922698 15989 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.016016 16127 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:01.017577 16127 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:01.018721 16310 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:22:01.021502 16309 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:22:01.021283 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2311) took 2174 ms. Trace:
I20260709 12:22:01.022189 13793 rpcz_store.cc:276] 0709 12:21:58.847116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.540164 (+693048us) service_pool.cc:224] Handling call
0709 12:21:59.540540 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:21:59.540986 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:21:59.541053 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:00.501740 (+960687us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:00.502142 (+   402us) log.cc:844] Serialized 26 byte log entry
0709 12:22:00.613169 (+111027us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:00.613209 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.613281 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:01.020851 (+407570us) raft_consensus.cc:1685] finished
0709 12:22:01.020875 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:01.021204 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1746,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"spinlock_wait_cycles":21657856,"thread_start_us":96822,"threads_started":1,"wal-append.queue_time_us":276419}
I20260709 12:22:01.023849 16231 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:01.026283 16304 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:01.030809 15939 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:01.031700 15939 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.032167 15939 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.034646 16023 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.003717 16015 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:00.919037 16322 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.039527 15939 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:01.041199 16322 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:00.892069 16251 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:01.042482 16316 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
W20260709 12:22:01.018580 16306 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 12:22:01.059333 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 297301us
W20260709 12:22:01.064893 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 195638us
I20260709 12:22:01.071447 13801 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
W20260709 12:22:01.074610 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.018115 16127 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:01.077546 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 186323us
W20260709 12:22:01.078613 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126188us
I20260709 12:22:01.079782 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:01.057112 16086 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.565s	user 0.003s	sys 0.011s
I20260709 12:22:01.084136 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:01.085040 13798 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 3.
I20260709 12:22:01.069708 16288 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:01.098081 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.944s (SENT)
I20260709 12:22:01.099012 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:01.100374 13793 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:01.100466 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:00.901124 14420 rpcz_store.cc:276] 0709 12:21:59.687752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.689072 (+  1320us) service_pool.cc:224] Handling call
0709 12:22:00.244476 (+555404us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:22:00.244729 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:00.292340 (+ 47611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:00.292381 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:00.292452 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:00.292494 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.788689 (+496195us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b5400651dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:00.793072 (+  4383us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:00.819097 (+ 26025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109217408}
I20260709 12:22:00.974994 16156 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.080797 16127 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:01.116225 16288 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:01.102291 15891 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.635s	user 0.013s	sys 0.001s
W20260709 12:22:01.109479 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 494634us
W20260709 12:22:01.131054 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 262642us
I20260709 12:22:01.131903 16329 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:01.128335 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:01.132143 16317 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:01.132638 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:01.135422 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:01.153870 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:01.087715 16086 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:01.138195 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6210104440001487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4595s for a response
W20260709 12:22:01.156483 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.111902 16015 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:01.157188 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:01.172688 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62052.9us
I20260709 12:22:01.055588 16245 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:01.172720 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1658119690000603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:22:01.174407 16331 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:01.174607 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:01.183061 16254 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:01.183312 16321 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:22:01.183882 16254 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:01.184306 16254 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:01.184679 16254 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.069964 16228 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:01.175774 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44956.8us
I20260709 12:22:01.177043 13649 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:01.177204 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1514119690000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 12:22:01.188043 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1154119689999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
W20260709 12:22:01.188532 13693 connection.cc:256] RPC call timeout handler was delayed by -1.052411968999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
W20260709 12:22:01.188936 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0398119690003114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
W20260709 12:22:01.191540 13693 connection.cc:256] RPC call timeout handler was delayed by -1.007411969000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260709 12:22:01.146698 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:01.151068 16329 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:01.181804 16331 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:01.211095 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75745.4us
I20260709 12:22:01.151680 16317 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:01.205340 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.217864 16324 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:01.222782 16333 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:22:01.223176 16324 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.223711 16015 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:01.224300 16277 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:01.226450 16337 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:01.227006 16337 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:01.227248 13726 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:01.227597 16323 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:22:01.228785 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.229314 16086 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:01.230216 16176 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.005s
I20260709 12:22:01.225991 16324 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:01.230698 16176 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:01.231308 16286 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:01.232129 15844 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.229961 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.179152 16078 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:01.235025 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.237278 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.237673 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:01.238615 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.146289 15891 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.235338 13719 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:01.186834 16270 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:01.235613 14422 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:01.241380 15844 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:01.214464 16238 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.906s	user 0.014s	sys 0.000s
I20260709 12:22:01.243873 13723 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:01.237907 13725 raft_consensus.cc:2376] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6. Re-sending same reply.
I20260709 12:22:01.247963 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.227421 16337 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.263605 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:01.235644 16317 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:01.274580 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.219870 16331 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.216147 16329 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:01.275746 13719 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:01.229904 16258 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:01.294850 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.301851 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.302245 13799 raft_consensus.cc:2368] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:22:01.303754 13652 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:01.310806 13725 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:01.207502 16297 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.312484 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.243228 13646 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:22:01.321791 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.322746 13723 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.315552 13724 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:01.316313 13647 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:22:01.316094 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 128845us
I20260709 12:22:01.317744 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.324613 13726 raft_consensus.cc:2368] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4.
I20260709 12:22:01.327775 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:01.328018 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.329344 16334 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:01.329671 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:01.331296 14419 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.000s
I20260709 12:22:01.266637 16238 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.329700 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.333029 16279 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.424417 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.425128 13719 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:01.414574 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.333397 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:01.429647 13792 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:01.392932 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 129641us
W20260709 12:22:01.404799 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77575us
I20260709 12:22:01.436573 13647 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:01.452224 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:01.454118 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.454644 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.456851 13726 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:01.457013 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:01.460150 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:01.461772 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:01.457726 16328 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:22:01.331804 14419 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:01.395814 16333 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.499607 13644 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:01.450083 16293 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.334s
I20260709 12:22:01.426916 15989 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:01.509296 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78632.4us
I20260709 12:22:01.429822 16015 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:01.429572 16258 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:01.571609 13724 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:01.544757 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:01.553936 13647 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:01.511293 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:01.625922 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:01.623826 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:01.626386 13794 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:01.512686 13726 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:01.629520 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.631037 13724 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:01.564349 13723 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:01.635228 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:01.506054 16299 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.570s	user 0.005s	sys 0.003s
W20260709 12:22:01.511408 16282 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.021s	user 0.011s	sys 0.004s
I20260709 12:22:01.661244 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:01.664525 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.569253 16156 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:01.568264 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 419) took 1032 ms. Trace:
I20260709 12:22:01.543113 16324 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:01.676405 13726 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:22:01.679803 13726 pending_rounds.cc:85] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 1
I20260709 12:22:01.680644 16349 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:01.680526 13726 pending_rounds.cc:107] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:22:01.688855 13726 raft_consensus.cc:2889] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:01.515158 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:01.681771 16156 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:01.696617 16347 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:22:01.695305 13798 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:01.681363 16349 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.698460 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:01.712092 13647 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:01.657127 15649 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:22:01.641458 16258 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.699401 16318 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:01.738206 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.637182 16333 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.665730 16299 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:01.772312 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 135648us
I20260709 12:22:01.911194 13719 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:01.923022 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:01.911887 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1560) took 1173 ms. Trace:
I20260709 12:22:01.918956 13794 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:01.812007 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:01.923920 13645 rpcz_store.cc:276] 0709 12:22:00.738313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.867463 (+129150us) service_pool.cc:224] Handling call
0709 12:22:00.889115 (+ 21652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:01.074686 (+185571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:01.074726 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:01.075342 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 12:22:01.252084 (+176742us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:01.252160 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:01.252263 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:01.911573 (+659310us) raft_consensus.cc:1685] finished
0709 12:22:01.911620 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:01.911810 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45790,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7208,"thread_start_us":328148,"threads_started":2,"wal-append.queue_time_us":343055}
I20260709 12:22:01.930684 13719 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:01.929523 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:01.769598 16141 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.903s	user 0.013s	sys 0.003s
I20260709 12:22:01.678001 14419 rpcz_store.cc:276] 0709 12:22:00.508879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.517339 (+  8460us) service_pool.cc:224] Handling call
0709 12:22:01.540989 (+1023650us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:01.744289 15982 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:01.939375 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:01.943774 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:01.943789 13724 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:22:01.922015 16238 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:01.952049 16324 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:01.952378 15623 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:01.952926 15623 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:01.953070 13722 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:01.839864 15989 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:01.864190 16318 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:01.936012 16288 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:01.966511 16230 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:01.760890 16015 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:01.969295 13719 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:22:02.502629 13796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.021s	sys 0.029s
W20260709 12:22:02.911602 13796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.031s	sys 0.039s
I20260709 12:22:02.527107 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1550) took 1976 ms. Trace:
I20260709 12:22:02.231076 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1548) took 1700 ms. Trace:
I20260709 12:22:02.439591 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:02.912712 13644 rpcz_store.cc:276] 0709 12:22:00.550635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.554577 (+  3942us) service_pool.cc:224] Handling call
0709 12:22:00.554874 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:01.628255 (+1073381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.526599 (+898344us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54001b6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:02.526973 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084623616}
I20260709 12:22:02.233044 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1640) took 1289 ms. Trace:
W20260709 12:22:02.222143 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1435285500001555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 12:22:02.921260 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1183285500001148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260709 12:22:02.322633 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.006s
I20260709 12:22:02.572053 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1672) took 1661 ms. Trace:
W20260709 12:22:02.924376 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.008s	sys 0.020s
I20260709 12:22:02.237660 13795 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:02.924661 13722 rpcz_store.cc:276] 0709 12:22:00.910678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.913138 (+  2460us) service_pool.cc:224] Handling call
0709 12:22:00.913473 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:01.953562 (+1040089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.571589 (+618027us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400173754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:02.571945 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2390343936}
I20260709 12:22:02.525012 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42338 (request call id 21) took 1047 ms. Trace:
W20260709 12:22:02.926016 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:02.926431 13721 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.005s	sys 0.004s
W20260709 12:22:02.926930 13721 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.005s	sys 0.004s
I20260709 12:22:02.448412 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1843) took 1636 ms. Trace:
W20260709 12:22:02.532714 13690 connection.cc:256] RPC call timeout handler was delayed by -1.4095114629999443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
I20260709 12:22:02.240021 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1633) took 1301 ms. Trace:
I20260709 12:22:02.928719 13801 rpcz_store.cc:276] 0709 12:22:00.938290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.938412 (+   122us) service_pool.cc:224] Handling call
0709 12:22:02.239766 (+1301354us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400103454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.239942 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302914432}
I20260709 12:22:02.917094 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1848) took 1608 ms. Trace:
W20260709 12:22:02.923190 13723 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.006s	sys 0.015s
I20260709 12:22:02.930079 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:02.930325 13647 rpcz_store.cc:276] 0709 12:22:01.308231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.634872 (+326641us) service_pool.cc:224] Handling call
0709 12:22:02.916822 (+1281950us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.917003 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174156672}
W20260709 12:22:02.930465 13723 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.006s	sys 0.015s
W20260709 12:22:02.177893 13725 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.853s	user 0.013s	sys 0.003s
I20260709 12:22:02.932686 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:02.933028 13725 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:22:02.251313 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 453) took 1730 ms (client timeout 1969 ms). Trace:
I20260709 12:22:02.470377 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1824) took 2219 ms. Trace:
W20260709 12:22:02.233919 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1638) took 1292 ms (client timeout 1519 ms). Trace:
I20260709 12:22:02.913985 13649 rpcz_store.cc:276] 0709 12:22:00.530884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.531198 (+   314us) service_pool.cc:224] Handling call
0709 12:22:00.531472 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:01.187541 (+656069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.230749 (+1043208us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b54001473d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:02.230995 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1484214656}
I20260709 12:22:02.920882 13792 rpcz_store.cc:276] 0709 12:22:00.943989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.944135 (+   146us) service_pool.cc:224] Handling call
0709 12:22:02.232659 (+1288524us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b5400288cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.232944 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103564032}
W20260709 12:22:02.928031 13690 connection.cc:256] RPC call timeout handler was delayed by -1.2731797999999799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 12:22:02.239171 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1551) took 1682 ms (client timeout 2173 ms). Trace:
W20260709 12:22:02.933925 13725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1686) took 1690 ms (client timeout 1645 ms). Trace:
I20260709 12:22:02.926353 13798 rpcz_store.cc:276] 0709 12:22:01.459458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.460308 (+   850us) service_pool.cc:224] Handling call
0709 12:22:02.477133 (+1016825us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b54003771a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b71520 00007f5393bee652 00007f5398bbc60a 00007f53921427d4 00007f5392142762 00007f5392142691 00007f539214260c 00007f53921418c2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:02.477650 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183263488}
W20260709 12:22:02.941828 13725 rpcz_store.cc:269] 0709 12:22:01.243975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.253915 (+  9940us) service_pool.cc:224] Handling call
0709 12:22:02.933822 (+1679907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:02.935182 13651 rpcz_store.cc:276] 0709 12:22:00.218764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.218924 (+   160us) service_pool.cc:224] Handling call
0709 12:22:00.219324 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:00.737113 (+517789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.437744 (+1700631us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b5400112954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:02.438166 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1188736768}
W20260709 12:22:02.936474 13799 rpcz_store.cc:269] 0709 12:22:00.941547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.941686 (+   139us) service_pool.cc:224] Handling call
0709 12:22:02.233658 (+1291972us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.233840 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766568832}
W20260709 12:22:02.934499 13720 rpcz_store.cc:269] 0709 12:22:00.521045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.521290 (+   245us) service_pool.cc:224] Handling call
0709 12:22:02.250985 (+1729695us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.251174 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631490944}
I20260709 12:22:01.954576 16361 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:02.026803 16345 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.458s	user 0.000s	sys 0.002s
W20260709 12:22:02.947278 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.0178e+06us
W20260709 12:22:02.100524 16127 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.003s
W20260709 12:22:02.223380 16122 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.096s	user 0.010s	sys 0.003s
I20260709 12:22:02.976917 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:02.978645 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:02.979357 13798 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
W20260709 12:22:02.254128 14422 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 543) took 2528 ms (client timeout 2114 ms). Trace:
I20260709 12:22:02.961496 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:02.980630 13644 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:02.974687 13799 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:02.961412 13647 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:02.982169 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2334) took 1355 ms. Trace:
I20260709 12:22:02.927701 13643 rpcz_store.cc:276] 0709 12:22:00.811729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.878710 (+ 66981us) service_pool.cc:224] Handling call
0709 12:22:00.879070 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:00.979342 (+100272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:00.979388 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:00.979445 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:00.979491 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.447962 (+1468471us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:02.448020 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:02.448328 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229915520}
I20260709 12:22:02.982596 13799 rpcz_store.cc:276] 0709 12:22:01.626373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.968909 (+1342536us) service_pool.cc:224] Handling call
0709 12:22:02.969249 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.981867 (+ 12618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:02.982103 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:02.791314 16336 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.469s	user 0.001s	sys 0.004s
I20260709 12:22:01.941411 16156 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:02.994174 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:02.995754 13648 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:02.074054 16086 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.006s	sys 0.056s
I20260709 12:22:03.004949 13644 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
W20260709 12:22:02.392797 16285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.016s	sys 0.026s
W20260709 12:22:02.563715 16358 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.633s	user 0.003s	sys 0.000s
W20260709 12:22:02.464610 16342 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.971s	user 0.002s	sys 0.001s
W20260709 12:22:02.496251 14416 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.208s	user 0.013s	sys 0.005s
W20260709 12:22:02.387199 16223 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.901s	user 0.015s	sys 0.022s
W20260709 12:22:02.234769 16303 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.329s	user 0.011s	sys 0.007s
W20260709 12:22:02.228094 16244 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.425s	user 0.016s	sys 0.009s
I20260709 12:22:02.973716 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:02.959543 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 520053us
W20260709 12:22:02.199103 16353 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
W20260709 12:22:02.233760 16305 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.513s	user 0.002s	sys 0.001s
W20260709 12:22:02.940330 16329 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.614s	user 0.010s	sys 0.008s
W20260709 12:22:02.940471 16317 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.613s	user 0.013s	sys 0.004s
W20260709 12:22:02.926054 14418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.002s	sys 0.011s
W20260709 12:22:02.227058 16280 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.480s	user 0.006s	sys 0.024s
W20260709 12:22:02.235356 16313 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.244s	user 0.011s	sys 0.008s
I20260709 12:22:02.974491 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:02.941625 13646 rpcz_store.cc:269] 0709 12:22:00.557203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.629309 (+ 72106us) service_pool.cc:224] Handling call
0709 12:22:02.238864 (+1609555us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54001e7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.239059 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014441216}
I20260709 12:22:02.964352 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:02.983881 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:01.942027 16141 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:02.990723 14424 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:02.991148 16127 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:02.992400 13793 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.017s	sys 0.025s
W20260709 12:22:03.075513 13793 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.018s	sys 0.025s
I20260709 12:22:02.976907 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:02.196206 16283 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:02.195017 16349 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:02.027074 15649 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.753499 16279 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:02.440943 16218 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.460983 15939 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:02.174177 16258 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:02.453890 16297 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:02.410583 16125 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:02.941169 16323 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.616s	user 0.018s	sys 0.012s
I20260709 12:22:02.024755 16362 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:02.981321 14422 rpcz_store.cc:269] 0709 12:21:59.725744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.887775 (+162031us) service_pool.cc:224] Handling call
0709 12:22:00.468126 (+580351us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:02.253878 (+1785752us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:02.254010 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774978688}
I20260709 12:22:02.975056 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:02.931048 13801 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:02.946879 16161 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:02.202157 16333 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:02.947576 16320 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:02.947242 16356 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:02.977553 16176 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.545s	user 0.015s	sys 0.017s
I20260709 12:22:02.966217 16321 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:02.970736 16242 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:02.965742 16339 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:03.167230 16320 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.167963 16339 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.165524 16161 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:03.014849 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.015s	sys 0.006s
W20260709 12:22:03.170424 16176 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.744s	user 0.016s	sys 0.018s
I20260709 12:22:03.169760 16320 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.171582 16320 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.172117 16321 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.174808 16321 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.174401 16356 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:01.958055 15982 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.177090 13644 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:03.177379 15982 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:02.970721 16217 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:03.160601 16258 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:03.139314 16362 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.181329 16362 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:03.007783 16285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.513s	user 0.016s	sys 0.053s
W20260709 12:22:03.183202 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 272143us
I20260709 12:22:03.184224 16127 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.185276 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:03.185737 16161 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.038669 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:03.186632 13648 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:03.038363 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:03.158447 16279 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:03.190735 16279 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:02.976625 16361 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.031311 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.023463 16237 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:03.071441 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:02.298504 16254 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.047s	user 0.010s	sys 0.003s
I20260709 12:22:03.023074 16156 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:03.022434 14416 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:03.034463 16370 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.034204 16282 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.041884 16223 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:03.121127 15649 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.044488 16244 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.034448 16327 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.191413 16285 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:03.073022 14418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.002s	sys 0.014s
I20260709 12:22:03.098883 16371 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:03.046578 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:03.086952 16337 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.790s	user 0.016s	sys 0.004s
I20260709 12:22:03.126096 13800 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:03.077066 16338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.745s	user 0.047s	sys 0.041s
W20260709 12:22:03.127714 16331 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.813s	user 0.016s	sys 0.005s
I20260709 12:22:03.078408 16283 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:03.134738 13798 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:03.135713 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1554) took 2545 ms. Trace:
I20260709 12:22:03.041541 16303 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:03.139989 16323 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.866s	user 0.018s	sys 0.013s
W20260709 12:22:03.151036 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 176049us
I20260709 12:22:03.172135 16320 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.352114 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:03.153947 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1674) took 2242 ms. Trace:
I20260709 12:22:03.062430 16317 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:03.333170 16338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.020s	user 0.048s	sys 0.042s
I20260709 12:22:03.061894 16329 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.072778 16280 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.149880 16125 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:03.170596 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.015s	sys 0.006s
I20260709 12:22:03.220777 16254 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:03.183115 16365 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:22:03.010444 15891 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.011523 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:03.385914 16377 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:22:03.284629 16327 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.390596 16327 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.391654 15649 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:03.005689 16086 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.550s	user 0.017s	sys 0.136s
I20260709 12:22:01.970769 15989 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:03.163944 15623 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.211s	user 0.012s	sys 0.005s
I20260709 12:22:03.207221 13646 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:03.208508 13648 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:03.340126 16337 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:03.394584 16367 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:22:03.183010 16141 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.394865 15989 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.393483 15623 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.396023 16337 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.396576 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2201) took 2178 ms. Trace:
I20260709 12:22:03.032778 16299 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:03.208819 16328 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.702s	user 0.011s	sys 0.005s
I20260709 12:22:03.397873 15623 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.238462 16370 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:03.398761 16375 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.264s	user 0.002s	sys 0.000s
I20260709 12:22:03.289930 13727 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:03.293771 16078 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:03.295495 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:03.295698 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.291644 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:03.322275 13643 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:03.323235 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 339395us
I20260709 12:22:03.408417 13797 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:03.209501 16242 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:03.340816 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2333) took 1717 ms. Trace:
I20260709 12:22:03.222024 16318 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.346486 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:03.157466 15939 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.147204 16122 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48736 (request call id 388) took 9079 ms. Trace:
W20260709 12:22:03.237620 14416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 547) took 2132 ms (client timeout 1552 ms). Trace:
I20260709 12:22:03.072427 16313 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.279964 16285 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:03.356453 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 387659us
I20260709 12:22:03.340546 13650 rpcz_store.cc:276] 0709 12:22:00.590735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.849197 (+258462us) service_pool.cc:224] Handling call
0709 12:22:00.874012 (+ 24815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:00.884408 (+ 10396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:00.884447 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:00.904343 (+ 19896us) log.cc:844] Serialized 26 byte log entry
0709 12:22:01.222137 (+317794us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:01.222202 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:01.222341 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:02.463746 (+1241405us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:03.131783 (+668037us) raft_consensus.cc:1685] finished
0709 12:22:03.131810 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.135619 (+  3809us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24336,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":16010,"spinlock_wait_cycles":12150016,"thread_start_us":258726,"threads_started":1,"wal-append.queue_time_us":382213}
I20260709 12:22:03.282927 16339 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.299890 16371 consensus_queue.cc:1243] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:03.206856 15982 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 12:22:03.353783 13718 rpcz_store.cc:276] 0709 12:22:00.911266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.913780 (+  2514us) service_pool.cc:224] Handling call
0709 12:22:00.914289 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:00.933171 (+ 18882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:00.933203 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:00.984376 (+ 51173us) log.cc:844] Serialized 26 byte log entry
0709 12:22:01.256706 (+272330us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:01.312687 (+ 55981us) log.cc:844] Serialized 14 byte log entry
0709 12:22:01.325375 (+ 12688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:01.325460 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:03.148146 (+1822686us) raft_consensus.cc:1685] finished
0709 12:22:03.152990 (+  4844us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.153801 (+   811us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13517,"prepare.queue_time_us":50413,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":18571,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":8431,"thread_start_us":170816,"threads_started":1,"wal-append.queue_time_us":286832}
I20260709 12:22:03.178247 16217 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:03.370462 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 159025us
I20260709 12:22:03.489068 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2326) took 1979 ms. Trace:
I20260709 12:22:03.261667 16237 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.263254 16282 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.263056 16223 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:03.374378 14390 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.014s	sys 0.009s
W20260709 12:22:03.498509 14390 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.014s	sys 0.010s
W20260709 12:22:03.498875 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 645409us
W20260709 12:22:03.499346 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:03.501453 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:03.237311 16156 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.345491 16356 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.384210 13799 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:03.359042 16161 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:22:03.359417 16245 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:03.134569 16218 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.397583 16361 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:03.398133 16328 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.891s	user 0.011s	sys 0.005s
I20260709 12:22:03.273631 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:03.400234 16370 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.357416 16383 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:03.403843 16394 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:03.369791 16258 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:22:03.521987 16218 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:03.404917 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 431206us
I20260709 12:22:03.522912 16383 raft_consensus.cc:2764] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:22:03.523597 16383 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:03.393430 16244 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.523888 16217 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.396960 13723 rpcz_store.cc:276] 0709 12:22:01.218327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.643577 (+425250us) service_pool.cc:224] Handling call
0709 12:22:01.644046 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:01.663336 (+ 19290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:01.663382 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:01.664026 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 12:22:02.047669 (+383643us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:02.932436 (+884767us) log.cc:844] Serialized 14 byte log entry
0709 12:22:03.159735 (+227299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.159821 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:03.396347 (+236526us) raft_consensus.cc:1685] finished
0709 12:22:03.396368 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.396512 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12988,"prepare.queue_time_us":205253,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":238941,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":10750,"thread_start_us":1296995,"threads_started":3,"wal-append.queue_time_us":1346332}
I20260709 12:22:03.416940 13793 rpcz_store.cc:276] 0709 12:22:01.622368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.227699 (+605331us) service_pool.cc:224] Handling call
0709 12:22:02.228061 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.228458 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:03.075970 (+847512us) log.cc:844] Serialized 14 byte log entry
0709 12:22:03.338768 (+262798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:03.338844 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:03.338907 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.339013 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.339193 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13374,"prepare.queue_time_us":913327,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":90,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3694,"thread_start_us":1108843,"threads_started":2}
I20260709 12:22:03.422480 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:03.369264 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.345475 16331 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:03.480029 13617 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.626s (SENT)
I20260709 12:22:03.526377 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:03.484009 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 140975us
I20260709 12:22:03.421821 15649 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.485347 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 423) took 2687 ms. Trace:
I20260709 12:22:03.485661 13792 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
W20260709 12:22:03.489498 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1635) took 2549 ms (client timeout 3089 ms). Trace:
W20260709 12:22:03.532743 13800 rpcz_store.cc:269] 0709 12:22:00.940176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.948560 (+  8384us) service_pool.cc:224] Handling call
0709 12:22:03.489314 (+2540754us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b5400102554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:03.489441 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4964721408}
I20260709 12:22:03.534415 14425 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:03.512686 16245 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.536129 16223 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.536734 16125 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.537377 16156 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.537662 13794 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:03.539079 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:03.489533 13795 rpcz_store.cc:276] 0709 12:22:01.509986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.540918 (+ 30932us) service_pool.cc:224] Handling call
0709 12:22:02.237365 (+696447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:02.925405 (+688040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.488659 (+563254us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400288cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:03.488869 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600222976}
I20260709 12:22:03.495653 15623 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:03.503453 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:03.548812 13718 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
W20260709 12:22:03.505729 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 464234us
I20260709 12:22:03.493466 14419 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 4: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:03.511114 14422 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:03.472169 14416 rpcz_store.cc:269] 0709 12:22:01.105117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.111661 (+  6544us) service_pool.cc:224] Handling call
0709 12:22:03.221402 (+2109741us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:03.436509 16337 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:22:03.517534 15939 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.518811 16237 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.525316 16329 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.521720 16394 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:03.404753 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52740.3us
I20260709 12:22:03.505482 16388 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.605777 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:03.607493 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:03.205950 16378 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:03.523959 16320 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:03.407640 13719 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
W20260709 12:22:03.509915 16384 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:22:03.617453 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:03.554003 15623 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:03.531069 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108724us
I20260709 12:22:03.401875 16279 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.621588 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.540619 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:03.623098 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:03.623411 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:03.545878 15649 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:03.546561 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66575.4us
I20260709 12:22:03.547770 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:03.552997 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:03.546140 16156 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:03.561900 16280 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.562556 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:03.538030 14418 rpcz_store.cc:276] 0709 12:22:00.784618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.788639 (+  4021us) service_pool.cc:224] Handling call
0709 12:22:00.876547 (+ 87908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:01.019885 (+143338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:01.019956 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:01.020950 (+   994us) log.cc:844] Serialized 26 byte log entry
0709 12:22:02.213305 (+1192355us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b18003c2440. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:03.189174 (+975869us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:03.341461 (+152287us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:03.346609 (+  5148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.354892 (+  8283us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:03.406534 (+ 51642us) raft_consensus.cc:1685] finished
0709 12:22:03.406562 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.471869 (+ 65307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33686,"raft.queue_time_us":50872,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":29666,"spinlock_wait_cycles":117071104,"thread_start_us":1068735,"threads_started":1,"wal-append.queue_time_us":2018057}
I20260709 12:22:03.512300 16242 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.530619 16327 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:03.564251 16317 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.582973 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:03.519799 16254 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:03.524170 16383 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.595068 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.595340 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.595232 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.538476 16370 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:03.596567 15939 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:03.592267 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:03.610982 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:03.545753 16331 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.614084 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1676) took 2702 ms. Trace:
I20260709 12:22:03.610278 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:03.529282 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:03.616868 16388 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:03.623737 14416 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:03.544726 16402 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:22:03.557549 16277 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:04.093827 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:03.631702 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.095005 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:22:04.095551 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.096591 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:03.613857 16405 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:03.681432 16392 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:22:03.430786 16122 rpcz_store.cc:276] 0709 12:21:54.085970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.100595 (+ 14625us) service_pool.cc:224] Handling call
0709 12:21:54.822933 (+722338us) ts_tablet_manager.cc:934] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075)
0709 12:22:00.466616 (+5643683us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:22:01.099624 (+633008us) tablet_metadata.cc:665] Metadata flushed
0709 12:22:01.099777 (+   153us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:22:03.007943 (+1908166us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:22:03.061072 (+ 53129us) tablet_replica.cc:206] Creating consensus instance
0709 12:22:03.131901 (+ 70829us) ts_tablet_manager.cc:989] Replica registered.
0709 12:22:03.146980 (+ 15079us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":51473,"tablet-copy.queue_time_us":415903,"thread_start_us":1125623,"threads_started":2}
I20260709 12:22:03.625098 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:03.621199 16337 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:04.051821 16125 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.661288 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:03.873930 14422 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:03.660032 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:03.618424 16141 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:03.899760 16394 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:03.616640 16378 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.135759 16378 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:04.139937 16383 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:04.098759 16405 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.140736 16383 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:04.141322 16383 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.141731 16383 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:04.133538 16388 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:03.683055 16303 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:04.093425 16277 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:04.144205 14420 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:03.807251 13726 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:03.898983 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:03.823101 13618 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.663s (SENT) [suppressed 18 similar messages]
I20260709 12:22:04.098315 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.146924 13719 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:22:04.147094 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 324022us
I20260709 12:22:03.615003 16334 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:04.106248 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 445054us
I20260709 12:22:04.149350 16317 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:04.150425 16317 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:04.152704 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:04.126647 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.099426 16237 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:04.101315 16320 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:04.105634 16361 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:04.116868 16015 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.133303 13795 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:04.103621 13721 rpcz_store.cc:276] 0709 12:22:00.911846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.926497 (+ 14651us) service_pool.cc:224] Handling call
0709 12:22:00.968554 (+ 42057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:01.048005 (+ 79451us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:01.083317 (+ 35312us) log.cc:844] Serialized 14 byte log entry
0709 12:22:02.249334 (+1166017us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b18003ae0a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:22:03.304233 (+1054899us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:03.355063 (+ 50830us) log.cc:844] Serialized 26 byte log entry
0709 12:22:03.355328 (+   265us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:03.489379 (+134051us) log.cc:844] Serialized 14 byte log entry
0709 12:22:03.613489 (+124110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.613629 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:03.613792 (+   163us) raft_consensus.cc:1685] finished
0709 12:22:03.613815 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:03.614008 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18713,"prepare.queue_time_us":103189,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":128,"raft.queue_time_us":301910,"raft.run_cpu_time_us":600,"raft.run_wall_time_us":12552,"spinlock_wait_cycles":32913664,"thread_start_us":1289806,"threads_started":4,"wal-append.queue_time_us":1957406}
I20260709 12:22:03.756827 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:03.744840 16393 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:22:04.107379 13718 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.171583 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:03.695510 16282 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:04.143419 16391 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:22:03.679628 16313 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:04.106547 13727 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:04.098210 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:04.154335 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:04.102236 13722 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:22:04.154388 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.000s	sys 0.027s
I20260709 12:22:04.155902 16405 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:04.102135 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 547985us
I20260709 12:22:04.185153 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:04.099623 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 546641us
I20260709 12:22:04.144982 16277 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.164829 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.167408 16361 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.170298 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2343) took 1126 ms. Trace:
I20260709 12:22:04.188544 13795 rpcz_store.cc:276] 0709 12:22:03.044404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.132731 (+1088327us) service_pool.cc:224] Handling call
0709 12:22:04.133100 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.169977 (+ 36877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.170211 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:04.101150 16122 tablet_copy_client.cc:806] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:22:04.172647 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2335) took 2496 ms. Trace:
I20260709 12:22:04.176955 13725 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:04.196242 13798 rpcz_store.cc:276] 0709 12:22:01.676794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.999066 (+1322272us) service_pool.cc:224] Handling call
0709 12:22:02.999363 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.345648 (+346285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.172380 (+826732us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003e17d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:04.172580 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249537920}
I20260709 12:22:04.179797 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.119177 15939 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:04.163043 16327 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:04.183288 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.000s	sys 0.028s
W20260709 12:22:04.208123 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 706657us
I20260709 12:22:04.183684 13723 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:04.185348 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:04.164563 16078 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.624058 16223 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.552s
I20260709 12:22:04.235428 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:04.236123 13651 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:04.162487 13799 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:04.204386 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:04.186789 16282 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:04.229378 16370 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:04.229435 15939 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.229976 16327 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.231113 14418 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.232808 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:04.233633 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:04.234222 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:04.235167 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:04.235991 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:04.237658 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:04.103006 16125 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.196058 16361 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:04.241104 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.245105 13799 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:22:04.251163 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:22:04.250662 13727 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:04.250670 13725 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:04.252439 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.175024 16015 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.199471 16405 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:04.242607 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:04.104463 15623 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.270385 13723 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:04.327052 13643 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:04.312355 13652 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:04.312832 13651 raft_consensus.cc:606] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:04.329037 13651 raft_consensus.cc:614] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:04.317198 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:04.319156 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:04.260018 16129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.021s
I20260709 12:22:04.321660 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2332) took 2695 ms. Trace:
I20260709 12:22:04.321586 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:04.238607 16331 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:04.202868 16156 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:04.329882 13727 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:04.355526 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 117971us
I20260709 12:22:04.356468 13693 leader_election.cc:400] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:04.334044 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:04.357436 13648 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:04.247112 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:04.335011 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:04.337520 13796 rpcz_store.cc:276] 0709 12:22:01.614679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.928916 (+314237us) service_pool.cc:224] Handling call
0709 12:22:01.944462 (+ 15546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:02.912102 (+967640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:02.912147 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:02.947080 (+ 34933us) log.cc:844] Serialized 26 byte log entry
0709 12:22:03.363568 (+416488us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:03.363609 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:03.363685 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:04.309899 (+946214us) raft_consensus.cc:1685] finished
0709 12:22:04.309919 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.310064 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":606112,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":1298681,"threads_started":2,"wal-append.queue_time_us":521000}
I20260709 12:22:04.325996 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:04.340057 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.341341 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.363266 13722 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:04.261762 16122 tablet_copy_client.cc:670] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:22:04.335423 16320 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:04.342706 13724 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.367456 13798 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:22:04.342950 13723 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:04.343609 13726 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:04.369323 13726 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:04.261375 16282 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.346493 15939 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:04.376192 14424 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.376001 16378 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.378854 16378 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:04.382081 16383 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.382735 16282 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:04.383925 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 550) took 1114 ms. Trace:
I20260709 12:22:04.384518 16383 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:04.359774 14417 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:04.246513 16078 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.351589 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:04.356779 13693 leader_election.cc:403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:04.393368 13646 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:04.393743 13724 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:22:04.337112 16129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.006s	sys 0.021s
I20260709 12:22:04.373101 13721 pending_rounds.cc:85] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 2
I20260709 12:22:04.398756 13721 pending_rounds.cc:107] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:22:04.399127 13721 raft_consensus.cc:2889] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:04.347337 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:04.400514 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:04.388988 13722 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:04.384310 14425 rpcz_store.cc:276] 0709 12:22:03.269226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.269359 (+   133us) service_pool.cc:224] Handling call
0709 12:22:03.333395 (+ 64036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.535061 (+201666us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.383615 (+848554us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b5400555fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:04.383843 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86106880}
I20260709 12:22:04.359118 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:04.394174 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:04.403887 13719 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:04.404742 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:04.405563 13648 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:04.394804 13726 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.410322 13726 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:04.396082 13718 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:04.414203 13693 leader_election.cc:400] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:04.420017 16176 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.420723 16176 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.424153 16125 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:04.374626 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:04.428946 16245 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:04.435597 15649 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:04.349480 16317 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:22:04.435972 16349 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:04.437408 16349 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:04.437789 16349 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:04.439265 16015 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:04.439370 16349 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.366501 16156 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.440850 16349 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:04.436329 15649 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:04.442090 15649 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.442497 15649 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:04.442941 16156 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:04.443219 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.448190 16176 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:04.448393 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2327) took 2937 ms (client timeout 2663 ms). Trace:
W20260709 12:22:04.450687 13797 rpcz_store.cc:269] 0709 12:22:01.510848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.511000 (+   152us) service_pool.cc:224] Handling call
0709 12:22:04.448154 (+2937154us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400287dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:04.448300 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4081042304}
I20260709 12:22:04.452101 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.453394 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.409579 16405 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:04.456486 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2331) took 2887 ms. Trace:
I20260709 12:22:04.456970 13794 rpcz_store.cc:276] 0709 12:22:01.568704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:01.933641 (+364937us) service_pool.cc:224] Handling call
0709 12:22:01.934082 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.538345 (+1604263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.456205 (+917860us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54002891d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:04.456412 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3687452928}
I20260709 12:22:04.449815 16277 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.467155 13727 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
W20260709 12:22:04.463963 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49961.8us
I20260709 12:22:04.462147 13719 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:04.470709 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:04.475317 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.476626 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.477451 13727 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 4: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:04.477023 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:04.478468 13794 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:04.468219 16298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.009s	sys 0.049s
I20260709 12:22:04.406975 16383 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
W20260709 12:22:04.523589 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48796.2us
I20260709 12:22:04.526185 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:04.412772 16334 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:22:04.535470 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.534061 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:04.575150 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:04.582329 13719 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:04.477311 16403 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:04.491230 16397 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.038s	sys 0.091s
W20260709 12:22:04.527958 16298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.012s	sys 0.052s
I20260709 12:22:04.576397 14420 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:04.644682 16383 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:22:04.651629 16397 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.039s	sys 0.103s
I20260709 12:22:04.660240 16352 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:04.660842 16352 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:04.662838 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:04.661813 16352 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.665190 16352 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:04.667948 16285 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.669831 16285 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:04.650794 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75745.7us
I20260709 12:22:04.675729 13693 leader_election.cc:400] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:22:04.676194 13693 leader_election.cc:403] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:22:04.676448 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 296) took 2417 ms (client timeout 2119 ms). Trace:
I20260709 12:22:04.421974 16283 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.668867 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.462610 16320 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:04.643448 13793 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:22:04.676893 13792 rpcz_store.cc:269] 0709 12:22:02.258656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:02.938569 (+679913us) service_pool.cc:224] Handling call
0709 12:22:04.676219 (+1737650us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54003c0354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:04.676368 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940253056}
I20260709 12:22:04.684136 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2342) took 1644 ms. Trace:
I20260709 12:22:04.684616 13793 rpcz_store.cc:276] 0709 12:22:03.039402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.100311 (+1060909us) service_pool.cc:224] Handling call
0709 12:22:04.642706 (+542395us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:22:04.643097 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:04.683738 (+ 40641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.684046 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59625344}
W20260709 12:22:04.692245 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 165016us
I20260709 12:22:04.705125 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:04.475025 16405 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.172s
I20260709 12:22:04.529157 16286 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:04.679171 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.045s	sys 0.109s
I20260709 12:22:04.671834 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2345) took 1178 ms. Trace:
W20260709 12:22:04.743136 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.045s	sys 0.110s
W20260709 12:22:04.743806 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 1.2145e+06us
I20260709 12:22:04.743191 13798 rpcz_store.cc:276] 0709 12:22:03.493562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.198454 (+704892us) service_pool.cc:224] Handling call
0709 12:22:04.198947 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.368074 (+169127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.671568 (+303494us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:04.671764 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386357632}
I20260709 12:22:04.483335 16176 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260709 12:22:04.744270 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 546) took 4497 ms. Trace:
I20260709 12:22:04.745258 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:04.746930 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:04.754483 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.749428 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.673699 16282 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.759668 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.655344 16078 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:04.771234 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:04.772830 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 97404.4us
W20260709 12:22:04.777439 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.778131 13693 leader_election.cc:400] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:22:04.778538 13693 leader_election.cc:403] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:04.770854 16283 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:04.781546 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.760751 16383 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:04.783480 16383 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:04.773197 16320 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.784754 16383 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:04.785779 16404 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.446s	user 0.000s	sys 0.002s
I20260709 12:22:04.787209 14418 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:04.760644 16298 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:04.788930 16298 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:04.740916 16402 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:22:04.791813 14416 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:04.796339 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2341) took 1774 ms. Trace:
I20260709 12:22:04.801800 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.803259 13792 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:04.739583 16156 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 12:22:04.816284 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.808557 13796 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:04.796862 13801 rpcz_store.cc:276] 0709 12:22:03.033276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.164674 (+131398us) service_pool.cc:224] Handling call
0709 12:22:04.676369 (+1511695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.676609 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:04.676637 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:04.676688 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:04.676719 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.676780 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.796047 (+119267us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54003c0350. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:04.796264 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3475346304}
W20260709 12:22:04.814046 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.824873 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:04.825685 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:04.831107 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.832022 13793 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:04.832587 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:04.834903 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.840926 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:04.844307 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:04.807974 16337 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 12:22:04.848596 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:04.849790 16334 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:22:04.787213 15623 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:04.808682 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.808218 14416 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 12:22:04.786178 16320 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:04.771809 16403 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:22:04.823192 16405 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5.
I20260709 12:22:04.745003 14423 rpcz_store.cc:276] 0709 12:22:00.155812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.250086 (+ 94274us) service_pool.cc:224] Handling call
0709 12:22:00.295101 (+ 45015us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:00.303817 (+  8716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:00.303850 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:00.311516 (+  7666us) log.cc:844] Serialized 26 byte log entry
0709 12:22:00.465844 (+154328us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:00.465887 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:00.465967 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:02.119117 (+1653150us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:03.156540 (+1037423us) raft_consensus.cc:1685] finished
0709 12:22:03.156574 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:04.652927 (+1496353us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54005e4f50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:04.653250 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8223,"raft.queue_time_us":16485,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":11253,"spinlock_wait_cycles":111403136,"thread_start_us":110684,"threads_started":1,"wal-append.queue_time_us":260351,"wal-append.run_cpu_time_us":14218,"wal-append.run_wall_time_us":3577430}
I20260709 12:22:04.875581 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:04.817963 13726 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:04.850025 16397 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:22:04.851068 16223 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:04.860169 16279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.035s	sys 0.067s
I20260709 12:22:04.831416 14417 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:04.915216 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 305) took 1382 ms. Trace:
I20260709 12:22:04.913560 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:04.916111 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.004s	sys 0.010s
I20260709 12:22:04.818491 16286 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:04.937110 16335 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:04.981247 13796 rpcz_store.cc:276] 0709 12:22:03.533198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.362356 (+829158us) service_pool.cc:224] Handling call
0709 12:22:04.390969 (+ 28613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.824263 (+433294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:04.914918 (+ 90655us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:04.915146 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959502208}
W20260709 12:22:04.982039 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.004s	sys 0.010s
W20260709 12:22:04.987540 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 730633us
W20260709 12:22:04.966393 16242 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.000s	sys 0.016s
W20260709 12:22:04.927744 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79263.5us
I20260709 12:22:04.853217 13801 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:04.998225 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.998515 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:04.931118 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 89713.3us
W20260709 12:22:04.932399 13795 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.006s	sys 0.008s
W20260709 12:22:04.999512 13795 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.007s	sys 0.008s
W20260709 12:22:04.925809 16370 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.006s	sys 0.009s
I20260709 12:22:04.947939 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.000399 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:05.001037 14423 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:05.001390 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.790153 16298 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:04.963308 13617 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.052s	sys 0.082s
I20260709 12:22:04.906587 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:05.004756 13617 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.053s	sys 0.082s
W20260709 12:22:05.005084 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 911107us
I20260709 12:22:05.008697 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:04.981048 16279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.035s	sys 0.067s
I20260709 12:22:05.008953 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:04.995007 16242 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.000s	sys 0.017s
I20260709 12:22:05.013741 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:04.965276 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:04.937443 16327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.009s	sys 0.006s
W20260709 12:22:04.937079 16344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.753s	user 0.013s	sys 0.015s
I20260709 12:22:04.937208 15939 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:04.911767 15649 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:04.986724 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:04.930024 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8984423570000217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:22:04.961198 16337 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.011518 13801 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:04.975021 14418 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:04.922168 16237 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.005s	sys 0.038s
W20260709 12:22:05.004303 16370 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.006s	sys 0.009s
W20260709 12:22:05.017716 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52126.3us
I20260709 12:22:04.992244 16335 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.982093 16223 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:04.969903 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.025s
I20260709 12:22:05.010130 16078 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:22:04.985001 13720 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:05.018832 16327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.009s	sys 0.006s
I20260709 12:22:05.019001 16317 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:05.019326 16344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.835s	user 0.014s	sys 0.016s
W20260709 12:22:04.980933 13616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.000s	sys 0.021s
W20260709 12:22:05.087601 13616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.000s	sys 0.021s
I20260709 12:22:05.022644 16415 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.073s
W20260709 12:22:05.029110 13693 connection.cc:256] RPC call timeout handler was delayed by -2.752642356999786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 12:22:05.091962 13693 connection.cc:256] RPC call timeout handler was delayed by -2.567242356999941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
I20260709 12:22:05.092087 16176 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:05.030071 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.093639 13796 raft_consensus.cc:2368] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
W20260709 12:22:05.035243 13721 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.011s	sys 0.011s
W20260709 12:22:05.094658 13721 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.011s	sys 0.011s
I20260709 12:22:05.038380 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.097566 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:05.042272 16237 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.006s	sys 0.038s
W20260709 12:22:05.092330 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4007423570001265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 12:22:05.043825 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.004s	sys 0.025s
W20260709 12:22:05.098871 13693 connection.cc:256] RPC call timeout handler was delayed by -2.3629423570000654s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 12:22:05.099050 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 773108us
W20260709 12:22:05.099252 13693 connection.cc:256] RPC call timeout handler was delayed by -2.3098423569999795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 12:22:05.099633 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9066423569997824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2621s for a response
W20260709 12:22:05.099896 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6042423569997482s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2957s for a response
W20260709 12:22:05.100172 13693 connection.cc:256] RPC call timeout handler was delayed by -1.501642357000037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30710000000000004s for a response
W20260709 12:22:05.100466 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4854423570000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3089s for a response
W20260709 12:22:05.100740 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3153423569997358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32780000000000004s for a response
I20260709 12:22:05.101511 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 308) took 1439 ms. Trace:
I20260709 12:22:05.092717 16176 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.109881 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:05.127200 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.128752 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:05.128983 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:05.122008 16361 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.004s
I20260709 12:22:05.134889 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.044638 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.052755 16282 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:05.077615 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:05.135869 13794 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
W20260709 12:22:05.136412 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91904.6us
I20260709 12:22:05.053226 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:05.113495 16277 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.000s
I20260709 12:22:05.058245 16023 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:22:05.098531 16237 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:05.116945 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.045503 16335 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:05.079797 16349 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.615s	user 0.013s	sys 0.008s
I20260709 12:22:05.119239 16415 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:05.119398 16286 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:05.045763 16223 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:05.122591 16283 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:05.123215 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:05.081476 13649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.004s	sys 0.010s
W20260709 12:22:05.080672 13799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.000s	sys 0.013s
I20260709 12:22:05.105269 13792 rpcz_store.cc:276] 0709 12:22:03.662082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.805077 (+1142995us) service_pool.cc:224] Handling call
0709 12:22:04.805606 (+   529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:04.812598 (+  6992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:04.812653 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:04.837661 (+ 25008us) log.cc:844] Serialized 26 byte log entry
0709 12:22:05.087377 (+249716us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:05.087733 (+   356us) log.cc:844] Serialized 14 byte log entry
0709 12:22:05.101159 (+ 13426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.101248 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:05.101291 (+    43us) raft_consensus.cc:1685] finished
0709 12:22:05.101307 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:05.101448 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12963,"raft.queue_time_us":19783,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":12855,"thread_start_us":172145,"threads_started":1,"wal-append.queue_time_us":196839}
I20260709 12:22:05.103040 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.086061 16317 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:05.136531 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61984.1us
I20260709 12:22:05.137351 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:05.066675 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:05.049674 15939 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:05.138571 13645 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:05.282095 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 257611us
I20260709 12:22:05.184243 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.178699 13646 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:05.292524 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:05.289191 13649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.005s	sys 0.010s
I20260709 12:22:05.180344 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.103387 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:05.080729 16394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.007s	sys 0.008s
W20260709 12:22:05.304641 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 201363us
W20260709 12:22:05.305897 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 177231us
W20260709 12:22:05.292693 13799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.001s	sys 0.015s
I20260709 12:22:05.306962 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:05.292980 14423 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:05.081414 16331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.000s	sys 0.013s
I20260709 12:22:05.309373 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:05.082543 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.177325 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:05.315640 13650 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:05.291571 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:05.308089 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.319113 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:05.184087 16416 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:22:05.179818 16285 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.501s	user 0.020s	sys 0.002s
I20260709 12:22:05.321738 13796 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:05.226963 16337 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:05.043917 16125 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:05.322808 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:22:05.309453 16394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.007s	sys 0.009s
I20260709 12:22:05.139680 16431 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:22:05.327406 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 190147us
I20260709 12:22:05.210945 16386 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:05.184697 16410 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:22:05.130147 16403 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:05.130386 16397 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:05.134049 15649 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:05.144726 16361 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.337412 16370 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:05.369199 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:05.140985 16413 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.356201 16023 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:05.185546 16141 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.337218 13792 raft_consensus.cc:606] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:05.390614 13792 raft_consensus.cc:614] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:05.198937 16352 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.004s
W20260709 12:22:05.018751 16313 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.006s
W20260709 12:22:05.403923 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.338816 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:05.294052 16317 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:05.322113 16331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.935s	user 0.003s	sys 0.014s
I20260709 12:22:05.291026 16415 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.211884 16411 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:05.370595 14420 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:05.340840 13797 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:05.347425 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.308948 16405 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:05.323675 16402 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:05.448029 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 129012us
I20260709 12:22:05.450717 14422 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:05.452090 16427 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:05.452611 14391 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.602s (SENT)
I20260709 12:22:05.453069 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:05.453392 16427 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:05.323536 16078 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:05.454840 16078 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.455339 16078 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:05.049520 16242 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.284s
I20260709 12:22:05.223479 16320 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.355618 16386 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:05.341979 16337 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.306593 16283 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.368520 16403 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:05.375448 16298 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:05.177505 16176 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.377071 15939 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.370244 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:05.380878 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1860) took 2356 ms. Trace:
I20260709 12:22:05.484768 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2351) took 1972 ms. Trace:
I20260709 12:22:05.345347 16431 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.357264 16406 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.391947 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1862) took 2360 ms. Trace:
I20260709 12:22:05.485504 13799 rpcz_store.cc:276] 0709 12:22:03.512699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.311937 (+799238us) service_pool.cc:224] Handling call
0709 12:22:04.312218 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.312432 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:22:05.307735 (+995303us) log.cc:844] Serialized 14 byte log entry
0709 12:22:05.483975 (+176240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:05.484088 (+   113us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:05.484167 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.484301 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:05.484673 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":754805,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":52,"raft.queue_time_us":3104,"thread_start_us":1130714,"threads_started":2}
I20260709 12:22:05.335266 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:05.413569 13793 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:05.414914 13800 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:05.346539 16378 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:22:05.354450 16156 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 12:22:05.441642 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.469594 16418 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:05.418215 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:05.418954 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:05.442971 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 302) took 2014 ms. Trace:
I20260709 12:22:05.371120 16370 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:05.372012 15623 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:05.442432 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 103684us
I20260709 12:22:05.475649 16337 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:05.503145 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:05.503805 13651 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:05.498397 13646 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:05.504654 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:05.446662 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.419365 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:05.411844 16313 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.008s	sys 0.007s
I20260709 12:22:05.390354 16023 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:05.461397 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:05.462991 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102494us
I20260709 12:22:05.463356 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:05.464174 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 116844us
I20260709 12:22:05.498236 13795 rpcz_store.cc:276] 0709 12:22:03.428091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.192433 (+764342us) service_pool.cc:224] Handling call
0709 12:22:04.192712 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:04.192941 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:04.999928 (+806987us) log.cc:844] Serialized 14 byte log entry
0709 12:22:05.442518 (+442590us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:05.442589 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:05.442638 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.442767 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:05.442904 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38886,"prepare.queue_time_us":841754,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":57918,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":5664,"thread_start_us":1191179,"threads_started":2,"wal-append.queue_time_us":495387}
I20260709 12:22:05.669482 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.465346 16415 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.454432 16430 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.467718 13652 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:05.391304 16141 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:05.411345 16331 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:05.471771 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.171s (SENT)
I20260709 12:22:05.415539 16411 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:05.440682 16405 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.460427 16427 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:05.387027 16420 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.485085 13647 rpcz_store.cc:276] 0709 12:22:03.024044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.024663 (+   619us) service_pool.cc:224] Handling call
0709 12:22:03.025113 (+   450us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:03.030566 (+  5453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:03.030621 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:03.042693 (+ 12072us) log.cc:844] Serialized 52 byte log entry
0709 12:22:04.669031 (+1626338us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b1800238f00. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:05.082849 (+413818us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:05.082909 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.083009 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:05.368545 (+285536us) raft_consensus.cc:1685] finished
0709 12:22:05.368566 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:05.380795 (+ 12229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":44820,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":12146,"spinlock_wait_cycles":14154112,"thread_start_us":413541,"threads_started":1,"wal-append.queue_time_us":2269177}
W20260709 12:22:05.486802 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 116633us
I20260709 12:22:05.467092 16329 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:05.468382 16245 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.471753 16352 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:05.461162 16320 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:05.491932 13649 rpcz_store.cc:276] 0709 12:22:03.033624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:03.073319 (+ 39695us) service_pool.cc:224] Handling call
0709 12:22:03.074312 (+   993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:03.206617 (+132305us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:03.262658 (+ 56041us) log.cc:844] Serialized 14 byte log entry
0709 12:22:04.429741 (+1167083us) spinlock_profiling.cc:243] Waited 8.12 ms on lock 0x7b1800238fc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:22:05.295387 (+865646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:05.295467 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:05.295532 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.391705 (+ 96173us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:05.391761 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:05.391887 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1471,"prepare.queue_time_us":58286,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":4136,"raft.queue_time_us":14394,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":43908,"spinlock_wait_cycles":288747136,"thread_start_us":863496,"threads_started":1,"wal-append.queue_time_us":1708550}
I20260709 12:22:05.624436 16317 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.501995 16418 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.501447 16370 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.633399 13650 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:05.560220 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94869.3us
I20260709 12:22:05.497146 16403 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:05.443959 16237 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.515799 16283 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:05.522882 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2353) took 1090 ms. Trace:
I20260709 12:22:05.659207 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2358) took 1225 ms. Trace:
I20260709 12:22:05.659374 13798 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:05.649389 13646 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:05.647166 16402 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:05.662426 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 215853us
I20260709 12:22:05.490137 16435 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:05.510735 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.507280 16386 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.501151 16406 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.472417 16448 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 12:22:05.665084 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 164247us
W20260709 12:22:05.519292 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.555161 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:05.673408 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.673647 13799 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:05.792188 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 306) took 2211 ms. Trace:
W20260709 12:22:05.792764 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119486us
I20260709 12:22:05.792138 13644 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:05.682134 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:05.712162 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:05.524477 16125 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.761358 13800 rpcz_store.cc:276] 0709 12:22:04.433488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.736703 (+303215us) service_pool.cc:224] Handling call
0709 12:22:05.539871 (+803168us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:05.659139 (+119268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460021120}
I20260709 12:22:05.762652 13797 rpcz_store.cc:276] 0709 12:22:04.432243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.453161 (+ 20918us) service_pool.cc:224] Handling call
0709 12:22:05.522690 (+1069529us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:05.522812 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2039474432}
W20260709 12:22:05.764458 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 457572us
I20260709 12:22:05.804133 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:05.515918 16286 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.261s
I20260709 12:22:05.722860 16415 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:05.519361 16361 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:05.536607 16378 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:05.663744 16176 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:05.658674 16437 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:05.536892 15623 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.721683 16331 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.792596 13798 rpcz_store.cc:276] 0709 12:22:03.581211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:04.791850 (+1210639us) service_pool.cc:224] Handling call
0709 12:22:05.659181 (+867331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:05.762357 (+103176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.791975 (+ 29618us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54000e47d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:05.792122 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993845504}
I20260709 12:22:05.676801 13795 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:22:05.843070 13795 pending_rounds.cc:85] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 7
I20260709 12:22:05.795547 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:05.770663 16427 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.805657 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:22:05.808296 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.806051 13794 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:05.845672 13795 pending_rounds.cc:107] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 7.8
W20260709 12:22:05.768149 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.860s (SENT)
I20260709 12:22:05.846181 13795 raft_consensus.cc:2889] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:05.807199 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:05.772419 16405 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:05.765209 16329 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:05.786715 16383 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:22:05.678756 16430 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:05.722425 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:05.844046 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49771.4us
I20260709 12:22:05.846181 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:05.745007 16223 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:05.852485 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45398.6us
I20260709 12:22:05.870857 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:05.771054 16352 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:05.855049 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:05.771632 16435 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:05.856427 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 384760us
I20260709 12:22:05.841786 16378 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:05.875043 13617 connection.cc:256] RPC call timeout handler was delayed by -2.0610790790001374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3352s for a response
I20260709 12:22:05.873749 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:05.680986 16394 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 12:22:05.789673 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:05.806342 16283 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:05.807245 16125 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:05.800047 16313 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:22:05.903286 16320 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.510900 16431 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.911360 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1647) took 1048 ms. Trace:
I20260709 12:22:05.807751 16448 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.844720 16349 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:22:05.877101 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.858039 16386 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:05.876372 16435 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.884092 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:05.929919 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45965.7us
I20260709 12:22:05.860466 15939 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:05.889496 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:05.934593 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45209.3us
I20260709 12:22:05.935704 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:05.936307 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:05.806089 16156 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:05.890429 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100810us
I20260709 12:22:05.720450 16141 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.864898 16329 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:05.861490 16334 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:05.898494 15623 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.899725 16378 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.889039 16283 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:05.714600 16279 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:05.917883 13794 rpcz_store.cc:276] 0709 12:22:04.863072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.136768 (+273696us) service_pool.cc:224] Handling call
0709 12:22:05.911162 (+774394us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54002887d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:05.911299 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177351552}
I20260709 12:22:05.872352 16361 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:05.879982 16352 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:05.926671 13617 connection.cc:256] RPC call timeout handler was delayed by -1.8164485299998887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 12:22:05.951738 13617 connection.cc:256] RPC call timeout handler was delayed by -1.7804485299998305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:22:05.952119 13617 connection.cc:256] RPC call timeout handler was delayed by -1.6031485299999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 12:22:05.789412 16420 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.885504 16383 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.908290 16451 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:22:05.925626 16452 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:22:05.913925 16282 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:05.924212 16245 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:05.856272 16023 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.786865 16403 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.956940 16378 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:05.933914 16402 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:05.933794 15939 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:05.944196 16320 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.941433 16413 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:05.946384 16286 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:05.945881 16283 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:05.951877 16141 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:05.970988 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111165us
I20260709 12:22:05.985215 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:05.949303 16349 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:05.970212 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 201862us
I20260709 12:22:05.981755 13727 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:22:05.991935 16349 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:05.992892 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:05.993892 13727 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:05.984179 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:05.994854 13722 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:22:05.985493 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:05.994181 16349 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:05.996609 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:05.997814 16349 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:05.959492 16451 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:05.987830 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49175.5us
I20260709 12:22:05.871814 16418 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:06.001127 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.106s (SENT)
I20260709 12:22:06.002241 16418 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:06.003726 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.119s (SENT)
I20260709 12:22:06.004302 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:06.005295 16327 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:06.005620 14391 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.220s (SENT)
W20260709 12:22:06.010851 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73813.1us
W20260709 12:22:05.975402 16078 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.005s
I20260709 12:22:06.011812 16078 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.011996 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:06.015223 16078 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:05.949003 16156 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.018460 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.019589 13723 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:06.021752 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64608.1us
I20260709 12:22:05.996640 16447 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:06.023317 16447 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:05.991959 16457 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:22:06.023829 16447 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:05.972010 15939 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.028678 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.032919 13644 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.033880 13718 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.028970 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.028678 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.036121 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.036528 13724 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:06.031327 13650 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:06.031824 13646 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:06.032779 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.035326 13726 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:06.025164 16447 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:05.953605 16361 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.054652 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:05.971036 16277 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:06.061631 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.062742 13724 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:06.063999 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.064968 13724 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:05.961447 16334 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:22:05.990713 16317 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:06.058177 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:06.071812 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.068698 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:06.088735 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.142236 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.101462 13644 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:22:06.143042 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:06.135201 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.008574 16023 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:06.144865 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76308.3us
I20260709 12:22:06.145928 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:06.149614 13723 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:06.151829 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.152441 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.153261 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:06.154189 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.154131 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.155382 13723 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:06.155555 13724 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 2.
I20260709 12:22:06.164814 16361 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:06.174763 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:05.978176 16337 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.179265 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.071292 16282 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:06.188596 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:06.182211 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 180547us
I20260709 12:22:06.208992 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.137743 16448 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:06.209729 13725 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.217644 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.138334 16406 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:06.211181 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:06.221560 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:06.223770 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.227306 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.229105 13650 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:05.953996 16430 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.175369 16463 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.177917 16447 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.236240 13795 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 12:22:06.240653 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48886.1us
W20260709 12:22:06.240834 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:06.287655 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138443us
W20260709 12:22:06.272475 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.289072 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.290072 13647 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:06.290247 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:06.247233 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.291450 13650 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.291711 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:06.291409 13798 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 9 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:06.234563 16237 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:06.236836 16468 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:06.304220 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:06.239478 16474 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.238929 16279 raft_consensus.cc:2764] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:06.295935 16237 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:06.292052 16430 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.249003 16223 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:06.233644 16313 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:22:06.237278 16448 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.361132 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 317) took 1030 ms. Trace:
I20260709 12:22:06.333786 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:06.380344 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 91352us
I20260709 12:22:06.363575 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:06.383352 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.362331 13799 rpcz_store.cc:276] 0709 12:22:05.330283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.792807 (+462524us) service_pool.cc:224] Handling call
0709 12:22:05.793181 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:05.800234 (+  7053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:05.800266 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:05.800774 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:22:05.967585 (+166811us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:05.967650 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:05.967761 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:06.360819 (+393058us) raft_consensus.cc:1685] finished
0709 12:22:06.360843 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:06.361050 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3703,"raft.queue_time_us":2468,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":35776,"thread_start_us":103964,"threads_started":1,"wal-append.queue_time_us":513398}
I20260709 12:22:06.358989 16469 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:22:06.386438 16469 consensus_queue.cc:1243] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 5.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:06.386653 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:06.387313 13794 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:06.246022 16394 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:22:06.397521 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:06.398134 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 85867.4us
W20260709 12:22:06.352619 16370 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 12:22:06.402077 13718 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:06.311015 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:06.351716 16156 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:06.405619 14423 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:06.307546 16464 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:06.328141 16237 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:06.342793 16465 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:06.413002 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:06.342824 16337 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:06.380858 16402 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:06.407253 16335 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.078s	user 0.015s	sys 0.000s
I20260709 12:22:06.455899 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:06.362026 16477 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:06.465756 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.288229 16463 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.400731 16313 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:22:06.471814 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58948.2us
W20260709 12:22:06.451861 16125 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.002s
I20260709 12:22:06.472715 13798 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260709 12:22:06.473896 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.014s	sys 0.018s
W20260709 12:22:06.504817 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.014s	sys 0.018s
I20260709 12:22:06.451361 16277 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:06.472721 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:06.510848 13799 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:06.511135 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.512054 13648 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:06.406689 16420 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:06.451543 16469 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.501783 13647 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.487622 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.500638 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:06.505187 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 699839us
I20260709 12:22:06.476413 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2361) took 1157 ms. Trace:
W20260709 12:22:06.505776 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 139332us
I20260709 12:22:06.360333 16282 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.475530 16471 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:06.548702 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.568265 13650 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:06.566954 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.312608 16474 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.425042 16370 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:06.570124 13793 rpcz_store.cc:276] 0709 12:22:05.318903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.793722 (+474819us) service_pool.cc:224] Handling call
0709 12:22:05.842212 (+ 48490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:05.842510 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:05.952695 (+110185us) log.cc:844] Serialized 14 byte log entry
0709 12:22:06.476018 (+523323us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:06.476090 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:06.476135 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:06.476220 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:06.476350 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64455,"prepare.queue_time_us":184733,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":12603,"thread_start_us":551743,"threads_started":2,"wal-append.queue_time_us":550731}
W20260709 12:22:06.503237 16415 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.005s
I20260709 12:22:06.503602 16317 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:06.645915 13650 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:06.475870 16464 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:06.470798 16335 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.514127 16277 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:06.654590 14420 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:06.641443 16469 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:06.498369 16454 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.053s	sys 0.075s
I20260709 12:22:06.506999 16125 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.674357 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.684958 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.658353 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:06.671396 16472 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:22:06.746088 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.804258 13725 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:06.805789 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.690909 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.470570 16475 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.140s
W20260709 12:22:06.571872 16405 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.001s
W20260709 12:22:06.563720 16286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.011s	sys 0.032s
W20260709 12:22:06.705565 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52948us
I20260709 12:22:06.570902 16298 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:06.642707 16471 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.830135 16298 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.832435 16298 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:06.715842 14423 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:06.467120 16402 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.722695 13796 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:06.669976 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.834431 13720 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:06.836575 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.680641 14418 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.669755 14421 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:22:06.819360 15623 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:06.738564 16317 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:06.840462 15623 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.841307 16317 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.746600 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:06.844938 13693 leader_election.cc:400] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:06.845367 13647 raft_consensus.cc:2376] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:22:06.845404 13693 leader_election.cc:403] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:06.750463 16454 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.053s	sys 0.088s
I20260709 12:22:06.749684 16427 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.655879 16479 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:06.754041 16464 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.755823 16469 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:06.757162 16370 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:06.687122 16431 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.762370 16474 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:06.764931 15939 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.000s
I20260709 12:22:06.766687 16335 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:06.699671 16397 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:06.772986 16435 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.828s	user 0.008s	sys 0.002s
W20260709 12:22:06.773679 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:06.773985 16386 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.002s
I20260709 12:22:06.774312 14422 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:06.775715 16352 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.813s	user 0.010s	sys 0.001s
I20260709 12:22:06.778364 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:07.095988 13718 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:06.778801 13646 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:06.722045 16420 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:06.781708 16463 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:06.783452 16482 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:22:06.791268 16334 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:06.792080 16448 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:06.797288 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:07.106705 13799 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:06.802906 13650 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:07.108417 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:07.109251 13650 raft_consensus.cc:2376] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5. Re-sending same reply.
W20260709 12:22:06.667994 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.112780 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.113364 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.810271 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:06.802459 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:07.123591 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.125708 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.126713 13650 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:07.129729 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:06.818370 16286 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:06.820442 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:06.812382 16378 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.840s	user 0.006s	sys 0.006s
I20260709 12:22:06.829319 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.734051 16176 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:06.864295 14424 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:06.868484 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:06.866153 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:06.875555 16298 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
W20260709 12:22:06.873837 16418 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.871s	user 0.011s	sys 0.004s
W20260709 12:22:06.888427 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43625.9us
I20260709 12:22:06.895727 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1658) took 1964 ms. Trace:
W20260709 12:22:06.993088 16237 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.000s
I20260709 12:22:06.836005 16473 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:22:06.942984 13725 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:06.947916 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:06.886857 16427 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:06.951289 14423 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:06.784178 16411 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:07.000882 16283 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.972s	user 0.003s	sys 0.009s
I20260709 12:22:07.218477 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2357) took 2785 ms. Trace:
I20260709 12:22:06.884145 16479 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:07.009936 16329 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.061s	user 0.010s	sys 0.001s
I20260709 12:22:06.993989 15623 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:06.953320 15939 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:06.833523 16405 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.669361 16451 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:06.996245 14420 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:22:07.007009 16023 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.841s	user 0.010s	sys 0.002s
W20260709 12:22:07.069546 16383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.086s	sys 0.108s
W20260709 12:22:07.092986 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 240980us
W20260709 12:22:06.674065 16320 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.698s	user 0.006s	sys 0.001s
W20260709 12:22:07.141068 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 338677us
I20260709 12:22:07.145434 13719 raft_consensus.cc:2410] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:07.153120 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:07.320843 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 167853us
I20260709 12:22:07.320570 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:07.155232 13722 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:07.197847 13645 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:07.200680 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.322373 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:07.322875 13650 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:22:07.157662 16476 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.004s	sys 0.009s
I20260709 12:22:07.203593 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:07.330642 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 127164us
I20260709 12:22:07.331732 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:07.202921 13801 rpcz_store.cc:276] 0709 12:22:04.931482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.755293 (+823811us) service_pool.cc:224] Handling call
0709 12:22:05.755898 (+   605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:06.011378 (+255480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:06.011412 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:06.012052 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 12:22:06.186524 (+174472us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:06.186600 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:06.743314 (+556714us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000d25d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:06.743415 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:06.895258 (+151843us) raft_consensus.cc:1685] finished
0709 12:22:06.895284 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:06.895627 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54698,"raft.queue_time_us":169863,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":24473,"spinlock_wait_cycles":495394048,"thread_start_us":207646,"threads_started":2,"wal-append.queue_time_us":389562}
W20260709 12:22:06.888190 16141 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.879s	user 0.010s	sys 0.002s
I20260709 12:22:07.229501 13796 rpcz_store.cc:276] 0709 12:22:04.433254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.337380 (+904126us) service_pool.cc:224] Handling call
0709 12:22:05.337777 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:06.833810 (+1496033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.218021 (+384211us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:07.218372 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3184355712}
I20260709 12:22:07.335894 13796 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:07.223338 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.157827 16465 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.000s	sys 0.019s
I20260709 12:22:07.219503 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:07.339180 13801 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:07.341931 13643 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:07.343520 13801 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:07.304083 16402 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.201527 16481 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.116962 16435 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.140466 16352 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:07.307129 16447 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.012s	user 0.012s	sys 0.002s
I20260709 12:22:07.311298 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 561) took 1318 ms. Trace:
I20260709 12:22:07.357070 14422 rpcz_store.cc:276] 0709 12:22:05.993288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.995265 (+  1977us) service_pool.cc:224] Handling call
0709 12:22:05.995617 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.095387 (+1099770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.310932 (+215545us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54007a0154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:07.311207 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1790088576}
I20260709 12:22:07.145819 16420 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.381572 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.382882 13800 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:07.301196 16411 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.323647 13726 raft_consensus.cc:2368] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6.
I20260709 12:22:07.202565 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:07.307467 16277 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
W20260709 12:22:07.341251 16465 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.000s	sys 0.020s
I20260709 12:22:07.324452 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:07.328802 13724 raft_consensus.cc:2410] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:07.320156 15623 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:07.340091 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:07.094681 16397 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.344041 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:07.160039 16492 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:07.229097 16237 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.399830 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:07.400832 16490 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:22:07.401083 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.401289 16237 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.401911 13643 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:07.223146 16176 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.359526 16481 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.404352 16176 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.404647 16481 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:07.405769 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81424.4us
W20260709 12:22:07.339140 16476 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.004s	sys 0.010s
I20260709 12:22:07.342576 16463 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:07.357586 13796 pending_rounds.cc:85] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 6
I20260709 12:22:07.410671 13796 pending_rounds.cc:107] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:07.411052 13796 raft_consensus.cc:2889] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:07.411194 16492 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.232628 16477 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:07.411752 16492 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.412545 16397 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.412424 16477 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.291384 16283 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.414426 16413 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:07.414882 16413 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.415261 16413 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.416009 16283 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:07.363047 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.364044 16488 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:22:07.364972 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:07.388722 16465 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:07.151829 16334 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.367660 16405 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.369457 15939 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.345912 16286 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.301044 16329 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.374560 13649 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:07.353840 16430 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:22:07.339706 16418 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:07.375492 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:07.378245 16478 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.735s	user 0.004s	sys 0.014s
I20260709 12:22:07.378710 13801 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:07.463864 16430 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:07.464156 13801 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:07.464373 16430 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:07.379416 16462 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.138s	user 0.010s	sys 0.003s
I20260709 12:22:07.393402 16277 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.390089 13727 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:07.465452 16430 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.392138 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.467201 16430 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:07.467576 16277 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.406826 16378 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:22:07.468148 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.468597 16349 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:07.468914 16329 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.469038 16349 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:07.469766 13794 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:07.339855 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:07.470041 16349 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.470606 13650 raft_consensus.cc:2376] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4. Re-sending same reply.
I20260709 12:22:07.471697 16479 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.392203 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:07.394091 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 228010us
I20260709 12:22:07.473599 13718 pending_rounds.cc:85] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 1
I20260709 12:22:07.473922 13718 pending_rounds.cc:107] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 5.2
I20260709 12:22:07.469810 13798 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:07.474463 13718 raft_consensus.cc:2889] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:07.396118 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.340437 16451 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:07.395327 13799 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:07.397608 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57636us
W20260709 12:22:07.393056 16282 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.017s	sys 0.060s
I20260709 12:22:07.394397 16402 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.346984 16279 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:07.315634 16383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.106s	user 0.086s	sys 0.108s
I20260709 12:22:07.406158 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.346118 16497 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:22:07.408244 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.490082 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 145237us
I20260709 12:22:07.409235 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:07.428527 13797 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:07.393546 16447 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.414021 16492 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.406803 16156 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:07.406329 16141 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.386584 16313 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:22:07.407051 16435 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.508507 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:07.489732 16402 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.510886 16141 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.513242 16141 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:07.513846 16431 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.619s	user 0.007s	sys 0.000s
I20260709 12:22:07.456633 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.514972 16237 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:07.515700 13643 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:22:07.457806 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:07.462078 16418 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.340996 16448 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:07.475919 13648 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:07.483767 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:07.480356 13792 raft_consensus.cc:606] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:07.520843 13792 raft_consensus.cc:614] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:07.479872 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87798.5us
I20260709 12:22:07.492362 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:07.523451 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:07.494334 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:07.523844 16435 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:07.524403 16435 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:07.527467 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1903) took 1023 ms. Trace:
I20260709 12:22:07.497013 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:07.527875 13648 rpcz_store.cc:276] 0709 12:22:06.504169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.383194 (+879025us) service_pool.cc:224] Handling call
0709 12:22:07.527396 (+144202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:07.405864 15623 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.479041 16461 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:07.504138 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 94948.5us
I20260709 12:22:07.483022 16279 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.535998 16279 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.516525 16176 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:22:07.537617 16176 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:07.537966 16176 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:07.538446 16461 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:07.479223 16282 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.017s	sys 0.060s
I20260709 12:22:07.516006 16277 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:07.529132 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:07.533826 16156 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:07.517796 16492 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:07.539852 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:07.519596 16418 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.500769 16473 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.500391 16481 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.520941 16497 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:07.517293 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:07.485900 16479 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:07.550295 16245 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.501032 16447 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.544621 13801 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:07.573082 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:07.548590 16465 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.525535 16435 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:07.544354 16431 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:07.540905 16476 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:07.554921 16361 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:07.584275 16386 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.561913 16405 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:07.544540 16430 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:22:07.547194 16463 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:22:07.548874 16448 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:22:07.572819 16492 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:07.553246 16277 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.577551 16431 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:07.589113 16476 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:07.599313 14419 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:07.599677 16476 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.599999 16431 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.601328 16431 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:07.594877 16386 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.565512 16473 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.590453 16405 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:07.605760 16405 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.606956 16435 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:07.608809 16402 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.608636 16497 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.613111 16461 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.603861 16277 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.613788 16463 consensus_queue.cc:1243] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:07.615605 16473 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.571447 16481 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.627131 16471 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.629544 16420 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.617332 16492 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.632244 16352 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.633337 16334 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.634464 16447 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:07.636525 16491 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.345s	user 0.002s	sys 0.000s
I20260709 12:22:07.636835 16481 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.633162 16492 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.649462 16479 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:07.650017 16479 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:07.651091 16479 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.652561 16479 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.668215 16476 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:07.748078 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:07.557626 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.752933 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1734) took 1058 ms. Trace:
I20260709 12:22:07.753927 13718 rpcz_store.cc:276] 0709 12:22:06.694607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.473024 (+778417us) service_pool.cc:224] Handling call
0709 12:22:07.473351 (+   327us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:07.498318 (+ 24967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.498367 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:07.499630 (+  1263us) log.cc:844] Serialized 78 byte log entry
0709 12:22:07.751695 (+252065us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:07.751879 (+   184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.752100 (+   221us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:07.752195 (+    95us) raft_consensus.cc:1685] finished
0709 12:22:07.752222 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.752510 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11183,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":12338,"thread_start_us":236318,"threads_started":1,"wal-append.queue_time_us":102421}
I20260709 12:22:07.758647 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:07.760113 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1905) took 1251 ms. Trace:
I20260709 12:22:07.764600 13650 rpcz_store.cc:276] 0709 12:22:06.508427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.471537 (+963110us) service_pool.cc:224] Handling call
0709 12:22:07.471910 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:07.472146 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.472182 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:07.486497 (+ 14315us) log.cc:844] Serialized 26 byte log entry
0709 12:22:07.759278 (+272781us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:07.759362 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.759503 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:07.759602 (+    99us) raft_consensus.cc:1685] finished
0709 12:22:07.759619 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.759848 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":271973,"threads_started":1,"wal-append.queue_time_us":155485}
I20260709 12:22:07.761400 13721 raft_consensus.cc:2410] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:22:07.775905 13799 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:07.776750 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:07.780400 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 491) took 1022 ms. Trace:
I20260709 12:22:07.780896 13718 rpcz_store.cc:276] 0709 12:22:06.757794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.778795 (+1021001us) service_pool.cc:224] Handling call
0709 12:22:07.779495 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.779944 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.779979 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:07.780059 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:07.780114 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.780178 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.780295 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:07.614071 16176 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.784924 15623 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.785912 15623 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.786068 16176 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.787017 15623 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:07.783772 14391 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.508s (SENT)
I20260709 12:22:07.790652 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:07.793202 14391 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.722s (SENT)
I20260709 12:22:07.793737 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:07.794730 16317 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.004s	sys 0.008s
I20260709 12:22:07.574586 16245 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.794929 16254 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:07.795378 16317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.004s	sys 0.008s
I20260709 12:22:07.806699 16402 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.807828 16402 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:07.838362 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.843432 16245 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.845204 16476 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:07.843452 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.847551 16334 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.848547 16334 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.850471 16446 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.851397 16446 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.853565 16402 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.855334 16446 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:07.856769 16413 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:07.860260 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.860881 16413 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.861179 16479 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.862130 16479 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.867527 16479 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:07.876389 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 392433us
W20260709 12:22:07.881950 16520 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:22:07.885017 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 365593us
I20260709 12:22:07.886703 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:07.887415 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94329.4us
I20260709 12:22:07.887668 16448 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:07.887475 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.888305 16438 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:07.893213 16023 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:07.893522 16334 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.894117 16023 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:07.894572 16448 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:07.895810 15623 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.897639 16023 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:07.888110 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:07.921159 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:07.924785 13648 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:22:07.905691 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:07.899591 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:07.933362 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 401) took 1052 ms. Trace:
I20260709 12:22:07.927099 13648 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 5
I20260709 12:22:07.933965 13643 rpcz_store.cc:276] 0709 12:22:06.831579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.405827 (+574248us) service_pool.cc:224] Handling call
0709 12:22:07.877939 (+472112us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54001487d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:07.883663 (+  5724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66394880}
I20260709 12:22:07.934671 13648 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:22:07.935045 13648 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:07.935433 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:07.936337 13643 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:07.938985 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:07.900882 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 392) took 1726 ms (client timeout 2054 ms). Trace:
I20260709 12:22:07.901559 14420 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:22:07.942890 13646 rpcz_store.cc:269] 0709 12:22:06.174242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.174415 (+   173us) service_pool.cc:224] Handling call
0709 12:22:07.899737 (+1725322us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400147dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:07.900402 (+   665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385781504}
I20260709 12:22:07.902522 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:07.898092 16448 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:07.905457 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:07.949064 14419 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
W20260709 12:22:07.963086 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2219) took 1979 ms (client timeout 1506 ms). Trace:
W20260709 12:22:07.963686 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58324.5us
W20260709 12:22:07.963793 13719 rpcz_store.cc:269] 0709 12:22:05.983160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.983291 (+   131us) service_pool.cc:224] Handling call
0709 12:22:07.962304 (+1979013us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400174654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:07.962663 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544333696}
I20260709 12:22:07.964947 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:07.965603 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:07.967554 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 486) took 1283 ms. Trace:
I20260709 12:22:07.952590 15623 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:07.970172 13718 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:07.969789 13723 rpcz_store.cc:276] 0709 12:22:06.683657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.686811 (+  3154us) service_pool.cc:224] Handling call
0709 12:22:06.708461 (+ 21650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:06.834229 (+125768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:06.834311 (+    82us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:06.834386 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:06.834435 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.966907 (+1132472us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54003fe8d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:07.967016 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.967302 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288618240}
I20260709 12:22:07.978886 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.980253 16334 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:07.981117 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:07.981513 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.818s (SENT)
W20260709 12:22:07.980506 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2238) took 1813 ms (client timeout 2030 ms). Trace:
W20260709 12:22:07.983475 13724 rpcz_store.cc:269] 0709 12:22:06.167527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.167811 (+   284us) service_pool.cc:224] Handling call
0709 12:22:07.979989 (+1812178us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003bea54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:07.980277 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2549412736}
I20260709 12:22:07.982218 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:07.982193 13652 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:22:07.984845 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1575) took 1130 ms. Trace:
I20260709 12:22:07.987972 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:07.994946 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:07.995687 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2234) took 1958 ms (client timeout 1902 ms). Trace:
W20260709 12:22:07.996245 13726 rpcz_store.cc:269] 0709 12:22:06.037159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.039151 (+  1992us) service_pool.cc:224] Handling call
0709 12:22:07.994824 (+1955673us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:07.995254 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712940928}
I20260709 12:22:07.998852 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1906) took 1481 ms. Trace:
I20260709 12:22:07.999295 13645 rpcz_store.cc:276] 0709 12:22:06.517004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.515459 (+998455us) service_pool.cc:224] Handling call
0709 12:22:07.515987 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:07.538815 (+ 22828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.538849 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:07.539346 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:22:07.764121 (+224775us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:07.764203 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.998164 (+233961us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b54001e6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:07.998215 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:07.998309 (+    94us) raft_consensus.cc:1685] finished
0709 12:22:07.998324 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.998636 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51839360,"thread_start_us":224150,"threads_started":1,"wal-append.queue_time_us":224644}
W20260709 12:22:08.001628 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2216) took 2019 ms (client timeout 2537 ms). Trace:
W20260709 12:22:08.002094 13721 rpcz_store.cc:269] 0709 12:22:05.981916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.982064 (+   148us) service_pool.cc:224] Handling call
0709 12:22:08.001345 (+2019281us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54002f5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.001554 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3965260032}
I20260709 12:22:08.003074 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1732) took 1310 ms. Trace:
I20260709 12:22:08.003399 14422 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:07.982467 16479 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:07.972312 15623 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:08.004220 15623 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:07.997973 16427 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.765s	user 0.008s	sys 0.008s
W20260709 12:22:08.012117 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 125709us
I20260709 12:22:08.003523 13727 rpcz_store.cc:276] 0709 12:22:06.692475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.466590 (+774115us) service_pool.cc:224] Handling call
0709 12:22:07.466954 (+   364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:07.477812 (+ 10858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.477843 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:07.482010 (+  4167us) log.cc:844] Serialized 52 byte log entry
0709 12:22:07.557052 (+ 75042us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:07.960792 (+403740us) log.cc:844] Serialized 14 byte log entry
0709 12:22:08.002417 (+ 41625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.002541 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:08.002649 (+   108us) raft_consensus.cc:1685] finished
0709 12:22:08.002666 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.002845 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21693,"raft.queue_time_us":30630,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":10920,"thread_start_us":239164,"threads_started":2,"wal-append.queue_time_us":110414}
I20260709 12:22:08.018625 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 389) took 1936 ms. Trace:
I20260709 12:22:08.018857 13646 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:08.019219 13649 rpcz_store.cc:276] 0709 12:22:06.032398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.034407 (+  2009us) service_pool.cc:224] Handling call
0709 12:22:06.034704 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.455141 (+1420437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.922770 (+467629us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001e5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:07.969227 (+ 46457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3080849280}
I20260709 12:22:08.022388 16176 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:08.023129 16176 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:08.024510 16176 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.035100 13644 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:22:08.037039 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 323) took 1243 ms. Trace:
W20260709 12:22:08.037864 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.048372 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:08.038652 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 319) took 2533 ms. Trace:
I20260709 12:22:08.040512 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 318) took 2700 ms. Trace:
W20260709 12:22:08.042043 16279 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.505s	user 0.014s	sys 0.007s
I20260709 12:22:08.042392 13719 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:08.046087 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:08.047501 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66083us
I20260709 12:22:08.033921 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.050820 13800 rpcz_store.cc:276] 0709 12:22:05.339547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.803444 (+463897us) service_pool.cc:224] Handling call
0709 12:22:05.847711 (+ 44267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.387453 (+1539742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.039903 (+652450us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54003d4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.040267 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3530124032}
I20260709 12:22:08.049122 13793 rpcz_store.cc:276] 0709 12:22:06.793035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.794554 (+  1519us) service_pool.cc:224] Handling call
0709 12:22:06.795248 (+   694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.384002 (+588754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:07.384035 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:07.384076 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:07.384104 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.036373 (+652269us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54000e56d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.036458 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.036800 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353475456}
I20260709 12:22:08.050393 13797 rpcz_store.cc:276] 0709 12:22:05.504834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.803765 (+298931us) service_pool.cc:224] Handling call
0709 12:22:05.804130 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.491633 (+1687503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.038037 (+546404us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54002cbfd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.038438 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3712570624}
I20260709 12:22:08.064472 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:08.067325 16481 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.067037 13649 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.068244 16481 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:08.071417 14424 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 533) took 1705 ms (client timeout 1592 ms). Trace:
I20260709 12:22:08.072149 14425 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:08.071902 14424 rpcz_store.cc:269] 0709 12:22:06.365643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.365791 (+   148us) service_pool.cc:224] Handling call
0709 12:22:08.070866 (+1705075us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.071126 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070552320}
I20260709 12:22:08.048169 16023 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:07.990682 13650 rpcz_store.cc:276] 0709 12:22:06.854760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.767572 (+912812us) service_pool.cc:224] Handling call
0709 12:22:07.768260 (+   688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.768770 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:07.826584 (+ 57814us) log.cc:844] Serialized 14 byte log entry
0709 12:22:07.973741 (+147157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:07.973815 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:07.984380 (+ 10565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:07.984508 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:07.984710 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7906,"prepare.queue_time_us":63330,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":13234,"raft.run_cpu_time_us":517,"raft.run_wall_time_us":512,"thread_start_us":180980,"threads_started":2}
I20260709 12:22:08.085796 13649 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:08.094873 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:08.100790 13718 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: [4.3-6.5]
I20260709 12:22:08.106534 15623 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:08.111299 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77717.9us
I20260709 12:22:08.112242 16394 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:08.100239 14391 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.842s (SENT)
I20260709 12:22:08.114037 16125 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.118213 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:08.120850 16405 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.014s
I20260709 12:22:08.121193 16511 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:08.121762 16511 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:08.121570 16405 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.122229 16511 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:08.123296 13722 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:08.126758 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:08.128192 16471 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.128592 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.129048 16471 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.129707 13793 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:08.131282 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.134032 16471 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.134349 16394 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.134768 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.135017 16394 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.135311 16125 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:08.131160 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48807us
I20260709 12:22:08.132333 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:08.136824 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61479.1us
I20260709 12:22:08.137624 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.138674 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:08.131312 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.140211 16435 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:08.139601 13797 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
I20260709 12:22:08.140754 16435 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:08.141263 16435 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:08.142689 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:08.144515 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.150763 16479 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:08.152034 16411 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:08.152433 16411 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:08.060087 16279 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.153644 16411 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.155884 16411 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:08.156481 16279 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:08.156602 16479 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:08.168524 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:08.171574 16298 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:08.172068 16298 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.172382 16298 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.173707 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:08.174809 16479 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:08.175387 16479 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.175799 16479 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:08.176369 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.177084 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.177743 16521 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.178021 14421 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:08.178483 16521 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.181682 16521 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.183600 16405 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.184582 16521 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.185655 16521 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.188282 16521 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.173740 16435 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.189121 16521 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:08.191490 16405 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:08.192749 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42338 (request call id 23) took 2656 ms. Trace:
I20260709 12:22:08.192869 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.193183 13792 rpcz_store.cc:276] 0709 12:22:05.535829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:05.776341 (+240512us) service_pool.cc:224] Handling call
0709 12:22:08.192311 (+2415970us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bee853 00007f5398bbc60a 00007f53921427d4 00007f5392142762 00007f5392142691 00007f539214260c 00007f53921418c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.192509 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3759566592}
I20260709 12:22:08.195152 16435 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:08.197084 13644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 391) took 2046 ms (client timeout 1818 ms). Trace:
W20260709 12:22:08.197763 13644 rpcz_store.cc:269] 0709 12:22:06.151180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.151448 (+   268us) service_pool.cc:224] Handling call
0709 12:22:08.196758 (+2045310us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.196977 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4296884992}
I20260709 12:22:08.199338 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:08.199514 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:08.199985 13644 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:08.201305 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:08.202643 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 436) took 1555 ms. Trace:
I20260709 12:22:08.202925 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.203101 14425 rpcz_store.cc:276] 0709 12:22:06.647284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.647546 (+   262us) service_pool.cc:224] Handling call
0709 12:22:08.071660 (+1424114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:08.072693 (+  1033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.202375 (+129682us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54006c1e50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.202558 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3273519872}
I20260709 12:22:08.204090 14419 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:08.204734 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.205492 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:08.201344 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:08.195143 14391 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.001s (SENT)
W20260709 12:22:08.209721 14391 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.054s (SENT)
I20260709 12:22:08.215696 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:08.219101 16438 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:08.221562 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.783s (SENT)
I20260709 12:22:08.222096 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:08.222930 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 495) took 1223 ms. Trace:
I20260709 12:22:08.223421 13722 rpcz_store.cc:276] 0709 12:22:06.999229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.965623 (+966394us) service_pool.cc:224] Handling call
0709 12:22:07.966206 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:08.123761 (+157555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.222631 (+ 98870us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54002f6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.222845 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360321792}
I20260709 12:22:08.224203 15623 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.227551 16386 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.232172 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:08.236001 16349 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:08.237455 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1576) took 1382 ms. Trace:
I20260709 12:22:08.238061 13648 rpcz_store.cc:276] 0709 12:22:06.855182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.831495 (+976313us) service_pool.cc:224] Handling call
0709 12:22:07.901385 (+ 69890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.978418 (+ 77033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.237119 (+258701us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b5400147dd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.237361 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132821376}
I20260709 12:22:08.238433 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:08.187953 16479 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:08.241611 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.242484 13650 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:08.246490 13648 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:08.261746 13643 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
W20260709 12:22:08.266831 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:08.245694 16418 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.007s
I20260709 12:22:08.269645 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:08.284142 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.285096 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1904) took 1780 ms. Trace:
I20260709 12:22:08.286273 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:08.286739 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148723us
I20260709 12:22:08.288565 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:08.291172 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 325) took 1497 ms. Trace:
I20260709 12:22:08.298869 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:08.291742 13794 rpcz_store.cc:276] 0709 12:22:06.793749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.799564 (+  5815us) service_pool.cc:224] Handling call
0709 12:22:06.799913 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.470214 (+670301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.290476 (+820262us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.290886 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540061568}
I20260709 12:22:08.308234 13650 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:08.321894 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:08.285641 13647 rpcz_store.cc:276] 0709 12:22:06.504486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.469141 (+964655us) service_pool.cc:224] Handling call
0709 12:22:07.469456 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.502419 (+ 32963us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:07.992685 (+490266us) log.cc:844] Serialized 14 byte log entry
0709 12:22:08.268373 (+275688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:08.282396 (+ 14023us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:08.282579 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.284420 (+  1841us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b5400097c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.284479 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.284701 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":329414,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":81,"raft.queue_time_us":7705,"spinlock_wait_cycles":75142528,"thread_start_us":725806,"threads_started":2}
I20260709 12:22:08.327929 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:08.327442 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:08.298439 16448 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:08.323863 15623 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.339390 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.335630 16405 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:08.344537 16405 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:08.327133 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.346112 14419 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:08.346369 16430 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:08.346853 16430 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:08.347267 16430 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:08.347313 16405 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.350553 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:08.351019 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1661) took 1878 ms. Trace:
I20260709 12:22:08.351416 13798 rpcz_store.cc:276] 0709 12:22:06.472220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.505496 (+ 33276us) service_pool.cc:224] Handling call
0709 12:22:06.505897 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:07.480177 (+974280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.350464 (+870287us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:08.350824 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2216263424}
I20260709 12:22:08.352380 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:08.335783 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.354808 16329 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.305728 16418 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.358647 13647 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:08.360034 16418 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.360288 13647 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:22:08.360703 13647 pending_rounds.cc:85] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 3
I20260709 12:22:08.360962 13647 pending_rounds.cc:107] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:22:08.361205 13647 raft_consensus.cc:2889] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:08.361390 13645 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:08.362689 16282 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:08.363406 16282 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.364674 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.364807 16413 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:08.365877 16282 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.369481 16479 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.345026 16334 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.369546 13644 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:08.349596 16298 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.327919 16386 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.405557 16405 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.410796 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1912) took 1003 ms. Trace:
I20260709 12:22:08.411860 16405 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:08.413322 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.416052 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:08.411547 13649 rpcz_store.cc:276] 0709 12:22:07.407120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.114392 (+707272us) service_pool.cc:224] Handling call
0709 12:22:08.242974 (+128582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:08.243312 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:08.244154 (+   842us) log.cc:844] Serialized 14 byte log entry
0709 12:22:08.403817 (+159663us) log.cc:844] Serialized 14 byte log entry
0709 12:22:08.404217 (+   400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:08.404298 (+    81us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:08.410413 (+  6115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.410526 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.410710 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":113,"raft.queue_time_us":10284,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":8368,"spinlock_wait_cycles":16896,"thread_start_us":133930,"threads_started":1}
I20260709 12:22:08.421335 13793 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:08.423024 13645 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:08.422828 16386 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.424011 16405 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.425274 16386 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.386277 16394 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.428578 16394 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:08.429736 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.431048 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.432204 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.433100 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.433902 16279 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:08.434382 16279 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.434722 16283 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:08.435405 16283 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:08.435436 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:08.436408 14421 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:08.436689 16283 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.439919 16298 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.442473 16448 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.445290 16448 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.445930 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:08.446691 16471 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:08.447274 16471 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:08.447383 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52592.5us
I20260709 12:22:08.448027 16471 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.448679 16464 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:08.449381 16464 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:08.392258 15623 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.451323 13796 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:08.358122 16511 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:08.463143 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 172878us
I20260709 12:22:08.450771 16464 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.469033 13650 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 12:22:08.470908 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:08.471511 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.450824 16279 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:08.396524 16334 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.469666 13650 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 4
I20260709 12:22:08.445932 16329 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:08.477262 13650 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:22:08.477711 13650 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:08.478659 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.487469 13722 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:08.486294 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.489414 13649 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:08.479532 16329 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.512635 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.502858 16329 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.526945 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.527858 13720 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:22:08.528606 16313 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.529240 16313 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.531754 16334 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.530004 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.538933 13724 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:08.541954 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:08.545715 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:08.546877 16528 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:22:08.548090 13651 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:08.548805 16418 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:08.549353 16418 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:08.549427 16511 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.549562 16435 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.550597 16418 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.551102 16481 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.555495 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:08.561192 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.558871 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:08.542315 16488 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:22:08.559758 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:08.570616 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 392885us
I20260709 12:22:08.579566 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:08.581542 13798 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:08.582464 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:08.585398 13718 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.596155 16477 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.581300 16334 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.659804 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 437) took 2011 ms. Trace:
I20260709 12:22:08.660403 14416 rpcz_store.cc:276] 0709 12:22:06.648740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:06.655188 (+  6448us) service_pool.cc:224] Handling call
0709 12:22:06.674200 (+ 19012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:06.725605 (+ 51405us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:08.068465 (+1342860us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b18003c2320. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:22:08.068744 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 12:22:08.587893 (+519149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:08.659169 (+ 71276us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:08.659265 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.659398 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:08.659606 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":338083,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":18533,"spinlock_wait_cycles":63969664,"thread_start_us":813849,"threads_started":2}
W20260709 12:22:08.662479 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 107267us
I20260709 12:22:08.663412 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:08.664736 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.667300 13647 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:08.669194 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 1916) took 1260 ms. Trace:
I20260709 12:22:08.669739 13651 rpcz_store.cc:276] 0709 12:22:07.408518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:07.457557 (+ 49039us) service_pool.cc:224] Handling call
0709 12:22:08.668966 (+1211409us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b5400095454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:08.669119 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2253669632}
I20260709 12:22:08.674853 15623 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.677326 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:08.678244 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 81223us
I20260709 12:22:08.680387 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:08.684002 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:08.684342 16313 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.693950 16438 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.697817 16438 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:08.699196 16438 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:08.700624 16438 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.710701 16476 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:08.711958 16476 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.712627 16431 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:08.712983 16488 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:22:08.713344 16431 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.716729 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.717653 13723 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:22:08.688090 13550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 236) took 14104 ms (client timeout 15000 ms). Trace:
W20260709 12:22:08.721474 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 147772us
I20260709 12:22:08.722946 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:08.723901 16298 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.722112 13545 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:08.664669 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:08.731133 13798 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:08.733583 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:22:08.736115 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53697us
I20260709 12:22:08.740543 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:08.783270 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 103030us
W20260709 12:22:08.783655 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45243us
W20260709 12:22:08.803251 14417 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.063s	sys 0.112s
I20260709 12:22:08.803568 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:08.804013 14417 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.063s	sys 0.112s
I20260709 12:22:08.803610 13618 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:08.804397 13792 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:08.804567 13618 leader_election.cc:403] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 12:22:08.718990 13550 rpcz_store.cc:269] 0709 12:21:54.583690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:54.583862 (+   172us) service_pool.cc:224] Handling call
0709 12:22:08.687346 (+14103484us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:01.508084 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:01.931179 (+423095us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:01.941434 (+ 10255us) write_op.cc:435] Acquired schema lock
0709 12:22:01.945576 (+  4142us) tablet.cc:662] Decoding operations
0709 12:22:01.968064 (+ 22488us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:01.968158 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:01.968190 (+    32us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:22:02.860581 (+892391us) tablet.cc:701] Row locks acquired
0709 12:22:02.909189 (+ 48608us) write_op.cc:260] PREPARE: finished
0709 12:22:02.989638 (+ 80449us) write_op.cc:270] Start()
0709 12:22:03.026694 (+ 37056us) write_op.cc:276] Timestamp: P: 1783599722967010 usec, L: 0
0709 12:22:03.026741 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 12:22:03.107642 (+ 80901us) log.cc:844] Serialized 10885 byte log entry
0709 12:22:04.233462 (+1125820us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b
0709 12:22:05.939734 (+1706272us) op_driver.cc:464] REPLICATION: finished
0709 12:22:06.842459 (+902725us) write_op.cc:301] APPLY: starting
0709 12:22:06.842576 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:06.843192 (+   616us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:06.843238 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:07.368151 (+524913us) spinlock_profiling.cc:243] Waited 9.87 ms on lock 0x7b18001f5c40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:07.403912 (+ 35761us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:07.404448 (+   536us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=28
0709 12:22:07.404502 (+    54us) write_op.cc:312] APPLY: finished
0709 12:22:07.405125 (+   623us) log.cc:844] Serialized 188 byte log entry
0709 12:22:07.461504 (+ 56379us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf132a 00007f5393b76f14 00007f5393b76ec3 00007f5393b76e38 00007f5393b76df3 00007f5393b760dc 00007f5393b9fe16 00007f5393b87cbb 00007f5393b7239d 00007f5393b75d75
0709 12:22:07.568864 (+107360us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:07.577477 (+  8613us) write_op.cc:454] Released schema lock
0709 12:22:07.585181 (+  7704us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:07.615027 (+ 29846us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:07.615064 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2644474,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":877177,"apply.run_cpu_time_us":32086,"apply.run_wall_time_us":772557,"num_ops":28,"prepare.queue_time_us":319888,"prepare.run_cpu_time_us":48733,"prepare.run_wall_time_us":3651185,"raft.queue_time_us":372857,"raft.run_cpu_time_us":158612,"raft.run_wall_time_us":1570013,"replication_time_us":2907194,"spinlock_wait_cycles":69395328,"thread_start_us":3422554,"threads_started":5,"wal-append.queue_time_us":2120543,"wal-append.run_cpu_time_us":53588,"wal-append.run_wall_time_us":2863100}]]}
I20260709 12:22:08.805954 16176 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.806509 13545 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 6, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:08.808940 13545 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 4 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.812950 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:08.814494 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137287us
I20260709 12:22:08.814819 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:08.815565 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:08.816529 16481 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:08.817781 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.819142 13721 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:08.791116 16334 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.822952 16334 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.816567 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.823421 16435 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:08.823899 16435 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.824280 16435 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:08.641652 16448 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:08.827690 16334 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.776785 15623 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:08.828568 15623 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:08.828632 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:08.829252 15623 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.830274 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:08.830973 13718 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:08.831779 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.832031 16334 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:08.736060 16298 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 12:22:08.833611 16298 consensus_queue.cc:1243] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:08.792809 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:08.834918 14425 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.007s	sys 0.005s
W20260709 12:22:08.835321 14425 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.007s	sys 0.005s
I20260709 12:22:08.831241 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.839418 13724 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:08.834156 13725 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:08.838974 16386 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:08.854655 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.778777 16431 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:08.859968 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:08.861774 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:08.867566 13724 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:22:08.867875 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 75229.2us
W20260709 12:22:08.867929 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 475304us
I20260709 12:22:08.863099 13545 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 6, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 6 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.854681 16386 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:08.880223 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:08.885334 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:08.876026 16431 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:08.880719 16430 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.533s	user 0.017s	sys 0.000s
W20260709 12:22:08.933001 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54681.9us
I20260709 12:22:08.936178 16334 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:08.930485 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:08.937896 13725 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:08.910735 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 589029us
I20260709 12:22:08.938401 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:08.882539 13545 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 6, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 6 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:08.968169 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:08.918821 16431 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:08.968323 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:08.976858 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 483223us
I20260709 12:22:08.974244 13545 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:09.003690 13719 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:09.005510 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:09.006623 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:09.007493 13726 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.009179 13723 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:22:09.009738 13723 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 3
I20260709 12:22:09.009996 13723 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:22:08.979938 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.207s (SENT) [suppressed 33 similar messages]
I20260709 12:22:09.014228 13723 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:09.021622 13766 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.026s	sys 0.066s
W20260709 12:22:09.022812 13766 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.026s	sys 0.067s
W20260709 12:22:09.011984 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43792.7us
I20260709 12:22:09.024602 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:09.024780 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 527915us
I20260709 12:22:08.941311 16430 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.020500 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:08.932253 16471 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:08.968001 16334 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:09.044857 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.045403 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59694.3us
W20260709 12:22:09.028648 16413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.021s	sys 0.034s
I20260709 12:22:09.029637 16298 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:09.034046 16282 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.019s	sys 0.037s
I20260709 12:22:09.028647 16349 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:08.992976 16431 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:09.067070 15623 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.069700 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.071714 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:09.072443 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 195584us
I20260709 12:22:09.075315 13719 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.073983 13651 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:22:09.058696 16413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.022s	sys 0.035s
I20260709 12:22:09.074787 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:09.029070 16313 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:09.068446 16282 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.020s	sys 0.037s
I20260709 12:22:09.083439 13723 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:09.081409 13794 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:09.087090 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.066627 16298 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.065627 16511 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.066103 16535 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:09.110376 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.122867 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:09.130822 13796 raft_consensus.cc:2368] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7.
I20260709 12:22:09.086710 13545 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 5, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 5 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.152015 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:09.041050 16386 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:09.087213 13727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.008s	sys 0.003s
W20260709 12:22:09.153633 13727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.009s	sys 0.003s
W20260709 12:22:09.155191 13648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.004s	sys 0.004s
W20260709 12:22:09.155730 13648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.005s	sys 0.005s
W20260709 12:22:09.071214 16283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.015s	sys 0.046s
I20260709 12:22:09.091797 16394 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:09.107815 16298 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.164350 13718 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:09.168628 13647 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:09.165467 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.085063 16476 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 12:22:09.081766 16334 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.081842 16349 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:09.176498 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 513155us
I20260709 12:22:09.174374 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.152480 16413 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:09.092284 16313 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:09.167382 16283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.015s	sys 0.047s
I20260709 12:22:09.181715 13724 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:09.186890 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:09.219290 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.220393 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.220595 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.221519 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.221755 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.222680 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.223471 13793 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:09.224452 16334 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:09.225091 16334 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.225598 16334 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.202548 16378 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:09.207355 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.207978 16413 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:09.228579 16435 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.228854 16413 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:09.229969 16329 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.001s
I20260709 12:22:09.210847 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:09.213649 16535 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:09.213513 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.214823 16471 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:09.214720 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.200402 16394 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:09.224349 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.210718 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.187911 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.206542 16349 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:09.238951 13726 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:09.236931 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:09.243197 13800 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:09.126706 16511 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:09.243572 13545 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:09.245414 16539 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.304s	user 0.001s	sys 0.007s
I20260709 12:22:09.243394 16282 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:22:09.245447 16386 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.247171 13798 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:09.246996 16394 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.248294 13794 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:09.265863 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.256840 13725 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:09.240195 16329 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.265719 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.270298 16471 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:09.262928 16176 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:09.272145 16512 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:09.272609 16512 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:09.273625 13650 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.251089 16349 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:09.264504 16279 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.769s	user 0.013s	sys 0.000s
I20260709 12:22:09.274289 13545 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index -1 to 4, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 4 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.284952 13800 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:09.292202 13792 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:09.292838 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.299680 16334 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.302129 13797 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:09.303238 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1926) took 1448 ms. Trace:
I20260709 12:22:09.263258 16476 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:09.303681 13648 rpcz_store.cc:276] 0709 12:22:07.855272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.270867 (+415595us) service_pool.cc:224] Handling call
0709 12:22:08.318506 (+ 47639us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:08.407279 (+ 88773us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.407310 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:08.485943 (+ 78633us) log.cc:844] Serialized 52 byte log entry
0709 12:22:09.156242 (+670299us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:22:09.156305 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:09.156410 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:09.302962 (+146552us) raft_consensus.cc:1685] finished
0709 12:22:09.302987 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:09.303168 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":678,"raft.run_wall_time_us":14284,"thread_start_us":641450,"threads_started":1,"wal-append.queue_time_us":778793}
I20260709 12:22:09.309892 13719 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:09.300617 16334 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:09.294759 16435 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:09.337109 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 138753us
I20260709 12:22:09.302318 16430 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.330853 13545 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.311461 16476 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:09.298794 16298 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.347378 13725 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 12:22:09.301551 16282 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:22:09.348965 16245 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:22:09.360145 16245 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:09.360733 16245 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:09.353080 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:09.364375 16540 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:09.364851 16540 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.365221 16540 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.367062 16464 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:09.368636 16551 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:22:09.370319 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.370321 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:09.371238 13648 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:09.364981 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:09.372973 16430 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:09.373095 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.373456 16430 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.373826 13645 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.374727 13643 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:22:09.362159 16245 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.376426 16245 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:09.376471 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.377326 13646 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:09.379740 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.379359 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 572) took 1145 ms. Trace:
I20260709 12:22:09.380623 13649 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 9 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:09.379402 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.381026 16298 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:09.381526 13651 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:09.380344 14423 rpcz_store.cc:276] 0709 12:22:08.234143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.247147 (+ 13004us) service_pool.cc:224] Handling call
0709 12:22:08.247570 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.247994 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.248025 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.248652 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:22:08.642487 (+393835us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:08.642525 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.642605 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:09.379127 (+736522us) raft_consensus.cc:1685] finished
0709 12:22:09.379153 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:09.379296 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17170,"raft.queue_time_us":5376,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":8546,"thread_start_us":393431,"threads_started":1,"wal-append.queue_time_us":609339}
I20260709 12:22:09.383064 16329 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:09.383939 16544 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 12:22:09.373888 16430 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:09.386231 16537 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.295s	user 0.000s	sys 0.001s
I20260709 12:22:09.387854 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 539) took 1190 ms. Trace:
I20260709 12:22:09.388195 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.378656 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.388248 14417 rpcz_store.cc:276] 0709 12:22:08.197744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.206651 (+  8907us) service_pool.cc:224] Handling call
0709 12:22:08.207197 (+   546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.207898 (+   701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.207937 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.208656 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 12:22:08.804685 (+596029us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:08.804744 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.804833 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:09.385314 (+580481us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:09.387510 (+  2196us) raft_consensus.cc:1685] finished
0709 12:22:09.387543 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:09.387773 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10991,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":160,"spinlock_wait_cycles":166162176,"thread_start_us":595504,"threads_started":1,"wal-append.queue_time_us":819211}
I20260709 12:22:09.389894 13644 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:09.390918 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 573) took 1141 ms. Trace:
I20260709 12:22:09.391363 14425 rpcz_store.cc:276] 0709 12:22:08.249522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.257509 (+  7987us) service_pool.cc:224] Handling call
0709 12:22:08.266079 (+  8570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:08.290574 (+ 24495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:08.290610 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:08.291225 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:22:08.835789 (+544564us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:08.835835 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:08.835904 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:09.383242 (+547338us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:09.390599 (+  7357us) raft_consensus.cc:1685] finished
0709 12:22:09.390625 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:09.390856 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13076,"raft.queue_time_us":17248,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":7747,"spinlock_wait_cycles":3771648,"thread_start_us":544139,"threads_started":1,"wal-append.queue_time_us":775375}
I20260709 12:22:09.393891 15623 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.351432 13725 pending_rounds.cc:85] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:22:09.394546 13725 pending_rounds.cc:107] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 12:22:09.394862 13725 raft_consensus.cc:2889] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:09.352490 16349 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.354975 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.307631 16279 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.398478 16329 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:09.399780 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44890.3us
I20260709 12:22:09.401613 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:09.402783 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.403650 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51591us
I20260709 12:22:09.404371 15623 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:09.406177 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.407083 13648 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:09.408077 13796 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:09.403306 13545 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 7, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 7 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.415970 16313 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.353245 16476 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:09.417660 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:09.344765 16334 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.423250 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.425482 13650 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:09.398140 16349 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.427641 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.413138 16481 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:22:09.432808 16557 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.108s	user 0.001s	sys 0.001s
I20260709 12:22:09.433373 16563 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.433969 16561 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
W20260709 12:22:09.441354 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 460799us
I20260709 12:22:09.346524 16512 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:09.467887 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 148638us
I20260709 12:22:09.467144 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.436085 16481 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:09.480695 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:09.466717 16448 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.640s	user 0.021s	sys 0.013s
I20260709 12:22:09.483316 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:09.484786 13545 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 2 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:09.488847 14423 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:09.483318 14422 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.487313 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:09.464577 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:09.480559 16560 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
I20260709 12:22:09.462836 16329 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:09.506340 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.480185 13648 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.512403 13644 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:09.509997 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.501358 16448 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:09.509835 16511 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.506965 16562 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.509681 14417 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.510470 16512 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:09.539255 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 427059us
I20260709 12:22:09.541100 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.543030 16464 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:09.543766 16464 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.544752 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:09.546204 16464 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:09.548401 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:09.518747 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:09.556874 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.550696 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.562922 13644 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.487491 16563 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.566771 13693 leader_election.cc:400] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7.
I20260709 12:22:09.557688 13545 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 3, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:09.570765 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.532748 16554 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:09.576629 13648 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:09.577072 13650 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:09.578490 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:09.579187 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 215053us
I20260709 12:22:09.584653 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.585469 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:09.592736 13545 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 6, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 6 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:09.597282 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.603734 13647 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.604552 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.609871 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:09.595220 13545 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 3 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:09.569530 14421 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.620725 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.622144 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.528959 16481 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.635008 13651 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:09.649024 13650 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:09.575454 16438 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:22:09.619585 16511 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:09.647991 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:09.626560 16464 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:09.549794 16562 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.657104 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.655807 13722 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:09.598297 16413 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:22:09.661049 13767 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:09.665043 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.665699 13651 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.666477 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.618548 16471 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:22:09.658326 16521 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.652396 16334 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.664445 16418 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:09.675417 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82217.6us
I20260709 12:22:09.671608 13644 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.638398 16554 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:22:09.667943 16481 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:09.669209 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89606.5us
I20260709 12:22:09.672842 16575 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.698226 16329 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:09.699026 16329 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:09.700223 16329 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.672425 13648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e9c30191f356495b9732659f925abc9f"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:38410
I20260709 12:22:09.703293 13648 raft_consensus.cc:606] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:09.703733 13648 raft_consensus.cc:614] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:09.705420 16579 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:22:09.676421 16541 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.180s
W20260709 12:22:09.706758 16572 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:22:09.696768 16575 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.707432 16125 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:09.707902 16125 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:09.708462 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:09.708925 16125 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.710731 16547 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:09.712071 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:09.713488 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.715046 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:09.709450 16575 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:09.719756 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67074.5us
I20260709 12:22:09.678663 16418 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:09.672194 16543 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 12:22:09.729274 13651 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.729456 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:09.729283 14422 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:09.734776 14423 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:09.741659 16471 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:22:09.746179 16471 consensus_queue.cc:1243] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:22:09.747290 16577 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:22:09.738888 16125 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:09.751127 13644 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:22:09.756525 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.733439 16430 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.760121 16563 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.760552 16279 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:09.761008 16430 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:09.751010 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55677.7us
I20260709 12:22:09.689129 16521 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.733103 16418 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:09.720507 16585 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:22:09.786037 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.782827 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:09.813128 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.818243 13650 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:09.819924 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 172187us
I20260709 12:22:09.829246 13645 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:22:09.829672 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.506s (SENT)
I20260709 12:22:09.830492 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:09.781279 14417 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:22:09.834743 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139005us
I20260709 12:22:09.836930 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.839013 16562 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:09.839912 16588 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:09.843242 13693 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.561s (SENT)
I20260709 12:22:09.843778 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:09.845114 13693 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.578s (SENT)
I20260709 12:22:09.845677 16587 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:09.825655 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:09.850200 13648 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:09.848685 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:09.849929 13693 leader_election.cc:336] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.633s (SENT)
I20260709 12:22:09.854048 13644 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:09.854789 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:09.858194 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.846489 16588 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:09.850737 16562 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:09.866463 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:09.866737 16576 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:22:09.869135 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43582.4us
I20260709 12:22:09.869815 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:09.872395 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:09.881608 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:09.883543 16521 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:09.884234 16589 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:09.885874 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49053.2us
W20260709 12:22:09.886662 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 183193us
I20260709 12:22:09.868345 16561 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:09.836149 16476 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.889659 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:09.883576 16582 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:22:09.822785 16540 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.891664 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:09.874023 16588 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:09.872011 16576 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:09.895704 16581 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:09.893522 16282 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:22:09.893471 16279 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:22:09.898110 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:09.899044 16588 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:09.899501 16334 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:09.900166 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:09.901937 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:09.902623 16551 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:09.887938 16589 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.875734 16562 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:09.866531 16568 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:09.912859 16540 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:09.920372 16592 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:09.933192 13650 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:09.914671 16476 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:09.858500 16587 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.947726 16593 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:09.921576 16576 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:09.943020 14417 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:09.901810 16561 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:09.946596 16562 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:09.946648 16589 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:09.952610 16592 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:09.989805 16431 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.898s	user 0.015s	sys 0.000s
I20260709 12:22:09.990197 16334 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:09.990486 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:09.982805 16562 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:09.962533 16418 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:09.985968 16593 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:10.027518 16334 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.044682 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:10.047680 14423 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:10.049491 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:09.990486 16431 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:10.055558 16594 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:22:10.055902 16431 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:10.056919 14421 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:10.062960 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77556.9us
W20260709 12:22:10.064523 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83524.6us
I20260709 12:22:10.037575 16521 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:10.071262 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:10.078737 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:10.079701 13719 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:10.080324 14423 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:10.060652 16334 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:10.057590 16593 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:10.090447 16604 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:10.091313 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:10.092020 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 193977us
I20260709 12:22:10.094826 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:10.104005 14423 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.104823 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:10.108475 13617 leader_election.cc:400] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6.
I20260709 12:22:10.108990 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:10.111132 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:10.111917 13720 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.109318 13693 leader_election.cc:400] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:10.120191 13693 leader_election.cc:403] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:10.125967 14423 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:10.138221 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:10.139202 13724 raft_consensus.cc:2376] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6. Re-sending same reply.
I20260709 12:22:10.141489 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:10.146086 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:10.146775 13726 raft_consensus.cc:2376] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6. Re-sending same reply.
I20260709 12:22:10.107478 16521 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:10.107297 16551 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:10.166327 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.176640 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:10.108395 16591 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:10.139724 16512 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.604s	user 0.014s	sys 0.000s
I20260709 12:22:10.050702 16592 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:10.188614 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 88460.9us
I20260709 12:22:10.194989 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:10.119454 16581 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:10.196632 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87487.1us
I20260709 12:22:10.197229 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:10.193095 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:10.146464 16580 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.395s	user 0.000s	sys 0.004s
I20260709 12:22:10.143766 16603 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:10.174124 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:10.144909 16431 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:10.212117 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:10.236873 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:10.203231 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:10.216132 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:10.238431 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:10.203572 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:10.239398 13726 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5. Re-sending same reply.
I20260709 12:22:10.214643 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:10.225090 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:10.218981 13644 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:10.212504 16606 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:22:10.128963 16604 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.245198 16604 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:10.246099 16581 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.246711 16592 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:10.250782 16587 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:10.228297 16591 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:10.252170 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:10.252835 13725 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.242179 16603 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.282244 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:10.282967 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68437.6us
I20260709 12:22:10.198204 16609 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:10.254146 16418 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:10.262471 14416 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:10.289642 16568 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:10.336171 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:10.363530 16568 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:10.365015 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 153353us
I20260709 12:22:10.287806 16431 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:10.310536 16606 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:10.363078 16551 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:10.382138 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 144152us
I20260709 12:22:10.401791 13725 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:10.411901 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:10.421835 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:10.431008 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:10.381088 16329 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.337049 16561 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:10.434717 13726 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 2
I20260709 12:22:10.469136 13726 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:22:10.469484 13726 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:10.390015 16568 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.252224 16591 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.433998 16581 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:10.425558 16606 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:10.567253 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 136327us
I20260709 12:22:10.425339 16609 raft_consensus.cc:2764] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:10.579809 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:10.427570 16562 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.570313 16581 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.604882 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:10.622416 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:10.623468 16603 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:10.626127 16464 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:10.626660 16464 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:10.626720 16606 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.627703 16464 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:10.629194 16464 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.629541 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:10.578290 16611 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:10.630311 16549 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:22:10.630503 16611 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:10.630954 16611 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:10.631145 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:10.631844 13645 raft_consensus.cc:2376] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7. Re-sending same reply.
I20260709 12:22:10.632712 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:10.589783 16608 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:22:10.634361 16607 raft_consensus.cc:2764] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:10.634657 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:10.618664 16581 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:10.603961 16571 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:10.608623 16431 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:10.647331 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1748) took 1836 ms (client timeout 2100 ms). Trace:
W20260709 12:22:10.647855 13722 rpcz_store.cc:269] 0709 12:22:08.811320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:08.829908 (+ 18588us) service_pool.cc:224] Handling call
0709 12:22:10.647131 (+1817223us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54003fe8d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:10.647258 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628513536}
I20260709 12:22:10.648252 16604 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:10.648864 16604 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:10.648797 16562 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:10.631421 16601 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:22:10.656342 13727 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:10.657297 16435 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:10.669600 13720 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.000s
I20260709 12:22:10.674175 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:10.674845 13795 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:10.666908 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 264149us
I20260709 12:22:10.679919 13720 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:10.717949 16448 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:10.718750 16448 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:10.723804 16620 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:22:10.665062 16565 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:10.749871 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:10.598057 16329 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.722064 16448 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:10.757944 13801 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:10.775328 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:10.776495 13724 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:10.796175 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:10.797186 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:10.791774 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:10.799723 13798 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:10.672698 16435 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:10.736017 16245 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:10.649837 16604 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.812708 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:10.815819 16418 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:10.816251 16418 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:10.816601 16418 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:10.819468 16616 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:22:10.820531 16589 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.795s	user 0.013s	sys 0.000s
I20260709 12:22:10.821301 16589 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.822933 14425 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:10.824829 16589 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:10.813304 16511 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:10.831568 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:10.836475 14391 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.689s (SENT)
I20260709 12:22:10.837052 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:10.831563 16122 tablet_copy_client.cc:538] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:22:10.756608 16438 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:10.891090 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:10.901162 14425 pending_rounds.cc:85] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 5
I20260709 12:22:10.901696 14425 pending_rounds.cc:107] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:22:10.902031 14425 raft_consensus.cc:2889] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:10.902444 14425 pending_rounds.cc:107] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:22:10.902724 14425 raft_consensus.cc:2889] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:10.856061 16613 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:10.891909 16592 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
W20260709 12:22:10.911103 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74686.5us
I20260709 12:22:10.868498 16562 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:10.912896 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:10.835320 16245 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:10.886730 13652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.007s	sys 0.004s
I20260709 12:22:10.858827 16329 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:10.890666 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:10.752931 16565 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:10.949191 16565 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:10.950450 16624 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:22:10.953644 16334 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.000s
I20260709 12:22:10.934886 16574 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:10.936190 13650 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:10.893396 16438 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:10.886472 13766 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.028s	sys 0.063s
W20260709 12:22:10.942085 16619 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:22:10.942515 16435 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:10.945184 16581 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:10.956581 16611 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:10.957350 16245 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:10.939396 16568 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:10.971554 13766 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.029s	sys 0.064s
W20260709 12:22:11.006425 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 686313us
W20260709 12:22:10.939934 13652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.007s	sys 0.004s
I20260709 12:22:10.867139 16125 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:22:10.974370 13691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.010s	sys 0.084s
W20260709 12:22:11.050906 13691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.011s	sys 0.084s
W20260709 12:22:10.988696 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 263388us
W20260709 12:22:10.914878 16471 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.007s	sys 0.004s
I20260709 12:22:11.056416 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:11.026485 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:11.029527 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47219.7us
I20260709 12:22:11.036284 13720 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:11.037278 13725 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:11.022378 16613 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:10.974291 16431 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:11.090927 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:11.092495 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:10.978358 16334 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.067504 13718 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:22:11.091920 14391 leader_election.cc:336] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.927s (SENT)
W20260709 12:22:11.029121 16576 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.038s	user 0.012s	sys 0.007s
I20260709 12:22:11.129040 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1952) took 1364 ms. Trace:
I20260709 12:22:11.060446 16386 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.105584 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.024796 16329 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:22:11.138303 13647 rpcz_store.cc:276] 0709 12:22:09.764884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.765593 (+   709us) service_pool.cc:224] Handling call
0709 12:22:09.766002 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:09.824404 (+ 58402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:09.824444 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:10.250683 (+426239us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:10.251151 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:22:10.327288 (+ 76137us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:10.327347 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.327463 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.106523 (+779060us) raft_consensus.cc:1685] finished
0709 12:22:11.106548 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.128954 (+ 22406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35086,"raft.queue_time_us":37299,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":37091,"spinlock_wait_cycles":112357760,"thread_start_us":75670,"threads_started":1,"wal-append.queue_time_us":460529}
I20260709 12:22:11.156591 13721 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:10.967854 16574 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.054844 16565 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:11.161409 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56011.7us
I20260709 12:22:11.107296 16575 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.098840 16607 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.167090 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.133045 16589 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.175597 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:11.183439 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 91541.8us
I20260709 12:22:11.089715 16125 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.193800 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.165292 16298 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:11.103488 16613 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.184803 16604 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:11.205929 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:11.208781 13798 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
W20260709 12:22:11.210855 16617 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.321s	user 0.000s	sys 0.004s
W20260709 12:22:11.224910 16600 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.821s	user 0.000s	sys 0.002s
I20260709 12:22:11.228214 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 507) took 1678 ms. Trace:
I20260709 12:22:11.228724 16334 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.229494 16568 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.192534 16588 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:11.230850 16636 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:22:11.231633 16574 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:11.151443 16431 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.195438 16589 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.232678 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1960) took 1051 ms. Trace:
I20260709 12:22:11.173900 16565 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:11.233848 16633 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:22:11.153910 16576 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:11.235298 14416 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.847s	user 0.013s	sys 0.000s
I20260709 12:22:11.103060 16438 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.156806 16386 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.239588 13650 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:11.239692 14424 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:11.244282 13647 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:11.246060 13727 rpcz_store.cc:276] 0709 12:22:09.549596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.549733 (+   137us) service_pool.cc:224] Handling call
0709 12:22:10.650011 (+1100278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:10.667764 (+ 17753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.227945 (+560181us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003fe8d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:11.228129 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2484313984}
I20260709 12:22:11.248266 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.236928 16575 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:11.276254 13648 rpcz_store.cc:276] 0709 12:22:10.180880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.180999 (+   119us) service_pool.cc:224] Handling call
0709 12:22:10.181276 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:10.181498 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:10.201015 (+ 19517us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.643418 (+442403us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18003678a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:11.231689 (+588271us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b18003678a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:22:11.232016 (+   327us) log.cc:844] Serialized 14 byte log entry
0709 12:22:11.232226 (+   210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:11.232285 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:11.232341 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.232415 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.232586 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":609050,"prepare.queue_time_us":1070735,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":93,"raft.queue_time_us":15311,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11191,"spinlock_wait_cycles":55920640}
W20260709 12:22:11.283878 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.189051 16607 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.287016 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.291437 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:11.291466 13650 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:11.293720 13650 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:11.275679 16568 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.297236 13647 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:11.259737 16418 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.307842 16607 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.309814 16386 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.310853 16610 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:11.311527 16610 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:11.312904 16611 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:11.312644 16610 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.314721 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:11.315690 14421 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:11.306082 16418 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:11.316655 16576 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.316920 13651 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:11.281509 16125 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:11.318512 16634 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:11.318571 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1955) took 1143 ms. Trace:
I20260709 12:22:11.319150 16634 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.319185 13652 rpcz_store.cc:276] 0709 12:22:10.175645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.175807 (+   162us) service_pool.cc:224] Handling call
0709 12:22:10.202222 (+ 26415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:10.225916 (+ 23694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:10.225959 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:10.229077 (+  3118us) log.cc:844] Serialized 26 byte log entry
0709 12:22:11.008368 (+779291us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:11.050411 (+ 42043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.050504 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.318289 (+267785us) raft_consensus.cc:1685] finished
0709 12:22:11.318322 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.318500 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40494,"raft.queue_time_us":25502,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":12853,"thread_start_us":688373,"threads_started":1,"wal-append.queue_time_us":737652}
I20260709 12:22:11.321564 16634 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:11.322986 16631 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 12:22:11.324586 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1783) took 1130 ms. Trace:
I20260709 12:22:11.285173 14416 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:11.325306 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 453) took 1222 ms. Trace:
I20260709 12:22:11.325798 14416 rpcz_store.cc:276] 0709 12:22:10.102903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.107350 (+  4447us) service_pool.cc:224] Handling call
0709 12:22:11.325238 (+1217888us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:11.295029 16588 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.324963 13726 rpcz_store.cc:276] 0709 12:22:10.194069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.388803 (+194734us) service_pool.cc:224] Handling call
0709 12:22:10.434404 (+ 45601us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:10.481502 (+ 47098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:10.481587 (+    85us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:10.559904 (+ 78317us) log.cc:844] Serialized 52 byte log entry
0709 12:22:10.859059 (+299155us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:10.859121 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.859234 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.324325 (+465091us) raft_consensus.cc:1685] finished
0709 12:22:11.324359 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.324521 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5760,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":92175,"thread_start_us":236964,"threads_started":1,"wal-append.queue_time_us":443306}
I20260709 12:22:11.322139 13652 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:22:11.338919 13652 pending_rounds.cc:85] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 4
I20260709 12:22:11.339502 13652 pending_rounds.cc:107] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:22:11.340762 13652 raft_consensus.cc:2889] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:11.292174 16642 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:22:11.351207 13647 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:22:11.355926 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:11.299151 16581 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.314424 16568 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.390290 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:11.349133 16576 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:22:11.398568 16476 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:22:11.403388 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1945) took 1665 ms. Trace:
I20260709 12:22:11.410006 13649 rpcz_store.cc:276] 0709 12:22:09.738177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.738850 (+   673us) service_pool.cc:224] Handling call
0709 12:22:09.778495 (+ 39645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:09.779004 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:09.779050 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:09.790659 (+ 11609us) log.cc:844] Serialized 26 byte log entry
0709 12:22:10.144280 (+353621us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:10.144326 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.144400 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.223930 (+1079530us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:11.385870 (+161940us) raft_consensus.cc:1685] finished
0709 12:22:11.385906 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.403296 (+ 17390us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1261,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":35312,"spinlock_wait_cycles":19410688,"thread_start_us":271241,"threads_started":1,"wal-append.queue_time_us":436528}
W20260709 12:22:11.408771 16649 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:22:11.385071 16634 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:22:11.402910 16413 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:11.423298 16548 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:11.457911 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1785) took 1257 ms. Trace:
W20260709 12:22:11.475404 16653 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:22:11.476692 16639 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:22:11.403044 16329 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:22:11.458590 13722 rpcz_store.cc:276] 0709 12:22:10.200569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.648912 (+448343us) service_pool.cc:224] Handling call
0709 12:22:10.649318 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:10.649746 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:10.649777 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:10.656402 (+  6625us) log.cc:844] Serialized 52 byte log entry
0709 12:22:10.976289 (+319887us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:11.054309 (+ 78020us) log.cc:844] Serialized 14 byte log entry
0709 12:22:11.055330 (+  1021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.055427 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.229994 (+174567us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:11.441318 (+211324us) raft_consensus.cc:1685] finished
0709 12:22:11.457581 (+ 16263us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.457819 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10208,"prepare.queue_time_us":73878,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":53927,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":16831,"spinlock_wait_cycles":58460160,"thread_start_us":319381,"threads_started":1,"wal-append.queue_time_us":446552}
I20260709 12:22:11.442700 16565 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.436363 16551 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:22:11.457127 16548 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:11.442835 16413 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:11.475435 16176 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:11.511471 16413 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:11.513751 16548 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.514604 16657 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:11.503211 16561 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.515417 16657 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:11.515743 16561 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.517712 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:11.518456 14416 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:11.523133 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1784) took 1328 ms. Trace:
I20260709 12:22:11.523607 13725 rpcz_store.cc:276] 0709 12:22:10.194402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.621041 (+426639us) service_pool.cc:224] Handling call
0709 12:22:10.621451 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:11.059640 (+438189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.522786 (+463146us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54000f6754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:11.523046 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955489024}
W20260709 12:22:11.524513 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 91039.3us
I20260709 12:22:11.524910 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1957) took 1346 ms. Trace:
I20260709 12:22:11.525391 13651 rpcz_store.cc:276] 0709 12:22:10.178417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.178655 (+   238us) service_pool.cc:224] Handling call
0709 12:22:10.179305 (+   650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:11.317390 (+1138085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.522075 (+204685us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400115154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:11.524838 (+  2763us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2615713664}
I20260709 12:22:11.512068 16176 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:11.532339 16574 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:22:11.559132 16176 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:11.561218 16643 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:22:11.560930 16626 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:11.562441 16626 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:11.563616 16626 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.565786 16626 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:11.564766 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1948) took 1810 ms. Trace:
I20260709 12:22:11.567368 13646 rpcz_store.cc:276] 0709 12:22:09.754260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.782375 (+ 28115us) service_pool.cc:224] Handling call
0709 12:22:09.830183 (+ 47808us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:09.861925 (+ 31742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:09.861977 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:11.212829 (+1350852us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b1800389080. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a
0709 12:22:11.213468 (+   639us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:11.214312 (+   844us) log.cc:844] Serialized 26 byte log entry
0709 12:22:11.349278 (+134966us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:11.349338 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.349440 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.564535 (+215095us) raft_consensus.cc:1685] finished
0709 12:22:11.564558 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.564702 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23693,"raft.queue_time_us":34114,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":3711,"spinlock_wait_cycles":34048768,"thread_start_us":134145,"threads_started":1,"wal-append.queue_time_us":264479}
I20260709 12:22:11.561080 16176 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:11.572563 16562 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.573173 16562 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.573688 16606 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:11.574107 16606 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:11.575073 16606 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.576923 16606 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:11.588879 16654 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:11.589589 16654 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:11.626574 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:11.667104 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:11.667644 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.560s (SENT)
W20260709 12:22:11.668223 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 420035us
W20260709 12:22:11.675845 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.669047 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.666860 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:11.685657 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.683480 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:11.692538 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1779) took 1500 ms. Trace:
I20260709 12:22:11.693811 13646 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:11.693418 13721 rpcz_store.cc:276] 0709 12:22:10.192678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.192841 (+   163us) service_pool.cc:224] Handling call
0709 12:22:10.215829 (+ 22988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:11.159956 (+944127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.692174 (+532218us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54000f5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:11.692445 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162898176}
I20260709 12:22:11.695302 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.698789 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2402) took 1030 ms. Trace:
I20260709 12:22:11.699453 13792 rpcz_store.cc:276] 0709 12:22:10.660949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.661100 (+   151us) service_pool.cc:224] Handling call
0709 12:22:11.622233 (+961133us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:11.690923 (+ 68690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5223680}
W20260709 12:22:11.712502 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.710371 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.719854 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:11.722163 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 366333us
I20260709 12:22:11.724951 13798 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:11.703055 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:11.690111 16418 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.732988 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:11.731117 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:11.692807 16548 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:11.517091 16657 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.736480 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:11.736903 13792 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:11.737763 13649 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:22:11.739585 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 383130us
I20260709 12:22:11.741896 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.745172 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:11.745858 13794 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:11.703469 16591 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.749426 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:11.750154 13652 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:11.755684 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.759605 13649 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 2.
I20260709 12:22:11.720993 16431 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:11.760941 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.012s	sys 0.009s
W20260709 12:22:11.780650 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65851us
W20260709 12:22:11.783696 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.012s	sys 0.009s
I20260709 12:22:11.785817 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:11.788535 13650 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:11.789355 13649 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:11.789652 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:11.790518 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:11.791189 13649 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:11.791999 16619 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:22:11.793375 16637 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.277s	user 0.003s	sys 0.002s
I20260709 12:22:11.731747 16565 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:11.695672 16551 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:22:11.799180 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:11.801573 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60393.2us
I20260709 12:22:11.802417 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:11.801084 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1614) took 1173 ms. Trace:
I20260709 12:22:11.821599 13645 rpcz_store.cc:276] 0709 12:22:10.627565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.633587 (+  6022us) service_pool.cc:224] Handling call
0709 12:22:10.633945 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:10.650291 (+ 16346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:10.650323 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:10.660179 (+  9856us) log.cc:844] Serialized 26 byte log entry
0709 12:22:11.079279 (+419100us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:11.439484 (+360205us) log.cc:844] Serialized 14 byte log entry
0709 12:22:11.585624 (+146140us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:11.702804 (+117180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.704596 (+  1792us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:11.797534 (+ 92938us) raft_consensus.cc:1685] finished
0709 12:22:11.800843 (+  3309us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:11.801017 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204017,"prepare.queue_time_us":195497,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":56,"raft.queue_time_us":328697,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":44270,"spinlock_wait_cycles":3308800,"thread_start_us":842563,"threads_started":3,"wal-append.queue_time_us":618992}
I20260709 12:22:11.745781 16634 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:11.847151 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 93742.4us
I20260709 12:22:11.852541 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.858706 13644 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:11.865180 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.801035 16613 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.801226 16629 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:11.875231 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.877096 13645 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:11.878947 13645 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:11.879549 16629 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.882135 16629 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.709419 16654 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:11.884684 16629 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.885291 16654 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:11.888392 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:11.899053 16670 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:22:11.899168 16629 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:11.795683 16511 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.900067 16511 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.902792 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:11.752231 16418 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:11.911855 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109524us
I20260709 12:22:11.913347 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:11.896881 13649 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:11.922742 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:11.924077 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:11.933866 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:11.943912 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:11.901676 16561 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:11.909303 16629 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:11.942938 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:11.922103 16574 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:11.873875 16660 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.233s
I20260709 12:22:11.936666 16605 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.913740 16634 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:11.949854 16563 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.950083 16581 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:22:11.954888 16122 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
I20260709 12:22:11.966329 16666 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:11.985556 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:11.963198 16574 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:11.978674 16591 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:11.999900 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:11.981623 16629 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:11.997570 16574 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:12.017354 16668 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:22:11.996580 16551 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:11.980151 16605 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.000694 16591 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:11.981652 16563 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.040181 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.021229 16629 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:12.016484 16471 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:12.002242 14416 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:11.993396 16666 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.040817 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.068245 13649 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:12.045951 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.030602 16687 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:12.047117 16612 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.104s	user 0.004s	sys 0.000s
I20260709 12:22:12.074616 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.075078 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1949) took 2320 ms. Trace:
I20260709 12:22:12.076177 13646 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:12.076574 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:12.077677 16674 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:22:12.075997 13643 rpcz_store.cc:276] 0709 12:22:09.754746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:09.781987 (+ 27241us) service_pool.cc:224] Handling call
0709 12:22:09.829434 (+ 47447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:09.852222 (+ 22788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:09.852272 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:10.627781 (+775509us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b180028a620. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a
0709 12:22:11.402938 (+775157us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:11.442499 (+ 39561us) log.cc:844] Serialized 26 byte log entry
0709 12:22:11.685939 (+243440us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:11.685983 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.686073 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:12.074547 (+388474us) raft_consensus.cc:1685] finished
0709 12:22:12.074570 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:12.074904 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":209,"raft.queue_time_us":33867,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":6338,"spinlock_wait_cycles":75737856,"thread_start_us":196720,"threads_started":1,"wal-append.queue_time_us":288974}
I20260709 12:22:12.075870 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 447) took 2031 ms. Trace:
I20260709 12:22:12.049268 16591 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:12.086756 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.082278 14422 rpcz_store.cc:276] 0709 12:22:10.043970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.049263 (+  5293us) service_pool.cc:224] Handling call
0709 12:22:10.049674 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:10.050276 (+   602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:10.050309 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:10.051043 (+   734us) log.cc:844] Serialized 52 byte log entry
0709 12:22:10.133966 (+ 82923us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:10.283857 (+149891us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.284630 (+   773us) log.cc:844] Serialized 14 byte log entry
0709 12:22:10.284861 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:10.284947 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:10.645715 (+360768us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800378640. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:10.894813 (+249098us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b1800378640. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:11.311658 (+416845us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b1800378640. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:11.921427 (+609769us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:12.075485 (+154058us) raft_consensus.cc:1685] finished
0709 12:22:12.075511 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:12.075672 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1232299,"prepare.run_cpu_time_us":749,"prepare.run_wall_time_us":8654,"raft.queue_time_us":4549,"raft.run_cpu_time_us":625,"raft.run_wall_time_us":17187,"spinlock_wait_cycles":153333120,"thread_start_us":215126,"threads_started":2,"wal-append.queue_time_us":257229}
I20260709 12:22:12.099802 13643 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.091820 16476 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:12.136761 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:12.138093 13645 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.139950 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:12.069715 16687 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:12.149539 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:12.150604 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 233910us
W20260709 12:22:12.143018 16626 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.016s
I20260709 12:22:12.154474 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:12.160389 13652 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
W20260709 12:22:12.165992 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 223129us
I20260709 12:22:12.169457 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.170953 13646 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:12.166306 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.176179 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:12.179033 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 241223us
I20260709 12:22:12.180687 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:12.178860 16626 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:12.181962 16666 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.190435 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:12.190721 13794 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:22:12.194200 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.195441 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:12.196185 13652 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:12.192811 16176 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.622s	user 0.018s	sys 0.003s
I20260709 12:22:12.168292 16476 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:12.194759 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.197672 13795 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:12.220587 16684 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:22:12.221762 16282 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:12.222294 16671 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:22:12.224076 16282 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:12.224339 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:12.224792 16282 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:12.225189 13800 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:12.225479 16282 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:12.226872 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:12.228610 14422 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:12.198333 13796 raft_consensus.cc:2368] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
I20260709 12:22:12.235662 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:12.236450 14425 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:12.237460 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:12.177079 16581 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 12:22:12.241571 16650 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.931s	user 0.002s	sys 0.004s
I20260709 12:22:12.243598 16701 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:12.183111 16605 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.189639 16634 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:12.246042 16413 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.733s	user 0.019s	sys 0.011s
I20260709 12:22:12.246385 16626 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:12.246513 16413 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.247699 16626 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.248947 16413 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:12.249989 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:12.250562 13798 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:12.251009 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:12.250614 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:12.255024 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 396013us
I20260709 12:22:12.254127 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:12.256654 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 89369.9us
W20260709 12:22:12.260583 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68682us
I20260709 12:22:12.261796 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.267012 13721 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:12.193075 16279 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:12.268709 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:12.270556 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.270393 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.273818 13792 raft_consensus.cc:2410] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:22:12.254760 16696 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.002s
I20260709 12:22:12.274917 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.278942 13646 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:12.279569 13643 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:22:12.281576 16692 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:22:12.282589 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.285110 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.287356 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.287933 13723 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:12.233091 16576 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:12.283192 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:12.290875 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.299546 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.300595 13726 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:12.301091 13721 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:12.267565 16666 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.257727 16610 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:12.317041 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.317595 13652 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:12.277005 16471 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.323868 13722 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:12.349134 13646 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:12.288887 16176 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:22:12.352475 16703 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.353040 16703 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.284458 16683 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:12.309557 16576 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.376015 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 548) took 1514 ms. Trace:
I20260709 12:22:12.379349 16608 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:12.379976 16608 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:12.380402 16608 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:12.307099 16676 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:12.286443 16551 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:22:12.349817 16563 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.385699 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.393170 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.393985 13645 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:12.394361 13722 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:22:12.385686 16606 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.808s	user 0.017s	sys 0.005s
I20260709 12:22:12.367741 16683 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:12.402504 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.406795 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.403824 13645 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:12.350042 16666 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.391220 16279 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:12.360345 16471 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.361629 16610 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.412765 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.419917 13721 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.415819 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:12.395550 16676 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:12.404407 16683 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:12.428164 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.432893 13645 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.431838 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.438849 13721 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:22:12.439715 16704 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.440438 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.440357 16704 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.441378 13645 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:22:12.432380 16691 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 12:22:12.435550 16562 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.858s	user 0.015s	sys 0.013s
I20260709 12:22:12.425586 16687 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.451473 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.468354 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:12.407891 16476 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:12.487761 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:12.484113 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.488718 13726 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:12.488590 14422 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:12.489487 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.489668 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.483644 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.490624 13726 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.490796 13725 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:12.484315 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:12.492120 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:12.487653 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.493254 13721 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:12.484763 16245 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.502023 13797 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:12.502694 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:12.507452 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 219570us
I20260709 12:22:12.507520 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.467626 16125 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.507761 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.516381 13723 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:12.480156 16471 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.528410 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.479739 16704 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:12.536852 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.537660 13794 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:12.536873 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.525645 13727 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.481220 16703 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.471009 16689 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:12.535898 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.467494 16562 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.511348 16606 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:12.555850 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 514) took 2290 ms. Trace:
I20260709 12:22:12.565976 13720 rpcz_store.cc:276] 0709 12:22:10.265882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.703797 (+437915us) service_pool.cc:224] Handling call
0709 12:22:10.731645 (+ 27848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:11.059229 (+327584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.555437 (+1496208us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:12.555753 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700071680}
I20260709 12:22:12.376830 14417 rpcz_store.cc:276] 0709 12:22:10.861259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:10.866734 (+  5475us) service_pool.cc:224] Handling call
0709 12:22:10.890647 (+ 23913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:10.950518 (+ 59871us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:11.194571 (+244053us) log.cc:844] Serialized 14 byte log entry
0709 12:22:11.443905 (+249334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:11.457423 (+ 13518us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:11.463582 (+  6159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.374520 (+910938us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b18002b4620. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:22:12.375680 (+  1160us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b5400516b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:12.375727 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:12.375934 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":331718,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":115,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":18194,"spinlock_wait_cycles":141579904,"thread_start_us":423413,"threads_started":2,"wal-append.queue_time_us":330529}
I20260709 12:22:12.561867 13723 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:12.511507 16576 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.568818 13645 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:22:12.576478 16698 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:22:12.563609 16606 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:12.587062 16606 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:12.527415 16125 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.590243 13648 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:12.591452 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:12.547991 16471 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.595819 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:12.551584 16704 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.597913 13719 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.605679 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.598786 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.479946 16279 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.609912 13725 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:12.510892 16245 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.598785 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:12.607647 16549 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.354s	user 0.005s	sys 0.003s
I20260709 12:22:12.606741 13645 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:12.516391 16700 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:22:12.622388 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.599076 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.615239 13720 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:12.621443 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:12.610847 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.615309 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.624982 13648 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.623433 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.625713 13726 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:12.632404 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.626123 13727 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:12.608429 16703 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.634965 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.563964 16689 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.641132 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.651154 13795 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:12.655220 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.658039 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.658987 13727 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:12.661098 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:12.662067 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.662748 13725 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:12.609185 16471 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.613071 16704 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.643397 16687 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.632198 16693 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:22:12.593267 16676 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.688346 13721 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:12.680577 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.683812 16125 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:12.684742 16245 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:12.672103 16707 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:22:12.684937 16699 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.160s
I20260709 12:22:12.641422 16700 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:22:12.705077 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.706126 13725 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:12.656730 16610 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.689570 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.714182 13723 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:12.715472 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134974us
W20260709 12:22:12.690104 16654 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.804s	user 0.016s	sys 0.001s
I20260709 12:22:12.650070 16703 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:12.687778 16657 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.729727 13721 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:12.760620 13720 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:12.760596 13725 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:12.766733 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.698885 16687 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.778050 16608 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.714850 16554 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:22:12.731810 16699 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:12.751612 16438 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:12.792215 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.733021 16657 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:12.794193 16548 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.047s	user 0.004s	sys 0.012s
W20260709 12:22:12.794175 16574 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.748s	user 0.015s	sys 0.000s
I20260709 12:22:12.804399 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.790676 16699 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:12.789157 16562 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.823796 13648 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:22:12.825852 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.828269 16471 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:12.831835 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69696.8us
W20260709 12:22:12.832854 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:12.841068 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.841992 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:22:12.843163 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.847963 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.849153 13721 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:12.850824 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.851817 13645 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:12.833039 16712 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:22:12.855448 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:12.859498 13649 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:12.880167 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.812585 16574 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:12.883530 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.506s (SENT)
I20260709 12:22:12.885061 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:12.887697 13645 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:12.887446 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:12.855991 16608 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.859522 16689 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.876859 16563 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:12.906054 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:12.884343 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:12.907198 13793 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:12.911335 13792 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:12.914181 16563 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.915647 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:12.916075 16574 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:12.916854 16563 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.916970 16610 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:12.917605 16610 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:12.919546 13723 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:12.919705 16610 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:12.920895 16716 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 12:22:12.924012 16717 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:22:12.926944 16732 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:12.891139 16704 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:12.896953 16726 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:12.934068 13727 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:12.940018 16657 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.943190 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1983) took 1258 ms. Trace:
I20260709 12:22:12.943918 13726 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:12.944639 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61172.4us
I20260709 12:22:12.945204 16673 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:12.945778 16673 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.891574 16699 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.948788 16589 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:12.945770 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.546s (SENT)
I20260709 12:22:12.951059 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:12.913357 16591 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.807s	user 0.016s	sys 0.000s
I20260709 12:22:12.958088 13645 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:12.943667 13651 rpcz_store.cc:276] 0709 12:22:11.684600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.684838 (+   238us) service_pool.cc:224] Handling call
0709 12:22:11.685157 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:11.781297 (+ 96140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:11.781332 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:11.782072 (+   740us) log.cc:844] Serialized 26 byte log entry
0709 12:22:12.942545 (+1160473us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b18001df320. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1
0709 12:22:12.942721 (+   176us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:12.942760 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:12.942837 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:12.942881 (+    44us) raft_consensus.cc:1685] finished
0709 12:22:12.942892 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:12.943123 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21131,"raft.queue_time_us":167725,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":53252,"spinlock_wait_cycles":4970240,"thread_start_us":259024,"threads_started":2,"wal-append.queue_time_us":271634}
I20260709 12:22:12.962065 16563 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:12.962893 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.963908 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:12.970301 13723 raft_consensus.cc:2376] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4. Re-sending same reply.
W20260709 12:22:12.930789 16714 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
I20260709 12:22:12.966815 13721 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:12.974555 16563 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:12.975829 16687 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.973912 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:12.954519 16703 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:12.986562 16568 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:12.967412 16591 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:13.007587 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61871.7us
I20260709 12:22:13.007766 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.012887 13726 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:13.022876 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.023994 13723 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:12.933671 16683 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:13.009889 16703 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.044418 16703 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.042553 13724 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.998471 16687 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:22:13.006819 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:13.065191 16606 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:13.066704 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.743s (SENT)
I20260709 12:22:13.067126 16703 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.067353 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.067852 16606 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.068216 16736 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.071970 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.072934 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.075487 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:13.075605 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:13.076867 14425 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:13.080099 13721 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:13.081322 13726 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:12.977223 16726 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.081717 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:13.082795 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.083053 16738 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.078008 16563 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.086247 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.085618 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:13.094138 13726 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:13.094426 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.095484 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.094677 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.096004 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.096268 13651 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:13.085445 14421 raft_consensus.cc:2410] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:22:13.082469 16660 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.101s	user 0.010s	sys 0.008s
I20260709 12:22:13.097628 13648 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:13.100333 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.098428 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.099502 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:13.100083 16660 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.120s	user 0.010s	sys 0.008s
I20260709 12:22:13.105659 16591 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.107316 16568 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:22:13.109285 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.087334 16687 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:22:13.110836 16568 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.110962 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.113473 16418 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.114643 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.115978 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.125274 16726 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.104393 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.130883 16683 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.133283 16666 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.099432 16676 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.111938 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1977) took 1815 ms. Trace:
I20260709 12:22:13.140072 13726 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:13.132925 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.112442 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.142565 13726 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:13.139740 13647 rpcz_store.cc:276] 0709 12:22:11.296190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.382925 (+ 86735us) service_pool.cc:224] Handling call
0709 12:22:11.667320 (+284395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:11.667709 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:11.667740 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:11.682105 (+ 14365us) log.cc:844] Serialized 26 byte log entry
0709 12:22:11.788804 (+106699us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:11.788844 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:11.788912 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:13.109900 (+1320988us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b1800413ce0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:13.111569 (+  1669us) raft_consensus.cc:1685] finished
0709 12:22:13.111595 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:13.111864 (+   269us) inbound_call.cc:177] Queueing success response
0709 12:22:13.132410 (+ 20546us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b1800413ce0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
Metrics: {"raft.queue_time_us":131147,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":37547,"spinlock_wait_cycles":36464128,"thread_start_us":61913,"threads_started":1,"wal-append.queue_time_us":95757}
I20260709 12:22:13.146930 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:13.149902 16666 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:13.150503 16666 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:13.138602 14417 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:22:13.151486 16725 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.309s	user 0.003s	sys 0.003s
I20260709 12:22:13.145395 16431 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.142021 16676 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.154945 16676 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.158470 13718 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:13.159945 16438 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:13.161002 16673 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:13.161528 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2412) took 1447 ms (client timeout 1511 ms). Trace:
I20260709 12:22:13.160928 16438 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:13.162489 16438 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:13.163050 16477 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:13.163412 16477 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:13.163607 16438 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.164878 16438 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.161729 16627 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:13.161963 13797 rpcz_store.cc:269] 0709 12:22:11.705965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.706122 (+   157us) service_pool.cc:224] Handling call
0709 12:22:13.153149 (+1447027us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:13.153281 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586532224}
I20260709 12:22:13.150871 16666 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.160920 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.145668 16683 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:13.153492 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:13.182598 16122 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:22:13.187500 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.188383 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:13.189936 13721 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:13.191751 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.191419 16673 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:13.192492 16660 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.192917 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.193181 16660 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:13.193295 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.193562 16660 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:13.193713 16329 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:13.194003 16660 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.194191 16673 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:13.196709 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.194748 16683 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:13.203229 16683 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:13.204635 16683 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.194186 16329 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:13.206885 16329 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.210903 16673 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:13.212972 16673 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.196026 13721 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:13.194082 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.218025 16685 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:13.218930 16685 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.219199 16122 tablet_bootstrap.cc:492] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:22:13.219453 16685 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:13.219439 16742 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:13.220178 16122 ts_tablet_manager.cc:1403] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 1.338s	user 0.046s	sys 0.039s
I20260709 12:22:13.220329 16742 consensus_queue.cc:1243] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:22:13.186573 16593 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:13.230152 16593 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:13.231288 16593 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:13.239777 16680 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.387s	user 0.005s	sys 0.000s
I20260709 12:22:13.243731 16476 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:13.244501 16476 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:13.254506 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163812us
I20260709 12:22:13.254985 13795 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:13.188560 13718 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:13.190493 14421 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
W20260709 12:22:13.223335 16727 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:22:13.224783 16606 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:13.226437 16122 raft_consensus.cc:359] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:13.220388 16627 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:22:13.226837 16742 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:13.227985 16664 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:13.228863 16699 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.229043 16676 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.255944 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.222503 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.272810 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.275508 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:13.276715 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54353.1us
I20260709 12:22:13.281627 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.283005 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:13.288703 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.261210 16431 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:13.292821 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.284775 16701 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:13.295223 16677 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.295397 16724 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:13.261000 16565 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:22:13.295579 16701 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:13.245709 16476 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.271241 16122 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:22:13.295986 16591 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:13.301465 16743 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:22:13.295887 16677 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:13.301920 16701 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.314289 16677 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.297514 16703 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:13.312371 16660 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.286475 16726 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:13.297350 16563 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:13.285415 16176 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:13.333711 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:13.344245 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.355006 13644 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:13.316697 16724 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:13.356591 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:13.356830 16724 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.364277 16676 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.368413 16660 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.316037 16591 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:13.372115 16591 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.375453 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2416) took 1639 ms. Trace:
I20260709 12:22:13.375912 13798 rpcz_store.cc:276] 0709 12:22:11.736024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:11.736179 (+   155us) service_pool.cc:224] Handling call
0709 12:22:13.375159 (+1638980us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54000d25d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:13.375369 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179140480}
I20260709 12:22:13.377310 13801 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:13.378634 16563 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:13.379664 16563 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.320286 16703 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:13.404420 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.340842 16476 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.409967 13721 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:13.422864 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.423629 13721 raft_consensus.cc:2376] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4. Re-sending same reply.
I20260709 12:22:13.424944 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.396729 16726 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:13.409411 16703 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.444537 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.314280 16122 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:13.482625 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.416481 16666 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.482898 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:13.436434 16329 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.497341 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.499747 13721 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:13.504170 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.505088 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.506022 13721 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:13.506846 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:13.508401 16744 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:22:13.510542 16122 ts_tablet_manager.cc:1434] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.290s	user 0.012s	sys 0.001s
W20260709 12:22:13.514834 14391 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.207s (SENT)
I20260709 12:22:13.515384 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.515602 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.516324 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.516985 13647 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:13.520139 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.520845 13646 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:13.489758 14421 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:13.524657 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.525405 13721 raft_consensus.cc:2376] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7. Re-sending same reply.
I20260709 12:22:13.526358 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.526969 13721 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:13.527801 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.528484 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:13.529291 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.531220 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.529829 13721 raft_consensus.cc:2376] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6. Re-sending same reply.
I20260709 12:22:13.499449 14416 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.540004 16435 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:13.542520 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.546593 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.548933 13727 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:13.550429 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:13.563791 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.568244 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.521894 16438 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:13.568343 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53550.9us
I20260709 12:22:13.572069 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.573662 13721 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
W20260709 12:22:13.566556 16739 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:22:13.575734 16542 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:13.486675 16629 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:13.582945 16750 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:22:13.585808 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.581488 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.592296 13721 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:13.563031 16435 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:13.568006 16589 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:13.593142 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.581959 16329 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.598374 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.603440 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:13.604744 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:13.580518 16677 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.611658 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.612522 13727 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:13.612586 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.614158 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.616894 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.617826 13798 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:13.533130 16726 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.610888 16676 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:13.622980 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:13.610566 16747 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:22:13.579457 16438 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.596995 13661 tablet_copy_service.cc:342] P 4dafe08a4f404502822a0732fe570de4: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-56f25c6fe9f245d99ae467bdd3948582 received from {username='slave'} at 127.0.0.1:60832
I20260709 12:22:13.587128 16685 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.634847 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:13.622279 16667 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:13.616266 16431 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.626226 16673 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:13.590517 16542 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.642549 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:13.624780 16676 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.619917 16629 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.637548 13661 tablet_copy_service.cc:434] P 4dafe08a4f404502822a0732fe570de4: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-56f25c6fe9f245d99ae467bdd3948582 on tablet 56f25c6fe9f245d99ae467bdd3948582 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:13.647462 16592 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.649458 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2422) took 1118 ms. Trace:
I20260709 12:22:13.650614 16435 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.652105 16477 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.652930 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1691) took 1143 ms. Trace:
I20260709 12:22:13.669940 13721 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:13.672375 13795 rpcz_store.cc:276] 0709 12:22:12.530637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.662370 (+131733us) service_pool.cc:224] Handling call
0709 12:22:13.254768 (+592398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.255431 (+   663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.649205 (+393774us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54001004d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:13.649376 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361472256}
I20260709 12:22:13.674650 13801 rpcz_store.cc:276] 0709 12:22:12.509888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.510029 (+   141us) service_pool.cc:224] Handling call
0709 12:22:13.377014 (+866985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:13.377817 (+   803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.652592 (+274775us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54000d25d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:13.652828 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993060608}
I20260709 12:22:13.678335 16477 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:13.677894 16435 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.669523 16593 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.670719 16676 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.792613 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.749766 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.777047 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:13.758773 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.808718 13801 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:13.751065 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.705264 13723 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:13.816941 13795 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:13.731455 16754 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:22:13.732102 16756 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:22:13.819010 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:13.702109 16666 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.680536 16673 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:13.744841 14417 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:13.714401 16724 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.728397 16563 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:13.744997 14421 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:13.846731 16511 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.815263 16476 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:13.848924 16476 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:13.849351 16476 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.834539 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:13.850301 14424 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:13.835681 16438 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.729368 16542 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.730834 16685 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:13.743780 16576 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:22:13.841966 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2267) took 1360 ms. Trace:
I20260709 12:22:13.864307 13719 rpcz_store.cc:276] 0709 12:22:12.481219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.489165 (+  7946us) service_pool.cc:224] Handling call
0709 12:22:13.841669 (+1352504us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400174154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:13.841876 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247033728}
I20260709 12:22:13.785701 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:13.843046 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.852361 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:13.868021 14423 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:13.846772 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:13.872813 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:13.879194 13646 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:13.858180 16759 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:22:13.815824 16593 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.858551 16654 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:13.873872 16431 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.858155 16673 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:22:13.879817 16666 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:13.885330 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:13.907449 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 335519us
W20260709 12:22:13.916780 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 293904us
I20260709 12:22:13.918076 13617 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:13.876912 16435 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:22:13.908452 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:13.863338 16563 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:13.903218 14423 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:13.902879 16685 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.919225 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.930235 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.938565 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:13.922521 16746 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.661s	user 0.000s	sys 0.005s
I20260709 12:22:13.941916 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:13.927024 16591 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.917521 16666 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:13.927563 16751 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
W20260709 12:22:13.926594 16699 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.649s	user 0.015s	sys 0.000s
I20260709 12:22:13.978348 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 1994) took 1608 ms. Trace:
I20260709 12:22:13.929971 16565 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:13.962901 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:13.964022 16592 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:13.942764 16438 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:13.908459 16654 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.941757 16761 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:13.987030 14424 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:13.946058 16589 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:13.978891 13652 rpcz_store.cc:276] 0709 12:22:12.370276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.370437 (+   161us) service_pool.cc:224] Handling call
0709 12:22:12.370798 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:12.373960 (+  3162us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:12.627430 (+253470us) log.cc:844] Serialized 14 byte log entry
0709 12:22:12.756571 (+129141us) log.cc:844] Serialized 14 byte log entry
0709 12:22:12.756713 (+   142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:12.756758 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:12.756803 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:13.129951 (+373148us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b18001eb140. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:13.132283 (+  2332us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b18001eb140. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:13.250187 (+117904us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18001eb140. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:22:13.977945 (+727758us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b54001478d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:13.978004 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:13.978227 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43747,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":94,"raft.queue_time_us":100062,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":19352,"spinlock_wait_cycles":37544064,"thread_start_us":350882,"threads_started":2,"wal-append.queue_time_us":180854}
I20260709 12:22:13.940438 16703 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.011855 13795 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.012241 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.012959 13801 raft_consensus.cc:2376] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7. Re-sending same reply.
I20260709 12:22:14.000458 16283 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:14.014051 16283 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.002856 16699 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.015542 16699 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.017578 16283 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:13.991497 16592 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:14.000680 16551 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.018477 16705 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:14.018849 16551 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.019165 16705 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:13.990422 16726 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:14.021603 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58821.7us
I20260709 12:22:14.001363 16589 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:14.025542 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:14.027748 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:13.947535 16685 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.022691 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.032281 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.033968 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:14.032476 16685 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.039990 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:14.041445 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:14.044283 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:14.045138 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.046716 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.051570 16703 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.044534 16761 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.053179 16761 raft_consensus.cc:2764] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:14.060633 16563 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:13.992020 16565 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.001580 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:13.967450 16591 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.060873 16699 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:14.081338 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.036106 16589 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.095916 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:14.071045 16760 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:22:14.103029 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:14.111460 16431 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:14.062026 16438 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:14.114213 16592 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:14.071764 16551 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:14.063510 16565 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.126938 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:14.129977 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2268) took 1648 ms. Trace:
I20260709 12:22:14.133154 13718 rpcz_store.cc:276] 0709 12:22:12.481564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.486796 (+  5232us) service_pool.cc:224] Handling call
0709 12:22:14.129711 (+1642915us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b54002f6f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:14.129883 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506231552}
I20260709 12:22:14.134877 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.141692 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.144148 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:14.144819 13650 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.147615 13651 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:14.148115 13651 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:14.158617 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 239448us
I20260709 12:22:14.162683 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:14.149129 13651 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.167445 13651 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:14.174073 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.097172 16724 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.175081 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:14.139912 16708 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.504s	user 0.000s	sys 0.008s
I20260709 12:22:14.180172 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.066339 16283 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:22:14.201618 14416 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:22:14.204603 14416 pending_rounds.cc:85] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 6
I20260709 12:22:14.205324 14416 pending_rounds.cc:107] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:22:14.205615 14416 raft_consensus.cc:2889] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:14.203508 13651 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:22:14.206743 13651 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 3
I20260709 12:22:14.207098 13651 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:14.207439 13651 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:14.208750 16667 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.015s	sys 0.031s
I20260709 12:22:14.140663 16705 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.166525 16644 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:14.215096 13797 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:14.222167 13652 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.169952 16761 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:14.218502 13650 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
W20260709 12:22:14.218029 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 171364us
I20260709 12:22:14.191605 16686 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:14.224866 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:14.229174 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:14.093624 16666 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:14.219077 16667 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.016s	sys 0.031s
I20260709 12:22:14.233439 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.234151 13645 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:22:14.235459 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72947.1us
I20260709 12:22:14.210858 16593 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:14.259203 13644 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.236512 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:14.223948 16477 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.002s
I20260709 12:22:14.233862 16655 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:22:14.231115 16438 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:14.220705 16726 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:22:14.244489 16761 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:14.245226 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:14.263768 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.270431 13646 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:14.246191 16629 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.002s
I20260709 12:22:14.246577 16686 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.231067 16644 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.249271 16654 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.253458 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:14.256705 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2251) took 1995 ms (client timeout 1546 ms). Trace:
I20260709 12:22:14.236097 16685 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.266655 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.268718 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.271199 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.275820 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.287726 13645 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:22:14.278123 13724 rpcz_store.cc:269] 0709 12:22:12.261269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.261438 (+   169us) service_pool.cc:224] Handling call
0709 12:22:14.256457 (+1995019us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b5400026cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:14.256624 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1791752064}
I20260709 12:22:14.257334 13725 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:22:14.258189 16667 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:14.292038 13722 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:22:14.269325 16676 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.295691 14391 leader_election.cc:400] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
I20260709 12:22:14.299805 14391 leader_election.cc:403] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:14.273195 16418 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:14.307169 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.296713 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:14.271603 16761 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.295871 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.301731 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:14.325927 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 184188us
I20260709 12:22:14.307940 13643 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:14.337456 14416 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.328253 13799 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:14.334575 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.340543 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.347285 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.347579 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.348356 14423 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:14.300275 16776 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:14.355490 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:14.356112 13690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.702s (SENT) [suppressed 38 similar messages]
I20260709 12:22:14.353190 14416 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:14.294651 16685 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:14.363157 16476 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.001s
I20260709 12:22:14.367000 13617 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 4 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5.
I20260709 12:22:14.379164 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.380514 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.351413 16762 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:14.389413 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.343461 16775 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:14.394089 16667 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.394637 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.359428 16431 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:14.395547 16283 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:14.369091 16736 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:14.367322 16329 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.367445 16772 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:14.410151 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:14.406512 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50450.1us
I20260709 12:22:14.373741 16511 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:14.390300 16591 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.419294 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:14.419989 14423 raft_consensus.cc:2376] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6. Re-sending same reply.
I20260709 12:22:14.421078 13617 leader_election.cc:400] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:14.409530 14421 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:14.414237 16773 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:22:14.414662 16329 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.409991 16431 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:14.429291 16431 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:14.409603 16283 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.431551 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:14.413885 16772 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:14.372179 16476 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:14.453305 16774 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:22:14.443195 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:14.456847 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:14.410543 16726 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:22:14.423377 16673 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.429915 16329 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:14.446902 16438 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
I20260709 12:22:14.373900 16644 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.450956 16676 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:14.461789 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.368438 16686 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.424113 16511 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:14.439569 16283 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:14.449239 16589 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.478628 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:14.458881 16772 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:14.481353 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.516611 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:14.517095 16673 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:14.517482 14421 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:14.517611 16511 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.519726 14418 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:22:14.520349 16783 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:22:14.527786 14423 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:14.529702 14423 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:14.529644 13646 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:22:14.546900 16784 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:22:14.547053 16787 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:22:14.472936 16591 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:22:14.478305 16706 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
W20260709 12:22:14.550722 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92526.9us
W20260709 12:22:14.551956 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.517897 16673 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.555836 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.546953 14418 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:22:14.556387 16605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.156s	user 0.011s	sys 0.022s
W20260709 12:22:14.573859 16605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.174s	user 0.011s	sys 0.023s
I20260709 12:22:14.570712 13792 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:14.572945 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:14.575979 13648 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.577538 14425 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:14.578280 16706 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:14.578758 16706 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:14.580742 13647 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:14.567098 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.582774 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 461) took 1433 ms. Trace:
I20260709 12:22:14.579640 16706 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.583146 14422 rpcz_store.cc:276] 0709 12:22:13.149512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.150679 (+  1167us) service_pool.cc:224] Handling call
0709 12:22:14.566934 (+1416255us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:14.582693 (+ 15759us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6302976}
I20260709 12:22:14.583881 16706 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:14.587612 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:14.589221 16418 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.590601 16589 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:14.572906 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.592908 13648 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:14.596220 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.601033 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.602722 16626 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:14.605094 16777 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:14.605278 16776 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:14.599968 16629 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:22:14.564755 16764 raft_consensus.cc:2764] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:22:14.606195 16738 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:22:14.600854 16589 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:14.608567 16629 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:14.608325 16701 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:22:14.606668 16591 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:22:14.647153 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.650084 13648 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:14.653901 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.629916 16629 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:14.675498 16629 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.676848 16629 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.659981 16581 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:22:14.668620 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2258) took 2273 ms (client timeout 1513 ms). Trace:
I20260709 12:22:14.678506 16581 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:14.678648 13722 rpcz_store.cc:269] 0709 12:22:12.395553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.395878 (+   325us) service_pool.cc:224] Handling call
0709 12:22:14.668342 (+2272464us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:14.668517 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4333344512}
I20260709 12:22:14.679483 16581 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.680603 16581 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.671655 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1796) took 1720 ms. Trace:
I20260709 12:22:14.659183 16713 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:14.695422 13647 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:14.644910 16431 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.633509 16477 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.684368 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.686573 13725 rpcz_store.cc:276] 0709 12:22:12.951014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.951185 (+   171us) service_pool.cc:224] Handling call
0709 12:22:14.257142 (+1305957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:14.289327 (+ 32185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:14.671435 (+382108us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400026cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:14.671587 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2975006976}
I20260709 12:22:14.654863 14417 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:14.697022 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.710850 13646 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:14.673866 16283 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.649258 16738 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:14.722332 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2274) took 2102 ms (client timeout 1985 ms). Trace:
W20260709 12:22:14.723169 13720 rpcz_store.cc:269] 0709 12:22:12.619517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:12.631970 (+ 12453us) service_pool.cc:224] Handling call
0709 12:22:14.722081 (+2090111us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:14.722232 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292099840}
I20260709 12:22:14.734591 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:14.698055 14421 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:14.740829 16593 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:14.741539 16283 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.742898 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.743767 16431 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.743808 16768 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:14.744809 16768 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.745025 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:14.745738 14425 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:14.745857 16768 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.729593 16738 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.733114 16626 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:22:14.749840 16800 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:14.750229 16800 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.751180 16800 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.752426 16800 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:14.754123 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:14.756412 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.03137e+06us
I20260709 12:22:14.720098 16591 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:14.759276 16591 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.759894 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.722530 16761 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.763340 16677 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.764182 16701 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:14.764703 16701 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.765187 16576 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:14.765616 16576 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:14.766589 16576 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.766992 16701 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.767616 16576 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.717360 16477 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.770406 16477 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.709069 16713 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.772159 16685 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.773134 16713 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.773442 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.773738 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.775360 14424 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:14.778426 16673 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.777539 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.780159 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:14.786151 14425 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:14.787683 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 146009us
I20260709 12:22:14.790563 14424 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEARNER]: Advancing to term 5
I20260709 12:22:14.724756 16686 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
W20260709 12:22:14.798192 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60282.9us
I20260709 12:22:14.781517 16776 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:14.801905 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:14.789188 16605 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:14.806234 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:14.803836 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:14.810379 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.819556 14424 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:14.794548 16713 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:22:14.817793 16777 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:14.804303 16761 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.832600 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.808516 16605 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.835616 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.837570 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:14.826723 16660 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:14.826562 16329 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:14.830817 16176 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:14.862056 16591 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.839275 16605 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:14.844123 16706 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:14.867880 16706 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.848212 16713 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:22:14.844636 16667 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:22:14.881784 16788 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.466s	user 0.003s	sys 0.001s
I20260709 12:22:14.881934 16667 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.883703 16713 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:14.884660 16626 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:14.885016 16626 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.885089 13798 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:14.885881 16626 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.887763 16644 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:14.888180 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.888322 16644 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.888804 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.889278 16644 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.879290 16592 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:14.890465 16329 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.891607 16329 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:14.845382 16660 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:14.894656 16660 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:14.899292 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:14.899847 16589 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:14.900643 16589 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:14.901605 16589 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:14.902746 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:14.902956 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.903043 16589 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:14.907619 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:14.909240 16775 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:14.909704 16775 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:14.910059 16775 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:14.900075 16673 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:14.910660 16673 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:14.911092 16807 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:22:14.833532 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.913800 16673 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.837903 16738 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:14.918705 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:14.919260 16736 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.502s	user 0.013s	sys 0.000s
I20260709 12:22:14.919696 16736 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:14.921388 16551 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:14.922189 16551 consensus_queue.cc:1243] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:14.922588 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:14.922724 16551 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:14.923272 16551 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.924140 16551 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.925515 16782 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:14.926442 16782 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:14.926873 16782 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.927999 13795 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.925433 16551 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.926611 16713 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 12:22:14.931186 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.931270 16706 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.933260 16686 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:14.933751 16686 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.935698 16686 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:14.937306 16686 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:14.937839 16775 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:14.938584 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.940184 16775 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.941404 13799 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:14.943711 13720 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.944525 16283 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:14.946195 13798 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:22:14.946872 16667 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:14.937322 16738 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:14.948037 16738 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.948349 16593 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:14.948887 13720 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:14.949095 16593 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:14.950241 16593 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.952111 16593 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:14.957036 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:14.957742 13644 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:14.954698 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:14.961707 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:14.962839 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.961751 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:14.963796 13725 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:22:14.965257 13724 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:14.950470 16654 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:22:14.967698 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:14.968360 13722 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:14.973342 13651 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:14.974653 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:14.978789 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:14.979699 13724 raft_consensus.cc:2410] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:22:14.972307 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:14.982348 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:14.833184 16772 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:14.992408 13651 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:14.992940 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:14.950215 16738 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:15.004899 13725 pending_rounds.cc:85] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 1
I20260709 12:22:15.005470 13725 pending_rounds.cc:107] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:22:15.005795 13725 raft_consensus.cc:2889] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:14.988533 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89341.7us
I20260709 12:22:15.009199 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:22:15.009892 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 251616us
I20260709 12:22:15.009944 13799 raft_consensus.cc:606] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:15.011498 13644 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:15.034749 13727 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:15.036530 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.037376 13727 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:15.038688 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.040416 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.042543 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.044661 13719 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:15.046474 13723 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:15.053472 13648 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:15.055603 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.057372 13727 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.056735 13726 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:15.002095 16736 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:14.973670 16655 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:22:15.014694 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:15.026891 16713 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.073827 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.076813 16817 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.069345 16591 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:15.094678 16785 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:22:15.110960 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:15.122565 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:15.071441 16176 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.080314 16629 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:15.121477 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.081740 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:15.123971 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:15.056437 16606 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:22:15.061250 16667 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:22:15.139195 13644 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:15.133944 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.069016 16775 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:22:15.062551 16660 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.133065 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.149272 13719 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:15.150992 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:14.989457 16776 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.144393 13646 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:15.098548 16713 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.156512 16713 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:15.159296 16713 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:15.159919 16713 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:15.168390 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:15.178099 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:15.179708 13647 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:15.183825 13719 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:15.191107 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:15.192193 13647 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:15.193398 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:15.063922 16782 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.202879 16738 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:15.203712 16738 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.208931 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:15.153050 13726 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:22:15.212183 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 202962us
I20260709 12:22:15.217643 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.227990 13726 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:15.129823 16605 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.128008 16591 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.207162 16738 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:15.138043 16176 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.100747 16817 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.230212 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:15.141641 16629 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.261580 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.262794 13719 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:22:15.188205 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3650062120000257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
W20260709 12:22:15.268453 13693 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.861s (SENT)
I20260709 12:22:15.269158 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:15.293375 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.296389 13726 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
W20260709 12:22:15.294209 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.297555 16738 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:15.309242 16738 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:15.310117 16738 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:15.313181 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.257958 16777 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:15.262318 16782 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.161312 16713 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:15.288899 16576 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.005s
I20260709 12:22:15.325735 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.329597 13726 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:15.341521 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.342429 13719 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:22:15.331296 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.317484 16542 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:15.349078 16660 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:15.349275 16476 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:15.331710 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0881926669999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:22:15.315351 16738 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:15.360196 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.367693 16660 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.368353 16660 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.365494 13726 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:15.366755 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:15.375331 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:15.379551 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:15.359131 16815 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
W20260709 12:22:15.406777 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.345955 16772 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:15.409823 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.430379 13726 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:22:15.356029 16713 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:15.435385 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:15.437484 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:15.437968 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:15.422973 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43514.3us
W20260709 12:22:15.365706 16821 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.073s	user 0.001s	sys 0.001s
I20260709 12:22:15.440090 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:15.402838 16776 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.434482 16283 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:15.434918 16810 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
W20260709 12:22:15.445528 16820 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:22:15.453891 16591 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:15.455650 16629 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:15.365305 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:15.457933 16626 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:15.458529 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:15.529194 14423 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:15.458967 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:15.427971 16814 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:22:15.356210 16644 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:22:15.508659 13719 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:15.408672 16768 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:15.502384 16593 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.004s
W20260709 12:22:15.534457 16477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.012s	sys 0.026s
W20260709 12:22:15.569516 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 130930us
I20260709 12:22:15.606081 14423 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:15.534793 16629 raft_consensus.cc:2764] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:22:15.535360 16591 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:15.610174 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 170238us
I20260709 12:22:15.611521 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:15.596907 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:15.628062 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:15.628636 13799 raft_consensus.cc:606] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:15.562666 13721 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.014s	sys 0.017s
I20260709 12:22:15.615224 13726 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
W20260709 12:22:15.629956 13721 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.014s	sys 0.018s
I20260709 12:22:15.635888 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 600) took 1010 ms. Trace:
I20260709 12:22:15.636446 14422 rpcz_store.cc:276] 0709 12:22:14.626066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.626285 (+   219us) service_pool.cc:224] Handling call
0709 12:22:14.626685 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:14.631500 (+  4815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:14.631548 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:14.632171 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:22:14.797906 (+165735us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:14.797953 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:14.798043 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:15.635622 (+837579us) raft_consensus.cc:1685] finished
0709 12:22:15.635646 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.635815 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1469,"raft.queue_time_us":46424,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":165353,"threads_started":1,"wal-append.queue_time_us":438627}
I20260709 12:22:15.513041 16776 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:15.616192 16477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.012s	sys 0.026s
I20260709 12:22:15.648468 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:15.568063 16848 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:15.643328 16589 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.005s
W20260709 12:22:15.662652 16832 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:22:15.663285 13726 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:22:15.664199 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.680891 13727 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:15.665907 16800 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:15.641314 16176 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:15.653180 16629 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:15.666942 16824 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:22:15.682768 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:15.683635 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45dc2f34593649778ec4b80d29638572"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:15.691169 13800 raft_consensus.cc:606] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:15.654733 16705 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:15.699501 16705 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:15.710062 16705 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.711397 16705 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:15.715299 16834 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:22:15.717067 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.695083 16660 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:15.717818 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
I20260709 12:22:15.677683 16854 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:15.723964 13719 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:15.611513 16768 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.685747 16817 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:15.702975 16819 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 12:22:15.687876 16800 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:15.730568 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:15.743885 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.763219 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2028) took 1112 ms. Trace:
I20260709 12:22:15.763777 13645 rpcz_store.cc:276] 0709 12:22:14.650373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.651059 (+   686us) service_pool.cc:224] Handling call
0709 12:22:14.651482 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:14.651898 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:14.651937 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:14.655525 (+  3588us) log.cc:844] Serialized 26 byte log entry
0709 12:22:14.831510 (+175985us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:14.842970 (+ 11460us) log.cc:844] Serialized 14 byte log entry
0709 12:22:14.846404 (+  3434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:14.846482 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:14.902666 (+ 56184us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b1800167140. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:15.731121 (+828455us) raft_consensus.cc:1685] finished
0709 12:22:15.762911 (+ 31790us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.763132 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7207,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":1140,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":244,"spinlock_wait_cycles":7927424,"thread_start_us":151184,"threads_started":1,"wal-append.queue_time_us":380382}
W20260709 12:22:15.767160 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.767745 13792 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:22:15.743233 16551 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.806s	user 0.006s	sys 0.009s
I20260709 12:22:15.736402 16589 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.731488 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 599) took 1126 ms. Trace:
I20260709 12:22:15.747741 16701 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:15.707530 16329 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:22:15.744349 16755 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:15.820749 13726 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
W20260709 12:22:15.831784 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 149116us
I20260709 12:22:15.837014 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:15.793601 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 467) took 1821 ms. Trace:
W20260709 12:22:15.839191 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:15.835672 16686 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.898s	user 0.014s	sys 0.000s
I20260709 12:22:15.849709 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:15.795568 16589 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:15.856470 16842 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:22:15.825892 16800 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:15.892020 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:15.909344 16838 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:22:15.895843 16701 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.915221 16701 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:15.917752 16701 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:15.906558 16849 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:15.921092 16768 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:15.927152 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.927932 13718 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:15.928426 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:15.929018 13722 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:15.932245 13720 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:22:15.932736 13720 pending_rounds.cc:85] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:22:15.933033 13720 pending_rounds.cc:107] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:22:15.933341 13720 raft_consensus.cc:2889] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:15.935226 16835 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:22:15.936151 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:15.936476 16776 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:15.936880 13722 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:15.891341 16833 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:15.896355 16755 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:15.943041 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.943866 13724 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:15.944448 13718 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:22:15.948946 13720 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:15.907958 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:15.958686 14422 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:22:15.962222 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59096.2us
W20260709 12:22:15.976678 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.979455 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:15.980598 13726 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
W20260709 12:22:15.969064 16860 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:22:15.985147 16329 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.976758 13722 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:15.983449 16701 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
W20260709 12:22:15.989658 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:16.018839 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:16.019708 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:15.954604 16861 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:15.989658 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.022689 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
W20260709 12:22:15.986742 16847 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:22:15.993162 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:15.982206 16833 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:15.998531 16644 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 12:22:16.008077 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 386184us
I20260709 12:22:15.975641 16849 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:15.990129 16686 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:16.028882 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.031368 13799 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:16.034293 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.032650 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.037607 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:16.029510 16782 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.707s	user 0.015s	sys 0.000s
I20260709 12:22:16.010031 16547 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:22:16.041392 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.014271 16836 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:22:16.053719 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1824) took 1070 ms. Trace:
W20260709 12:22:16.054168 16713 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.003s
I20260709 12:22:15.842965 14419 rpcz_store.cc:276] 0709 12:22:13.966250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:13.969500 (+  3250us) service_pool.cc:224] Handling call
0709 12:22:14.000080 (+ 30580us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:14.012936 (+ 12856us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:14.012979 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:14.014113 (+  1134us) log.cc:844] Serialized 52 byte log entry
0709 12:22:15.076153 (+1062040us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800399280. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:15.278178 (+202025us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:15.289488 (+ 11310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.316395 (+ 26907us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:15.725121 (+408726us) raft_consensus.cc:1685] finished
0709 12:22:15.725141 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.787910 (+ 62769us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7690,"raft.queue_time_us":5315,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":27371392,"thread_start_us":187020,"threads_started":1,"wal-append.queue_time_us":1339725}
I20260709 12:22:16.054311 13721 rpcz_store.cc:276] 0709 12:22:14.983102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.983236 (+   134us) service_pool.cc:224] Handling call
0709 12:22:14.983570 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:14.983963 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:22:15.630479 (+646516us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.797778 (+167299us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.826697 (+ 28919us) log.cc:844] Serialized 26 byte log entry
0709 12:22:15.826940 (+   243us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:15.826997 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.827105 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.053399 (+226294us) raft_consensus.cc:1685] finished
0709 12:22:16.053431 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.053637 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47735,"prepare.queue_time_us":664051,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":6191,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":18948,"thread_start_us":711629,"threads_started":2,"wal-append.queue_time_us":219692}
I20260709 12:22:15.793830 14418 rpcz_store.cc:276] 0709 12:22:14.600605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.606872 (+  6267us) service_pool.cc:224] Handling call
0709 12:22:14.648002 (+ 41130us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:14.804386 (+156384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:14.807660 (+  3274us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:14.868436 (+ 60776us) log.cc:844] Serialized 52 byte log entry
0709 12:22:15.204072 (+335636us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:15.238662 (+ 34590us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.288616 (+ 49954us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.322940 (+ 34324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.333296 (+ 10356us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:15.703931 (+370635us) raft_consensus.cc:1685] finished
0709 12:22:15.707490 (+  3559us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:15.727136 (+ 19646us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9375,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":203,"raft.queue_time_us":77839,"raft.run_cpu_time_us":612,"raft.run_wall_time_us":42147,"thread_start_us":482059,"threads_started":2,"wal-append.queue_time_us":445262}
I20260709 12:22:16.004601 16329 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.028771 16833 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:16.034224 16606 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:16.032552 16476 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:16.029098 16851 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:22:16.040391 16830 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:16.073855 16782 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:16.097615 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.098510 14419 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:16.016436 16859 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:22:16.099813 16776 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:16.100473 16776 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.034166 16849 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.045222 16738 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:16.083206 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2442) took 1198 ms. Trace:
I20260709 12:22:16.102555 13797 rpcz_store.cc:276] 0709 12:22:14.884328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.889138 (+  4810us) service_pool.cc:224] Handling call
0709 12:22:14.889464 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:15.111763 (+222299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:15.111804 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:15.112661 (+   857us) log.cc:844] Serialized 52 byte log entry
0709 12:22:15.427407 (+314746us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:15.427634 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.428064 (+   430us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.082924 (+654860us) raft_consensus.cc:1685] finished
0709 12:22:16.082946 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.083132 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":390,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":54,"thread_start_us":504555,"threads_started":2,"wal-append.queue_time_us":688254}
I20260709 12:22:16.082937 16863 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:22:16.034680 16861 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.075012 16833 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:16.114826 16655 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.087240 16606 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.035717 16686 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.072981 16589 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.129282 16861 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.135025 16776 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:16.118412 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80897.3us
I20260709 12:22:16.111480 16738 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.112361 16871 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.119694 16606 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:16.145808 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.148741 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:16.155216 16849 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.155575 16713 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.156281 16713 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:16.156776 16713 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.157817 16329 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:16.159406 16855 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:22:16.142663 16738 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:16.174719 16826 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.809s	user 0.004s	sys 0.000s
I20260709 12:22:16.176617 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 582) took 1242 ms. Trace:
I20260709 12:22:16.177091 14416 rpcz_store.cc:276] 0709 12:22:14.933958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.970317 (+ 36359us) service_pool.cc:224] Handling call
0709 12:22:14.970651 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:14.986654 (+ 16003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:14.986688 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:14.987160 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:22:15.245541 (+258381us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:15.245592 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.245670 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.165330 (+919660us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:16.176176 (+ 10846us) raft_consensus.cc:1685] finished
0709 12:22:16.176208 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.176531 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15520,"raft.queue_time_us":16880,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":86,"spinlock_wait_cycles":14364032,"thread_start_us":257829,"threads_started":1,"wal-append.queue_time_us":345431}
I20260709 12:22:16.184094 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:16.188028 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108880us
I20260709 12:22:16.190214 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.190709 16654 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.191171 16654 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.191552 16654 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.191751 16849 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:16.192171 16849 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.192512 16849 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.195288 16551 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.195209 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:16.195825 16551 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:16.196401 16551 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.199635 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.200364 13725 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:16.169862 16755 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:16.186975 16626 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.002s
I20260709 12:22:16.146965 16871 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.211318 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:16.215137 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.217140 16589 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:22:16.219492 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.238550 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.239073 13648 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:16.241890 16593 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:16.246996 16605 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.252570 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:16.254050 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.254789 13649 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:16.258158 16660 raft_consensus.cc:993] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:22:16.258783 16660 raft_consensus.cc:1081] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:22:16.232494 16686 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.236864 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.272413 14420 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:16.273741 16871 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:16.274740 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.267388 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.276226 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.269909 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:16.271101 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:16.208122 16775 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.277865 13797 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:16.218477 16576 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.232342 16176 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.232131 16329 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:16.279919 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.292768 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:16.293538 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.275730 13651 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:16.294140 13649 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:16.283939 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.283962 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:16.287712 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72693.4us
I20260709 12:22:16.253861 16833 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.299556 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.278138 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.285780 16816 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:16.286149 16859 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.218713 16644 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.279088 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.288932 13648 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:16.290849 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52396.2us
W20260709 12:22:16.300657 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106866us
W20260709 12:22:16.310992 13617 connection.cc:256] RPC call timeout handler was delayed by -1.036251902999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 12:22:16.311961 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.263160 16626 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:22:16.293555 13643 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:16.301985 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.219585 16870 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:22:16.302656 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:16.312647 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.301715 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.317027 13645 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:16.317739 13648 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:16.296217 16686 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.296797 16329 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:16.328589 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:16.325804 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.321497 13647 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:16.220430 16811 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:22:16.310397 16816 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.288278 16775 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.305199 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.306057 16589 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:16.292132 16576 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:16.351621 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49080.8us
I20260709 12:22:16.311328 16755 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:16.359323 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57734.6us
I20260709 12:22:16.353757 13648 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:16.298626 16176 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.357673 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:16.358466 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.364972 13645 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:16.291563 16764 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:22:16.357558 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.361088 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.327235 16686 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:16.363521 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.371176 13652 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:16.339425 16329 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:16.350878 16816 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.352759 16431 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.356051 16673 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:16.362095 16705 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:16.379753 13650 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:16.372555 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:16.359694 16755 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.318186 16644 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.362442 16677 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:16.318193 16859 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.369333 16626 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:16.366587 16873 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.370900 16853 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.359150 14420 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:16.372849 16686 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.395684 13643 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:16.376492 16768 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:16.360193 16589 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.373775 16547 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:22:16.386564 16673 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.408761 16768 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.413869 16673 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.414232 16768 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.385200 16705 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:16.393358 16755 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.399646 16874 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:16.400274 16782 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:16.400038 16626 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.399729 16852 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:22:16.409379 16677 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:16.395803 16660 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:22:16.389793 16431 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:22:16.441496 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70889.8us
I20260709 12:22:16.432554 16782 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.384790 16830 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:22:16.443110 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:16.481953 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124457us
I20260709 12:22:16.435166 16626 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:16.491156 16775 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.408813 16853 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.492985 16833 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.432411 16874 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:16.494014 16869 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:22:16.442564 16606 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:16.502717 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 137978us
I20260709 12:22:16.427307 16654 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.409058 16873 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.522979 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.500020 16418 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:16.527437 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 84428.5us
I20260709 12:22:16.523698 16666 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:16.547032 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.518728 16782 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.552006 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:16.539439 16871 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:16.559273 13792 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:22:16.553416 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.524632 16705 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.550201 16849 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.561755 16705 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.564502 16849 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.565528 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:16.567715 16655 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:16.568478 16655 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:16.570677 13799 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:16.576897 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:16.580215 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.574100 16677 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:16.570860 16879 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:22:16.572957 16581 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:16.555544 16775 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:22:16.592068 16873 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.570101 16871 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.596226 16887 raft_consensus.cc:2764] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:22:16.600472 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.590734 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:16.605791 16884 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:22:16.612363 13725 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
W20260709 12:22:16.608990 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.636482 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.615480 16176 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.554610 16418 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.630365 16871 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:16.611788 16890 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:22:16.614797 16881 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:22:16.598335 16576 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.558842 16666 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.633370 16854 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:16.664144 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.666551 14419 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:16.666751 16654 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.670746 16809 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:16.671241 16809 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:16.671728 16809 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.636147 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.594491 16644 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.599501 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.674964 16874 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.676245 16551 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.646761 16885 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:22:16.674563 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60213.1us
I20260709 12:22:16.680938 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:16.661434 16869 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:22:16.691442 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 318951us
I20260709 12:22:16.694768 16551 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:16.670002 16713 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.004s
I20260709 12:22:16.604475 16738 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.706655 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.697180 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:16.678709 16666 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.658455 16859 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.656124 16863 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:16.699344 16872 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.125s
I20260709 12:22:16.690169 16676 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:16.673658 16854 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:16.699409 16755 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.700263 16885 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:16.699365 16576 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:22:16.700347 16589 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:16.727190 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90695.2us
W20260709 12:22:16.734794 13693 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.610s (SENT)
I20260709 12:22:16.733361 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2449) took 1175 ms. Trace:
I20260709 12:22:16.735472 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:16.736613 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.615s (SENT)
I20260709 12:22:16.736665 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.737052 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:16.735538 13794 rpcz_store.cc:276] 0709 12:22:15.557680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:15.557819 (+   139us) service_pool.cc:224] Handling call
0709 12:22:15.606207 (+ 48388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:15.633770 (+ 27563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:15.633809 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.999247 (+365438us) spinlock_profiling.cc:243] Waited 8.51 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:15.999747 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:22:16.146950 (+147203us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:16.150203 (+  3253us) log.cc:844] Serialized 14 byte log entry
0709 12:22:16.170710 (+ 20507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:16.170828 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.727918 (+557090us) raft_consensus.cc:1685] finished
0709 12:22:16.733105 (+  5187us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.733270 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33075,"prepare.queue_time_us":273,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":73518,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":37729,"spinlock_wait_cycles":19572480,"thread_start_us":146737,"threads_started":1,"wal-append.queue_time_us":263582}
I20260709 12:22:16.739168 16849 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:16.740012 16849 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:16.708820 16887 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:22:16.710922 16176 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:22:16.732223 16666 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:16.748486 16431 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.748879 16755 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.718145 16863 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.759037 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:16.760133 14417 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.748647 16418 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.762676 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:16.709671 16713 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.769546 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:16.776794 14417 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:16.778419 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.717358 16768 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.782826 13725 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:16.785341 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:16.786607 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.784775 16581 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:16.788008 16581 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:16.788937 16777 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:16.789449 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.790665 13722 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:16.789011 16581 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:16.790845 16843 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.126s	user 0.007s	sys 0.000s
I20260709 12:22:16.764394 16606 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.771185 16431 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:16.797485 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.797741 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.763892 16853 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.780964 16755 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:16.806305 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.807049 16895 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:16.783587 16666 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:16.814642 16894 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:16.830614 16894 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.794062 16438 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:16.825232 16895 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:16.840353 16895 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:16.829505 16666 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:16.825924 16777 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:16.825508 16776 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.820154 16755 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.834488 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.803067 16863 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.838442 13719 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:16.837045 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.841430 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 601) took 2032 ms. Trace:
I20260709 12:22:16.852909 14425 rpcz_store.cc:276] 0709 12:22:14.809651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:14.810010 (+   359us) service_pool.cc:224] Handling call
0709 12:22:14.810462 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:14.810918 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:14.835031 (+ 24113us) log.cc:844] Serialized 14 byte log entry
0709 12:22:15.414944 (+579913us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:15.415609 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 12:22:15.415754 (+   145us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:15.415787 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:15.415861 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:16.790171 (+1374310us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:16.841127 (+ 50956us) raft_consensus.cc:1685] finished
0709 12:22:16.841235 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:16.841370 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23881,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":5479,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":6820,"spinlock_wait_cycles":4957568,"thread_start_us":424666,"threads_started":1,"wal-append.queue_time_us":702725}
I20260709 12:22:16.858070 16755 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.840384 16768 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.866487 13644 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:16.869055 16666 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.870350 16673 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:16.871244 16892 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.179s	user 0.000s	sys 0.002s
I20260709 12:22:16.842133 16894 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.875705 16673 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:16.875430 16626 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.878337 16776 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.880808 16776 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.882125 16713 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.882611 16782 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.887853 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.888628 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:16.895644 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:16.896817 13644 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:16.898685 13796 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:16.837657 16438 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:16.911123 16738 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:16.845444 16686 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.783771 16854 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:16.899855 16776 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:16.897424 16816 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.006s
I20260709 12:22:16.947926 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:16.945958 16283 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:16.964280 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:16.978180 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.985853 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:16.843209 16329 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.001600 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:17.003486 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:16.966193 16673 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:16.989631 14419 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:16.985082 16283 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:17.015583 13801 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:17.019177 14422 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:16.994029 16854 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:17.005509 16542 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:17.036659 16329 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:16.979494 16776 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:17.040309 16776 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:17.042472 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:17.043223 13644 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:17.045657 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:17.017027 16777 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:16.994575 16871 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:17.055562 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:16.986985 16816 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:16.991215 16655 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:17.079558 13693 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.712s (SENT)
I20260709 12:22:17.081276 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:17.088155 13646 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:17.087795 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:17.084553 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:17.011554 16673 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:17.052886 16776 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:17.099346 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:17.100215 13643 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:17.089501 16683 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.101310 16863 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:17.101627 16683 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.104313 16683 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:17.104960 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:17.105895 13727 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:17.106875 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:17.107535 13720 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:17.110522 14416 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:17.112932 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:17.021754 16438 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:17.116971 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:17.118108 13724 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.089298 16883 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.143798 16883 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:17.122234 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:17.146858 13723 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:17.138178 13549 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 4 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:17.149210 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:17.149592 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:17.149988 13726 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:17.100355 16776 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:17.156311 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 459193us
W20260709 12:22:17.134044 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 620669us
I20260709 12:22:17.136538 16547 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.138087 16654 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:17.139292 16660 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.150532 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:17.170054 13722 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:17.085448 16777 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:17.120010 16865 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:17.114042 16887 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:17.131525 14419 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:17.147423 16848 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.161157 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:17.164412 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:17.098847 16773 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:22:17.141556 16626 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:17.182144 13724 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:17.219611 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.177834 13646 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:17.116840 16176 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:22:17.162114 16776 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.160550 16431 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:17.195791 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.166194 16859 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:17.161273 16782 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:17.305859 16676 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:17.307165 16871 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:17.163290 16764 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
W20260709 12:22:17.314881 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 155492us
I20260709 12:22:17.318403 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.288367 16885 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:17.202600 16686 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:17.328980 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.174470 16547 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.319941 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.331871 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:17.332504 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 172337us
I20260709 12:22:17.288687 16666 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:17.298667 16877 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.919s	user 0.005s	sys 0.000s
I20260709 12:22:17.291131 16683 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:17.352500 13794 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:17.353897 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:17.354480 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.335479 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.356731 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.348222 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:17.290884 16701 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:17.185105 16654 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:17.363018 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.369197 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1656) took 1093 ms. Trace:
I20260709 12:22:17.369855 13650 rpcz_store.cc:276] 0709 12:22:16.276076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.278839 (+  2763us) service_pool.cc:224] Handling call
0709 12:22:17.368920 (+1090081us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b5400221fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:17.369109 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161954560}
I20260709 12:22:17.371409 13792 raft_consensus.cc:2368] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:17.366163 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.364279 13799 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:17.374240 13796 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:17.365481 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.376420 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.189258 16660 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:17.379278 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.380542 13793 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:17.380864 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.381234 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.381863 13799 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:17.337697 16885 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:17.371206 16764 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:17.386628 13795 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 12:22:17.388743 16898 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:22:17.388188 14417 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:17.365860 16859 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:17.359297 16431 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:17.418802 16431 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:17.419607 16800 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.419931 16431 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.421101 16431 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:17.405126 16885 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:17.422931 16859 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.427528 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2455) took 1138 ms. Trace:
I20260709 12:22:17.428462 13801 rpcz_store.cc:276] 0709 12:22:16.289196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.289362 (+   166us) service_pool.cc:224] Handling call
0709 12:22:16.289621 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:17.016081 (+726460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:17.427210 (+411129us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:17.427446 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1669021952}
I20260709 12:22:17.431586 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.411787 13799 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:17.415227 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.291363 16848 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.440112 16848 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:17.447893 16654 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:17.393316 16904 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:22:17.391304 16777 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:17.376816 16701 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:17.436442 16547 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:17.405879 16764 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:17.389632 16899 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 12:22:17.450603 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:17.460539 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.462201 13799 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:17.460638 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:17.394109 16809 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.722s	user 0.015s	sys 0.000s
I20260709 12:22:17.376428 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:17.416931 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52987.4us
I20260709 12:22:17.576251 13792 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:17.591998 13792 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:17.612406 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 324377us
W20260709 12:22:17.639669 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:17.659596 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.660189 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.660959 13801 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 3.
I20260709 12:22:17.661859 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.475915 16605 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.258s
I20260709 12:22:17.922837 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:17.716856 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:17.971979 13727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.019s	sys 0.015s
W20260709 12:22:17.918473 14422 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.002s	sys 0.001s
W20260709 12:22:18.344096 14422 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.004s	sys 0.004s
I20260709 12:22:18.343611 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:17.938241 13797 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.583s	user 0.003s	sys 0.004s
I20260709 12:22:18.344926 13799 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:18.345504 13797 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:18.346067 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1715) took 1029 ms. Trace:
W20260709 12:22:18.343698 13727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.034s	sys 0.029s
I20260709 12:22:18.346472 13797 rpcz_store.cc:276] 0709 12:22:17.316645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.318029 (+  1384us) service_pool.cc:224] Handling call
0709 12:22:18.346006 (+1027977us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:17.754684 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:17.671070 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 469165us
I20260709 12:22:18.349452 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:17.923341 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:18.336148 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 758197us
I20260709 12:22:17.957376 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:18.175491 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:17.893675 13792 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:18.357949 13718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.011s	sys 0.002s
W20260709 12:22:18.358394 13718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.914s	user 0.011s	sys 0.002s
W20260709 12:22:18.339572 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.361615 14417 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:18.365579 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.385663 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.386991 13797 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:18.395995 14391 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.009s	sys 0.022s
W20260709 12:22:18.396762 14391 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.009s	sys 0.022s
W20260709 12:22:18.397161 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 517882us
W20260709 12:22:18.420384 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52276us
I20260709 12:22:17.659354 14419 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:18.429450 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.430827 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:17.538602 16816 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:17.597462 16830 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:17.795511 16895 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.938s	user 0.013s	sys 0.002s
W20260709 12:22:17.894438 16808 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.023s	sys 0.084s
W20260709 12:22:17.730844 16898 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:22:17.754936 16849 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:18.321732 16431 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.900s	user 0.011s	sys 0.000s
I20260709 12:22:18.355280 16654 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:18.317327 16885 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.894s	user 0.011s	sys 0.000s
I20260709 12:22:18.447463 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:17.669373 16655 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.113982 16701 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:17.851758 16871 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:17.848348 16773 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:17.725075 16576 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.770s
W20260709 12:22:18.476922 16736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.476s	user 0.039s	sys 0.069s
I20260709 12:22:17.615770 16809 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:18.248387 16908 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
W20260709 12:22:18.473685 16800 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.021s	sys 0.027s
I20260709 12:22:18.348551 16660 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:18.438463 16906 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:22:17.864507 16764 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.478037 16808 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.495088 16773 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.517163 16808 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.491120 16654 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:18.504110 16800 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.021s	sys 0.027s
W20260709 12:22:18.572557 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.576086 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2333) took 1467 ms. Trace:
W20260709 12:22:18.577752 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 156109us
I20260709 12:22:18.439004 13793 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:18.656890 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.657593 16654 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.579995 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2335) took 1470 ms. Trace:
I20260709 12:22:18.468236 13799 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:18.588914 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:18.666777 13797 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:18.476328 16431 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.671337 16431 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.419993 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.675273 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:18.676863 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.508677 16576 consensus_queue.cc:1243] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:18.652187 13727 rpcz_store.cc:276] 0709 12:22:17.108736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.111611 (+  2875us) service_pool.cc:224] Handling call
0709 12:22:17.111953 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.120113 (+  8160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.120141 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.120899 (+   758us) log.cc:844] Serialized 26 byte log entry
0709 12:22:17.387784 (+266885us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b18003ee000. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:17.443176 (+ 55392us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:22:18.347315 (+904139us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:18.347363 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.347444 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.574937 (+227493us) raft_consensus.cc:1685] finished
0709 12:22:18.575063 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.575668 (+   605us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7773,"raft.queue_time_us":8353,"raft.run_cpu_time_us":2635,"raft.run_wall_time_us":301191,"spinlock_wait_cycles":62421632,"thread_start_us":959304,"threads_started":1,"wal-append.queue_time_us":1356919}
I20260709 12:22:18.533566 16773 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.682535 16773 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.683926 16773 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:18.685880 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 336511us
W20260709 12:22:18.531138 16776 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.222s	user 0.014s	sys 0.001s
I20260709 12:22:18.691288 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.691915 16776 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.523489 16808 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.514148 16809 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.450668 16830 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.469766 16849 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.711386 14416 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:18.713639 16773 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:18.714706 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55768.7us
W20260709 12:22:18.534901 16854 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.502s	user 0.015s	sys 0.000s
I20260709 12:22:18.719774 16859 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:18.489498 16885 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:18.496001 16871 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.722517 16926 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.723852 16885 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.724591 16871 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.456768 16895 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.725204 16629 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:18.497659 16736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.504s	user 0.039s	sys 0.069s
I20260709 12:22:18.507273 16912 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:18.632287 16701 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:18.730818 16701 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.579746 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 455) took 1757 ms. Trace:
I20260709 12:22:18.661314 13718 rpcz_store.cc:276] 0709 12:22:17.109708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.111349 (+  1641us) service_pool.cc:224] Handling call
0709 12:22:17.111708 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.115608 (+  3900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.115637 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.116193 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:22:17.444333 (+328140us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b180023fec0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:18.358863 (+914530us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:18.358919 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.359007 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.578725 (+219718us) raft_consensus.cc:1685] finished
0709 12:22:18.578778 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.579565 (+   787us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3332,"raft.queue_time_us":7724,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7283,"spinlock_wait_cycles":16183680,"thread_start_us":914247,"threads_started":1,"wal-append.queue_time_us":1405098}
I20260709 12:22:18.705511 13799 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:18.705163 16923 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.733666 16816 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:18.734493 13801 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:18.734742 16923 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.714752 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.736614 13793 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:18.723929 16926 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.728500 16895 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.738832 16431 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.719928 16907 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:18.738088 16924 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.740098 16924 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.733889 14425 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:18.734937 16816 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.717679 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:18.736562 16808 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:22:18.746359 16776 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.747361 16808 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:18.753417 16849 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.750870 16871 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.726171 16629 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.761075 16773 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.731812 13651 rpcz_store.cc:276] 0709 12:22:16.822546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.822673 (+   127us) service_pool.cc:224] Handling call
0709 12:22:16.823046 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:16.828098 (+  5052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:16.828148 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:16.833847 (+  5699us) log.cc:844] Serialized 26 byte log entry
0709 12:22:17.006454 (+172607us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:17.006814 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 12:22:17.007124 (+   310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:17.007228 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:18.578971 (+1571743us) raft_consensus.cc:1685] finished
0709 12:22:18.579020 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.579432 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":112,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":5575,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":4595,"thread_start_us":172191,"threads_started":1,"wal-append.queue_time_us":274909}
I20260709 12:22:18.776158 16907 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:18.784837 16816 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:18.787853 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 487) took 1717 ms. Trace:
I20260709 12:22:18.790498 16773 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.790982 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:18.810161 16924 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.795966 16673 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.798581 16923 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.770825 16871 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.800010 16701 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.808601 16775 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:18.813212 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.825944 16926 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:18.827306 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 109927us
I20260709 12:22:18.759713 16808 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:18.828778 13797 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:18.831167 16581 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.837702 16936 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:18.838995 16936 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.837298 16581 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.842520 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:18.843446 16701 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:18.844257 16701 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.844839 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:18.846000 13798 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:18.847222 16701 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.843741 13796 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:18.847587 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.848192 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2462) took 1515 ms. Trace:
I20260709 12:22:18.848874 16673 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.855556 16910 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:18.856153 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2464) took 1197 ms. Trace:
I20260709 12:22:18.856626 13801 rpcz_store.cc:276] 0709 12:22:17.658817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.661616 (+  2799us) service_pool.cc:224] Handling call
0709 12:22:18.855854 (+1194238us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b5400287dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:18.856068 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903162240}
I20260709 12:22:18.856554 16910 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.857596 16871 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.860227 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:18.844623 16581 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.760185 16849 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:18.848827 13796 rpcz_store.cc:276] 0709 12:22:17.332813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.382097 (+ 49284us) service_pool.cc:224] Handling call
0709 12:22:18.842038 (+1459941us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:18.848063 (+  6025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4330240}
I20260709 12:22:18.846791 16935 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.860090 16431 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.868500 16923 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.868973 16736 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.869962 16576 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:18.871604 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2459) took 1545 ms (client timeout 1642 ms). Trace:
I20260709 12:22:18.872738 16418 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.873615 16676 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.849697 16673 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:18.890699 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:18.884815 16576 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.890717 16418 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.892481 16883 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:18.882292 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 93326.6us
I20260709 12:22:18.893030 16576 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:18.883227 13798 rpcz_store.cc:269] 0709 12:22:17.325745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.380289 (+ 54544us) service_pool.cc:224] Handling call
0709 12:22:18.844498 (+1464209us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:18.871530 (+ 27032us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8350080}
I20260709 12:22:18.875428 16885 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:18.897534 16431 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.901633 16431 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.897604 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:18.903472 13724 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:18.903928 16910 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.894768 16926 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:18.882622 16935 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.922490 16923 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.923851 16935 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.898449 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:18.908385 16912 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:18.924973 16903 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.611s	user 0.004s	sys 0.012s
I20260709 12:22:18.913993 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:18.925937 13723 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.926429 16676 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.929090 16676 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.907871 14419 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
W20260709 12:22:18.930066 14419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 619) took 1604 ms (client timeout 1847 ms). Trace:
I20260709 12:22:18.897652 16885 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:18.930526 14419 rpcz_store.cc:269] 0709 12:22:17.325777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.331533 (+  5756us) service_pool.cc:224] Handling call
0709 12:22:18.929913 (+1598380us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:18.924901 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46967.5us
I20260709 12:22:18.931663 16418 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.933048 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:18.933775 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 593) took 1012 ms. Trace:
I20260709 12:22:18.938532 14416 rpcz_store.cc:276] 0709 12:22:17.921834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.923059 (+  1225us) service_pool.cc:224] Handling call
0709 12:22:18.933426 (+1010367us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400651dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:18.933664 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821290880}
W20260709 12:22:18.919108 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 161147us
I20260709 12:22:18.893692 16736 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:18.925124 16859 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:18.914825 13719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:22:18.906530 16849 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.934360 16677 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:18.935819 13724 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:22:18.914887 13549 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 6, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 6 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:18.944842 13795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1720) took 1623 ms (client timeout 1615 ms). Trace:
I20260709 12:22:18.945231 16885 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.893157 16673 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.925179 16923 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.819757 14422 rpcz_store.cc:276] 0709 12:22:17.070623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.070866 (+   243us) service_pool.cc:224] Handling call
0709 12:22:17.071559 (+   693us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:17.292060 (+220501us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:22:17.345796 (+ 53736us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.492074 (+1146278us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.701204 (+209130us) log.cc:844] Serialized 14 byte log entry
0709 12:22:18.767664 (+ 66460us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:18.770119 (+  2455us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:18.774497 (+  4378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:18.787070 (+ 12573us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:18.787125 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:18.787623 (+   498us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14927,"prepare.queue_time_us":273591,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":183,"raft.queue_time_us":133994,"raft.run_cpu_time_us":545,"raft.run_wall_time_us":59480,"spinlock_wait_cycles":302380800,"thread_start_us":1271658,"threads_started":4,"wal-append.queue_time_us":1139603}
I20260709 12:22:18.926002 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:18.954655 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:18.963137 16936 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.967386 14419 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:18.970587 16677 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.019719 16773 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:18.975261 13719 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.020613 16677 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.020972 13719 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.025327 16283 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.026779 16283 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.028741 14420 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:19.030968 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.031886 13724 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:19.033172 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.034585 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.036391 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.037073 13726 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:18.972218 16808 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:18.965881 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:18.978730 13723 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:19.038733 13724 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:19.046440 16789 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:19.047580 16789 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.049203 16589 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:19.049697 16589 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:19.050120 16589 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:19.063014 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87425.9us
I20260709 12:22:18.952490 16859 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.067978 16859 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:19.072397 16800 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:19.073460 16800 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:19.075495 16800 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.078933 16476 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:19.079600 16476 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:19.080894 16476 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:18.954205 13795 rpcz_store.cc:269] 0709 12:22:17.321479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.331598 (+ 10119us) service_pool.cc:224] Handling call
0709 12:22:18.944270 (+1612672us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:18.944647 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87177088}
I20260709 12:22:19.016767 16849 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:18.957903 16431 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.038758 16808 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.012049 16775 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.040129 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:19.040082 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74567.9us
I20260709 12:22:19.099552 13726 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:19.105381 13724 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:19.107573 16677 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.109496 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.110460 13724 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:19.110805 16736 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.112640 16874 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:19.113111 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.114056 16874 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:19.114423 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.115300 13724 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.115175 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.115448 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.109807 16677 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.116956 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.114341 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.109447 13723 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.108139 16789 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:19.137463 13724 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:19.139178 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.140416 13724 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:19.141476 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.142302 13724 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:19.142443 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.143710 13800 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:19.144539 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2461) took 1812 ms. Trace:
I20260709 12:22:19.144465 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:19.145458 14423 raft_consensus.cc:2376] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6. Re-sending same reply.
I20260709 12:22:19.147373 16775 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.149871 16849 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.150593 16849 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.153086 16849 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.095939 16438 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.171555 16830 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:19.174670 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.145121 13800 rpcz_store.cc:276] 0709 12:22:17.332555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.384066 (+ 51511us) service_pool.cc:224] Handling call
0709 12:22:19.085357 (+1701291us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:22:19.143214 (+ 57857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:19.144196 (+   982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:19.144459 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50003200}
I20260709 12:22:19.101555 16673 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:19.105989 16476 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:19.180908 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.186399 13724 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:22:19.174134 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.114482 16874 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:19.103384 16885 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.179649 13719 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.196367 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:19.205369 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.205217 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.196831 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:19.208775 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.211566 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.211606 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.212985 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.216447 13651 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:22:19.218068 13724 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:19.218446 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63244.7us
I20260709 12:22:19.194022 16849 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.223124 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:19.223621 16666 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.224599 13643 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:19.224479 16666 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.223651 13794 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:19.226481 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:19.227428 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1725) took 1904 ms (client timeout 2210 ms). Trace:
W20260709 12:22:19.227896 13794 rpcz_store.cc:269] 0709 12:22:17.323309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.375706 (+ 52397us) service_pool.cc:224] Handling call
0709 12:22:19.204950 (+1829244us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:19.227336 (+ 22386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53486336}
I20260709 12:22:19.230557 16885 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.231469 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.232759 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.233847 13793 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:19.234562 13726 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:19.226492 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.235975 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.236284 13796 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:19.226361 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.236747 13726 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:22:19.239192 16943 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:22:19.239360 16942 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.240545 16942 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.242555 16923 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:19.244156 16677 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.245051 16677 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:19.246671 16872 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:19.246381 16673 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.236758 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.247545 16872 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.248292 16941 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:19.249253 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.249403 16941 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:19.249929 16941 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:19.209422 16773 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:19.247272 13797 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:19.121224 16431 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.252489 13725 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:19.250646 13718 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.258104 13723 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:19.267284 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:19.268709 13645 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:19.269177 16666 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.233065 16885 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.249408 16673 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:19.245834 16677 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.274698 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2332) took 2166 ms. Trace:
I20260709 12:22:19.275328 13720 rpcz_store.cc:276] 0709 12:22:17.108371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:17.109124 (+   753us) service_pool.cc:224] Handling call
0709 12:22:17.109578 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:17.110097 (+   519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:17.110145 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:17.110894 (+   749us) log.cc:844] Serialized 26 byte log entry
0709 12:22:18.931206 (+1820312us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b1800042220. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:19.099634 (+168428us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:19.203956 (+104322us) log.cc:844] Serialized 14 byte log entry
0709 12:22:19.205296 (+  1340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:19.221110 (+ 15814us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:19.221185 (+    75us) raft_consensus.cc:1685] finished
0709 12:22:19.260345 (+ 39160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:19.274438 (+ 14093us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121039,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":49,"raft.queue_time_us":459,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":36923,"spinlock_wait_cycles":11967872,"thread_start_us":88105,"threads_started":2,"wal-append.queue_time_us":2039137}
I20260709 12:22:19.249413 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.278004 16942 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.252375 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.279234 13650 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:19.279960 14419 raft_consensus.cc:2368] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:19.251221 16773 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.278539 16859 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.285240 16887 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:19.285697 16887 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:19.278532 16849 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.286947 16887 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.288172 16887 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.288524 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.288297 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.289179 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.289431 14423 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:19.289621 14425 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:19.283061 14416 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:19.290547 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:19.291229 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57895.3us
W20260709 12:22:19.292747 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5389216789999409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2554s for a response
W20260709 12:22:19.293380 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4278008069995849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17110000000000003s for a response
W20260709 12:22:19.293721 13693 connection.cc:256] RPC call timeout handler was delayed by -1.425796234000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
I20260709 12:22:19.291297 14422 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:22:19.294201 13693 connection.cc:256] RPC call timeout handler was delayed by -1.361621679000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2751s for a response
W20260709 12:22:19.295308 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3490216790000886s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2765s for a response
W20260709 12:22:19.295640 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2964008069998272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
W20260709 12:22:19.295976 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2433008069997413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:22:19.296335 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1686008069996205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
W20260709 12:22:19.296681 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1326008070000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
W20260709 12:22:19.297044 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1290008070000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 12:22:19.277905 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.293587 13719 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:19.305661 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.247730 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:19.329140 13726 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:19.339655 13724 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:19.375875 13650 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:19.376461 16431 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.359390 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.395709 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.390790 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:19.405666 13718 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:19.377362 16666 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:19.413594 16666 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:19.413506 16576 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:19.414580 16576 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.415211 16576 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:19.414990 16666 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:19.420279 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.518s (SENT) [suppressed 16 similar messages]
I20260709 12:22:19.391052 16936 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.251505 16941 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.422796 16936 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.423820 16941 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:19.425271 16936 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.397161 16859 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:19.411650 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:19.408705 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 161118us
I20260709 12:22:19.411717 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.409235 13549 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 3, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 3 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:19.379004 16764 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.394420 16755 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.433838 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:19.377452 16872 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.384848 16848 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.451799 13718 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:19.452361 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:19.454962 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.466316 13800 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:19.474056 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.475525 14418 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:22:19.477418 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.478049 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.478595 16762 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.479511 16762 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.442576 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.448613 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:19.478811 13726 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:19.475669 16789 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:19.483546 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.387182 16589 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.486371 13727 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:19.391327 16830 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:22:19.447641 16677 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:19.440318 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.492978 13792 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:19.493505 13798 raft_consensus.cc:606] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:19.493633 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.493989 13798 raft_consensus.cc:614] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:19.494771 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.495558 13724 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:19.517825 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.518316 14422 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:22:19.518808 13645 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:19.523576 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:19.525532 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83402.1us
I20260709 12:22:19.528039 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.529241 13723 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:19.533177 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:19.488624 16666 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:19.517839 16431 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:22:19.542860 13719 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:19.579246 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.531096 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:19.579506 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 145801us
I20260709 12:22:19.539224 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.588107 13723 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:19.589977 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 70319.6us
I20260709 12:22:19.497371 16773 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:19.582760 13722 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:19.592867 14422 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:19.488860 16848 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.578850 16438 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.607064 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:19.601588 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.496869 16755 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.580956 16431 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:19.585882 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.619603 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.625813 16605 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.626503 13646 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:19.628211 13722 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:19.491245 16874 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:19.635070 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86806.3us
W20260709 12:22:19.639372 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.594991 16629 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.647167 13724 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:19.638412 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:19.648913 13721 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:19.605770 16936 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:19.660071 13719 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:19.662055 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.667961 13718 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:19.617254 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.641788 16605 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.674072 16576 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.675671 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.676462 13651 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:19.635170 13549 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 5, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 5 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:19.684623 13549 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 4 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:19.627272 16431 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.601581 16283 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:19.662688 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.691110 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:19.692956 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.694859 13645 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:19.695425 14416 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:19.637921 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:19.601914 16849 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.701138 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:19.708675 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46123.7us
I20260709 12:22:19.655238 14419 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:19.700538 13800 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:19.699373 16654 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:19.722864 16773 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:22:19.728142 16887 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:19.728402 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:19.714988 13722 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:22:19.730038 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.730187 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.730532 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.730669 13722 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:19.731076 13644 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:19.731966 13722 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:19.732982 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.733362 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.733762 13645 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:19.735070 13545 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 250) took 20132 ms (client timeout 15000 ms). Trace:
I20260709 12:22:19.736137 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.736929 13645 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:19.698588 13726 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:19.738224 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.738349 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.739037 13645 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:19.739780 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.741459 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.742144 13647 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:19.708779 16283 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.624343 16438 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.744551 16283 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:19.745707 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107926us
I20260709 12:22:19.707751 14422 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:19.535346 16789 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.715356 13549 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:19.708309 16418 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:19.726778 13799 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:19.728541 16887 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:19.725068 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.738961 13643 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:19.759169 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:19.768623 13799 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:19.749090 16418 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:19.776119 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46268.1us
I20260709 12:22:19.776998 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.787161 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.788153 16629 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:19.788378 13690 leader_election.cc:400] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:19.789762 13644 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:19.788729 13690 leader_election.cc:403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:19.752189 16438 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:19.794808 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:19.796722 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46324us
I20260709 12:22:19.799984 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.800133 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.781502 16418 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:19.799674 16872 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:19.788738 16629 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:19.805186 16887 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:19.807137 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.806808 16887 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:19.806898 16629 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:19.811358 13549 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 1 to 3, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 3 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:19.814134 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.801616 13647 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:19.826752 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60177.3us
I20260709 12:22:19.831317 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:19.832242 13801 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.806753 16911 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:19.838608 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 418366us
I20260709 12:22:19.782051 16589 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.794736 16830 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:19.840044 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.847090 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.855082 13643 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:19.847661 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.848717 13724 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:19.843178 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:19.753650 16849 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:22:19.858601 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:19.835134 16677 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:19.856096 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:19.870723 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.847374 16438 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:19.844543 16911 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.868112 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.880918 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.883714 16477 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:19.884553 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:19.884770 16283 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:19.887503 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:19.888021 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:19.888797 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 115995us
I20260709 12:22:19.889281 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:19.890021 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:19.890573 16848 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.802214 16800 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:22:19.871987 16644 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:19.872400 16677 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.867048 16936 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
W20260709 12:22:19.895226 16476 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.701s	user 0.009s	sys 0.004s
W20260709 12:22:19.896232 16808 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.002s
I20260709 12:22:19.896109 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.813642 16654 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:22:19.852807 16830 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:19.896745 16755 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:19.897372 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1742) took 1294 ms (client timeout 1653 ms). Trace:
I20260709 12:22:19.902577 16911 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:19.950495 13792 rpcz_store.cc:269] 0709 12:22:18.603203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:18.676439 (+ 73236us) service_pool.cc:224] Handling call
0709 12:22:19.897205 (+1220766us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54000d20d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:19.897310 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1875082496}
I20260709 12:22:19.955125 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.916772 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:19.917337 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:19.918483 16476 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:19.918473 13795 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:19.919232 16808 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:19.922878 16762 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.932384 16789 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:19.912103 16677 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:19.938866 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.851663 16907 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:22:19.948058 13721 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:19.914850 16773 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:19.866861 16859 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:19.952831 13801 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:19.903735 16477 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:19.958648 13799 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:19.912901 13724 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:19.960001 13644 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:19.903595 16283 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:19.908805 16644 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:19.968030 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:19.969697 16576 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:19.949829 16848 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:19.975170 16418 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:19.982479 16941 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.005s
I20260709 12:22:19.983203 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:19.984058 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99684us
I20260709 12:22:19.985805 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:20.022317 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:19.986756 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:19.988544 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:19.989279 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.024786 13797 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:19.992522 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:19.996510 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 140511us
W20260709 12:22:19.998035 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 410413us
I20260709 12:22:20.025736 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:19.909057 14419 raft_consensus.cc:2376] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8. Re-sending same reply.
I20260709 12:22:19.950165 16830 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:19.735543 13545 rpcz_store.cc:269] 0709 12:21:59.666897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:21:59.667075 (+   178us) service_pool.cc:224] Handling call
0709 12:22:19.734740 (+20067665us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:10.146933 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:10.247855 (+100922us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:10.247906 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:22:10.247924 (+    18us) tablet.cc:662] Decoding operations
0709 12:22:10.249521 (+  1597us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:10.249612 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:10.249643 (+    31us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:22:10.620712 (+371069us) tablet.cc:701] Row locks acquired
0709 12:22:10.660433 (+ 39721us) write_op.cc:260] PREPARE: finished
0709 12:22:10.672037 (+ 11604us) write_op.cc:270] Start()
0709 12:22:10.740490 (+ 68453us) write_op.cc:276] Timestamp: P: 1783599730660544 usec, L: 0
0709 12:22:10.750701 (+ 10211us) op_driver.cc:348] REPLICATION: starting
0709 12:22:10.936529 (+185828us) log.cc:844] Serialized 5961 byte log entry
0709 12:22:11.901151 (+964622us) op_driver.cc:464] REPLICATION: finished
0709 12:22:12.359860 (+458709us) write_op.cc:301] APPLY: starting
0709 12:22:12.370879 (+ 11019us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:12.432123 (+ 61244us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:12.495244 (+ 63121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:12.578219 (+ 82975us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:12.675371 (+ 97152us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 12:22:12.686995 (+ 11624us) write_op.cc:312] APPLY: finished
0709 12:22:12.841127 (+154132us) log.cc:844] Serialized 107 byte log entry
0709 12:22:12.908140 (+ 67013us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:12.932553 (+ 24413us) write_op.cc:454] Released schema lock
0709 12:22:12.967666 (+ 35113us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:13.034191 (+ 66525us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:13.035267 (+  1076us) write_op.cc:454] Released schema lock
0709 12:22:13.304753 (+269486us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b1800337240. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
Metrics: {"mutex_wait_us":1304,"reported_tablets":40,"child_traces":[["op",{"apply.queue_time_us":382704,"apply.run_cpu_time_us":18225,"apply.run_wall_time_us":690933,"num_ops":15,"prepare.queue_time_us":252553,"prepare.run_cpu_time_us":42743,"prepare.run_wall_time_us":1082555,"raft.queue_time_us":420931,"raft.run_cpu_time_us":5243,"raft.run_wall_time_us":275959,"replication_time_us":1150217,"spinlock_wait_cycles":20581632,"thread_start_us":1014491,"threads_started":5,"wal-append.queue_time_us":625726,"wal-append.run_cpu_time_us":31474,"wal-append.run_wall_time_us":2413381}]]}
I20260709 12:22:19.999056 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.012897 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.034691 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
W20260709 12:22:20.026968 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.027220 13650 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:20.030187 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.002291 16859 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.042094 16859 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.011994 16644 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:20.043599 16477 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.045159 16477 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.046603 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:19.998181 16773 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:20.048111 16773 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.048750 16418 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.051331 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:20.052012 14418 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:20.036343 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.039354 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:20.054812 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:20.041565 13720 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:20.028262 16476 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:20.058612 13647 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:20.065639 16928 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:22:20.068452 13797 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:20.073989 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.083133 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.094007 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.098474 16589 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:20.099116 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:20.102150 16476 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.103504 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.105463 16848 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:20.106237 16848 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:20.107298 16848 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.108294 16874 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:20.093971 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:20.088697 16764 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.003s
W20260709 12:22:20.115178 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.130761 13799 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:20.085166 16666 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:20.088208 16849 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:20.144562 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.147629 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.081900 16762 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:22:20.149633 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:20.155001 13649 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.151044 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.157419 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.143370 16872 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:20.173600 13792 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.185235 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.178558 13795 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:20.147177 16605 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:20.204789 16644 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.207340 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:20.207542 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.146867 16666 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.212020 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.191974 16654 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:20.210461 13727 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.028s	sys 0.056s
W20260709 12:22:20.215662 13727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.028s	sys 0.057s
I20260709 12:22:20.216048 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.216674 13795 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:20.216547 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1882) took 1151 ms. Trace:
I20260709 12:22:20.219244 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.220083 13798 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:20.219455 13727 rpcz_store.cc:276] 0709 12:22:19.064911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.521695 (+456784us) service_pool.cc:224] Handling call
0709 12:22:19.541764 (+ 20069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:19.573028 (+ 31264us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:22:19.603588 (+ 30560us) log.cc:844] Serialized 14 byte log entry
0709 12:22:20.216174 (+612586us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:20.216237 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:20.216290 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:20.216355 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:20.216485 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36070,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":96,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":105,"thread_start_us":545562,"threads_started":1}
W20260709 12:22:20.227262 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 133415us
I20260709 12:22:20.228721 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.230850 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.232546 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:20.238400 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.221246 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.233892 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.251544 13799 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:20.180948 16431 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.255951 13794 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:20.215365 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.158034 16283 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.211066 16764 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:20.266875 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.266929 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.254429 16849 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:20.268431 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.271234 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.274088 13646 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:20.275348 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.276883 13793 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:20.277752 13652 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:20.286744 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.290794 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:20.291535 16666 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:20.293470 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.294766 16755 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:20.295670 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.296480 16755 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:20.277261 16764 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.298080 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.298717 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.300655 16283 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.300998 16764 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.301575 16755 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:20.304021 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.304960 16755 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.252825 16654 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.306911 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.307578 16431 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:20.307778 16755 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:20.263151 16859 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.309247 13796 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:20.310201 16859 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.310917 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.310240 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:20.318238 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.331871 13649 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:20.328851 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:20.341830 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:20.342787 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.345968 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.279211 16872 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:20.350762 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:20.351742 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.354506 13794 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:20.345883 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:20.355834 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 505228us
I20260709 12:22:20.356220 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.357582 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:20.370222 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.370240 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:20.376513 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:20.377522 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:20.378597 16907 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:20.379284 16907 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:20.379685 16800 raft_consensus.cc:2764] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:20.380400 16907 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.382632 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:20.382723 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103214us
I20260709 12:22:20.383189 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:20.383543 14423 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:20.279110 14418 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:20.385557 13798 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:20.363188 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:20.387274 13792 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:20.309330 16654 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.360901 13796 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:20.389782 13797 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:20.287148 16438 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:20.391927 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.348523 16431 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:20.364019 16629 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.428566 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:20.430514 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 273193us
I20260709 12:22:20.382647 16605 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:22:20.436323 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 126152us
I20260709 12:22:20.393098 16764 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:20.442495 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.398082 16800 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:20.404630 16654 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:20.404896 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:20.430688 16431 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:20.454306 13793 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 3.
I20260709 12:22:20.454816 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:20.457327 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:20.462061 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.446031 16764 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.464020 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.455508 16654 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.453701 16605 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:20.460232 16874 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:20.483937 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.463490 16644 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.465144 16764 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.474251 16848 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.495242 16962 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:20.494824 16431 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.495844 16941 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:20.496483 16941 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:20.498478 16957 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:22:20.481002 16928 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:22:20.499542 16576 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:20.500038 16576 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.463199 16677 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.500465 16576 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.466109 16961 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:22:20.463917 16965 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:20.495849 16962 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.478853 16654 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:20.503209 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47295.2us
I20260709 12:22:20.513072 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:20.502609 13644 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:22:20.506847 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.511083 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
W20260709 12:22:20.513626 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 184728us
I20260709 12:22:20.489681 16971 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:20.512805 16911 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.005s
I20260709 12:22:20.504823 16874 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:20.528566 16970 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:20.510797 16800 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:20.525765 16971 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:20.497758 16941 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.489604 16773 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.518986 16762 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:20.535087 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:20.538694 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:20.473658 16438 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.544493 16874 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.545544 16773 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.531066 16677 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.566941 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.567962 13649 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:22:20.550359 14417 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:20.569314 16960 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:22:20.531147 16800 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.574585 16800 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:20.553808 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.559053 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:20.576361 13646 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:20.525795 16848 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:20.563917 16605 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.565296 16629 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:20.529469 16911 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.565555 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:20.582005 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:20.584923 16967 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:22:20.586680 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.546619 16762 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.565207 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.558003 16644 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:20.626642 13648 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:20.616428 13652 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:20.535574 16971 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.637475 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:20.639153 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 179633us
W20260709 12:22:20.648092 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 243289us
W20260709 12:22:20.639700 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 155865us
I20260709 12:22:20.708810 13651 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
W20260709 12:22:20.709203 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59928.3us
W20260709 12:22:20.710426 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 145312us
W20260709 12:22:20.711803 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57139.7us
I20260709 12:22:20.646668 16644 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:20.717758 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:20.720718 13646 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:20.634512 16762 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:20.725514 16576 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:20.735349 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:20.739202 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:20.709740 16980 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.148s	user 0.000s	sys 0.002s
I20260709 12:22:20.744171 14417 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:20.745254 14422 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:20.711629 16476 raft_consensus.cc:2764] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:20.711620 16849 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:20.751343 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:20.755432 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:20.742874 16978 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:20.759126 16978 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:20.759573 16978 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:20.761029 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 184128us
I20260709 12:22:20.747762 16848 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:20.748667 16970 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:20.753724 16887 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:20.765568 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 179010us
I20260709 12:22:20.754933 16987 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:20.772547 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:20.762439 16848 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:20.772082 16830 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.709s	user 0.011s	sys 0.002s
W20260709 12:22:20.772447 16974 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
W20260709 12:22:20.804153 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.766638 16982 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:20.783613 16589 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:20.752975 16773 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:22:20.831106 16963 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:20.836143 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55917.4us
I20260709 12:22:20.836125 13727 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:20.836261 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55604.6us
W20260709 12:22:20.816462 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:20.780125 16911 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:20.786842 16848 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:20.845745 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134646us
I20260709 12:22:20.834602 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:20.851915 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:20.837846 16987 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:20.842869 16589 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:20.840132 16962 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:20.856597 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.832720 16644 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.864674 16477 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:20.839146 16830 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:20.882979 16789 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:20.883829 16789 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:20.885105 16789 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.889564 13801 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:20.891294 16830 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:20.877476 16755 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:20.895961 16755 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:20.897742 13727 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:20.899814 16957 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:20.902755 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53609.7us
I20260709 12:22:20.906659 16576 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:20.906525 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 141219us
I20260709 12:22:20.907583 16576 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:20.908121 16773 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:20.908483 16576 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:20.908771 16773 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:20.907022 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 134566us
I20260709 12:22:20.909695 16773 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.911528 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:20.911957 16773 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:20.914661 16975 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 12:22:20.916931 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.917265 16987 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.919126 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.919998 13796 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:20.922770 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:20.882790 16808 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:20.938145 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:20.941308 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.906126 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:20.944494 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2086) took 1084 ms. Trace:
I20260709 12:22:20.848970 16963 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:20.952769 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:20.953079 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47038.5us
I20260709 12:22:20.954370 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:20.956871 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:20.956084 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 91106.8us
I20260709 12:22:20.848971 16982 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:21.006927 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52713.9us
W20260709 12:22:21.003028 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61870.5us
I20260709 12:22:21.004446 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:20.909618 16576 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.016573 13800 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:20.944460 16677 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:21.018671 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.026966 16677 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:21.027191 13720 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 3.
I20260709 12:22:21.027733 16677 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.028661 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:21.029335 13720 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:21.029700 16941 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:21.030475 16941 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.031004 13722 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:21.030973 16941 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:20.924484 16666 consensus_queue.cc:1175] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 10 status { last_received { term: 9 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:22:21.021252 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.022565 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:20.950786 16830 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:20.947027 16962 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:20.948846 16808 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:21.022732 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.048856 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:21.038441 13719 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:20.945147 13643 rpcz_store.cc:276] 0709 12:22:19.859908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:19.860164 (+   256us) service_pool.cc:224] Handling call
0709 12:22:19.899367 (+ 39203us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:20.278197 (+378830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:20.278244 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:20.365133 (+ 86889us) log.cc:844] Serialized 52 byte log entry
0709 12:22:20.510286 (+145153us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:20.749187 (+238901us) log.cc:844] Serialized 14 byte log entry
0709 12:22:20.749708 (+   521us) log.cc:844] Serialized 14 byte log entry
0709 12:22:20.752397 (+  2689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:20.752496 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:20.904990 (+152494us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b18001ea780. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5397ac858f 00007f5390b07571 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd
0709 12:22:20.944081 (+ 39091us) raft_consensus.cc:1685] finished
0709 12:22:20.944129 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:20.944379 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2095,"prepare.queue_time_us":257990,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":25826,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":3109,"spinlock_wait_cycles":12653056,"thread_start_us":662305,"threads_started":3,"wal-append.queue_time_us":151148}
I20260709 12:22:21.043565 13721 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:21.051539 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:21.051429 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:21.047832 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.052937 13724 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:21.009548 16872 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:20.951068 16911 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 12:22:21.036737 16477 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:21.036569 16849 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:21.005330 16283 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:22:21.075932 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2471) took 1080 ms. Trace:
I20260709 12:22:21.080209 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.091032 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:21.054064 16438 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:21.090752 13801 rpcz_store.cc:276] 0709 12:22:19.989578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.028838 (+ 39260us) service_pool.cc:224] Handling call
0709 12:22:20.040343 (+ 11505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:20.890303 (+849960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:21.070018 (+179715us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b5400129654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:21.070209 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952379648}
I20260709 12:22:21.068418 16789 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:21.096694 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49579.8us
I20260709 12:22:21.062340 16962 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:21.101013 16962 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.099275 13796 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:21.104691 13720 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:21.095503 16654 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 12:22:20.898025 16755 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.083177 16849 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:20.951274 16970 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.096915 16773 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:21.064246 16762 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.130400 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:21.121378 16849 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:21.116848 16477 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:21.116300 16830 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:21.094487 16978 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.106602 16981 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.275s
I20260709 12:22:21.125537 16773 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.125901 16941 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:21.128609 16666 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.127836 16911 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:21.146673 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55761.7us
I20260709 12:22:21.120386 16654 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:21.170475 16990 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:22:21.174767 16477 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:21.163141 16995 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:22:21.146543 17000 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:21.144485 16993 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.141216 17001 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:21.156059 16773 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.149595 16808 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.225837 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:21.231745 14422 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:21.179080 16996 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:22:21.227614 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100737us
I20260709 12:22:21.231292 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.233980 13719 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
W20260709 12:22:21.194753 16994 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:22:21.223551 16654 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:21.242874 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:21.247022 16764 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.734s	user 0.013s	sys 0.000s
I20260709 12:22:21.247481 16764 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.254563 16677 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.256103 13794 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:21.258363 16677 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.250137 16764 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.259023 16978 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.243057 13801 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:21.231556 16993 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.186481 16666 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.262002 16830 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.245430 17005 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.238358 16418 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:21.267268 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 187145us
I20260709 12:22:21.268383 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.269351 16438 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:21.269714 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.269901 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.270707 16438 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:21.271139 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:21.272032 16965 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:22:21.272503 16965 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.272883 16965 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:21.233426 16996 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:21.275365 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.275643 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:21.220355 16911 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.278441 13794 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.280339 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:21.288391 14417 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:22:21.289125 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.294678 13797 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:21.296219 13793 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:21.298415 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.298534 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.299669 13643 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:21.305032 13794 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:21.306979 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.308787 13643 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:21.314757 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:21.239918 16982 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:21.314776 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:21.319160 13800 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.265921 16963 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.277763 16981 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:21.326217 17008 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.319963 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:21.339942 13801 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.322937 17001 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:21.321475 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:21.312161 16438 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:21.273835 16764 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:22:21.318887 16808 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:22:21.357717 16800 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.783s	user 0.012s	sys 0.002s
I20260709 12:22:21.351444 17000 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.358566 17006 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:21.358218 16800 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.359109 17000 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.360584 16800 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:21.272080 16418 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.361488 17000 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.351321 16961 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:22:21.361626 17001 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:21.311563 16654 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:22:21.351980 16762 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.363698 16849 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:21.364946 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43586.8us
I20260709 12:22:21.366400 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:21.352154 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82330.9us
W20260709 12:22:21.352952 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73955.2us
I20260709 12:22:21.353225 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:21.356281 16887 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.586s	user 0.013s	sys 0.000s
W20260709 12:22:21.356585 16848 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.004s
I20260709 12:22:21.362715 17001 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:21.351608 17002 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:22:21.367208 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.373152 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:21.402580 16764 consensus_queue.cc:1243] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:22:21.376629 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.378582 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:21.363713 16418 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.351771 16666 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:21.274413 17005 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.399580 14417 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:21.376915 16887 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:21.376296 16789 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:21.410570 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:21.377478 16961 raft_consensus.cc:2764] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:22:21.412587 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.416326 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.406133 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:21.421631 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80006.5us
I20260709 12:22:21.344702 17008 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:21.468324 17022 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:22:21.468519 17008 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:21.478132 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54709.3us
I20260709 12:22:21.427656 13801 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:21.407104 16476 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:21.382192 16848 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.437525 17012 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.438681 14417 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:21.441452 16666 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:21.447482 16993 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.448284 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:21.450074 16849 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.451241 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.380136 16996 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.514745 16996 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.459272 17016 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:22:21.519366 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 463) took 1512 ms (client timeout 1885 ms). Trace:
W20260709 12:22:21.519832 13651 rpcz_store.cc:269] 0709 12:22:20.006726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.008194 (+  1468us) service_pool.cc:224] Handling call
0709 12:22:21.519107 (+1510913us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001b5654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:21.519261 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426732800}
I20260709 12:22:21.521529 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.459870 17000 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:21.523242 16808 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:21.461356 16589 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.523679 16808 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.524121 16808 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:21.525871 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74710.5us
I20260709 12:22:21.526444 16973 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:21.526960 16973 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:21.527031 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:21.527488 16973 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.528344 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:21.407253 16654 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:21.529219 16654 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:21.376072 16989 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:22:21.464514 16971 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.754s	user 0.003s	sys 0.010s
W20260709 12:22:21.530985 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82814.2us
I20260709 12:22:21.488422 13800 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:21.531888 14419 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:21.504536 16666 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.532933 16666 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.533277 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:21.524132 16589 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.535640 17012 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:21.507315 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 215259us
I20260709 12:22:21.418354 16789 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:21.509994 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 104160us
W20260709 12:22:21.462440 17009 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:22:21.530680 16971 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.417984 16872 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:21.546214 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:21.553110 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:21.559531 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:21.560395 13727 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.546136 16789 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.551062 16849 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:21.545934 16978 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:21.552268 17006 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:21.551852 17008 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:21.552121 16677 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:21.508591 16887 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:22:21.610302 13723 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:21.610184 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55341us
I20260709 12:22:21.557247 16982 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:21.615199 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:21.631347 13726 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:21.617597 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.557456 16997 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:21.555809 17004 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:21.622581 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.633800 13801 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:22:21.622054 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44535us
I20260709 12:22:21.578141 17012 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:21.613880 17009 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:22:21.530189 16654 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.642946 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.638077 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.642130 13721 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:21.644155 13724 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:21.638322 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:21.640172 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97089.4us
I20260709 12:22:21.647622 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.614562 16849 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:21.508109 17001 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.651811 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.625406 17008 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:21.617182 17006 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:21.670701 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.653991 16971 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.549836 16961 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:22:21.629562 16978 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:21.656086 16848 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.679945 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.639452 16677 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:21.686211 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.687121 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:21.662436 16849 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.675130 17008 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:21.671984 16976 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.709371 16976 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.710773 17014 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.711388 17014 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.711943 16976 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:21.712734 13723 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:21.657984 16418 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:22:21.713699 17014 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:21.713914 16764 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:21.714334 16764 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:21.714810 16764 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:21.714855 16677 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.708382 17019 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:21.661913 16755 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:22:21.717597 16755 consensus_queue.cc:1243] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:22:21.717788 16978 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.706938 16773 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.722715 16830 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:21.723587 16830 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:21.724818 16830 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.700851 17010 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.178s
I20260709 12:22:21.676949 17006 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:21.734738 14418 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:21.748899 17005 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:21.775777 13726 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:21.782603 17001 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:21.800243 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.801227 13643 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:21.804114 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.805043 13647 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:21.805073 16666 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:21.806093 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:21.806711 13651 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:21.793603 16997 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:21.811151 16986 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.974s	user 0.004s	sys 0.001s
I20260709 12:22:21.812194 16997 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:21.807669 16666 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:21.825271 16911 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:21.827275 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.830394 13645 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 6: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:21.831447 17031 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.127s	user 0.003s	sys 0.000s
I20260709 12:22:21.835048 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.837546 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.838702 13645 raft_consensus.cc:2410] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:21.840713 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.856299 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.857486 13794 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:21.857618 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 344898us
I20260709 12:22:21.858505 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.858913 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.863294 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:21.865079 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.867918 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:21.868626 13798 raft_consensus.cc:2376] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7. Re-sending same reply.
I20260709 12:22:21.873175 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.874146 13792 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:21.859145 16979 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:21.882552 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:21.854975 16830 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:21.882000 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 251184us
I20260709 12:22:21.893996 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:21.894325 17004 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:21.898806 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.899786 13792 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:22:21.900157 13643 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:21.910137 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.915131 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:21.924180 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 128984us
I20260709 12:22:21.883989 16979 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:21.924305 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.925622 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:21.930671 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:21.931504 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2952614380001251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
I20260709 12:22:21.931820 14416 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:21.934756 13651 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:21.935168 13801 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:21.930338 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:21.909710 16629 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:21.939055 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:21.940850 13801 raft_consensus.cc:2368] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:21.940845 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.941741 13726 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:21.931901 16962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.012s	sys 0.012s
I20260709 12:22:21.940192 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.945644 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.947402 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.947606 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.947870 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:21.948073 13724 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:21.948652 13801 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:21.946285 13721 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:21.959568 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:21.960161 13721 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:21.961524 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.964459 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.967950 13792 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:22:21.968400 13792 pending_rounds.cc:85] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 6
W20260709 12:22:21.949199 16962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.012s	sys 0.013s
I20260709 12:22:21.978272 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:21.980083 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.984122 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:21.987869 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:21.985869 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:21.968617 13792 pending_rounds.cc:107] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:22:21.993822 13792 raft_consensus.cc:2889] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:21.995612 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:21.996738 13718 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 3: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:21.999681 16961 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:22.004745 16961 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:22.005796 16961 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.011019 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.012048 13720 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:22.013025 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.013672 13720 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.014716 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.016250 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.017097 13719 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:21.957861 16629 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:22.027354 16973 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:22.029196 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.518s (SENT)
I20260709 12:22:22.029757 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:22.030529 14422 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:22.030685 16911 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:22:22.031870 16996 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.517s	user 0.011s	sys 0.000s
I20260709 12:22:22.017911 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.039600 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:22.034950 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.041257 13724 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:22.042320 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.043308 13724 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:22.044390 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.025899 16629 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.045423 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.048522 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.049758 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.050292 14416 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:21.998379 16972 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:22.050693 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:22.052726 13727 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:22.051955 13724 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:22.061915 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.063402 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:22.056842 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:22.072481 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.073146 13719 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:22.032624 16808 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.075899 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.071385 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.079720 13721 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:22.081013 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.081931 13796 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:22.084889 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:21.998155 16907 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:22:21.993150 17004 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:22.318611 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 255399us
I20260709 12:22:22.142201 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.326445 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.381755 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:22.382055 13721 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7. Re-sending same reply.
I20260709 12:22:22.349887 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.345850 13720 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
W20260709 12:22:22.222509 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.087726 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.350556 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:22.380152 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 350977us
W20260709 12:22:22.059257 16965 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.786s	user 0.007s	sys 0.007s
I20260709 12:22:22.389103 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.389439 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.059350 16997 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:22.071653 16972 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.397696 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.410337 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.411118 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.106982 16629 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:22.232363 16808 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.403381 13719 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:22.225090 16764 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.510s	user 0.016s	sys 0.005s
W20260709 12:22:22.284602 17006 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.552s	user 0.013s	sys 0.000s
W20260709 12:22:22.119411 17046 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:22:22.207515 16789 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.001s
I20260709 12:22:22.397665 13724 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:22.078640 16996 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:22.118789 17023 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.075968 17033 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:22.416661 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:22.417049 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:22.119534 17037 raft_consensus.cc:2764] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:22:22.431816 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.432582 13726 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:22.205718 16962 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.289328 16874 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:22.424561 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.425334 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.399423 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.426743 13724 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:22.441643 13727 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.443215 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:22.403712 16989 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.406342 17052 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.407327 17019 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:22.435490 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.507s (SENT)
I20260709 12:22:22.436124 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.391490 16973 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:22.427415 16849 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
I20260709 12:22:22.087801 17008 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.412956 16972 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:22.446197 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 64506.3us
I20260709 12:22:22.461668 13724 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:22.438557 16978 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:22.478483 13724 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:22.398793 16965 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.483831 13721 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:22.456066 17052 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.465744 13727 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:22.428592 16764 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.490051 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:22.490186 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54762.2us
I20260709 12:22:22.428774 17006 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.433284 16789 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.493399 13726 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:22.495410 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.506237 13798 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 4: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:22.438494 17023 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.505523 13724 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:22.492600 13727 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:22.447767 16874 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.453751 16849 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.513823 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:22.514881 13647 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:22.520666 13726 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:22.455848 16989 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.519204 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.521341 13725 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:22.517874 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.445659 16962 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:22.522859 17048 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:22:22.549913 13720 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:22.489781 16978 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.519167 17019 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:22.525454 17016 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:22.554495 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:22.525960 16996 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:22.549747 17033 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:22.571556 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:22.583209 13722 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.568493 17019 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:22.611726 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:22.615674 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61700.5us
I20260709 12:22:22.616514 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:22.620288 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:22.600092 16629 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:22.613744 17008 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:22.622437 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:22.624845 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:22.613221 17025 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.622242 17001 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.625417 17033 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:22.648728 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.643533 16764 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.629444 16789 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:22.642712 17016 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:22.668059 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.670739 16978 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:22.648746 17033 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.671597 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:22.671702 17056 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:22:22.672662 16830 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.009s
W20260709 12:22:22.673004 17054 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:22:22.646524 16965 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:22.668782 13726 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 4.
W20260709 12:22:22.680764 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46867.2us
I20260709 12:22:22.681706 13693 leader_election.cc:400] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:22.653504 16644 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:22.682215 13693 leader_election.cc:403] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:22:22.659076 17052 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:22.664851 17016 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:22.655680 17063 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:22.679947 16874 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.629829 16996 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.674785 16849 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.648185 17001 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.666965 17006 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:22.671572 17033 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:22.703882 16962 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:22.679613 17023 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.645030 17025 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:22.675468 16629 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:22.716702 17068 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:22.678699 17008 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:22:22.705204 17016 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:22.702733 16644 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:22.743966 16629 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.747300 17001 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:22.747678 16644 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.749768 17023 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.750335 17023 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:22.752106 17050 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
I20260709 12:22:22.770013 17068 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:22:22.786302 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 104699us
I20260709 12:22:22.793264 17019 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:22.806226 17057 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:22:22.792583 17063 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:22.839761 17073 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:22:22.807542 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:22.863459 13648 raft_consensus.cc:1217] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-7.8]   Dedup: 3.6->[5.7-7.8]
I20260709 12:22:22.865664 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:22.788839 17068 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:22.832445 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:22.807629 17065 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:22.865056 17063 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:22.809397 17064 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 12:22:22.787920 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 79728.2us
W20260709 12:22:22.876332 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 216711us
I20260709 12:22:22.871345 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:22.885259 13722 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:22.871165 13648 pending_rounds.cc:85] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 6
I20260709 12:22:22.906064 13648 pending_rounds.cc:107] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:22:22.906636 13648 raft_consensus.cc:2889] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:22.911785 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:22.912773 14416 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:22.920634 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 613) took 1983 ms. Trace:
I20260709 12:22:22.921382 14419 rpcz_store.cc:276] 0709 12:22:20.937109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.946440 (+  9331us) service_pool.cc:224] Handling call
0709 12:22:21.061692 (+115252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.532317 (+470625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.920314 (+1387997us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400498c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:22.920557 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080241408}
I20260709 12:22:22.936863 17069 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:22.937258 16982 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:22.938687 16830 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:22.940474 17066 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:22.940979 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 641) took 1045 ms. Trace:
I20260709 12:22:22.941401 14422 rpcz_store.cc:276] 0709 12:22:21.896067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.896192 (+   125us) service_pool.cc:224] Handling call
0709 12:22:21.918861 (+ 22669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.031002 (+112141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:22.940741 (+909739us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:22.940916 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256204544}
I20260709 12:22:22.942240 17071 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.942919 17052 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.942793 17071 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.943626 17052 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:22.944404 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1690) took 1115 ms. Trace:
I20260709 12:22:22.944803 13647 rpcz_store.cc:276] 0709 12:22:21.828643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.834607 (+  5964us) service_pool.cc:224] Handling call
0709 12:22:22.944186 (+1109579us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:22.944331 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1560998144}
I20260709 12:22:22.945405 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.945768 17052 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:22.946066 13726 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.937736 17069 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:22.948503 17069 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:22.954166 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:22.955394 13797 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:22.970570 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.971572 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:22.972676 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:22.962361 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:22.986223 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:22.987109 13794 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:22.985137 13721 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:23.003933 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:23.005385 13719 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:23.020246 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:23.021178 13719 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:23.022699 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.023507 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:23.024976 13721 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:23.029371 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:23.030350 13792 raft_consensus.cc:2368] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:23.032438 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.036423 13799 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:22:23.036684 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:22.937788 16982 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.041672 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.066799 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.067303 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.068022 13797 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:22.984117 16963 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.061071 14417 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:22.945462 17071 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:22.952804 17025 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.079286 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.091387 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.159053 13650 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.160382 14422 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:23.177681 13643 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:23.153388 16654 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.388s	user 0.011s	sys 0.044s
I20260709 12:22:22.981354 17070 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.238s
I20260709 12:22:23.181807 13719 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:23.053270 13726 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:23.079507 17001 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:22.906389 17065 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.031908 17069 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 12:22:23.193871 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:23.202690 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:23.185933 16654 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.480s	user 0.012s	sys 0.044s
I20260709 12:22:23.179694 17025 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:23.193645 17051 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:22:23.155572 16963 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.277477 13793 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:22:23.033834 17068 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:23.271392 16972 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.788s	user 0.021s	sys 0.000s
I20260709 12:22:23.203963 17001 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.268231 17071 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:22:23.291267 16629 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.274525 17025 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.298178 16972 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.338948 17068 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:23.379040 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 219305us
I20260709 12:22:23.386390 14425 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:23.380957 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 179642us
I20260709 12:22:23.372232 17063 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.336683 16629 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.391404 14424 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:22:23.380957 17016 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.003s
W20260709 12:22:23.385666 17076 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:22:23.380160 17086 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:23.403512 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 200926us
I20260709 12:22:23.405884 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:23.398952 17063 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:23.417187 17081 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.133s	user 0.000s	sys 0.002s
I20260709 12:22:23.401306 17088 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:23.408025 17086 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.422653 17094 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:23.416085 17078 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:22:23.418596 17063 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.456979 13650 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
W20260709 12:22:23.429026 17077 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:22:23.430234 17100 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:23.457944 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 190500us
I20260709 12:22:23.462167 17097 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:22:23.463613 17083 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:22:23.467011 17065 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.402895 17016 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.438529 17063 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.480862 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:23.427505 17094 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.483412 16830 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:22:23.484514 17082 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:22:23.484856 17094 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.486765 16773 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.486693 17065 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:23.488639 17033 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.000s
I20260709 12:22:23.489112 17094 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:23.489085 17033 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.489656 17094 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:23.490417 16963 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.488271 17067 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:22:23.491062 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80277.4us
I20260709 12:22:23.497372 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:23.506865 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:23.509755 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:23.510677 13720 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:23.512121 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:23.483455 17085 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:22:23.517673 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:23.513684 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.518927 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:23.524451 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.525084 13646 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.525346 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:23.525936 13644 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:23.526688 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:23.490465 17016 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.534198 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.503358 16830 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:23.540627 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.543334 13646 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:23.514582 16789 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:23.549832 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.517017 17071 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.551128 13649 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:23.518013 16963 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:23.551679 13643 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:22:23.553604 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:23.558876 13799 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
I20260709 12:22:23.558019 13643 pending_rounds.cc:85] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 4
I20260709 12:22:23.559934 13643 pending_rounds.cc:107] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:22:23.479820 17100 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.559746 13801 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:23.527983 16972 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.560243 13643 raft_consensus.cc:2889] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:23.556463 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.538717 17001 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:23.568171 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:23.543836 13720 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:23.552081 16789 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:23.562388 13800 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:23.559149 17071 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.550467 17025 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:23.575989 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.575737 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:23.555579 16629 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.481742 17097 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.581622 13647 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:23.600549 17067 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:22:23.600947 17033 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.569721 16800 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.612264 16800 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.614861 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.615731 14421 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:23.562053 17106 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:23.622745 17106 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.622897 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 638) took 2331 ms. Trace:
I20260709 12:22:23.623219 17106 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.623320 14423 rpcz_store.cc:276] 0709 12:22:21.286707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.288474 (+  1767us) service_pool.cc:224] Handling call
0709 12:22:21.288887 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.289179 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:21.498684 (+209505us) log.cc:844] Serialized 14 byte log entry
0709 12:22:21.504780 (+  6096us) log.cc:844] Serialized 14 byte log entry
0709 12:22:23.614705 (+2109925us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b180046cc60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f53937aad70 00007f539379e293 00007f5393793cea 00007f539379423a 00007f5393bfea2a 00007f5393c0e0bf 00007f5393c0e038
0709 12:22:23.617353 (+  2648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:23.617410 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:23.617461 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.617555 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.617666 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2860,"prepare.queue_time_us":415035,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":6312,"raft.queue_time_us":8497,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":15284,"spinlock_wait_cycles":9533184,"thread_start_us":209067,"threads_started":1}
I20260709 12:22:23.625578 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.628911 17097 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.629514 17097 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:23.631568 17100 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.633062 17086 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.594323 17071 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.634621 16800 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.638449 16973 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:23.639194 16973 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:23.626204 14423 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:23.640194 16973 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.641065 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:23.642241 16800 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.642796 16800 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:23.643252 16800 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.645303 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2510) took 1956 ms. Trace:
I20260709 12:22:23.645711 13793 rpcz_store.cc:276] 0709 12:22:21.688726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.688933 (+   207us) service_pool.cc:224] Handling call
0709 12:22:21.689427 (+   494us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:23.278041 (+1588614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.644995 (+366954us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400129654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:23.645236 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3651187456}
W20260709 12:22:23.648517 16982 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.002s
I20260709 12:22:23.648969 16982 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.650077 14424 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:23.651784 16800 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.583370 17102 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:23.584816 16789 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.658699 17102 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.587718 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.659933 13800 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:23.588748 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.491914 17094 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.661562 17102 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.661883 17094 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.617683 16874 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:23.662909 16874 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.591382 13650 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:23.595908 13794 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:23.565986 16859 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:23.666482 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 473) took 1279 ms. Trace:
I20260709 12:22:23.666677 16859 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.667147 16859 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.667014 13645 rpcz_store.cc:276] 0709 12:22:22.386740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.386928 (+   188us) service_pool.cc:224] Handling call
0709 12:22:22.406823 (+ 19895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:22.566017 (+159194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:22.566058 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:22.569153 (+  3095us) log.cc:844] Serialized 26 byte log entry
0709 12:22:22.606247 (+ 37094us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:22.606334 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.666066 (+1059732us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:23.666120 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:23.666165 (+    45us) raft_consensus.cc:1685] finished
0709 12:22:23.666177 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:23.666409 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122,"raft.queue_time_us":742,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":7830,"spinlock_wait_cycles":364881280,"thread_start_us":19883,"threads_started":1,"wal-append.queue_time_us":52226}
I20260709 12:22:23.652410 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:23.659126 16789 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.586822 13652 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:22:23.590968 17093 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:22:23.565222 16963 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:23.566442 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 372691us
I20260709 12:22:23.676769 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:23.680205 16963 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:23.680663 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.682628 13652 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:23.683327 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.684376 13645 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:23.686196 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.687196 16982 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.688539 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:23.714442 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 195596us
I20260709 12:22:23.727861 13647 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:23.691190 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:23.729312 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:23.732705 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56036.9us
I20260709 12:22:23.733723 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:23.735064 13649 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:23.737535 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:23.671684 17104 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.672298 17111 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:23.745311 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 123694us
I20260709 12:22:23.730283 17016 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:23.730316 17099 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:23.754350 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:23.750900 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:23.763866 13793 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:23.763753 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:23.766825 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:23.768643 13793 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:22:23.775369 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44376.4us
I20260709 12:22:23.747650 17108 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:22:23.749326 17111 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.824162 17111 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.826617 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:23.827497 13799 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:23.828444 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:23.730309 17025 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 12:22:23.829165 13549 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 151) took 23586 ms (client timeout 15000 ms). Trace:
I20260709 12:22:23.799628 17013 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:23.799919 17068 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:23.675237 16979 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.831108 16979 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.763527 16982 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:23.730489 16654 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:22:23.767923 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:23.836233 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.836969 14416 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:23.737218 16907 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:22:23.775833 16963 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:22:23.813841 16874 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.763057 17099 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.817713 17039 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:23.863399 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95541.4us
I20260709 12:22:23.829020 13799 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:23.839458 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:23.837990 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.751796 17016 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:23.762480 17001 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:23.817147 17040 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:23.823393 16800 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:23.865259 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:23.874631 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:23.875202 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:23.777421 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:23.777406 16677 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.747767 17110 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 12:22:23.880302 13799 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:22:23.877362 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100020us
W20260709 12:22:23.878461 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:23.864953 17099 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.867749 16773 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.866777 16887 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.863471 17068 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:23.750576 17104 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.868903 17033 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:23.868780 17016 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.869186 16654 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:23.886502 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:23.904633 14416 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:23.855592 16961 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:22:23.869436 17001 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.889602 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:23.890592 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 161366us
I20260709 12:22:23.879580 17012 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:23.883994 17108 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:23.894088 16887 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.892841 16773 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.885073 17013 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:23.922466 17012 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.944830 17012 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:23.945456 16773 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.946185 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:23.946808 13649 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:23.947571 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:23.948438 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.948915 13649 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:23.949434 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:23.949810 13649 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:23.952844 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:23.954355 17023 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.912698 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 576) took 2332 ms. Trace:
I20260709 12:22:23.913452 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 581) took 1859 ms. Trace:
I20260709 12:22:23.885686 16477 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:23.954779 13643 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:23.991394 13725 rpcz_store.cc:276] 0709 12:22:22.054111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.450118 (+396007us) service_pool.cc:224] Handling call
0709 12:22:22.478033 (+ 27915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:22.537101 (+ 59068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.913238 (+1376137us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:23.913396 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83519360}
I20260709 12:22:23.956404 16859 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.896121 17004 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:22:23.955384 13649 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:23.916582 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:23.897770 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 503) took 3181 ms. Trace:
I20260709 12:22:23.918335 17001 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:23.917941 17068 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.885262 16677 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:23.903880 16605 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:22:23.936743 13693 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:23.914714 16654 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:23.897567 17033 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:23.921690 17039 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:22:23.929489 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1756) took 2083 ms (client timeout 1613 ms). Trace:
I20260709 12:22:23.935233 17104 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:23.937916 13726 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 12:22:23.866835 17106 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:23.939345 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
W20260709 12:22:23.998649 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82161.9us
I20260709 12:22:23.997550 13723 rpcz_store.cc:276] 0709 12:22:21.580701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.640234 (+ 59533us) service_pool.cc:224] Handling call
0709 12:22:21.671581 (+ 31347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.713115 (+ 41534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.912430 (+2199315us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b54000276d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:23.912630 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93768448}
W20260709 12:22:23.829649 13549 rpcz_store.cc:269] 0709 12:22:00.243300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.243652 (+   352us) service_pool.cc:224] Handling call
0709 12:22:23.829083 (+23585431us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:20.290741 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:20.291484 (+   743us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:20.291560 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:22:20.291581 (+    21us) tablet.cc:662] Decoding operations
0709 12:22:20.292594 (+  1013us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:20.292685 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:20.292716 (+    31us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:22:20.293523 (+   807us) tablet.cc:701] Row locks acquired
0709 12:22:20.293581 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:22:20.293781 (+   200us) write_op.cc:270] Start()
0709 12:22:20.293945 (+   164us) write_op.cc:276] Timestamp: P: 1783599740293708 usec, L: 0
0709 12:22:20.293970 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:22:20.448164 (+154194us) log.cc:844] Serialized 2810 byte log entry
0709 12:22:21.131557 (+683393us) op_driver.cc:464] REPLICATION: finished
0709 12:22:21.317867 (+186310us) write_op.cc:301] APPLY: starting
0709 12:22:21.322031 (+  4164us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:21.322323 (+   292us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:21.322500 (+   177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:21.446249 (+123749us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:21.513611 (+ 67362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:22:21.513671 (+    60us) write_op.cc:312] APPLY: finished
0709 12:22:21.514095 (+   424us) log.cc:844] Serialized 59 byte log entry
0709 12:22:21.514342 (+   247us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:21.514492 (+   150us) write_op.cc:454] Released schema lock
0709 12:22:21.514571 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:21.514970 (+   399us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:21.515006 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4326232,"reported_tablets":23,"spinlock_wait_cycles":4480,"child_traces":[["op",{"apply.queue_time_us":145728,"apply.run_cpu_time_us":9233,"apply.run_wall_time_us":200648,"num_ops":7,"prepare.queue_time_us":405710,"prepare.run_cpu_time_us":21320,"prepare.run_wall_time_us":451952,"raft.queue_time_us":231562,"raft.run_cpu_time_us":20085,"raft.run_wall_time_us":129909,"replication_time_us":822532,"thread_start_us":756984,"threads_started":4,"wal-append.queue_time_us":100537,"wal-append.run_cpu_time_us":141742,"wal-append.run_wall_time_us":1964326}]]}
I20260709 12:22:24.008455 14418 rpcz_store.cc:276] 0709 12:22:20.713612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:20.725094 (+ 11482us) service_pool.cc:224] Handling call
0709 12:22:20.725527 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:21.735508 (+1009981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:23.882496 (+2146988us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:23.895266 (+ 12770us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2320256768}
W20260709 12:22:24.012423 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1763) took 2121 ms (client timeout 2257 ms). Trace:
I20260709 12:22:24.015214 13650 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:22:24.055084 13650 pending_rounds.cc:85] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 2
I20260709 12:22:24.055586 13650 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 4.3
W20260709 12:22:24.054952 13798 rpcz_store.cc:269] 0709 12:22:21.890691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.890971 (+   280us) service_pool.cc:224] Handling call
0709 12:22:24.012075 (+2121104us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54001bbcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:24.012299 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1368336768}
I20260709 12:22:24.055929 13650 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:24.056406 13650 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:22:24.056687 13650 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:24.019886 14416 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:24.054061 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:24.032603 13693 leader_election.cc:403] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:24.043608 13796 rpcz_store.cc:269] 0709 12:22:21.846156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:21.847853 (+  1697us) service_pool.cc:224] Handling call
0709 12:22:23.929319 (+2081466us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:23.929422 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387541888}
I20260709 12:22:23.997251 16859 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:23.990586 16887 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:24.048337 13647 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:23.977656 17013 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:24.016633 16477 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:22:24.067382 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 201024us
I20260709 12:22:24.047784 17033 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:24.041267 16605 consensus_queue.cc:1243] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:24.073788 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:24.081941 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 149831us
I20260709 12:22:24.075659 14418 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:24.077374 16677 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:24.058238 17039 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:24.100982 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:24.078030 17063 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.004s
I20260709 12:22:24.102963 17001 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.103070 17039 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.103402 17063 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.104229 13800 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:24.105672 17063 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.106357 17092 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:24.106980 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.106894 17092 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:24.103091 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 1916) took 2044 ms (client timeout 2078 ms). Trace:
W20260709 12:22:24.107802 13718 rpcz_store.cc:269] 0709 12:22:22.059364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.100673 (+2041309us) service_pool.cc:224] Handling call
0709 12:22:24.103020 (+  2347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:24.108510 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:24.109117 17092 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.078604 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.078493 17013 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.111037 17001 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:24.083374 13793 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:24.068187 13649 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:24.071192 16961 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:22:24.112493 17013 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.112740 16961 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:22:24.113075 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.113125 16961 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:24.113389 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.114217 16961 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.115717 16961 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.084975 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:24.086791 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:24.123893 17013 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:24.124356 17013 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.124701 17013 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.125082 17086 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:24.125517 17086 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:24.126456 17086 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.127310 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.127704 17086 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.128611 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:24.129416 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44553.4us
W20260709 12:22:24.130107 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.132730 14424 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:24.133386 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.078763 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.097425 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:24.099795 17070 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:24.100929 16654 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:24.101169 17108 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:24.109192 13800 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.111127 16849 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:24.074110 14423 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:24.128677 16887 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:24.134359 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:24.136891 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.084534 16477 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:24.093713 14418 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:24.068681 17040 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:24.041513 17071 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.138721 13644 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:24.142177 17070 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:24.182353 17008 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:24.182827 17070 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.144109 17073 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:24.143215 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 582) took 1621 ms (client timeout 2104 ms). Trace:
I20260709 12:22:24.138870 13801 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:24.144115 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 304732us
I20260709 12:22:24.132820 13649 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:24.146239 16849 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:24.102020 17012 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.144413 17010 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.147619 16887 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:24.226577 16887 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:24.142589 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:24.227985 14416 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:24.203074 17106 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:24.148236 17092 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:24.215265 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:24.215437 13722 rpcz_store.cc:269] 0709 12:22:22.517035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:22.517181 (+   146us) service_pool.cc:224] Handling call
0709 12:22:24.137974 (+1620793us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540033fed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:24.138148 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127681792}
I20260709 12:22:24.235355 13795 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 9. Ops: []
I20260709 12:22:24.236322 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.236661 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:24.214769 17108 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:24.280181 13799 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:24.284301 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.264523 16859 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.289108 13793 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:24.160089 16677 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 12:22:24.185977 16979 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:22:24.299949 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.299463 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:24.408628 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 193462us
I20260709 12:22:24.409365 13643 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:24.416952 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:24.283974 16808 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:24.227219 16849 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:24.441839 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 284328us
I20260709 12:22:24.224853 17073 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.279546 17108 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:24.454520 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.438482 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:24.281680 17092 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.456954 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.449373 13800 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:24.243377 17010 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.311120 14421 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:24.438053 16808 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:24.311084 16859 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:24.478609 13767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.870s (SENT) [suppressed 30 similar messages]
I20260709 12:22:24.458135 17012 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:24.502521 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.514533 13793 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
W20260709 12:22:24.485661 16789 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.021s
I20260709 12:22:24.479133 16808 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:24.433152 16477 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:24.535609 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 390718us
I20260709 12:22:24.536231 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:24.537045 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 409635us
I20260709 12:22:24.539498 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:24.543942 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 281002us
I20260709 12:22:24.548558 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:24.550773 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:24.551831 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:24.552446 13725 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.555732 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:24.527894 16849 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.567775 16859 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:24.568810 17071 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:24.570354 17092 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.545260 17040 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:24.546141 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:24.574887 16644 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:24.593935 17111 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.769s	user 0.014s	sys 0.000s
I20260709 12:22:24.519234 17108 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.614482 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:24.598687 17039 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:24.602350 17092 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:24.636335 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.638355 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:24.628881 16961 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.513s	user 0.018s	sys 0.014s
I20260709 12:22:24.641443 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.642158 13644 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:24.643121 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.639770 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.643792 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:24.639156 13643 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:24.590728 17040 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.653780 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.657725 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.659915 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:24.659804 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:24.661684 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:24.657537 13645 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:24.601715 17013 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.671357 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:24.662345 13652 raft_consensus.cc:2376] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7. Re-sending same reply.
I20260709 12:22:24.629740 17039 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:24.675118 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:24.649099 17108 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:24.674041 13644 raft_consensus.cc:2376] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10. Re-sending same reply.
I20260709 12:22:24.672021 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:24.621150 16644 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.625770 17111 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.698962 13796 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:24.687659 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:24.700199 13797 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:22:24.703395 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.707629 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:24.684212 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 279088us
W20260709 12:22:24.726087 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.727423 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:24.730293 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:24.642194 17092 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.739693 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:24.740315 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:24.740495 14423 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:24.741539 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:24.744788 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:24.745563 14417 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:24.685298 17073 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:24.750898 13794 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:24.683442 17039 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:24.754598 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:24.744516 16965 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:22:24.722801 17010 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.733582 17065 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.742199 16961 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:24.707268 16989 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:24.767813 17099 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.873s	user 0.014s	sys 0.000s
I20260709 12:22:24.780471 14417 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:24.823357 16961 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:24.785372 13617 connection.cc:256] RPC call timeout handler was delayed by -2.2127233839996734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
W20260709 12:22:24.791826 17130 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:22:24.811283 13796 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
W20260709 12:22:24.839571 13617 connection.cc:256] RPC call timeout handler was delayed by -2.2115864150000561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2391s for a response
I20260709 12:22:24.782212 17071 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:24.816417 17010 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.881394 17149 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:24.883391 17144 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:24.884193 17144 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:24.884565 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 157322us
I20260709 12:22:24.888900 17144 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.892246 17144 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:24.893904 13725 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
W20260709 12:22:24.899367 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 353305us
I20260709 12:22:24.900378 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:24.774849 17137 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 12:22:24.793473 17065 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:24.828017 17099 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.874228 17071 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:24.909278 13798 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:24.910554 17099 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:24.911399 17137 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:24.897269 17128 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.304s	user 0.001s	sys 0.001s
W20260709 12:22:24.914984 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 187656us
W20260709 12:22:24.900961 17122 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:22:24.894186 17033 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.807s	user 0.008s	sys 0.004s
I20260709 12:22:24.911808 17137 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:24.939896 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:24.940659 13645 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:24.940934 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:24.939689 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95958.6us
W20260709 12:22:24.941959 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.556s (SENT)
I20260709 12:22:24.942513 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:24.914093 17071 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:24.886632 17010 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:24.948676 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48437.1us
I20260709 12:22:24.948900 17149 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:24.979174 17129 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:24.984930 17129 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:24.984968 17010 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.945502 17040 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:24.991389 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:24.995324 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46485.3us
I20260709 12:22:24.992591 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:24.987324 17129 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:24.996680 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.816s (SENT)
I20260709 12:22:24.997205 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:25.002135 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 623) took 1438 ms. Trace:
I20260709 12:22:24.949612 16996 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:24.938972 17033 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:25.002746 14419 rpcz_store.cc:276] 0709 12:22:23.563962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.565375 (+  1413us) service_pool.cc:224] Handling call
0709 12:22:23.593825 (+ 28450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:23.644283 (+ 50458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:23.644338 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:24.134901 (+490563us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b2c000d57a8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:24.135384 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:22:24.458949 (+323565us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:24.459001 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:24.459078 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:25.001735 (+542657us) raft_consensus.cc:1685] finished
0709 12:22:25.001768 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.002053 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5167,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":69088000,"thread_start_us":317520,"threads_started":1,"wal-append.queue_time_us":411672}
W20260709 12:22:25.014173 13617 connection.cc:256] RPC call timeout handler was delayed by -2.1771898360002524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 12:22:24.990948 17013 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:24.995079 16989 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:25.024576 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.853s (SENT)
I20260709 12:22:25.026335 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:25.028043 13617 connection.cc:256] RPC call timeout handler was delayed by -1.99529944599999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
I20260709 12:22:25.030167 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:22:25.060950 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50738.9us
I20260709 12:22:25.063019 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:25.059886 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.065270 13645 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:25.012856 17131 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:25.064033 14419 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:25.003722 17143 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.021250 17129 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.032368 17040 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:24.949699 16961 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.083547 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:25.081260 17129 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.089418 17148 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:25.091200 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.022667 17092 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.018316 17111 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:25.074687 17013 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:25.109514 16849 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
W20260709 12:22:25.110671 17086 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.983s	user 0.015s	sys 0.000s
I20260709 12:22:25.110850 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2117) took 1576 ms. Trace:
I20260709 12:22:25.102397 17129 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.112118 13722 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:25.110154 16808 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.004s
I20260709 12:22:25.111296 13646 rpcz_store.cc:276] 0709 12:22:23.534520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.109941 (+1575421us) service_pool.cc:224] Handling call
0709 12:22:25.110320 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.110516 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.110541 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:25.110584 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:25.110617 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.110671 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.110788 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:25.098476 16887 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.871s	user 0.007s	sys 0.012s
I20260709 12:22:25.034515 16989 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:25.137293 14418 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.959s	user 0.016s	sys 0.016s
I20260709 12:22:25.139314 17033 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.143692 17065 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:25.096233 17040 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:25.149197 13646 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:22:25.095285 17143 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.103818 17013 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.183287 17143 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:25.074170 17137 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:25.189571 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.189321 14418 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:25.203025 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 520) took 1182 ms. Trace:
W20260709 12:22:25.203760 17152 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:22:25.133395 16808 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.147305 16887 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:25.221833 17016 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.307s	user 0.013s	sys 0.022s
I20260709 12:22:25.222437 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.222369 17016 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.223898 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.224650 13646 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:25.224985 17016 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.226526 14420 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:25.227346 17033 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.227995 17143 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.233522 13724 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:25.233769 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.228838 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.234588 13646 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:25.235241 13725 raft_consensus.cc:2376] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6. Re-sending same reply.
I20260709 12:22:25.235786 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.229468 13718 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:25.236697 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.237293 13652 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:25.234508 13645 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:25.237974 13718 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:25.229962 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.239100 13719 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
W20260709 12:22:25.210783 17153 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.099s	user 0.000s	sys 0.003s
I20260709 12:22:25.179003 17013 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:25.240434 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.246848 13726 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:25.241936 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:25.208523 17129 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.258685 13648 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:25.243842 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.244729 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.262045 13720 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:22:25.262281 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.265501 13724 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:25.203614 14418 rpcz_store.cc:276] 0709 12:22:24.020822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.093091 (+ 72269us) service_pool.cc:224] Handling call
0709 12:22:24.093528 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.202584 (+1109056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.202940 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:25.218187 17146 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:25.239702 13646 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:25.268746 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:25.281955 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.286646 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 617) took 2243 ms. Trace:
I20260709 12:22:25.283092 13719 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:22:25.286872 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 206155us
I20260709 12:22:25.290786 14416 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:25.279458 17065 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:25.241462 17103 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.233526 17006 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.242705 17137 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:25.258235 17156 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:22:25.288872 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:25.293550 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:25.296960 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 620) took 2109 ms. Trace:
I20260709 12:22:25.287039 14425 rpcz_store.cc:276] 0709 12:22:23.043580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.044110 (+   530us) service_pool.cc:224] Handling call
0709 12:22:23.075328 (+ 31218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:23.386839 (+311511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.285310 (+1898471us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:25.286576 (+  1266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714802048}
I20260709 12:22:25.306903 14422 rpcz_store.cc:276] 0709 12:22:23.187632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.188029 (+   397us) service_pool.cc:224] Handling call
0709 12:22:25.296130 (+2108101us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:22:25.296357 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.296573 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.296600 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:25.296654 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:25.296691 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.296753 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.296892 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56223488}
I20260709 12:22:25.297849 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:25.227936 17033 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.308748 14419 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:25.280277 17086 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.334944 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:25.423405 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88561.1us
W20260709 12:22:25.381265 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:25.308815 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 316258us
I20260709 12:22:25.255501 17143 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.432288 13646 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:22:25.434702 17147 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:22:25.439847 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:25.476429 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:25.440734 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.477759 13721 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:25.443712 17013 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.281031 16830 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.137s
I20260709 12:22:25.298400 17118 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:25.466825 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.281123 17129 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.312245 17099 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:25.488423 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:25.311977 17159 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:25.451037 17123 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:22:25.481876 17153 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:22:25.483321 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.486213 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 404) took 1029 ms. Trace:
I20260709 12:22:25.306511 17071 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.312484 17065 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:25.308907 17006 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.466629 16808 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.519971 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.516894 13793 rpcz_store.cc:276] 0709 12:22:24.456393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.456638 (+   245us) service_pool.cc:224] Handling call
0709 12:22:25.485975 (+1029337us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b54002cb0d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:25.486132 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129615104}
I20260709 12:22:25.521721 13797 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:25.306876 17103 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.457129 16849 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:22:25.482867 17165 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:25.449581 17086 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.548108 13721 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:25.466462 17150 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
W20260709 12:22:25.528645 17108 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.835s	user 0.013s	sys 0.000s
I20260709 12:22:25.544590 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.488664 17118 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.418635 17040 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.507174 16989 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:25.510694 17159 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.511400 17068 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.533583 17136 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:25.601004 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 590) took 1065 ms. Trace:
I20260709 12:22:25.534215 17005 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.605611 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.606673 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:25.574530 17148 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:25.514580 17016 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:22:25.546175 17123 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:25.579562 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2517) took 1814 ms. Trace:
I20260709 12:22:25.611478 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2122) took 1928 ms. Trace:
W20260709 12:22:25.591768 17163 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:22:25.593871 17006 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:25.594849 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 161819us
I20260709 12:22:25.599206 17013 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.601603 13722 rpcz_store.cc:276] 0709 12:22:24.535596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.535765 (+   169us) service_pool.cc:224] Handling call
0709 12:22:25.600800 (+1065035us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54001a5254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:25.600929 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322749312}
I20260709 12:22:25.572063 17165 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:25.534598 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 107132us
W20260709 12:22:25.606933 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.614650 13723 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:25.567973 17118 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:25.611858 13650 rpcz_store.cc:276] 0709 12:22:23.682595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.685347 (+  2752us) service_pool.cc:224] Handling call
0709 12:22:23.685663 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.066650 (+380987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.611208 (+1544558us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:25.611397 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725184384}
I20260709 12:22:25.601559 17136 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.596009 17159 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:25.611081 13799 rpcz_store.cc:276] 0709 12:22:23.764850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.903771 (+138921us) service_pool.cc:224] Handling call
0709 12:22:23.917979 (+ 14208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.285448 (+367469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.579299 (+1293851us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:25.579493 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832339200}
I20260709 12:22:25.609031 17148 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.629460 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:25.571187 16808 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:22:25.633714 17165 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.634671 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.636359 17013 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.655218 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.659691 13721 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:25.639242 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:25.644609 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.649013 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.663312 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:25.605024 17068 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.641860 17136 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.670977 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:25.641089 16887 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:25.646751 17148 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.613994 17005 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.645591 17161 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:25.657660 17168 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:25.703850 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.704707 13720 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:25.686501 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.687531 17129 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.683915 17161 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.695086 17086 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:25.672374 17150 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:25.666643 16989 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:25.717060 13720 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:25.694438 17123 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.676975 17099 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:25.732437 13721 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:25.710644 17005 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:25.730379 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1369919119997576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 12:22:25.734702 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:25.663617 17013 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.710175 17136 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:25.735924 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.735945 14422 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
W20260709 12:22:25.750408 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0811919120001221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
I20260709 12:22:25.752285 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:25.657434 17065 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:25.755637 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 279264us
I20260709 12:22:25.756691 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:25.758566 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.734901 17168 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:25.763379 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:25.746464 17150 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:25.758752 17071 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:25.757633 17086 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:25.761500 16654 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.775810 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.812983 13721 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:25.762789 16476 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:25.777810 17129 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.777798 16961 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:25.778832 16830 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:25.780087 13720 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:25.819873 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.821141 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.821947 13720 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:25.767338 17103 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.782411 13693 leader_election.cc:400] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:25.783125 16773 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:25.787262 17068 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:25.788188 17168 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.790412 17136 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:25.794528 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:25.743301 17099 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.760382 17137 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:25.801993 17150 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.805572 17013 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:25.806594 13617 connection.cc:256] RPC call timeout handler was delayed by -1.8648537860003671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 12:22:25.838647 13617 connection.cc:256] RPC call timeout handler was delayed by -1.764707547999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
W20260709 12:22:25.823629 17039 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.040s	user 0.015s	sys 0.000s
I20260709 12:22:25.809037 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:25.839886 13727 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:25.809896 14421 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 12:22:25.780198 16644 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:25.826422 13693 leader_election.cc:403] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:22:25.843649 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61426.1us
I20260709 12:22:25.828254 16773 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:25.827786 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.847404 13721 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:25.827669 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.831076 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:25.812302 17086 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:25.839049 13617 connection.cc:256] RPC call timeout handler was delayed by -1.7492792519997238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 12:22:25.808038 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 715571us
I20260709 12:22:25.848124 16677 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:25.836153 17137 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:25.842374 13727 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
I20260709 12:22:25.855721 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:25.845288 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:25.824944 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:25.845767 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.853960 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:25.850104 13617 connection.cc:256] RPC call timeout handler was delayed by -1.7355580670000563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
W20260709 12:22:25.840718 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77464.7us
I20260709 12:22:25.822639 16654 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.825304 16476 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:25.856341 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2145) took 1014 ms. Trace:
I20260709 12:22:25.860898 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.823089 16961 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.857060 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.885728 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:25.893412 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:25.886580 13643 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:25.894373 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:25.886406 13617 connection.cc:256] RPC call timeout handler was delayed by -1.6146851080002307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 12:22:25.823422 17129 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:25.895148 13617 connection.cc:256] RPC call timeout handler was delayed by -1.6035599370002274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
W20260709 12:22:25.895870 13617 connection.cc:256] RPC call timeout handler was delayed by -1.528172148000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 12:22:25.896126 13617 connection.cc:256] RPC call timeout handler was delayed by -1.5153224929999851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 12:22:25.896363 13617 connection.cc:256] RPC call timeout handler was delayed by -1.4851567779996913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2278s for a response
W20260709 12:22:25.896615 13617 connection.cc:256] RPC call timeout handler was delayed by -1.4564864149997447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.323s for a response
W20260709 12:22:25.896859 13617 connection.cc:256] RPC call timeout handler was delayed by -1.4373851080003988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
W20260709 12:22:25.897096 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3806851080003071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 12:22:25.897343 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0214721480001572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 12:22:25.838002 17071 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
W20260709 12:22:25.900223 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44608us
I20260709 12:22:25.852839 17086 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:25.848567 17013 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:25.898366 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:25.901007 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:25.904832 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.887245 17123 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:25.903041 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.843366 17039 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:25.890229 17136 raft_consensus.cc:2764] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:22:25.908461 13646 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:25.894556 17068 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:25.930137 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.917596 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.920495 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:25.920637 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:25.931746 13792 raft_consensus.cc:2376] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7. Re-sending same reply.
I20260709 12:22:25.932683 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.933779 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.934650 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.903355 17103 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:22:25.961606 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:25.964778 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:25.966709 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46377.1us
I20260709 12:22:25.967780 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:25.969933 13645 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:25.889302 17137 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:25.927554 13643 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:25.890630 13644 rpcz_store.cc:276] 0709 12:22:24.841743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.841894 (+   151us) service_pool.cc:224] Handling call
0709 12:22:24.842299 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:24.872299 (+ 30000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:24.872352 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:24.880491 (+  8139us) log.cc:844] Serialized 26 byte log entry
0709 12:22:25.696319 (+815828us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800450160. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:25.746595 (+ 50276us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:25.843917 (+ 97322us) log.cc:844] Serialized 14 byte log entry
0709 12:22:25.844592 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:25.844685 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:25.844725 (+    40us) raft_consensus.cc:1685] finished
0709 12:22:25.856107 (+ 11382us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:25.856277 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":110469,"raft.queue_time_us":2155,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":16743,"spinlock_wait_cycles":25234816,"thread_start_us":128234,"threads_started":2,"wal-append.queue_time_us":884945}
W20260709 12:22:26.005735 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:25.935276 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.854476 17040 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:22:25.935998 13792 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:25.908600 17008 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:25.982743 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.011423 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.012764 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:25.983124 13721 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:25.984056 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:25.984334 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:25.986382 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:25.989603 13725 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:25.999967 13648 raft_consensus.cc:2368] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9.
I20260709 12:22:26.005530 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:25.921499 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:26.005888 13643 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:26.021106 13801 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:26.021754 13801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 395) took 1786 ms (client timeout 2200 ms). Trace:
I20260709 12:22:26.024125 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:25.901911 16773 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.025614 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1707) took 2095 ms. Trace:
I20260709 12:22:26.025732 16677 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:26.026373 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 388) took 2494 ms. Trace:
I20260709 12:22:26.027029 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.026973 13795 rpcz_store.cc:276] 0709 12:22:23.516912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.517038 (+   126us) service_pool.cc:224] Handling call
0709 12:22:23.539305 (+ 22267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.281127 (+741822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.011464 (+1730337us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b540012a054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:26.011639 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586262272}
I20260709 12:22:26.026094 13647 rpcz_store.cc:276] 0709 12:22:23.930390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:23.935698 (+  5308us) service_pool.cc:224] Handling call
0709 12:22:23.936120 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.070039 (+133919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.010830 (+1940791us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400147dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:26.025553 (+ 14723us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257238272}
I20260709 12:22:26.030884 13647 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:26.031296 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:26.031675 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.032698 13643 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:26.033591 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:26.036636 17175 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.037276 17175 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:26.037657 17175 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.038937 13797 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: []
W20260709 12:22:26.039539 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 592) took 1503 ms (client timeout 1799 ms). Trace:
I20260709 12:22:26.039750 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.040228 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:26.041323 13792 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:26.039976 13723 rpcz_store.cc:269] 0709 12:22:24.536580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.536715 (+   135us) service_pool.cc:224] Handling call
0709 12:22:26.039358 (+1502643us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:26.039477 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338989952}
I20260709 12:22:25.904448 16830 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:22:26.042848 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.063982 13800 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:25.918206 17178 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:26.043596 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:26.022147 13801 rpcz_store.cc:269] 0709 12:22:24.235572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.238585 (+  3013us) service_pool.cc:224] Handling call
0709 12:22:26.010917 (+1772332us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:26.021680 (+ 10763us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992256}
I20260709 12:22:25.922747 17010 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:25.902098 16644 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:22:26.073453 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.061740 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.061779 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:26.075059 14422 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:26.042526 17157 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:22:26.069466 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:26.076297 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:26.078780 13795 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:22:26.079481 13693 leader_election.cc:400] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9.
I20260709 12:22:26.082897 13693 leader_election.cc:403] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:22:26.083767 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.012220 17131 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:25.928563 16996 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.140s
I20260709 12:22:26.042423 17121 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:26.066210 16773 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:26.114005 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.130048 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.081878 13726 raft_consensus.cc:2376] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7. Re-sending same reply.
I20260709 12:22:26.126847 13800 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:26.132434 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.132591 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:26.133330 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:26.134689 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:26.137007 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 150773us
I20260709 12:22:26.149720 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.152209 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.157356 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:26.083688 17178 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:26.089350 17103 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9.
W20260709 12:22:26.169534 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.112020 17099 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:26.093739 17131 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:26.229687 17131 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.136020 17060 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:26.234761 17060 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.206243 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.088838 17136 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 12:22:26.239918 17060 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.240571 17060 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.083923 16887 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 12:22:26.083305 17145 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 12:22:26.163218 17179 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:26.247665 17179 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.247985 17131 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.092536 17040 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:22:26.192359 16973 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:26.076514 17008 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:26.197603 17177 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.250217 17179 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.251084 16973 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:26.256081 16973 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.251250 17008 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.251443 17177 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.250721 17103 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:26.200222 17121 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:26.299971 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.301648 16654 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:26.301972 17185 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
W20260709 12:22:26.302467 17189 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:22:26.303076 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.322604 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.241729 17060 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.322888 17178 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:26.303848 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.304572 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.302975 13795 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:26.202687 17066 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:26.326464 17129 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.225607 16476 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:26.225111 16677 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:26.316033 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.316893 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:26.329228 13800 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.306317 17103 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:26.330919 17103 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.332930 17103 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.326807 17066 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.268540 16979 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:26.320807 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.327171 17060 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.339959 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.250425 17131 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.319641 17008 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.359165 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.362362 13650 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:26.376065 13801 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 12:22:26.378369 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.328962 17121 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:26.355962 16996 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.395403 13800 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:26.394740 17039 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.430771 16996 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.453377 16996 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:26.460707 17148 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.757s	user 0.009s	sys 0.001s
I20260709 12:22:26.466712 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.468772 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:26.388375 17168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.023s	sys 0.028s
W20260709 12:22:26.485147 17168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.023s	sys 0.034s
I20260709 12:22:26.487859 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:26.489108 14418 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:22:26.491618 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1950) took 1186 ms. Trace:
I20260709 12:22:26.492211 13719 rpcz_store.cc:276] 0709 12:22:25.304863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.305029 (+   166us) service_pool.cc:224] Handling call
0709 12:22:25.305395 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:25.751866 (+446471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.751898 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:25.759909 (+  8011us) log.cc:844] Serialized 26 byte log entry
0709 12:22:26.090870 (+330961us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:26.090907 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.490995 (+400088us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b5400173754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:26.491064 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:26.491151 (+    87us) raft_consensus.cc:1685] finished
0709 12:22:26.491171 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:26.491529 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26362,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6992,"spinlock_wait_cycles":611441536,"thread_start_us":452767,"threads_started":2,"wal-append.queue_time_us":547423}
I20260709 12:22:26.495075 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:26.496168 17191 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:22:26.501649 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1949) took 1233 ms. Trace:
I20260709 12:22:26.502163 13724 rpcz_store.cc:276] 0709 12:22:25.268068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.268187 (+   119us) service_pool.cc:224] Handling call
0709 12:22:25.303801 (+ 35614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:25.464421 (+160620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.464466 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:25.464513 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:25.464546 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.501371 (+1036825us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400155ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:26.501416 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:26.501581 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368874240}
I20260709 12:22:26.503654 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.504297 13724 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:26.505237 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.505790 13724 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:26.508175 17203 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:26.509097 17177 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:26.510154 17121 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.511472 17066 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:26.515148 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.515877 17023 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:26.516444 17023 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.518136 17161 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:26.520901 17086 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.005s
I20260709 12:22:26.522713 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:26.523542 16859 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.524230 16859 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.524860 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:26.523613 13801 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:26.525624 14416 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:26.527426 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.527634 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:26.528256 13797 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:26.528605 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:26.529183 17023 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.534233 16961 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.529093 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:26.559460 17159 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.913s	user 0.012s	sys 0.000s
I20260709 12:22:26.560276 13797 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:26.560017 17159 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.521432 17086 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.561192 13727 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:26.563051 17159 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:26.565774 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:26.569272 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:26.570641 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.571332 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:26.528277 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:26.573304 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:26.573706 17176 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.866s	user 0.005s	sys 0.000s
I20260709 12:22:26.509141 17203 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:22:26.577446 17086 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:26.579908 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:26.580678 13646 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:26.580595 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.581987 17066 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:26.582720 17066 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:26.585721 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.587743 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.589877 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:26.538797 17184 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.526s	user 0.002s	sys 0.000s
W20260709 12:22:26.510203 17118 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.863s	user 0.009s	sys 0.000s
I20260709 12:22:26.625340 17118 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.627295 13795 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:26.627365 13796 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:26.547371 13648 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:22:26.549903 17175 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.004s
I20260709 12:22:26.574124 17164 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:26.594103 17108 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:26.593914 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.583957 17066 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.595736 16859 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.461458 17148 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.596604 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:26.598814 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 76651.4us
I20260709 12:22:26.599258 17179 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:26.605842 13724 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:22:26.606135 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 377903us
I20260709 12:22:26.657238 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:26.608022 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1781) took 2528 ms. Trace:
I20260709 12:22:26.495987 13719 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:26.618952 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 406) took 2020 ms. Trace:
I20260709 12:22:26.658011 13798 rpcz_store.cc:276] 0709 12:22:24.080146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.110312 (+ 30166us) service_pool.cc:224] Handling call
0709 12:22:24.110663 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:24.909791 (+799128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:26.607699 (+1697908us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b54001ca7d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:26.607951 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836007168}
I20260709 12:22:26.620966 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.658525 13794 rpcz_store.cc:276] 0709 12:22:24.598371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:24.598609 (+   238us) service_pool.cc:224] Handling call
0709 12:22:26.618726 (+2020117us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:26.618874 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288823040}
W20260709 12:22:26.621552 17123 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.698s	user 0.008s	sys 0.004s
W20260709 12:22:26.622238 14422 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:22:26.660337 14422 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:22:26.622926 17137 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.004s
I20260709 12:22:26.651576 17108 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:26.638095 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 485999us
I20260709 12:22:26.661829 17137 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.662349 17179 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:26.662896 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.663226 17179 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.664287 17137 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.664606 17179 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:26.664958 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.668277 17161 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:26.651366 17175 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.669359 17108 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.670734 17175 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.646893 17164 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:26.671804 17164 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:26.638913 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.007s	sys 0.033s
W20260709 12:22:26.674608 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.007s	sys 0.033s
W20260709 12:22:26.674947 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 641403us
I20260709 12:22:26.640368 13795 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:22:26.677057 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80586.5us
I20260709 12:22:26.536283 13801 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:26.642122 13721 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:26.647850 13646 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:26.648859 13723 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:26.678697 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:26.651965 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:26.620582 13727 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:22:26.670805 17108 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:26.638003 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.688943 13718 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:26.691708 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53846us
I20260709 12:22:26.693050 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:26.654425 17124 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.031s	sys 0.060s
I20260709 12:22:26.696758 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:26.697515 13719 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.698812 14391 leader_election.cc:400] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:26.699249 14391 leader_election.cc:403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:26.694339 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.700898 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:26.702184 14422 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 4.
I20260709 12:22:26.703298 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.703614 13724 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:26.705734 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.675879 16773 raft_consensus.cc:2764] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:22:26.707545 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:26.708370 14422 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:22:26.709728 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52582.8us
I20260709 12:22:26.711258 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:26.701036 17008 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:26.708779 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:26.716722 17137 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:26.717110 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:26.717651 17192 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:26.718283 17192 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:26.718426 17008 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:26.719269 17192 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:26.720890 17033 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:26.689858 17025 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:26.704476 17124 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.031s	sys 0.061s
I20260709 12:22:26.688145 17123 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:26.723414 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44793.1us
I20260709 12:22:26.724834 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.728859 16982 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:26.729154 16957 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:26.729638 14420 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:26.728883 17193 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:26.713909 17148 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:26.702008 17068 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:26.735150 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:26.695060 17121 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:22:26.690810 17010 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:26.738869 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:26.733151 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:26.747195 13797 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:26.737893 16982 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:26.759140 16957 consensus_queue.cc:1243] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:26.725986 16677 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:26.762337 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.763325 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:26.766357 16477 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:22:26.767738 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.727540 17004 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:22:26.715022 16996 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:22:26.714625 17111 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:22:26.770546 17060 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:26.736294 13724 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:26.716777 17198 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 12:22:26.780876 17193 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:26.733345 17118 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:26.808439 17203 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:26.828989 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.809303 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:26.830165 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97107.3us
I20260709 12:22:26.836071 16830 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.878753 17070 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:26.879348 16830 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.879873 16830 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.848151 17168 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.881029 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.881385 17168 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.881891 17168 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:26.850015 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 86787.2us
I20260709 12:22:26.869378 13722 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.863242 16982 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:26.902408 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 177632us
W20260709 12:22:26.902732 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163936us
W20260709 12:22:26.904227 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 169139us
I20260709 12:22:26.905035 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:26.906597 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.909842 13719 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:26.863943 17202 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:22:26.912084 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 245886us
I20260709 12:22:26.922722 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:26.912638 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.917024 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:26.924021 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 108714us
I20260709 12:22:26.924455 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:26.919472 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.863566 17060 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:26.927510 13719 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:26.902514 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 167869us
I20260709 12:22:26.936875 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.939822 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.942080 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:26.952656 13795 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:26.954949 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.955930 13798 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:26.957070 13799 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:26.957322 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.960592 13722 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:26.889899 17118 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:22:26.919975 17203 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:26.983892 17203 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:26.984417 17203 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:26.986738 17092 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:26.916155 17123 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:26.988767 17123 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:26.992551 17123 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:26.992913 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:26.920246 17202 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:26.993862 17202 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:26.922106 16773 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:26.944321 17158 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:26.997658 17193 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:27.002704 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80076.2us
I20260709 12:22:27.004101 17198 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:26.973675 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:27.004611 17198 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:27.004889 13720 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:26.973948 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:27.006093 17158 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.006685 16887 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:26.925696 17103 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:27.007387 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:26.983420 17215 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:22:27.010922 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.958417 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:27.002483 17060 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.003921 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:27.021206 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.926017 17148 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:26.976354 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:26.946630 17008 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.005508 17198 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.017421 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:26.918205 16973 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:22:26.946301 17033 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:26.975657 17218 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:22:26.955873 17013 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:27.012836 17145 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:22:27.037262 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:26.959123 16982 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.036358 13644 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:26.967041 17150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.015s	sys 0.014s
I20260709 12:22:26.921733 17068 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:22:27.059849 13643 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:26.968431 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:26.969297 17004 consensus_queue.cc:1243] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:22:27.016772 14418 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.520s	user 0.017s	sys 0.004s
I20260709 12:22:27.063342 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:26.969938 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:27.026623 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:27.120319 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93791us
I20260709 12:22:27.121706 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:27.100087 13721 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:22:27.114244 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2165) took 1298 ms. Trace:
I20260709 12:22:27.122634 13721 pending_rounds.cc:85] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 4
I20260709 12:22:27.123092 13721 pending_rounds.cc:107] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260709 12:22:27.119256 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56031.7us
I20260709 12:22:27.123785 13721 raft_consensus.cc:2889] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:27.126910 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:27.065047 16887 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.129290 13651 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:27.136860 13720 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:22:27.022673 17097 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:22:27.053222 17103 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.063668 16773 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:27.102425 17150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.015s	sys 0.014s
I20260709 12:22:27.122715 13652 rpcz_store.cc:276] 0709 12:22:25.815449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.815731 (+   282us) service_pool.cc:224] Handling call
0709 12:22:25.830337 (+ 14606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:25.834975 (+  4638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:25.835038 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:25.836965 (+  1927us) log.cc:844] Serialized 26 byte log entry
0709 12:22:26.611320 (+774355us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800151d20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:26.724875 (+113555us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:22:27.103472 (+378597us) log.cc:844] Serialized 14 byte log entry
0709 12:22:27.113687 (+ 10215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.113803 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:27.113865 (+    62us) raft_consensus.cc:1685] finished
0709 12:22:27.113976 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:27.114162 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34777,"prepare.queue_time_us":21126,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":5289,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4087,"spinlock_wait_cycles":5008768,"thread_start_us":470072,"threads_started":2,"wal-append.queue_time_us":1049322}
I20260709 12:22:27.138670 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:27.146390 13721 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:22:27.146843 13651 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:22:27.147055 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 216658us
W20260709 12:22:27.148062 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.148856 13643 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:27.051085 17148 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.128592 17198 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.118033 14418 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:27.180853 13797 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:27.128712 17033 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:27.187592 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.548s (SENT)
I20260709 12:22:27.144886 17013 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:27.144577 17097 consensus_queue.cc:1243] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:27.192253 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:27.136140 17008 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.197209 13643 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:27.198622 13722 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.201607 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:27.175993 17179 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.511s	user 0.019s	sys 0.001s
I20260709 12:22:27.203914 13719 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:22:27.211743 14391 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.568s (SENT)
I20260709 12:22:27.162679 17010 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:27.237845 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:27.207372 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 184289us
I20260709 12:22:27.236790 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:27.200428 17164 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.528s	user 0.015s	sys 0.013s
I20260709 12:22:27.259775 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:27.190119 17175 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:27.206037 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:27.267532 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55846.7us
W20260709 12:22:27.118403 17222 consensus_queue.cc:1175] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 7 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:22:27.271787 14391 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.578s (SENT)
I20260709 12:22:27.272259 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:27.197203 17013 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.275224 17161 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.276657 17146 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:27.305146 14391 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.580s (SENT)
I20260709 12:22:27.278404 17097 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.266288 17158 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.263060 17010 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.288942 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:27.289613 16859 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:27.235944 16887 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.311203 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:27.196599 16773 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:27.195982 17124 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:27.299839 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:27.325618 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:27.311017 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:27.295027 17225 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:22:27.238651 16973 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.203670 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 637) took 1116 ms. Trace:
I20260709 12:22:27.303129 17161 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:27.317919 17068 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:27.322482 17150 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:27.321447 16989 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.321291 16887 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.333868 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:27.310123 17013 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:27.338848 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:27.349581 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 224673us
I20260709 12:22:27.319065 17143 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.351794 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:27.313623 17179 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:27.361541 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:27.299134 17164 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.337603 16982 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.352321 16989 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.363583 16773 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:27.364837 17238 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:27.365586 16887 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:27.334609 14418 rpcz_store.cc:276] 0709 12:22:26.080081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.083926 (+  3845us) service_pool.cc:224] Handling call
0709 12:22:26.487590 (+403664us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:27.196783 (+709193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64433408}
I20260709 12:22:27.346683 17203 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.352488 17144 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:27.367594 17179 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:27.369248 16789 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:27.369513 16677 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:27.369809 17148 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.370432 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:27.343976 17097 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:27.341450 17150 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:27.344470 17192 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:27.332906 17124 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.361239 17111 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:27.425141 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102091us
I20260709 12:22:27.394723 17144 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.392594 13724 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:27.344615 17158 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:22:27.404760 13792 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:22:27.362128 17175 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:22:27.377058 17238 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:27.421223 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76661.5us
I20260709 12:22:27.528393 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:27.421742 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51419.7us
I20260709 12:22:27.422473 17237 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:27.423292 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 116732us
I20260709 12:22:27.392130 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:27.415756 17192 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.416363 17150 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:27.427338 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:27.529521 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:27.536924 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.685s (SENT)
I20260709 12:22:27.535199 14420 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:27.551107 16859 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.551654 16859 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.553817 16859 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:27.554737 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:27.542539 17236 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:22:27.556594 17097 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:27.557072 17097 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:27.557782 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:27.558562 17179 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.562304 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:27.563252 17164 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:27.563733 16677 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:27.564226 16677 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.564566 16677 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:27.566018 17227 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:22:27.533231 17238 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:27.569541 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2174) took 1408 ms. Trace:
W20260709 12:22:27.570874 17232 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:22:27.570695 16887 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:22:27.572095 17233 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:22:27.497820 17144 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:27.569962 13648 rpcz_store.cc:276] 0709 12:22:26.161651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.165222 (+  3571us) service_pool.cc:224] Handling call
0709 12:22:26.165593 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:26.630125 (+464532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.569295 (+939170us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:27.569476 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876833664}
I20260709 12:22:27.574080 17236 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:27.574105 13649 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:27.574566 17236 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:27.574815 13649 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:27.575636 17236 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.575742 13649 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.577028 17236 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:27.577284 13649 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:22:27.545850 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 174424us
I20260709 12:22:27.544572 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:27.588811 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44353.1us
W20260709 12:22:27.590454 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.590606 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:27.592041 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:27.594781 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 167523us
I20260709 12:22:27.599627 13649 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:22:27.525388 16789 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:27.600941 13649 pending_rounds.cc:85] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 2
I20260709 12:22:27.604135 13649 pending_rounds.cc:107] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 12:22:27.601977 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65119.3us
I20260709 12:22:27.605901 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:27.606335 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76893.1us
I20260709 12:22:27.606858 13649 raft_consensus.cc:2889] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:27.607108 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:27.608062 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:27.609714 13794 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:27.545562 16982 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:27.546489 17192 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:27.612758 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44848.8us
W20260709 12:22:27.597532 17108 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.906s	user 0.020s	sys 0.001s
I20260709 12:22:27.627517 13649 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:27.593783 16979 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:27.629732 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:27.631886 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2522) took 2112 ms (client timeout 2083 ms). Trace:
W20260709 12:22:27.632297 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.633018 13800 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:22:27.529268 17143 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:22:27.643281 13720 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:27.643441 13796 rpcz_store.cc:269] 0709 12:22:25.519490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:25.519631 (+   141us) service_pool.cc:224] Handling call
0709 12:22:27.631619 (+2111988us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003c0354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:27.631796 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2515449600}
I20260709 12:22:27.610780 16973 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:27.600030 17241 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.642431 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.602157 16887 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:27.619895 17087 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:27.603813 16789 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:27.625285 17175 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:27.609766 17231 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:27.641119 17196 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 12:22:27.634555 17168 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.005s
I20260709 12:22:27.675911 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.679180 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.649513 17033 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.632975 17068 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:27.679435 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.633535 17246 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:27.693109 13647 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:27.694317 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:27.697893 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.710642 13644 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:22:27.701087 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.711874 13652 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:27.715651 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.717389 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:27.717382 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109789us
I20260709 12:22:27.718874 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:27.641099 17060 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:22:27.690920 17175 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:27.721346 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.739588 13647 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:27.682102 17241 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.727274 13801 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:27.747073 13801 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:27.727782 13644 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:27.730787 17243 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.170s	user 0.000s	sys 0.004s
I20260709 12:22:27.731659 17194 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:22:27.734738 17235 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 12:22:27.736135 17240 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:22:27.701349 16654 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:27.670996 17229 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:22:27.748428 13801 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.782079 13801 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:27.704854 17033 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.757494 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:27.787981 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.788727 17241 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:27.789974 17175 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.795691 13720 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:22:27.796029 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81783.2us
I20260709 12:22:27.797758 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:27.798511 17071 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:22:27.798463 17099 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:27.799474 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 419) took 1038 ms. Trace:
I20260709 12:22:27.799837 16859 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:27.801218 16859 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:27.800187 13801 rpcz_store.cc:276] 0709 12:22:26.761435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.761799 (+   364us) service_pool.cc:224] Handling call
0709 12:22:26.762164 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.727147 (+964983us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540040b8a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:22:27.798092 (+ 70945us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:27.798123 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:27.798230 (+   107us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:27.798268 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.798532 (+   264us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:27.799284 (+   752us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2833,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":255883136}
I20260709 12:22:27.802593 17123 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:27.803197 17123 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:27.801851 16859 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:27.762018 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.776525 17087 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:27.829629 17087 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.800436 17099 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:22:27.711441 17068 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.834668 17068 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.744843 17131 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:27.695889 17168 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.826933 17124 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.807050 16859 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.804772 17123 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.853058 13644 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:22:27.849102 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91703.8us
I20260709 12:22:27.867576 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:27.848836 17241 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:22:27.865468 17229 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.874243 17229 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.846968 17068 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:27.879667 17068 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.876371 17097 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.763397 17178 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.886569 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:27.887280 13719 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:27.761078 17203 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.756181 17194 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:27.899761 17025 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:27.903259 17068 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.903519 17254 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:22:27.904619 17229 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:27.887288 14420 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:27.865010 17192 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:27.831426 17198 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.642s	user 0.006s	sys 0.006s
I20260709 12:22:27.907742 13719 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:27.910606 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.911446 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:27.912598 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.838518 17131 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:27.916885 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.928814 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:27.921588 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:27.934428 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:27.909915 17202 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.909s	user 0.013s	sys 0.000s
I20260709 12:22:27.850479 17248 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:27.892555 17097 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:27.853305 17099 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 12:22:27.933542 13721 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:27.898356 16979 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:27.937464 13723 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:27.931823 16830 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.051s	user 0.008s	sys 0.012s
I20260709 12:22:27.922720 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:27.942408 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:27.834708 17060 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 12:22:27.944095 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:27.953208 13720 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:27.939088 13645 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:27.915645 16982 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:27.970170 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:27.971742 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49150.1us
I20260709 12:22:27.975961 13648 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:27.930800 17254 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:22:27.972157 17262 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:27.979063 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:27.980724 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:27.945353 17248 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:27.945820 17237 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:27.972234 17194 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:27.995231 17192 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:27.952276 17123 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:27.940342 17005 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:22:28.002774 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.010939 13644 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:28.009166 17262 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:27.989621 17145 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:28.010224 17013 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
W20260709 12:22:28.005570 17161 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.004s
I20260709 12:22:27.998684 17248 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:28.003259 17025 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:27.950382 17202 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:27.981456 16830 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.041545 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:28.044346 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:28.046928 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70240.4us
I20260709 12:22:28.014072 17033 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:28.076398 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.080626 13647 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:22:28.018217 17192 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.031107 17194 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:28.089890 13796 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:28.090835 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.091897 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:28.092774 13799 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:28.093600 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:28.094231 13794 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:28.098718 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.101203 13801 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:28.102859 13647 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
W20260709 12:22:28.090011 17144 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.517s	user 0.019s	sys 0.000s
I20260709 12:22:28.101188 17192 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.106540 17144 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.000835 17131 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.018020 16677 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:28.114194 17260 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:22:28.114889 17244 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:28.115162 17144 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:28.115276 16677 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:28.119525 13651 raft_consensus.cc:2368] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
I20260709 12:22:28.023377 17013 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.122114 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.010478 17238 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.032974 17161 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.043538 17025 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.119029 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:28.115629 17244 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.092448 17237 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:28.145076 17150 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.002s
I20260709 12:22:28.137243 17248 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.152225 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.142573 17131 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:28.145175 17258 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.143s
I20260709 12:22:28.123827 17168 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:28.083222 16982 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.146955 17124 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:28.116317 17230 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.241s
I20260709 12:22:28.124217 16677 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 12:22:28.201519 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.188108 16979 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.145792 17150 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.194104 17124 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:28.253660 14391 leader_election.cc:336] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.740s (SENT)
I20260709 12:22:28.261785 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:28.274816 17087 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:28.301144 14391 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.799s (SENT)
I20260709 12:22:28.305315 13798 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:22:28.153609 17237 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.239151 17231 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:28.328984 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.827s (SENT)
I20260709 12:22:28.263792 17124 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:28.355082 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:28.401572 13647 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 7
W20260709 12:22:28.288551 17267 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:22:28.361071 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:28.348590 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.262640 17263 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:28.234246 16979 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.417326 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:28.159998 17262 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.298367 17013 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:28.418159 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 89247.3us
I20260709 12:22:28.424727 17264 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:28.430096 17274 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:22:28.360743 17231 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:28.452633 13644 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:28.479085 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 366112us
I20260709 12:22:28.416936 17033 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:28.435667 16830 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:28.638615 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 157252us
I20260709 12:22:28.692588 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:28.487754 17202 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:28.739077 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.737290 17025 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:28.772974 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:28.782770 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.784281 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:28.785285 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:28.425508 17263 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.662467 17068 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:28.904700 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:28.904975 17068 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.906172 17144 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:28.906710 17244 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:28.907079 17144 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:28.907730 17068 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:28.910594 13794 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.015s	sys 0.003s
W20260709 12:22:28.911047 13794 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.015s	sys 0.004s
I20260709 12:22:28.743726 17161 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:28.744590 17238 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:28.662550 17033 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.909205 17144 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:28.956712 17033 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:28.963028 17248 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:28.964887 17150 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:28.913822 17231 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.924566 17068 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:28.967533 17068 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:28.969730 17237 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:28.971632 16979 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:28.977274 17178 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.091s	user 0.011s	sys 0.000s
I20260709 12:22:28.966861 17231 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:28.953765 17236 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.376s	user 0.007s	sys 0.007s
W20260709 12:22:28.913342 17270 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.285s	user 0.009s	sys 0.000s
I20260709 12:22:28.792301 16830 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.189s
I20260709 12:22:28.970119 17068 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.018882 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 345798us
I20260709 12:22:29.004499 17178 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:29.027220 17255 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:29.029412 17263 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.034801 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:29.035585 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.070410 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2003) took 1133 ms. Trace:
I20260709 12:22:29.070977 13721 rpcz_store.cc:276] 0709 12:22:27.937319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.950701 (+ 13382us) service_pool.cc:224] Handling call
0709 12:22:27.951015 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:27.954497 (+  3482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:27.954530 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:27.955054 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:22:28.289797 (+334743us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:28.289850 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:28.289944 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:29.070115 (+780171us) raft_consensus.cc:1685] finished
0709 12:22:29.070138 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.070311 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3089,"raft.queue_time_us":3215,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":16333,"thread_start_us":196555,"threads_started":1,"wal-append.queue_time_us":467111}
W20260709 12:22:29.090925 17123 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.081s	user 0.016s	sys 0.000s
I20260709 12:22:29.100721 17263 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:29.115029 17124 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.003s
I20260709 12:22:29.124964 17124 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.121320 17263 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:29.124580 17194 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.015s	user 0.014s	sys 0.000s
I20260709 12:22:29.135756 17263 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.171417 13618 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.025s	sys 0.033s
W20260709 12:22:29.172421 13618 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.026s	sys 0.033s
W20260709 12:22:29.173079 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 599122us
I20260709 12:22:29.174331 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.176616 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2540) took 1081 ms. Trace:
I20260709 12:22:29.177124 13794 rpcz_store.cc:276] 0709 12:22:28.095659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.172964 (+1077305us) service_pool.cc:224] Handling call
0709 12:22:29.176444 (+  3480us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.179878 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.181074 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1823) took 1082 ms. Trace:
I20260709 12:22:29.181548 13794 rpcz_store.cc:276] 0709 12:22:28.098656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.179307 (+1080651us) service_pool.cc:224] Handling call
0709 12:22:29.180925 (+  1618us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.183060 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.184095 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1821) took 1086 ms. Trace:
I20260709 12:22:29.184521 13794 rpcz_store.cc:276] 0709 12:22:28.098135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.182556 (+1084421us) service_pool.cc:224] Handling call
0709 12:22:29.184035 (+  1479us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:29.185222 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.185751 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.186687 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2542) took 1082 ms. Trace:
I20260709 12:22:29.187060 13794 rpcz_store.cc:276] 0709 12:22:28.111437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.185282 (+1073845us) service_pool.cc:224] Handling call
0709 12:22:29.186630 (+  1348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.188191 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:29.188830 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.197495 13796 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:29.207865 16773 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.209232 16773 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.207722 16979 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.210822 16979 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.216711 16773 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.217995 16773 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.218946 16773 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.214169 16979 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.222909 17237 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:29.223869 17131 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.032s	user 0.021s	sys 0.020s
I20260709 12:22:29.224205 17237 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:29.228901 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 2184) took 2218 ms (client timeout 2540 ms). Trace:
I20260709 12:22:29.229144 17237 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.229413 13650 rpcz_store.cc:269] 0709 12:22:27.009984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.010610 (+   626us) service_pool.cc:224] Handling call
0709 12:22:29.228359 (+2217749us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54002b48d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:29.228634 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978511232}
I20260709 12:22:29.229974 17231 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:29.189471 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2539) took 1094 ms. Trace:
I20260709 12:22:29.231351 17231 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.236591 16982 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:29.241020 13794 rpcz_store.cc:276] 0709 12:22:28.095294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.187745 (+1092451us) service_pool.cc:224] Handling call
0709 12:22:29.189412 (+  1667us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.247938 17231 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.250154 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2535) took 1156 ms. Trace:
I20260709 12:22:29.250728 13798 rpcz_store.cc:276] 0709 12:22:28.093653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.093769 (+   116us) service_pool.cc:224] Handling call
0709 12:22:28.094081 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.305825 (+211744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.249376 (+943551us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54001004d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.249848 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485003136}
I20260709 12:22:29.251706 17178 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.252581 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.252674 17178 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.253701 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2543) took 1142 ms. Trace:
I20260709 12:22:29.254135 13798 rpcz_store.cc:276] 0709 12:22:28.111746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.252124 (+1140378us) service_pool.cc:224] Handling call
0709 12:22:29.253632 (+  1508us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.254860 16982 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.255504 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.258873 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.260051 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1824) took 1161 ms. Trace:
I20260709 12:22:29.260398 13794 rpcz_store.cc:276] 0709 12:22:28.098913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.258362 (+1159449us) service_pool.cc:224] Handling call
0709 12:22:29.259968 (+  1606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.261541 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.262437 17236 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.263412 17236 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.266471 16982 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:29.257839 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.269170 17263 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.275767 16773 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.284564 17068 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.270164 17263 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.285825 17068 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.292639 17068 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.292840 17124 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.281661 17178 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.295745 17263 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.307514 14421 raft_consensus.cc:1217] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Deduplicated request from leader. Original: 4.6->[5.7-5.7]   Dedup: 5.7->[]
I20260709 12:22:29.308421 14421 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:22:29.310248 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.315050 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2545) took 1080 ms. Trace:
I20260709 12:22:29.315536 13794 rpcz_store.cc:276] 0709 12:22:28.234434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.309828 (+1075394us) service_pool.cc:224] Handling call
0709 12:22:29.314930 (+  5102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.316618 14422 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:29.316848 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.321152 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.322741 13794 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:29.323865 17198 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:29.324612 17198 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:29.325078 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.326665 13794 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:29.329454 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1822) took 1231 ms. Trace:
I20260709 12:22:29.329963 13794 rpcz_store.cc:276] 0709 12:22:28.098412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.324123 (+1225711us) service_pool.cc:224] Handling call
0709 12:22:29.329352 (+  5229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.349015 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:29.343909 17198 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.355515 17198 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:29.361397 13794 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:22:29.362712 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2537) took 1268 ms. Trace:
I20260709 12:22:29.365559 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:29.366613 14421 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:29.367333 14425 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:29.367827 14425 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:29.368130 17253 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.736s	user 0.013s	sys 0.005s
I20260709 12:22:29.370872 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 421) took 1819 ms. Trace:
I20260709 12:22:29.371389 13795 rpcz_store.cc:276] 0709 12:22:27.551703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.551855 (+   152us) service_pool.cc:224] Handling call
0709 12:22:27.552245 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:27.578547 (+ 26302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:27.578579 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:27.579128 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 12:22:27.638616 (+ 59488us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:27.638675 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.248099 (+1609424us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b54003d4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.248209 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:29.367103 (+118894us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:29.370301 (+  3198us) raft_consensus.cc:1685] finished
0709 12:22:29.370328 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.370614 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22428,"raft.queue_time_us":24198,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":51579136,"thread_start_us":58959,"threads_started":1,"wal-append.queue_time_us":51843}
W20260709 12:22:29.374783 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70044.6us
I20260709 12:22:29.376915 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:29.366062 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:29.378163 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:29.379446 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 482) took 1743 ms. Trace:
I20260709 12:22:29.379865 13648 rpcz_store.cc:276] 0709 12:22:27.635535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.635670 (+   135us) service_pool.cc:224] Handling call
0709 12:22:27.636110 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.976688 (+340578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.378841 (+1402153us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.379184 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780961152}
I20260709 12:22:29.363191 13794 rpcz_store.cc:276] 0709 12:22:28.094325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.340712 (+1246387us) service_pool.cc:224] Handling call
0709 12:22:29.360714 (+ 20002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:29.362024 (+  1310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.362572 (+   548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.387645 17196 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:29.388175 14419 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 LEARNER]: Advancing to term 8
I20260709 12:22:29.390849 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:29.393054 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:29.369272 14425 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:29.366060 17250 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.734s	user 0.000s	sys 0.031s
I20260709 12:22:29.394816 14425 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:29.394506 17248 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:29.394106 13795 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:22:29.396245 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2538) took 1301 ms. Trace:
I20260709 12:22:29.396684 13795 rpcz_store.cc:276] 0709 12:22:28.094903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.374188 (+1279285us) service_pool.cc:224] Handling call
0709 12:22:29.393500 (+ 19312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:29.395882 (+  2382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.396167 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.397955 17241 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:29.399380 13795 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:22:29.398444 17241 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:29.403148 17241 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:29.404302 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2541) took 1299 ms. Trace:
I20260709 12:22:29.404732 13795 rpcz_store.cc:276] 0709 12:22:28.103723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.397630 (+1293907us) service_pool.cc:224] Handling call
0709 12:22:29.398107 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:29.402297 (+  4190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.402541 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.405897 17146 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:29.406769 17146 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.409431 17146 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:29.415993 13795 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:29.416918 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2544) took 1304 ms. Trace:
I20260709 12:22:29.417301 13795 rpcz_store.cc:276] 0709 12:22:28.112677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.414963 (+1302286us) service_pool.cc:224] Handling call
0709 12:22:29.415590 (+   627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:29.416581 (+   991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.416821 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:29.418848 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:29.414454 17179 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.821s	user 0.021s	sys 0.039s
W20260709 12:22:29.420296 17179 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.859s	user 0.022s	sys 0.039s
I20260709 12:22:29.429713 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:29.430228 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:29.435278 13647 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.018s	user 0.010s	sys 0.016s
I20260709 12:22:29.435846 13647 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:29.436651 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 506) took 1243 ms. Trace:
I20260709 12:22:29.437114 13647 rpcz_store.cc:276] 0709 12:22:28.193173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.354675 (+161502us) service_pool.cc:224] Handling call
0709 12:22:29.436523 (+1081848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:29.438997 13718 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:29.438095 14419 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:29.449309 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.450356 17196 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:29.450742 17272 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.451332 17196 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.453377 14425 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
W20260709 12:22:29.458586 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81851.6us
I20260709 12:22:29.457748 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:29.454542 14425 pending_rounds.cc:85] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 8
I20260709 12:22:29.468720 14425 pending_rounds.cc:107] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:22:29.452327 17124 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.482349 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1979) took 2527 ms. Trace:
I20260709 12:22:29.483175 13724 rpcz_store.cc:276] 0709 12:22:26.954458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.956045 (+  1587us) service_pool.cc:224] Handling call
0709 12:22:26.956474 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:27.429912 (+473438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.481632 (+2051720us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54003fe3d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.482000 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002457344}
I20260709 12:22:29.485831 17248 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:22:29.488745 13793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1793) took 3566 ms (client timeout 2511 ms). Trace:
I20260709 12:22:29.477659 14425 raft_consensus.cc:2889] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:29.451639 17272 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.492926 17143 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.493643 17143 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.496840 17143 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.497601 17092 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.498677 13652 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:22:29.498456 17092 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.499897 17196 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.500908 17092 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.501801 17198 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:22:29.502038 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.491173 14421 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:29.502522 17124 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.503954 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2534) took 1410 ms. Trace:
I20260709 12:22:29.504722 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:29.462793 17179 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.506392 17271 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:29.507017 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 410) took 2986 ms. Trace:
I20260709 12:22:29.507040 17271 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:29.505728 17124 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.507484 13800 rpcz_store.cc:276] 0709 12:22:26.519974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.526516 (+  6542us) service_pool.cc:224] Handling call
0709 12:22:27.632738 (+1106222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:27.644439 (+ 11701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.506590 (+1862151us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003c0350. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:29.506785 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2543188736}
I20260709 12:22:29.506821 17179 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:29.489208 13793 rpcz_store.cc:269] 0709 12:22:25.922097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.023724 (+101627us) service_pool.cc:224] Handling call
0709 12:22:29.488268 (+3464544us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:29.488512 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3147597568}
I20260709 12:22:29.508534 17271 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.527732 17245 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.528523 17245 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.504555 13797 rpcz_store.cc:276] 0709 12:22:28.093242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.093388 (+   146us) service_pool.cc:224] Handling call
0709 12:22:28.100893 (+  7505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:28.167163 (+ 66270us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:22:28.356032 (+188869us) log.cc:844] Serialized 14 byte log entry
0709 12:22:28.454359 (+ 98327us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:28.454494 (+   135us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:28.454584 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.502595 (+1048011us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b180003c1c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:22:29.503391 (+   796us) spinlock_profiling.cc:243] Waited 65.8 ms on lock 0x7b5400287dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.503444 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.503749 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":433763,"raft.queue_time_us":194,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":99642,"spinlock_wait_cycles":353248128,"thread_start_us":261684,"threads_started":2,"wal-append.queue_time_us":590093}
I20260709 12:22:29.539872 17190 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:22:29.547101 16789 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.549446 16789 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:29.550697 17194 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:29.551541 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:29.526440 17198 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.555145 17198 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.540320 17272 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.527868 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1819) took 2346 ms. Trace:
I20260709 12:22:29.558841 13799 rpcz_store.cc:276] 0709 12:22:27.180957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.181414 (+   457us) service_pool.cc:224] Handling call
0709 12:22:27.182015 (+   601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.093173 (+911158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.514527 (+1421354us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.527674 (+ 13147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093950848}
I20260709 12:22:29.579330 17179 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.521235 17143 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.580845 17143 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.586949 17143 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.634429 17183 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:29.635145 17183 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:29.636544 17183 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:29.639369 17298 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:22:29.640910 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1820) took 1543 ms. Trace:
I20260709 12:22:29.641304 13794 rpcz_store.cc:276] 0709 12:22:28.097862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:29.398629 (+1300767us) service_pool.cc:224] Handling call
0709 12:22:29.419633 (+ 21004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:29.420346 (+   713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:29.420382 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:29.420993 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:22:29.510985 (+ 89992us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:29.511038 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.511138 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:29.640650 (+129512us) raft_consensus.cc:1685] finished
0709 12:22:29.640672 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:29.640830 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13130,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":250,"thread_start_us":89476,"threads_started":1,"wal-append.queue_time_us":164337}
W20260709 12:22:29.646387 13767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.583s (SENT) [suppressed 32 similar messages]
W20260709 12:22:29.655632 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:29.665035 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.624300 17190 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.676582 17190 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.678681 17241 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.679724 17190 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.681821 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80413.1us
I20260709 12:22:29.691318 17245 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.676613 17183 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:29.695109 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.696007 17137 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:29.696689 17137 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:29.697163 17239 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 2.133s	user 0.000s	sys 0.019s
I20260709 12:22:29.698192 17137 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.699981 17137 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:29.709472 17175 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.914s	user 0.020s	sys 0.048s
W20260709 12:22:29.711863 17175 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.918s	user 0.020s	sys 0.048s
I20260709 12:22:29.709713 16957 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.716030 16957 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:29.717162 17281 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.704s	user 0.000s	sys 0.002s
I20260709 12:22:29.718936 16957 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.727818 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1804) took 3424 ms (client timeout 3009 ms). Trace:
W20260709 12:22:29.730156 13792 rpcz_store.cc:269] 0709 12:22:26.303029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:26.304282 (+  1253us) service_pool.cc:224] Handling call
0709 12:22:29.727171 (+3422889us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400129b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:29.727541 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470998272}
I20260709 12:22:29.729275 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:29.732295 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 486) took 2061 ms (client timeout 1827 ms). Trace:
W20260709 12:22:29.732810 13649 rpcz_store.cc:269] 0709 12:22:27.671202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.671378 (+   176us) service_pool.cc:224] Handling call
0709 12:22:29.728841 (+2057463us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:29.732175 (+  3334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9843072}
I20260709 12:22:29.735282 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:29.736718 13647 raft_consensus.cc:2410] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:22:29.742250 17255 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.751120 13652 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:22:29.752111 13652 pending_rounds.cc:85] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 3
I20260709 12:22:29.755352 13652 pending_rounds.cc:107] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:22:29.755616 13652 raft_consensus.cc:2889] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:29.758020 17265 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:22:29.758601 17265 raft_consensus.cc:1068] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:22:29.759251 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.761436 14425 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:22:29.763619 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.777163 16476 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 12:22:29.778169 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.803493 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:29.804198 14420 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:29.792285 17241 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.765396 17255 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.829739 17236 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.829974 17190 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:29.830893 17190 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.833838 17190 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.834511 17198 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.834810 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2530) took 1746 ms. Trace:
I20260709 12:22:29.835276 13801 rpcz_store.cc:276] 0709 12:22:28.088025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.088224 (+   199us) service_pool.cc:224] Handling call
0709 12:22:28.088566 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:28.101668 (+ 13102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:29.827445 (+1725777us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:29.834606 (+  7161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28568448}
I20260709 12:22:29.835387 17198 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.838200 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:29.842489 17121 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.842658 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:29.843221 17121 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.845594 17121 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.846609 17196 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:29.847081 17196 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:29.847468 17196 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:29.865725 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.824682 16476 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:29.890600 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.890340 16476 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.893097 16476 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.894798 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:29.902923 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 1.39583e+06us
I20260709 12:22:29.881763 17198 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.861481 17121 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:29.925808 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87755.5us
I20260709 12:22:29.931732 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.939800 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.946898 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.947651 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.949700 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:29.951800 17121 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.956668 13797 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.958772 13794 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:29.960376 17255 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.960747 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.964097 17192 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:29.966401 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:29.966401 17229 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:29.967427 17229 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.969148 17286 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.969708 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:29.969781 17286 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.970467 13649 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.970682 17229 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.972348 17286 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.973621 17143 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.974005 17164 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.974434 17143 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.974825 17164 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.975608 13652 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:29.976977 17143 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.977655 17143 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:29.977523 17164 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.978462 17245 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:29.979261 17245 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.982236 17245 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:29.982715 17198 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:29.983530 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:29.984292 13647 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:29.984614 17198 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:29.985054 17111 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:29.986059 17190 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:29.986527 17190 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:29.987023 17236 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:29.987727 17190 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.988519 17278 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:29.988945 17278 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:30.001999 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.002380 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.990814 17286 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:30.003746 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.003901 17286 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:29.989995 17278 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:29.954205 17121 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:29.976929 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:29.990988 13650 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:29.996932 13644 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:22:30.011951 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.012409 17286 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:29.995968 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 489) took 2302 ms (client timeout 1799 ms). Trace:
I20260709 12:22:29.989923 17013 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:29.992074 17256 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 2.202s	user 0.015s	sys 0.022s
I20260709 12:22:29.999295 17164 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:29.993573 16989 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:30.014508 17164 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.013602 17013 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.015404 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 615) took 2385 ms. Trace:
I20260709 12:22:29.977847 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.016004 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.016443 13648 raft_consensus.cc:2376] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7. Re-sending same reply.
I20260709 12:22:30.016693 13652 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:30.000582 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.017477 17013 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:30.017887 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 485) took 2358 ms (client timeout 1580 ms). Trace:
I20260709 12:22:29.990720 17222 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:22:29.989836 17124 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:30.019340 17124 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:30.015821 13726 rpcz_store.cc:276] 0709 12:22:27.630234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.630442 (+   208us) service_pool.cc:224] Handling call
0709 12:22:27.630896 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:27.640586 (+  9690us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:27.640614 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:27.641134 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:22:27.702975 (+ 61841us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:27.703022 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:27.703096 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:29.991260 (+2288164us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:30.014720 (+ 23460us) raft_consensus.cc:1685] finished
0709 12:22:30.014743 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:30.015211 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9642,"raft.queue_time_us":10736,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":29978,"spinlock_wait_cycles":87307520,"thread_start_us":61478,"threads_started":1,"wal-append.queue_time_us":148042}
W20260709 12:22:30.013301 13643 rpcz_store.cc:269] 0709 12:22:27.693680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.693871 (+   191us) service_pool.cc:224] Handling call
0709 12:22:29.995541 (+2301670us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400147dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:29.995760 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701113728}
I20260709 12:22:30.022249 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.022552 17222 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:30.023149 17222 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:30.023679 17143 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.025405 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 491) took 2327 ms. Trace:
I20260709 12:22:30.025951 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.013557 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:29.992115 17245 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.025812 13651 rpcz_store.cc:276] 0709 12:22:27.698160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.714372 (+ 16212us) service_pool.cc:224] Handling call
0709 12:22:30.025004 (+2310632us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b54001e5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:30.025213 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915126016}
I20260709 12:22:30.032819 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.033114 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.033569 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.027525 17246 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:30.034585 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.036425 13793 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:30.037626 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:30.037886 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.035214 17246 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.038954 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.017486 17164 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:30.032203 17245 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.040077 17164 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.040298 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.040859 17246 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.040740 17164 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.041164 13720 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:30.018451 13646 rpcz_store.cc:269] 0709 12:22:27.663266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.663425 (+   159us) service_pool.cc:224] Handling call
0709 12:22:30.000220 (+2336795us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:30.017797 (+ 17577us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26697088}
I20260709 12:22:30.041541 17246 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:30.027794 17276 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.241s	user 0.009s	sys 0.010s
W20260709 12:22:29.993685 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.030570 13724 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:30.042552 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.034410 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.042871 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.034626 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.030812 17124 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.053452 17124 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:30.053638 14417 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 4. Current term is 5. Ops: []
W20260709 12:22:30.052825 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.027524 13721 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:22:30.056768 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.057616 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.058177 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.058857 14424 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:30.059306 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.060034 14422 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:30.045615 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2207) took 2153 ms. Trace:
I20260709 12:22:30.061035 13645 rpcz_store.cc:276] 0709 12:22:27.892234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.985299 (+ 93065us) service_pool.cc:224] Handling call
0709 12:22:28.019725 (+ 34426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:28.020160 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:28.020247 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:28.020990 (+   743us) log.cc:844] Serialized 26 byte log entry
0709 12:22:28.305154 (+284164us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:28.305213 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:28.305329 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:30.026931 (+1721602us) spinlock_profiling.cc:243] Waited 91.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:30.045152 (+ 18221us) raft_consensus.cc:1685] finished
0709 12:22:30.045172 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:30.045422 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":7864,"spinlock_wait_cycles":209610624,"thread_start_us":218581,"threads_started":1,"wal-append.queue_time_us":369514}
I20260709 12:22:30.063103 13727 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:30.042204 17245 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:30.063756 13645 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:22:30.046203 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.065344 13726 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:30.064210 13792 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:30.057756 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 509) took 1641 ms. Trace:
I20260709 12:22:30.048624 13722 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:30.049144 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.034439 13651 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:22:30.046355 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.049144 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.048060 14420 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:30.068532 13794 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:30.066998 13644 rpcz_store.cc:276] 0709 12:22:28.416803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.786025 (+369222us) service_pool.cc:224] Handling call
0709 12:22:29.996189 (+1210164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.011239 (+ 15050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.057540 (+ 46301us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400147dd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:30.057684 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2782347648}
I20260709 12:22:30.167615 13801 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:30.129532 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.112138 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.194293 14417 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:30.048611 17278 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.204180 13651 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:30.111140 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:30.192688 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.150914 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:30.202662 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.186863 17306 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:22:30.210685 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:30.192221 13646 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:30.026238 17143 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:30.226294 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.084249 17229 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:30.234701 17310 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:30.236267 17318 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:30.240557 17222 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.213755 17121 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.043957 17246 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.266525 13649 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:30.282788 13797 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:30.142695 17196 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.277920 13801 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:22:30.286873 13651 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:30.289204 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:22:30.295408 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.296540 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.296717 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108857us
I20260709 12:22:30.308514 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:30.309269 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.319409 13644 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:30.328802 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:30.298810 17229 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:30.362555 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.976s (SENT)
I20260709 12:22:30.367170 13646 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:30.386986 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.387817 13725 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:30.391391 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 1993) took 2460 ms. Trace:
I20260709 12:22:30.391790 13718 rpcz_store.cc:276] 0709 12:22:27.930834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:27.932157 (+  1323us) service_pool.cc:224] Handling call
0709 12:22:27.932381 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:29.439545 (+1507164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.391088 (+951543us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b540021fcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:30.391330 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3463666432}
I20260709 12:22:30.394774 16965 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
W20260709 12:22:30.399569 13617 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.540s (SENT)
I20260709 12:22:30.400245 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:30.400918 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47116.4us
W20260709 12:22:30.401969 14391 connection.cc:256] RPC call timeout handler was delayed by -1.3487864000003356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
W20260709 12:22:30.402597 14391 connection.cc:256] RPC call timeout handler was delayed by -1.0062378930001614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
I20260709 12:22:30.405122 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.406114 14416 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:22:30.406989 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2532) took 2315 ms (client timeout 1618 ms). Trace:
W20260709 12:22:30.407498 13796 rpcz_store.cc:269] 0709 12:22:28.091554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:28.091661 (+   107us) service_pool.cc:224] Handling call
0709 12:22:30.406714 (+2315053us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54003d54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:30.406901 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2297724800}
I20260709 12:22:30.314078 17284 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
W20260709 12:22:30.412590 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 64039.3us
I20260709 12:22:30.381713 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.423588 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.366377 17222 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:30.425331 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.426070 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.427544 17190 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:30.321019 17121 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.435364 17196 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.436127 17196 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.436594 17196 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:22:30.437549 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.437948 17143 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.423362 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:30.438570 17143 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.305454 17310 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.416651 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.441700 17322 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:30.444336 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.445043 13650 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:30.383764 17164 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.446205 17143 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:30.449873 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.438930 14419 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:30.451094 17246 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.353262 17318 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:30.453210 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.454595 17318 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.455277 17318 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:30.456986 17318 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:30.457327 17318 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:30.446038 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.459164 13801 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:30.446110 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.465569 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.465819 14417 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:30.466375 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:30.467031 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.468954 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.470136 13723 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:30.465873 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:30.473815 17311 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:22:30.476302 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.477336 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.477794 13646 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:30.477077 13550 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:30.479346 13550 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 6, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:30.481988 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.479369 13646 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:30.487002 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:30.487998 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.488416 16989 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:30.488698 13725 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:30.489169 16989 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:30.490429 17192 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:30.490541 16989 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.493170 17236 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:30.492877 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:30.498086 17111 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:30.498839 17111 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.498246 16476 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:30.499899 16476 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:30.500279 16476 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:30.500555 17121 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:30.501405 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.501127 16476 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.499249 17111 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:30.503867 17229 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.505045 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:30.476323 17258 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.506958 17258 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.509402 17258 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:30.510454 13644 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:30.511703 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:30.512081 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.514431 17025 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:30.515463 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:30.516109 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 116669us
I20260709 12:22:30.516434 13801 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:22:30.517401 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.518581 17025 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:30.523177 13801 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:30.524231 13796 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: [4.4-6.5]
I20260709 12:22:30.418223 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:30.529063 17196 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.529696 17196 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:30.532176 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:30.532927 13726 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:30.533221 17284 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:30.533636 17284 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.532006 16989 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:30.533982 17284 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:30.535087 16989 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:30.535785 16989 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:30.538957 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.539821 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:30.481433 13550 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 6, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 6 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.517290 16476 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:30.568601 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.569315 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.570650 14420 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:22:30.555872 17124 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.006s
I20260709 12:22:30.536995 16989 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.576462 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.574707 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.582484 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:30.591259 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 265346us
I20260709 12:22:30.594076 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.591985 13550 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 7, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:30.600471 13643 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:22:30.600801 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 272156us
I20260709 12:22:30.601680 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:30.602734 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.619266 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.613117 13550 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 7, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:30.637552 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.637552 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.638365 13726 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:30.641239 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:30.648051 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:30.650202 17137 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:30.650528 16957 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.615173 17310 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.643311 17325 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:30.644356 16989 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:30.576507 17318 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.663142 17318 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:30.668336 13648 raft_consensus.cc:2410] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:22:30.664172 13550 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 7, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 7 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:30.676352 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:30.676726 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.677675 14419 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:22:30.681124 17190 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:30.681746 17190 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:30.683806 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.692821 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:30.696208 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.698222 13796 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:30.698940 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.704399 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.701753 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:30.709018 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.709012 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:30.715068 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.715499 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:30.711158 13550 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:30.718012 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.722114 13550 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:30.718972 17111 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.732740 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.733887 17111 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:30.735183 13794 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:30.736578 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 219822us
I20260709 12:22:30.741437 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:30.743177 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.744043 13797 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:30.744129 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.744789 13648 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:30.745217 17284 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:30.748775 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.713656 17246 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:30.723582 17198 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:30.750140 13550 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 8, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:30.711364 17326 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:30.723308 17323 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:30.761114 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:30.765277 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:30.771603 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.788506 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.757040 17198 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:30.789378 13795 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:30.751421 17222 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:22:30.794896 17222 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:30.760852 17323 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:30.795884 13719 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:30.802559 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.806686 13550 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:30.815284 13723 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:22:30.783608 17322 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:30.813513 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7901234280002427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3009s for a response
W20260709 12:22:30.820135 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0643079940000462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
I20260709 12:22:30.826988 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.749243 17190 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:30.837839 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.727s (SENT)
I20260709 12:22:30.838572 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:30.832053 13652 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:30.792560 17198 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:30.795430 17222 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:30.783916 17246 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.847474 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.812939 17327 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:22:30.849937 17327 consensus_queue.cc:1243] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:22:30.851917 17328 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:30.856247 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.811515 17323 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:30.763082 17326 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:30.868245 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.846360 17278 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.001s
I20260709 12:22:30.860742 13550 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 8 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:30.882819 13800 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:30.885457 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.886472 13795 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
W20260709 12:22:30.890681 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59214.2us
I20260709 12:22:30.890054 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.834471 17322 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.893698 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:30.895056 13795 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:30.898713 17025 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:22:30.899384 17025 raft_consensus.cc:1068] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:22:30.891942 13723 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:30.902213 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:30.913560 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.913329 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:30.922092 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:30.930647 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.936829 13550 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:30.944610 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.764s (SENT)
I20260709 12:22:30.949043 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.948483 13550 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 4 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.954370 13723 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:30.956620 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:30.957845 13727 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:30.960021 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:30.960764 13652 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:30.961724 13550 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:30.941212 16957 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:30.971956 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:30.941217 17222 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:30.980106 17143 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:30.981187 17143 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:30.981547 17143 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:30.981870 17143 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:30.983548 13799 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:30.984266 17164 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:30.984710 17164 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:30.985828 17164 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:30.987136 17164 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:30.988474 17322 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:30.998138 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:31.001888 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.003641 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:30.944330 17137 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:31.020838 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.020419 13550 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 6, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 6 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.017323 13648 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:22:31.026378 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.027108 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.027581 13725 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:31.028028 13723 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:31.027582 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.023849 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:31.036689 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96995.4us
I20260709 12:22:31.007054 17284 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:31.023528 17236 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.054291 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.055123 17236 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:31.056030 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.057252 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:31.057533 17236 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:31.057837 13650 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:31.061338 17246 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:31.062587 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.063306 13643 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:31.065454 13643 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:31.069052 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.069518 17222 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:31.073063 17326 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:31.066828 13550 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 11 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.075258 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:31.075953 13800 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:31.075676 13550 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 4 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.077227 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:31.077805 13550 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 7, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 7 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.091739 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.094553 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:31.020143 17137 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:31.102418 13727 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:31.090162 13550 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 9, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 9 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:31.099555 13652 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:31.103655 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:31.128098 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.129247 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:31.101168 17326 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:31.132575 13652 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:22:31.125494 13550 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:31.105851 13650 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:22:31.140180 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.116405 17328 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:31.144833 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:31.142748 13550 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:31.165773 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.185542 13723 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:31.187165 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.188951 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.123590 17236 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:31.190380 13648 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:31.237952 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.239135 13648 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:31.134636 17326 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.199669 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:31.244841 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:31.121716 17323 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.247606 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.235881 13550 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:31.250465 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.259209 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:31.260253 13550 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 5, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 5 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:31.261099 17143 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:31.262507 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.907s (SENT)
I20260709 12:22:31.263200 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:31.265830 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.249825 17198 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.266850 17164 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:31.267513 17164 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.270468 16961 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:31.272351 17328 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:31.241995 17318 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.007s
I20260709 12:22:31.246652 17326 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:31.282316 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.256963 17236 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:31.282933 13651 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:22:31.283249 13644 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:31.283893 17236 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.286959 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2232) took 1256 ms. Trace:
I20260709 12:22:31.287691 17323 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:31.289045 17148 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.289609 17148 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.287416 13649 rpcz_store.cc:276] 0709 12:22:30.030950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.300204 (+269254us) service_pool.cc:224] Handling call
0709 12:22:30.307495 (+  7291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:30.543351 (+235856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.543413 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:30.572149 (+ 28736us) log.cc:844] Serialized 26 byte log entry
0709 12:22:30.925895 (+353746us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:22:30.925944 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:30.926034 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.286680 (+360646us) raft_consensus.cc:1685] finished
0709 12:22:31.286705 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.286884 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144076,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":548540,"threads_started":2,"wal-append.queue_time_us":593253}
I20260709 12:22:31.253433 17137 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:31.296680 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.262054 13550 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 7, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:31.299955 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2239) took 1134 ms. Trace:
I20260709 12:22:31.313575 13644 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:31.307940 13651 rpcz_store.cc:276] 0709 12:22:30.189314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.469972 (+280658us) service_pool.cc:224] Handling call
0709 12:22:30.470310 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:31.283490 (+813180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.299713 (+ 16223us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b5400114254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:31.299893 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867755392}
I20260709 12:22:31.315719 17198 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:31.320607 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75876.4us
I20260709 12:22:31.322607 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:31.335624 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:31.337644 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:31.320194 16961 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:31.329747 16989 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:31.401158 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.090s (SENT)
I20260709 12:22:31.406605 17325 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:22:31.396057 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55373us
I20260709 12:22:31.415958 17118 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:31.410493 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:31.410724 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.418987 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.428195 13723 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:31.430559 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.030s	sys 0.024s
W20260709 12:22:31.430970 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.030s	sys 0.024s
W20260709 12:22:31.431483 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 781131us
W20260709 12:22:31.417675 17329 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
W20260709 12:22:31.431707 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 459479us
I20260709 12:22:31.410183 16989 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.409739 17148 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:31.435036 17137 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:31.452142 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 361653us
I20260709 12:22:31.431784 17318 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:31.466827 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 125437us
I20260709 12:22:31.467096 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:31.467765 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.102s (SENT)
I20260709 12:22:31.468389 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:31.470045 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.186s (SENT)
W20260709 12:22:31.465268 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.472497 13799 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:22:31.453059 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 358413us
W20260709 12:22:31.481065 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.458057 17328 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.460719 16989 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:31.512212 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.520428 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.528756 13723 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:31.534247 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.539198 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69223.3us
I20260709 12:22:31.539471 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:31.514532 17190 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:31.584209 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50826.5us
I20260709 12:22:31.585812 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:31.531903 17124 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
W20260709 12:22:31.608155 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.216s (SENT)
I20260709 12:22:31.608806 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:31.609694 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 195263us
I20260709 12:22:31.519083 17328 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:31.602563 17246 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:31.597532 17143 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:31.630959 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2032) took 1092 ms. Trace:
I20260709 12:22:31.635347 14425 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:31.631426 13720 rpcz_store.cc:276] 0709 12:22:30.534518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.557439 (+ 22921us) service_pool.cc:224] Handling call
0709 12:22:30.557803 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:30.593718 (+ 35915us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:30.943344 (+349626us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.623844 (+680500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:31.626599 (+  2755us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:31.626687 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.626797 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.626987 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":527084,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":89219,"raft.run_cpu_time_us":1814,"raft.run_wall_time_us":20325,"thread_start_us":935721,"threads_started":3,"wal-append.queue_time_us":648190}
W20260709 12:22:31.637966 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 357380us
I20260709 12:22:31.620193 17346 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:31.639611 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:31.607470 17353 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:31.538609 16961 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:31.660972 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 132770us
I20260709 12:22:31.636534 17164 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:31.666898 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.675662 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 134652us
W20260709 12:22:31.677714 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:31.678009 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.643635 17310 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:31.683478 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 216497us
I20260709 12:22:31.648988 17346 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:31.649248 17353 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:31.704078 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99171.7us
I20260709 12:22:31.698443 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:31.644940 17148 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:22:31.694056 17322 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.678970 17190 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:31.685922 17340 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
W20260709 12:22:31.732194 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.220s (SENT)
W20260709 12:22:31.730953 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 108636us
I20260709 12:22:31.695163 17357 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:22:31.746709 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.746675 13643 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:31.701903 17346 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:31.707798 17353 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:31.747087 17356 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:22:31.775928 16989 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:31.723842 17328 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:31.738562 17357 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:31.770931 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.238s (SENT)
I20260709 12:22:31.770679 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:31.754050 17363 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:31.771927 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74222.4us
I20260709 12:22:31.746361 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2246) took 1158 ms. Trace:
I20260709 12:22:31.794590 13645 rpcz_store.cc:276] 0709 12:22:30.587969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.589117 (+  1148us) service_pool.cc:224] Handling call
0709 12:22:30.589628 (+   511us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:30.606992 (+ 17364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:30.607027 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:30.644643 (+ 37616us) log.cc:844] Serialized 52 byte log entry
0709 12:22:31.104415 (+459772us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:31.187392 (+ 82977us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.403809 (+216417us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b18003c3040. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:22:31.404090 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 12:22:31.404399 (+   309us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.404495 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:31.745958 (+341463us) raft_consensus.cc:1685] finished
0709 12:22:31.745987 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:31.746225 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157457,"prepare.queue_time_us":355587,"prepare.run_cpu_time_us":1384,"prepare.run_wall_time_us":6265,"raft.queue_time_us":493,"raft.run_cpu_time_us":565,"raft.run_wall_time_us":87548,"spinlock_wait_cycles":12675328,"thread_start_us":400112,"threads_started":1,"wal-append.queue_time_us":633146}
W20260709 12:22:31.758848 17222 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.689s	user 0.015s	sys 0.002s
W20260709 12:22:31.791520 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52937.6us
I20260709 12:22:31.797492 13649 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:22:31.806399 17148 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:31.805958 17350 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:31.816469 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:31.817800 17148 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:31.820256 17148 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:22:31.831081 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.836362 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:31.835456 17326 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.003s
I20260709 12:22:31.732949 17190 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:31.724026 17322 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.794072 17357 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:31.843572 17335 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:31.857995 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:31.865258 14425 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:31.841955 17341 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:31.863281 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:31.887162 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 116275us
W20260709 12:22:31.890561 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 120021us
W20260709 12:22:31.893362 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95915.3us
W20260709 12:22:31.898667 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 162147us
I20260709 12:22:31.862811 17335 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:31.856292 17322 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:31.907238 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63964.5us
I20260709 12:22:31.888608 17363 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:31.913589 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:31.914623 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75260us
I20260709 12:22:31.902846 17335 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:31.919757 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:31.904547 17371 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.898073 16989 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:31.850956 17326 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:31.924703 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:31.925848 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:31.926895 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.929015 13718 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:31.929333 13723 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:31.941062 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:31.941829 13721 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:31.943310 17348 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:31.943425 13725 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:22:31.954025 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45643.1us
I20260709 12:22:31.956956 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:31.957680 13646 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:31.962767 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:31.918704 17025 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:22:31.975687 17380 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:31.976586 17380 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:31.977023 17380 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:31.978622 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1764) took 1155 ms. Trace:
I20260709 12:22:31.979077 13643 rpcz_store.cc:276] 0709 12:22:30.823441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:31.082525 (+259084us) service_pool.cc:224] Handling call
0709 12:22:31.082866 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:31.746219 (+663353us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b54004a0220. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:22:31.753048 (+  6829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:31.978310 (+225262us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400115154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:31.978551 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861239424}
I20260709 12:22:31.979825 17326 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:31.981158 17190 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:31.953831 17325 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.981978 17325 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.983711 17357 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.984334 17325 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:31.985432 17349 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.337s	user 0.001s	sys 0.001s
W20260709 12:22:31.986683 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:31.986907 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:31.987119 17148 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:31.987802 13720 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:31.988366 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:31.991456 17348 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:31.992393 16965 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:31.992908 16965 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:31.995193 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:32.006708 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 107331us
W20260709 12:22:32.006733 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46735.9us
I20260709 12:22:32.010823 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:31.982002 17347 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.341s	user 0.003s	sys 0.001s
W20260709 12:22:32.015234 17137 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.003s
W20260709 12:22:32.015980 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90817.5us
I20260709 12:22:32.016355 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.020763 16957 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:22:32.024843 16989 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:22:32.025424 16957 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:32.025837 16957 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:32.026335 17335 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.026923 16957 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.006407 17363 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.031291 13652 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:22:32.033267 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.034870 14418 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:32.036075 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.018887 17025 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:32.021019 17236 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:32.038631 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:31.946038 17371 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.039103 17137 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:32.054626 17365 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:22:32.068814 14418 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:32.070648 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:32.073426 17335 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.042492 16957 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:32.075676 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59540us
I20260709 12:22:32.078552 17359 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:32.039443 17357 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.047140 17236 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.082412 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.028151 17346 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.091575 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.097754 13646 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:32.102599 13718 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:32.112466 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 186665us
I20260709 12:22:32.063562 17383 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:32.081146 17143 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.089205 17164 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:32.091369 17025 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:32.100813 17236 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:32.120204 17383 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.128537 17363 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.152971 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.151168 17359 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:22:32.167240 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103597us
I20260709 12:22:32.172379 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.173188 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:32.140866 17284 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:22:32.149966 17383 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.151610 16965 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:32.143124 17164 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.176566 17361 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:32.175314 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:32.170877 17372 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.158768 17371 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.156908 17363 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:32.189203 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106966us
I20260709 12:22:32.179270 17361 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.179186 17393 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.186084 17372 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:32.216362 17393 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:32.226104 17393 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.226300 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.226897 13643 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:32.227164 14418 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:32.227792 17362 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:32.228252 17362 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:32.228643 17362 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.231173 17346 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.241520 17353 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.244151 17353 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.248222 17372 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.249831 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:32.174249 17143 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:22:32.251303 17143 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:32.251771 17143 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:32.252957 17143 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.202569 17361 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:32.256362 17390 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:22:32.185508 17025 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.215176 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:32.258682 13646 raft_consensus.cc:2376] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7. Re-sending same reply.
I20260709 12:22:32.222730 17341 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:22:32.259738 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.260931 13720 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:32.271596 17143 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.276237 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:32.277161 17284 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:32.277791 17284 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.280256 17394 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:32.289618 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:32.298843 13799 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.291375 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:32.291991 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.292585 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.300580 13720 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.192174 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.297385 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:32.321359 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 146098us
I20260709 12:22:32.326179 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:32.325129 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 133771us
I20260709 12:22:32.332238 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:32.333277 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.338050 13794 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:32.338068 13652 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:32.321866 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.339742 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.324316 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.345307 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.346673 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.350010 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:32.350693 13799 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:32.343541 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:32.356959 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:32.359652 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:32.361030 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:32.364024 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.365504 13793 pending_rounds.cc:85] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 6
I20260709 12:22:32.311647 17394 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.365835 13793 pending_rounds.cc:107] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:32.366093 13793 raft_consensus.cc:2889] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:32.319339 17325 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:32.370611 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.373276 13795 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:32.364732 13798 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:32.364174 13792 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:32.374856 14425 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:32.369644 13718 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:32.363759 17363 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.379128 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:32.380381 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.326054 17377 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.388306 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.389226 17265 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.389698 17265 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.390223 17368 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.391177 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.392800 17380 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.334421 17284 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:32.376330 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:32.396775 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.388440 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.397902 13720 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:32.404305 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.390046 17265 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.404583 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:32.408914 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.409379 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:32.411644 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.411585 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.413546 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.372634 17325 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.416797 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:32.416812 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.421540 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45307.3us
I20260709 12:22:32.422399 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:32.435465 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.374419 17394 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:32.440687 17396 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:32.441291 17396 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:32.441784 17396 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:32.442349 13727 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:32.397256 17348 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.420549 17371 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.444612 17371 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.453116 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:32.446506 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:32.463311 13799 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:32.417321 17322 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:32.417821 17124 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:32.431223 17325 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:32.445791 17341 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:32.475358 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.474732 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:32.482892 13792 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.445575 16957 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:32.459059 17371 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.469012 17322 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:32.462464 17190 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:32.500087 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 177304us
I20260709 12:22:32.511791 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:32.451089 17350 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:22:32.408244 16989 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.498754 17341 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:32.502151 17322 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:32.500978 16957 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:32.520373 17368 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.516387 17284 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.538662 17284 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.541078 17284 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:32.514317 17190 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.549817 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.550243 17353 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:32.551328 17353 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.553591 17353 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:32.573939 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.581226 17380 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.582703 17202 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:22:32.584082 17372 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.586437 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255f10165e844f2cb50d654996d639cf"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:32.587105 13799 raft_consensus.cc:606] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:32.595592 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61950us
I20260709 12:22:32.597524 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:32.604841 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.605856 13720 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.610949 13643 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:32.617384 13794 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:32.621683 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.625974 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:32.627658 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:32.629040 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.629689 13726 raft_consensus.cc:2376] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10. Re-sending same reply.
I20260709 12:22:32.630630 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:32.631991 13723 pending_rounds.cc:85] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 9
I20260709 12:22:32.632296 13723 pending_rounds.cc:107] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 12:22:32.632550 13723 raft_consensus.cc:2889] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:32.634181 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:32.546016 17164 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.642760 13792 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:32.641921 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.645325 13727 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:22:32.617195 17190 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:32.651667 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:32.656670 13647 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.617704 16476 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:32.652943 13720 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:32.658684 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:32.662693 13794 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:32.637298 17323 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:32.594340 17341 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.638636 17372 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:32.642066 17375 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:32.642380 17335 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.671466 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:32.674407 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.598371 17383 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:32.659291 17236 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.000s
I20260709 12:22:32.635473 17380 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:22:32.686295 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:32.627060 17368 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:22:32.694852 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:32.702605 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 376455us
I20260709 12:22:32.636593 17361 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.719240 13650 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:32.685737 17377 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:32.719930 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:32.724107 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.683638 17323 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:32.748468 13720 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:32.750147 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.754192 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
W20260709 12:22:32.763686 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43837.6us
W20260709 12:22:32.757247 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 242669us
I20260709 12:22:32.766126 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.710753 17335 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.681134 16957 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.715626 17236 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:32.782907 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:32.783222 17409 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:22:32.784799 17335 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:32.741863 17403 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:32.791416 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:32.770571 17025 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:32.799685 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.770987 17377 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:32.777251 17372 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:32.827296 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.794015 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:32.849303 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55379.1us
I20260709 12:22:32.840122 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:32.854139 13726 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:22:32.841706 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50387.3us
W20260709 12:22:32.857798 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58202.2us
I20260709 12:22:32.858443 13795 raft_consensus.cc:2368] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10.
W20260709 12:22:32.843400 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46534.7us
I20260709 12:22:32.859939 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:32.867432 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.866545 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:32.865954 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.593s (SENT)
I20260709 12:22:32.900882 13722 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:32.901115 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.902563 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:32.903038 13724 raft_consensus.cc:2368] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:32.837195 17403 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:32.908000 13726 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:22:32.919268 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.927634 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.894744 17385 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:32.899706 17346 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:32.906662 17411 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.934116 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:32.944991 13801 raft_consensus.cc:606] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:32.934836 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:32.937011 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:32.864396 17420 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:32.946122 13724 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:22:32.909390 17363 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:32.949632 17363 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.952584 17363 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:32.955771 17420 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:32.957360 17236 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:32.959187 17393 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.732s	user 0.017s	sys 0.004s
I20260709 12:22:32.959712 17393 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.944585 17368 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:32.960801 17346 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:32.961812 17346 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:32.963120 17383 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:32.963018 17394 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:32.964218 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98330us
I20260709 12:22:32.965349 17394 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:32.965453 16476 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:32.935173 17407 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
W20260709 12:22:32.965463 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.618s (SENT)
I20260709 12:22:32.967744 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:32.904505 17375 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:32.932605 17143 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.657s	user 0.007s	sys 0.006s
I20260709 12:22:32.909585 17403 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:32.970247 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:32.999972 13801 raft_consensus.cc:606] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:33.000267 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:32.971316 14420 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:33.001119 13651 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:32.973484 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:32.966064 16476 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:32.894985 17323 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:32.977528 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:32.977795 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:32.979415 17396 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.537s	user 0.014s	sys 0.000s
I20260709 12:22:32.945086 17148 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:22:33.011083 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.015707 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:32.996510 17377 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.051945 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.052827 13724 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8.
I20260709 12:22:33.055655 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.059353 13722 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:33.008431 17368 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:33.078910 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.131664 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.046762 17148 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.145084 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.149176 13724 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10.
W20260709 12:22:33.135942 17362 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.907s	user 0.013s	sys 0.013s
I20260709 12:22:33.157264 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.159004 13726 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:33.156283 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.137387 17361 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:33.149520 17325 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.000s
I20260709 12:22:33.132682 17350 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:33.133486 17420 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:33.160282 13718 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:33.150823 17148 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:32.971679 17411 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.175040 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:33.054397 17396 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.179216 13718 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:22:33.178784 13795 raft_consensus.cc:606] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:33.132565 17414 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:22:33.163311 17190 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:33.191913 13726 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
W20260709 12:22:33.203902 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.194142 13718 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:33.210137 13718 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:22:33.216334 13719 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:33.224584 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:33.225379 13648 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:33.208343 17265 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.787s	user 0.009s	sys 0.004s
I20260709 12:22:33.167130 17362 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.176676 17325 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.204149 17372 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:33.243021 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:33.239936 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91471.3us
I20260709 12:22:33.241822 13648 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:33.250167 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.237000 17415 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.251677 17415 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.215879 17323 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:33.208077 17190 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.254874 13725 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:33.180532 17350 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.275326 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:33.285897 13652 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11.
I20260709 12:22:33.288335 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:33.256054 17362 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:33.291230 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53062.4us
I20260709 12:22:33.292100 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.292662 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.294723 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.295045 13722 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:33.296264 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.293872 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:33.272087 17424 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:22:33.297785 13727 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:33.271411 17426 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:22:33.163669 17375 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.307608 17236 raft_consensus.cc:993] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:22:33.270954 17417 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.324s	user 0.002s	sys 0.000s
W20260709 12:22:33.286520 17409 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:22:33.319525 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.263370 17396 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:33.254217 17415 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:33.354228 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.358942 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.363307 13722 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:33.241771 17265 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.254572 17323 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.379479 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.361039 17192 raft_consensus.cc:1081] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:22:33.383215 13718 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:22:33.385732 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.387342 13798 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:33.402248 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108889us
W20260709 12:22:33.393316 17371 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.882s	user 0.011s	sys 0.003s
I20260709 12:22:33.407728 17371 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:33.408782 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 477239us
I20260709 12:22:33.411552 13718 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:33.414222 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.409471 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:33.427871 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:33.428596 14425 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:33.435845 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.394387 16476 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.438552 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:33.439640 14418 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:22:33.441337 17425 raft_consensus.cc:993] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:22:33.442423 17435 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:22:33.442586 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:33.442655 17425 raft_consensus.cc:1081] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:22:33.443948 17404 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.695s	user 0.001s	sys 0.007s
I20260709 12:22:33.382041 17372 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:33.435036 17322 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.898s	user 0.012s	sys 0.003s
W20260709 12:22:33.448248 17428 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
I20260709 12:22:33.448364 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 431) took 1118 ms. Trace:
I20260709 12:22:33.447592 17350 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.458658 13793 rpcz_store.cc:276] 0709 12:22:32.329809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.330419 (+   610us) service_pool.cc:224] Handling call
0709 12:22:32.365287 (+ 34868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.366649 (+  1362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.366680 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:32.367221 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:22:32.520634 (+153413us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:32.656860 (+136226us) log.cc:844] Serialized 14 byte log entry
0709 12:22:32.677460 (+ 20600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:32.677577 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:33.436424 (+758847us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:33.448040 (+ 11616us) raft_consensus.cc:1685] finished
0709 12:22:33.448155 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:33.448300 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30027,"prepare.queue_time_us":214604,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":5179,"raft.queue_time_us":24307,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":28189,"spinlock_wait_cycles":19266944,"thread_start_us":232179,"threads_started":2,"wal-append.queue_time_us":276721}
I20260709 12:22:33.439072 17192 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:33.462949 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.479557 13720 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:33.457090 17403 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.483942 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.451442 17411 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.466574 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.469301 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.485978 13726 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:33.470327 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.471141 14416 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:22:33.475289 17371 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.483887 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.484146 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:33.486192 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.487433 14423 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:33.463711 14418 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:33.458321 17322 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.489686 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:33.490525 17431 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.255s	user 0.004s	sys 0.000s
I20260709 12:22:33.490986 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.491643 13718 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:33.489830 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.493331 13722 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:33.494565 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.495368 13722 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:33.496474 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.487560 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:33.498581 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.499480 13722 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:33.498684 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.499989 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.500854 13718 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:33.501073 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.501958 13722 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:33.504068 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.504174 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.505266 13720 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8.
I20260709 12:22:33.479197 17192 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.548173 17385 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.509004 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.484174 17325 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:33.565980 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.560974 13722 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:33.590180 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.502141 17403 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:33.667570 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:33.658365 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 157351us
I20260709 12:22:33.659082 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.658900 14425 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:33.660646 17447 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:22:33.662484 17322 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.666500 17371 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.549535 17192 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.657383 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.680691 13719 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:33.612457 17411 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.668632 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.549736 17265 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:33.673935 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.692510 13720 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:33.679185 17371 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.697396 17371 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:33.697880 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:33.698604 13793 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:33.699074 13797 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:33.701561 17450 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:22:33.709910 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:33.683416 13718 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:33.679868 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:33.671257 14423 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:33.684329 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 652) took 1103 ms. Trace:
W20260709 12:22:33.715592 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.716490 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:33.716436 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:33.717267 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.718211 13801 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:33.715068 13725 rpcz_store.cc:276] 0709 12:22:32.580952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.581115 (+   163us) service_pool.cc:224] Handling call
0709 12:22:32.581489 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:33.280159 (+698670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.684165 (+404006us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54000294d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:33.684281 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548152704}
I20260709 12:22:33.722606 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:33.723274 13797 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:33.549049 17385 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:33.726815 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47073.7us
I20260709 12:22:33.727378 13800 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:22:33.728356 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.733076 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:33.732079 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.724402 17325 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:33.704049 17411 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.725248 16476 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:33.716423 17278 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:22:33.732059 17423 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:33.739163 17350 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:33.750556 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:33.754163 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:33.749547 13725 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:33.745641 17325 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:33.756405 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 436) took 1341 ms (client timeout 1541 ms). Trace:
I20260709 12:22:33.756651 17325 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:33.756860 13798 rpcz_store.cc:269] 0709 12:22:32.414658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.414802 (+   144us) service_pool.cc:224] Handling call
0709 12:22:33.756128 (+1341326us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540012b454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:33.756301 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145040256}
I20260709 12:22:33.760949 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.761973 13722 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:33.762635 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.763310 13718 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:33.763563 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.764181 13722 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:33.737457 17458 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:22:33.767732 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:33.749229 17148 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.556s	user 0.006s	sys 0.006s
I20260709 12:22:33.769932 13797 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:33.772464 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.773051 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.771528 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.780048 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.788542 17393 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:33.774730 17411 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.767222 17350 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.772452 17148 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.795295 14420 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:33.797274 17350 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.798296 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.798930 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.800431 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.798979 13727 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:33.804317 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:33.804610 13718 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:33.805229 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.803187 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.806661 13722 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:33.825380 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.826215 13719 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.836928 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:33.851297 17385 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:33.825874 17346 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:22:33.853160 17346 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:22:33.853237 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 148142us
I20260709 12:22:33.853636 17346 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:33.811618 17372 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:33.865553 13801 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:33.861826 17413 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:33.854875 17346 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.825799 17442 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:22:33.857474 17192 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:33.867956 17413 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:33.871338 17419 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.000s	sys 0.018s
W20260709 12:22:33.878226 17419 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.000s	sys 0.019s
I20260709 12:22:33.873471 17393 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:33.880054 17148 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:33.875911 17346 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:33.881160 17393 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:33.885275 17393 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.886864 17393 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:33.891561 17192 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:33.907855 17413 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.911561 17413 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:33.928128 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.933269 17190 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:33.938923 13726 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:33.942040 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.943259 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.943859 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:33.950670 13693 connection.cc:256] RPC call timeout handler was delayed by -2.152408596999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 12:22:33.951220 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1119085969999105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 12:22:33.951637 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0966085969998858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 12:22:33.952004 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0363085969997883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.165s for a response
W20260709 12:22:33.952345 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9976085969997257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 12:22:33.952677 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8887085970000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260709 12:22:33.953043 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8473085969999374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18600000000000003s for a response
W20260709 12:22:33.953408 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7663085969998065s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 12:22:33.953717 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6970085969996944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
W20260709 12:22:33.954182 13693 connection.cc:256] RPC call timeout handler was delayed by -1.62950859700004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
W20260709 12:22:33.954584 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6169085970000197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 12:22:33.954928 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6160085970000182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 12:22:33.955390 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3559085970000524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24060000000000004s for a response
W20260709 12:22:33.955753 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2488085969998792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2525s for a response
W20260709 12:22:33.956072 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1480085969997162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2637s for a response
I20260709 12:22:33.956979 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:33.957883 17413 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:33.959333 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:33.960127 17413 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:33.963716 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:33.964524 17413 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:33.965792 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:33.966615 17413 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:33.967861 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:33.969427 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2046) took 1360 ms. Trace:
I20260709 12:22:33.976284 13721 rpcz_store.cc:276] 0709 12:22:32.609473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.609668 (+   195us) service_pool.cc:224] Handling call
0709 12:22:32.634315 (+ 24647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:32.647757 (+ 13442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.647802 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:32.665202 (+ 17400us) log.cc:844] Serialized 52 byte log entry
0709 12:22:32.861076 (+195874us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:32.907881 (+ 46805us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.968409 (+1060528us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b180011b660. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:22:33.968689 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.968929 (+   240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.969016 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:33.969161 (+   145us) raft_consensus.cc:1685] finished
0709 12:22:33.969179 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:33.969354 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1032072,"prepare.queue_time_us":1114005,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":90,"raft.queue_time_us":51803,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":16299,"spinlock_wait_cycles":35823488,"thread_start_us":133575,"threads_started":1,"wal-append.queue_time_us":444128}
I20260709 12:22:33.970316 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:33.970446 17411 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:33.982242 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.983664 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.984251 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.984951 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.985613 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:33.971652 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:33.986414 13721 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10. Re-sending same reply.
I20260709 12:22:33.986821 17411 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:33.996958 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:33.997666 13721 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:33.987870 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:33.985679 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:34.003204 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.003921 17411 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:34.006074 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:33.811803 13722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:22:34.005134 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.007994 13722 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.008466 13722 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:34.009452 17411 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.010977 17323 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.006978 17393 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:34.011603 17393 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.011804 13721 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:34.012040 17393 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.013003 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.014118 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.014668 13721 raft_consensus.cc:2376] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8. Re-sending same reply.
I20260709 12:22:34.015493 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.016232 13721 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:22:34.016928 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.017906 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.018464 13721 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:34.019727 13721 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:34.010914 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.006954 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.023769 14391 leader_election.cc:400] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10.
I20260709 12:22:34.023880 14425 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:34.024202 14391 leader_election.cc:403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:22:34.024889 13726 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:34.025890 17411 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:34.027676 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.021237 17393 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.030637 17411 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:34.033139 17442 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.033795 17442 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.034354 13721 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:22:34.034766 13721 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 4
I20260709 12:22:34.035033 13721 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:22:34.035339 13721 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:34.036453 17442 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.041064 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2047) took 1431 ms. Trace:
I20260709 12:22:34.042541 17148 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10.
I20260709 12:22:34.044044 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 530) took 1456 ms. Trace:
I20260709 12:22:34.044461 13646 rpcz_store.cc:276] 0709 12:22:32.587973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.588192 (+   219us) service_pool.cc:224] Handling call
0709 12:22:32.588643 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.589177 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.589222 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:32.589861 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:22:32.673160 (+ 83299us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:32.761487 (+ 88327us) log.cc:844] Serialized 14 byte log entry
0709 12:22:33.205760 (+444273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:33.205905 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:33.270211 (+ 64306us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:33.955698 (+685487us) spinlock_profiling.cc:243] Waited 4.9 ms on lock 0x7b2400196f88. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b45fb0 00007f53937bc488 00007f53937bcbe8 00007f5393794398 00007f539378f76a 00007f539378ebb6
0709 12:22:34.043802 (+ 88104us) raft_consensus.cc:1685] finished
0709 12:22:34.043829 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.043977 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":516413,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":29292,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":167,"spinlock_wait_cycles":42455168,"thread_start_us":569904,"threads_started":3,"wal-append.queue_time_us":286053}
I20260709 12:22:34.048131 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:34.053454 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:34.054553 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:34.055320 14418 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:34.056386 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:34.041940 13723 rpcz_store.cc:276] 0709 12:22:32.610156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.610307 (+   151us) service_pool.cc:224] Handling call
0709 12:22:32.631841 (+ 21534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:32.633195 (+  1354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:32.633245 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:32.633846 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:22:33.192771 (+558925us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:34.029605 (+836834us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.040710 (+ 11105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.040808 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.040853 (+    45us) raft_consensus.cc:1685] finished
0709 12:22:34.040865 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.040998 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":847171,"raft.queue_time_us":11829,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":112,"thread_start_us":480233,"threads_started":1,"wal-append.queue_time_us":649502}
I20260709 12:22:34.063750 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.065222 13647 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:22:34.071336 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.072160 13650 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:34.073128 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.075120 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.076891 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.079885 13643 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:34.081281 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.081914 13643 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:22:34.082618 17346 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:34.085800 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:34.086612 14423 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:34.089704 13645 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:33.843122 13719 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:34.091270 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50801.8us
I20260709 12:22:34.104503 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.107730 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.073953 13650 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:34.109179 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.109784 13648 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.112470 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:34.114184 17190 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:34.114841 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.114943 17190 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.114424 13723 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:34.115330 13727 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:34.115497 17190 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:34.115536 13723 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:34.116741 13723 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.117946 13723 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.122814 17325 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:34.123299 17325 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.123692 17325 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:22:34.124558 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.126502 17190 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:34.130829 17452 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.613s	user 0.001s	sys 0.003s
I20260709 12:22:34.135489 17411 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.136246 17413 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.137171 17413 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:34.137780 17403 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.138336 17403 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.139746 17413 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.140448 17403 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.136451 17411 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.142743 17411 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.162904 13723 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:34.163153 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:34.164156 14423 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:34.170892 13725 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:22:34.174979 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.109695 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.131883 17325 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.179342 13649 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:34.181046 17325 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:34.189549 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.617s (SENT)
I20260709 12:22:34.190140 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.194249 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77264.8us
I20260709 12:22:34.194795 17442 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:34.196448 17372 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:34.196797 17325 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:34.198609 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.751s (SENT)
W20260709 12:22:34.201867 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.888s (SENT)
I20260709 12:22:34.202898 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:34.203083 13617 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.901s (SENT)
I20260709 12:22:34.094748 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.204352 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:34.095537 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:34.205972 13719 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:34.207177 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 469317us
W20260709 12:22:34.209519 13652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 529) took 1698 ms (client timeout 1683 ms). Trace:
I20260709 12:22:34.213534 17423 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:34.095602 17393 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:34.214713 17423 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:34.210050 13652 rpcz_store.cc:269] 0709 12:22:32.511205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.511429 (+   224us) service_pool.cc:224] Handling call
0709 12:22:34.209207 (+1697778us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54001b5654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:34.209413 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774352768}
I20260709 12:22:34.219205 13645 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:22:34.223403 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:34.224234 17395 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:34.224817 17436 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:34.097549 14425 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:34.226462 17436 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.228633 17436 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.231472 17395 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:34.100416 13722 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.203644 14417 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:34.232548 17395 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.233887 13722 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.243983 13646 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:22:34.244609 13646 pending_rounds.cc:85] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 5
I20260709 12:22:34.244967 13646 pending_rounds.cc:107] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:22:34.261837 13646 raft_consensus.cc:2889] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:34.256901 13723 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:34.259008 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:34.265049 13722 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:34.260564 13727 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:34.253350 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:34.092655 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65078.8us
W20260709 12:22:34.294404 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.203644 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.304550 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 101493us
I20260709 12:22:34.305763 13719 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
W20260709 12:22:34.309206 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.286059 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.325752 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.326233 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:34.310223 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
W20260709 12:22:34.306721 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.052s (SENT)
I20260709 12:22:34.327239 13649 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:34.131965 17190 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.328658 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.330983 13643 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:34.344751 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:34.463135 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 156448us
W20260709 12:22:34.463927 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.788s (SENT)
W20260709 12:22:34.464092 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:34.464769 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:34.387059 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60924us
I20260709 12:22:34.462481 17190 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.465999 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.466575 17190 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.467347 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.468609 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:34.469043 17190 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.469584 17190 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:34.466005 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.394753 17403 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:34.380952 17474 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.431347 17124 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:34.426568 17476 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:34.473212 17474 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.473476 17476 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:34.473894 17474 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:34.474300 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.613s (SENT)
I20260709 12:22:34.454784 17367 raft_consensus.cc:993] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:22:34.478088 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.481698 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.705s (SENT)
I20260709 12:22:34.484380 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.487412 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.789s (SENT)
I20260709 12:22:34.480079 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:34.487489 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 260131us
I20260709 12:22:34.483454 17124 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:34.495055 17124 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:34.495781 17124 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.496279 17124 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.496891 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:34.497280 13693 leader_election.cc:400] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:34.497614 13693 leader_election.cc:403] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:34.488719 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:34.501041 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.504900 17372 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:34.505735 17372 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.489037 17413 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:34.464041 17419 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.506554 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:34.507118 17419 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:33.808531 13720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:22:34.493721 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.511332 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.905s (SENT)
I20260709 12:22:34.506204 17372 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:34.515580 17124 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.470137 17190 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:34.490018 17476 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.510051 13720 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.512184 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:34.511445 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.518214 17419 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.523243 17419 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:34.523774 17477 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.519192 13720 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.513571 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 657) took 1520 ms. Trace:
I20260709 12:22:34.506654 17413 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:34.530632 17413 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.531026 17477 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:34.531615 17442 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:34.532145 17442 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:34.531966 17477 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:34.533078 17442 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.534878 17323 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:34.535676 17323 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:34.536115 17323 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:34.536936 17323 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.518646 17190 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.537662 13724 rpcz_store.cc:276] 0709 12:22:32.993464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.171547 (+178083us) service_pool.cc:224] Handling call
0709 12:22:33.172645 (+  1098us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:33.173389 (+   744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:33.173430 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:33.218719 (+ 45289us) log.cc:844] Serialized 78 byte log entry
0709 12:22:33.400233 (+181514us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:34.130145 (+729912us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:34.341174 (+211029us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.383397 (+ 42223us) log.cc:844] Serialized 14 byte log entry
0709 12:22:34.513125 (+129728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:34.513203 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:34.513350 (+   147us) raft_consensus.cc:1685] finished
0709 12:22:34.513364 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:34.513509 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":840450,"prepare.queue_time_us":865700,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":176570,"raft.run_cpu_time_us":465,"raft.run_wall_time_us":20041,"spinlock_wait_cycles":124616064,"thread_start_us":365502,"threads_started":4,"wal-append.queue_time_us":211117}
I20260709 12:22:34.541831 14418 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:36658
I20260709 12:22:34.544021 14418 raft_consensus.cc:606] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.544559 14418 raft_consensus.cc:614] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:34.550647 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.553328 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:34.554772 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54711.6us
I20260709 12:22:34.555820 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:34.557323 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:34.544023 13720 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.540073 17419 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.543707 17477 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:34.539778 17413 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.589119 17413 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.589807 17413 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:34.591506 13726 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:34.566891 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55582.9us
I20260709 12:22:34.576607 13720 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:34.592826 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.938s (SENT)
I20260709 12:22:34.593456 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.601177 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.949s (SENT)
I20260709 12:22:34.601884 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.603657 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.040s (SENT)
I20260709 12:22:34.605726 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:34.713903 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 193870us
I20260709 12:22:34.579699 17190 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.591133 17477 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:34.719168 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 161926us
I20260709 12:22:34.728631 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:34.721810 17413 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:34.749089 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:34.749845 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 146236us
I20260709 12:22:34.750079 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.750749 13648 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:34.726244 17190 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.752334 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:34.752650 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:34.743643 17323 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:34.754408 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.754653 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.756042 13721 raft_consensus.cc:2376] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11. Re-sending same reply.
I20260709 12:22:34.756455 13643 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:34.753261 13724 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10. Re-sending same reply.
I20260709 12:22:34.757385 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.756995 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.758664 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.758847 13651 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:34.759017 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.759678 13725 raft_consensus.cc:2376] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8. Re-sending same reply.
I20260709 12:22:34.749042 17413 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.760598 17413 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.757395 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:34.762732 13720 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
W20260709 12:22:34.767203 14416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.031s	sys 0.028s
I20260709 12:22:34.768855 13651 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:34.770342 17403 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:34.771311 14416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.031s	sys 0.028s
W20260709 12:22:34.755007 17478 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.180s	user 0.002s	sys 0.000s
W20260709 12:22:34.772976 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.058s (SENT)
I20260709 12:22:34.774245 13648 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:22:34.775425 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.752107 17476 raft_consensus.cc:2764] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:22:34.777321 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:34.780736 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.074s (SENT) [suppressed 49 similar messages]
I20260709 12:22:34.780593 17190 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.782608 17190 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.771437 17480 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:34.783854 17480 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.784382 17480 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:34.785280 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.093s (SENT)
I20260709 12:22:34.785842 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.791399 17419 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:34.792124 17419 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.792793 17419 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:34.780269 17477 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.794723 17477 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:34.781476 17484 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:34.798761 17481 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:22:34.805260 17480 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:34.806041 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.802702 17485 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:34.810959 17411 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.770864 17413 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.811400 17485 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.811880 17485 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.811633 17411 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:34.813299 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.814214 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:34.814728 17411 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.815420 17411 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:34.816628 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.817355 13643 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:34.779560 17403 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:34.827661 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:34.828835 13652 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:34.835657 17403 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.838096 13643 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:34.853492 17474 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:34.854079 17474 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:34.855075 17474 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.856751 17474 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.854933 17485 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:34.868243 17485 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.816634 17411 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:34.873121 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:34.873889 13643 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.874502 17413 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:34.880435 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92526.1us
W20260709 12:22:34.883167 14423 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.006s
I20260709 12:22:34.891268 14423 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:34.897946 13643 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:34.901777 17480 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 12:22:34.928747 14420 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 702) took 1494 ms (client timeout 1665 ms). Trace:
W20260709 12:22:34.929781 14420 rpcz_store.cc:269] 0709 12:22:33.433812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:33.433965 (+   153us) service_pool.cc:224] Handling call
0709 12:22:34.927719 (+1493754us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54005546d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:34.927907 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803662208}
I20260709 12:22:34.937314 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:34.938863 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:34.940922 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:34.944048 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:34.950011 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:34.952872 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.954198 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.945825 17485 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:34.928859 17372 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:34.954690 17484 raft_consensus.cc:2764] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:34.954442 17413 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:34.950994 17479 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:34.971187 14417 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.738s	user 0.015s	sys 0.007s
W20260709 12:22:34.965915 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 861907us
I20260709 12:22:34.971223 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:34.973948 14417 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:22:34.953465 17476 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:34.928041 17411 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.975327 14425 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:34.975643 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:34.977977 17372 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:34.979084 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:34.979771 13720 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:34.971522 17482 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.966624 17419 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:34.989737 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:34.988101 14391 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11.
I20260709 12:22:34.991319 14391 leader_election.cc:403] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:22:34.955067 17480 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:22:35.000173 13720 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:34.983212 17124 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:35.002576 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:34.973577 17479 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:34.988790 17484 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:22:35.007059 17484 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:35.007416 17484 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:35.008167 17124 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:35.008291 17484 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.010176 17484 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.009917 17124 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.012882 17385 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11.
I20260709 12:22:35.004415 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.014709 17487 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:35.015316 17487 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:35.016228 17487 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.005146 17479 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:34.990513 17476 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:35.031575 17458 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:35.032215 17458 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:35.033113 17458 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:34.994452 17482 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.036266 17482 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.019047 17480 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:35.018765 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:35.019876 13720 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:35.018368 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:35.044792 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:35.048465 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:35.051352 17480 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:35.051477 17476 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.054200 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:35.057639 17480 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:35.060175 17480 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:35.063787 17480 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:35.069321 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:35.047983 17482 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:22:35.047230 17419 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:35.073582 17419 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:35.073984 17419 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.074506 17419 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:35.076798 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:35.077476 13647 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:35.077507 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:35.075958 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:35.078773 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:35.080274 13649 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:35.081071 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:35.081733 13643 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.082885 13647 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:35.040364 17413 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:35.101460 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:35.088932 17499 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:35.114984 14425 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
W20260709 12:22:35.124403 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 151248us
I20260709 12:22:35.124874 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.128494 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.130137 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:35.132623 13721 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:35.133811 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:35.133965 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.109937 17413 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:35.134511 13647 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:35.155754 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.157117 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.145058 13721 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:35.158161 17372 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:35.158682 17372 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.159103 17372 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:35.127918 17124 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.136198 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.167326 13718 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:35.174281 17190 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:35.179754 17496 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
I20260709 12:22:35.170413 17458 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.169106 17124 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.188036 17458 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.126124 17499 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:22:35.192878 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:35.167924 17479 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.191638 17458 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.198653 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:35.134385 17487 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:22:35.132265 17488 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:22:35.196902 17395 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.961s	user 0.004s	sys 0.020s
I20260709 12:22:35.192618 17124 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:35.200991 17325 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:35.211799 17479 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:35.202674 17395 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.005s	sys 0.020s
I20260709 12:22:35.214089 17372 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.201992 17323 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:22:35.201749 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:35.199287 17190 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:35.222481 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.226089 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:35.236090 13651 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.236308 17372 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.237296 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:35.237975 13648 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:35.235461 17190 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:35.241063 17190 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.241595 17190 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:35.250510 13648 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:35.252091 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.253507 17372 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:35.255143 17372 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:35.256685 17372 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.262147 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.262858 13726 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:35.264533 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:35.204098 17488 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.270865 13651 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:35.272887 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:35.230754 17325 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:35.293638 17442 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.294584 17442 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.295444 17482 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.295986 17482 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.232707 17419 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.299084 17442 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.300658 17419 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.272629 17488 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:35.304677 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:22:35.276762 17325 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.271561 17323 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:35.261317 17346 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:22:35.322821 17482 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.314741 17323 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.323827 17323 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:35.318794 17325 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:35.319433 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800254900 after lost signal to thread 12472
W20260709 12:22:35.325874 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08003058c0 after lost signal to thread 13584
W20260709 12:22:35.327419 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5140 after lost signal to thread 13676
W20260709 12:22:35.328851 13748 debug-util.cc:398] Leaking SignalData structure 0x7b080012c8c0 after lost signal to thread 13751
W20260709 12:22:35.329152 13748 debug-util.cc:398] Leaking SignalData structure 0x7b080012c8a0 after lost signal to thread 13752
I20260709 12:22:35.331429 17346 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:35.331887 17346 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:35.332773 17346 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.333014 17488 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.334749 17346 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:35.342744 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:35.343488 13649 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:35.344584 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.347173 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.347743 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:35.348353 13648 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:35.350090 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.350600 17367 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:35.351182 17367 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.352905 17488 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.353448 17367 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:35.354417 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 859832us
I20260709 12:22:35.355154 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.358220 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.359879 13799 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:22:35.360049 17277 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:35.360633 17277 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.361225 17202 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.361744 17202 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.362653 17423 raft_consensus.cc:1081] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:22:35.362798 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.363858 17202 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:35.363969 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.364399 17202 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:22:35.365151 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.365674 13794 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:35.366622 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.364064 17423 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:35.367785 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.367954 17423 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:35.368873 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.368912 17423 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.366401 17365 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:35.370635 17365 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:35.371760 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.371594 17365 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.372498 13800 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:35.372862 17365 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:22:35.373168 17469 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.469s	user 0.002s	sys 0.000s
I20260709 12:22:35.373286 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.370064 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:35.360637 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.375384 13792 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:35.376224 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.377658 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.379544 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.379828 13650 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:35.370200 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.381325 13652 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:35.351557 17482 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:35.383711 13652 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:35.385593 13646 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:35.386575 13644 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:35.386768 13646 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:35.389633 13797 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:35.390377 17325 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:35.390861 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.394686 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.396067 13793 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:22:35.382071 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.396067 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:35.397476 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124722us
I20260709 12:22:35.402422 13652 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:35.402851 13796 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:22:35.399453 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.404736 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.411999 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.407038 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:35.404783 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.416030 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.416218 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.418032 13651 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
W20260709 12:22:35.420145 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.420486 13797 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:35.427063 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.427282 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.383668 17202 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.396833 17482 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:35.444729 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.456683 13793 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:22:35.463146 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.464112 13793 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:35.464586 13793 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:35.437371 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.449376 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:35.468647 13726 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:35.399345 17499 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:35.450954 13795 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:35.454432 13794 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:35.470022 13800 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:35.470880 13794 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:35.471517 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:35.410600 17359 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:35.478931 17474 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:35.452422 17449 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:22:35.477955 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.481555 13720 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:35.482538 17325 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:35.485625 13795 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:35.493319 17458 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:35.494776 17403 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:35.495512 17403 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.424814 17190 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.503154 17190 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:35.505512 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:35.506451 13724 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:35.510607 17499 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:35.511997 17499 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.455106 17488 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:35.520785 17488 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.521297 17488 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:35.523901 17411 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:35.524544 17411 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:35.525430 17411 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.524070 13722 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:22:35.527865 17411 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:35.531733 13797 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:35.551970 13797 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:35.460002 17497 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.553071 13797 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.564774 13797 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:35.548864 13649 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:35.568979 13652 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:35.453783 17482 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:35.584920 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:35.591109 17372 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:35.591967 17372 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.593382 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:35.594821 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.595481 17190 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:35.595739 14425 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:35.596205 17190 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:35.597723 14416 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:35.561219 17393 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:22:35.600286 17393 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.600936 17393 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.604100 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:35.605517 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.620359 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:35.630972 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800392340 after lost signal to thread 13825
W20260709 12:22:35.631537 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800315f40 after lost signal to thread 13826
W20260709 12:22:35.633194 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08003bea00 after lost signal to thread 14449
W20260709 12:22:35.633591 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08003dcbc0 after lost signal to thread 14450
W20260709 12:22:35.636986 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8700 after lost signal to thread 17445
I20260709 12:22:35.466840 13793 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.725353 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.689844 13726 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:35.457345 17202 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.730120 13793 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:35.745072 13797 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:22:35.731123 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.750509 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.758378 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:35.638233 17485 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.000s	sys 0.047s
I20260709 12:22:35.760099 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.773887 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:35.795364 13652 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:35.562073 17359 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.798226 17202 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:35.798632 13792 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:35.799742 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.841259 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.842662 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.747236 17325 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:22:35.822638 17476 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:35.854977 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.816439 17440 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:22:35.659005 17277 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:35.637929 17458 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:35.638386 17487 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:35.597555 17190 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.806449 13793 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:35.615916 17491 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.826278 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:35.857007 17325 consensus_queue.cc:1243] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:22:35.878459 13801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.026s	sys 0.060s
W20260709 12:22:35.878891 13801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.026s	sys 0.060s
I20260709 12:22:35.890445 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.894125 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:35.831804 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75165.3us
W20260709 12:22:35.816987 17485 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.001s	sys 0.047s
W20260709 12:22:35.832533 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.583856 17497 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:35.838135 14423 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:35.857470 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:35.923839 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:35.931191 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:35.626354 17442 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.728456 17419 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:35.863243 17372 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:35.940059 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.944387 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:35.889524 17413 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:35.933844 17486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.046s	sys 0.079s
W20260709 12:22:35.980196 13643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.005s	sys 0.017s
W20260709 12:22:35.980651 13643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.005s	sys 0.017s
I20260709 12:22:35.860344 17476 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:35.923329 17485 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:35.956647 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:35.995571 17148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.001s	sys 0.010s
I20260709 12:22:35.994364 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:35.947216 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:35.998165 13648 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.012s	sys 0.011s
W20260709 12:22:36.018505 13648 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.012s	sys 0.011s
W20260709 12:22:35.998642 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.007s	sys 0.016s
W20260709 12:22:36.019446 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.007s	sys 0.017s
W20260709 12:22:36.019826 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 594420us
I20260709 12:22:35.924584 17491 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:35.942875 17346 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.608s	user 0.012s	sys 0.000s
I20260709 12:22:35.939121 17419 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:35.999670 17365 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.000s
I20260709 12:22:36.026984 14420 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:36.000597 17423 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.011s	sys 0.002s
I20260709 12:22:35.938932 17442 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:35.966172 17486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.046s	sys 0.080s
I20260709 12:22:36.037132 13798 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
W20260709 12:22:36.038705 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91596.2us
W20260709 12:22:36.028200 13724 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:22:36.043753 13724 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:22:36.050746 13647 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.012s
W20260709 12:22:36.051496 13647 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.000s	sys 0.013s
I20260709 12:22:36.053987 14416 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
W20260709 12:22:36.024294 17148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.002s	sys 0.010s
W20260709 12:22:36.056300 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 308927us
W20260709 12:22:36.057440 13799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.046s	sys 0.094s
I20260709 12:22:35.960554 17413 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:36.059383 13799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.046s	sys 0.094s
W20260709 12:22:36.038036 17423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.011s	sys 0.003s
I20260709 12:22:36.033619 17393 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.037505 17325 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.049868 17277 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:36.064352 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:36.074494 13724 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:36.055675 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.053442 17359 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:36.124184 13795 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.036s	sys 0.023s
W20260709 12:22:36.125903 13795 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.036s	sys 0.024s
I20260709 12:22:36.126332 17514 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.131883 17514 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.136705 17530 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.136727 17514 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.137573 17530 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.145464 17530 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.147289 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 542) took 1036 ms. Trace:
I20260709 12:22:36.147603 17536 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:36.148499 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:36.149524 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98264.9us
W20260709 12:22:36.150127 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99452.9us
W20260709 12:22:36.127701 13693 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.003s	sys 0.016s
I20260709 12:22:36.147986 13643 rpcz_store.cc:276] 0709 12:22:35.111396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.147274 (+ 35878us) service_pool.cc:224] Handling call
0709 12:22:35.147643 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:35.147897 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:35.428767 (+280870us) log.cc:844] Serialized 14 byte log entry
0709 12:22:36.146864 (+718097us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:36.146949 (+    85us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:36.147002 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:36.147075 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:36.147219 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":450538,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":13091,"raft.queue_time_us":148142,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"thread_start_us":991354,"threads_started":3,"wal-append.queue_time_us":705640}
W20260709 12:22:36.164176 13693 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.003s	sys 0.016s
W20260709 12:22:36.164724 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 609066us
W20260709 12:22:36.165699 13722 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.007s	sys 0.016s
W20260709 12:22:36.166352 13722 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.007s	sys 0.016s
I20260709 12:22:36.184970 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.187870 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:36.192718 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:36.211988 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47724.6us
W20260709 12:22:36.203884 17323 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.880s	user 0.010s	sys 0.003s
I20260709 12:22:36.078404 17325 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.202414 17458 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:36.202100 17487 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:36.205919 17519 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:36.227509 17453 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:22:36.120080 17485 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.261739 17442 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.261492 17487 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.262162 14423 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
W20260709 12:22:36.263161 17516 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:22:36.225906 17372 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.262887 17458 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.265775 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:36.266633 17515 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.265s	user 0.000s	sys 0.001s
W20260709 12:22:36.268407 17488 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.000s
I20260709 12:22:36.269366 17491 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.277002 17413 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.230907 17474 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:36.278895 17513 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.284s	user 0.001s	sys 0.001s
I20260709 12:22:36.227604 17423 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:22:36.263265 13539 maintenance_manager.cc:419] P d5e54c6d2c954fbf93d5136276b7b78c: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033404
I20260709 12:22:36.296047 17530 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.297024 17372 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.281771 17514 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.297562 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:36.299425 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2598) took 1942 ms. Trace:
I20260709 12:22:36.352552 14424 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:36.256655 17497 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.229947 17323 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.225534 17529 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 12:22:36.284097 17476 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:36.342643 17499 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.312809 17277 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:22:36.337643 17419 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:36.361757 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 189043us
W20260709 12:22:36.360936 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 176722us
I20260709 12:22:36.290441 17519 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:36.343014 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:36.289553 17440 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:22:36.315280 17491 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.318442 17474 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.350553 13721 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:36.352415 13801 rpcz_store.cc:276] 0709 12:22:34.356560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.470776 (+114216us) service_pool.cc:224] Handling call
0709 12:22:34.471172 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:35.348387 (+877215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:35.348420 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:35.349053 (+   633us) log.cc:844] Serialized 52 byte log entry
0709 12:22:35.879339 (+530286us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:35.879416 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:35.879518 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:36.299129 (+419611us) raft_consensus.cc:1685] finished
0709 12:22:36.299154 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:36.299343 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":876673,"raft.queue_time_us":890783,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":529839,"threads_started":1,"wal-append.queue_time_us":706354}
I20260709 12:22:36.317337 17413 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.344947 17346 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.314103 17488 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:36.381242 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163500us
I20260709 12:22:36.383474 17419 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:36.385340 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:36.383699 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:36.386893 17419 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:36.387424 17419 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:36.387507 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:36.388351 17419 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.371346 17486 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:36.372565 17277 consensus_queue.cc:1243] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:22:36.379709 17411 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.803s	user 0.011s	sys 0.002s
I20260709 12:22:36.352453 17530 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:36.395203 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 168620us
I20260709 12:22:36.401681 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:36.402542 14425 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:36.357386 17514 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.407006 17449 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:36.407424 17277 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.398434 17519 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.369621 17499 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.374392 17534 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:22:36.397621 14421 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:36.372794 17491 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.394212 17486 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:36.463827 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2608) took 1095 ms. Trace:
W20260709 12:22:36.469691 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68127.6us
I20260709 12:22:36.382364 17413 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.469446 13794 rpcz_store.cc:276] 0709 12:22:35.368321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.471858 (+103537us) service_pool.cc:224] Handling call
0709 12:22:35.472203 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:35.841462 (+369259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:35.841499 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:35.841562 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:35.841607 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:36.463498 (+621891us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54000e60d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:36.463555 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:36.463746 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848680832}
I20260709 12:22:36.391971 17346 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:36.459875 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84350.4us
I20260709 12:22:36.479977 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:36.492053 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:36.399103 17541 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:22:36.473721 17486 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:36.493924 13796 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:36.494843 13796 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:36.495803 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106765us
I20260709 12:22:36.480310 17365 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:36.478431 17546 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:36.497161 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111909us
I20260709 12:22:36.471593 17542 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.470109 17442 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 12:22:36.508126 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 430278us
I20260709 12:22:36.500586 13796 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.484131 17325 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.489063 17528 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.515969 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:36.496227 17541 consensus_queue.cc:1243] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:22:36.515251 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.517804 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.523025 13796 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:36.500002 17503 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.506973 17365 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:36.525441 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.521984 17546 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:36.550362 17560 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.551309 17560 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.553893 17560 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.540027 17365 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.525125 17542 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.555742 17546 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.558612 17488 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.558837 14424 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:36.558980 17499 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:36.542924 17323 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.559944 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 443) took 1203 ms. Trace:
I20260709 12:22:36.560561 13798 rpcz_store.cc:276] 0709 12:22:35.356838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.394414 (+ 37576us) service_pool.cc:224] Handling call
0709 12:22:36.559753 (+1165339us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54002cb0d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:36.559878 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475692800}
I20260709 12:22:36.539513 17403 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:36.564472 17482 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.929s	user 0.012s	sys 0.001s
I20260709 12:22:36.539940 17503 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.565063 17482 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.564499 17556 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:36.568495 17503 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.574591 17372 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.583328 17557 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:36.583660 17545 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.003s
I20260709 12:22:36.584223 17346 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.583982 17557 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:36.582623 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57263.8us
I20260709 12:22:36.586395 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.587622 14424 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:22:36.588078 14424 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 6
I20260709 12:22:36.588681 14424 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.7
I20260709 12:22:36.589318 17559 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:36.589660 14424 raft_consensus.cc:2917] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Aborting config change with OpId 8.7: Aborted: Op aborted by new leader
I20260709 12:22:36.590735 14424 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:36.581619 17277 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:22:36.592314 17277 consensus_queue.cc:1243] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:36.591281 17491 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:36.591445 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1858) took 1215 ms. Trace:
I20260709 12:22:36.598207 13792 rpcz_store.cc:276] 0709 12:22:35.375591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.375974 (+   383us) service_pool.cc:224] Handling call
0709 12:22:36.591267 (+1215293us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:36.591381 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970123008}
I20260709 12:22:36.601801 17440 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:36.603173 17474 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.590095 17559 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:36.608855 13796 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:22:36.616282 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1864) took 1200 ms. Trace:
I20260709 12:22:36.617792 13796 rpcz_store.cc:276] 0709 12:22:35.415526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.444326 (+ 28800us) service_pool.cc:224] Handling call
0709 12:22:36.493791 (+1049465us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b54003771a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf7e18 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:36.616177 (+122386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137096576}
I20260709 12:22:36.618968 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:36.603207 17557 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:36.617175 17544 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.003s
I20260709 12:22:36.558663 17542 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:36.671520 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45068.6us
I20260709 12:22:36.785205 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:36.694324 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:36.645148 17449 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:22:36.610886 17440 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:36.629346 17190 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:36.798833 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 281123us
W20260709 12:22:36.832623 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47564.9us
I20260709 12:22:36.834681 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:36.839022 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.842211 13649 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:36.802632 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:36.844300 13651 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:36.846877 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.847705 13651 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:36.783751 17477 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:36.851749 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.857275 13651 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:36.853809 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 366508us
I20260709 12:22:36.793174 17474 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:36.874017 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71516.8us
I20260709 12:22:36.876567 14417 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:36.877830 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:36.800046 17559 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:36.879875 13646 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:36.880383 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:36.647368 17365 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.881457 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.882215 13796 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:36.882622 14419 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:36.882412 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:36.889266 13796 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:36.884361 14423 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:36.890918 13796 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:36.892225 13796 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:36.830571 17503 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:36.833045 17403 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.793591 17325 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:22:36.920982 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.921878 13643 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:36.853505 17514 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:36.924309 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.851130 17536 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.883970 17190 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.929123 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.849721 17557 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.880717 17533 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:36.927065 13648 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:36.952328 13796 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:36.951395 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.962163 13649 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:36.963594 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.964311 13649 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:36.923321 17458 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.921322 17564 raft_consensus.cc:2764] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:22:36.940310 17395 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.952136 17556 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:22:36.976493 17561 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:22:36.975952 17557 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.952550 17562 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.966894 17533 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:36.941929 17549 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:22:36.978001 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 558) took 1385 ms. Trace:
I20260709 12:22:36.926590 17503 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.982045 17568 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.982865 17323 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:36.879709 17477 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.023173 17540 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:22:37.024549 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.030s	sys 0.029s
I20260709 12:22:36.886018 17474 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:36.928191 17560 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:36.927644 17403 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.043696 14416 rpcz_store.cc:276] 0709 12:22:35.604965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.605191 (+   226us) service_pool.cc:224] Handling call
0709 12:22:36.977764 (+1372573us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54006e0fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:36.977915 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993607552}
I20260709 12:22:36.977885 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.088726 13649 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:37.047725 13648 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:37.045360 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.091008 13652 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 5.
I20260709 12:22:36.989176 17482 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.050140 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.025559 17411 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:37.067044 17365 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:37.070571 17503 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.029683 17458 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:37.071695 17511 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.996s	user 0.013s	sys 0.000s
I20260709 12:22:37.035238 17514 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.986951 17190 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.048205 17533 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:37.120278 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 81310.1us
I20260709 12:22:37.035499 17124 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:37.081046 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.030s	sys 0.029s
I20260709 12:22:37.046754 17395 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.126685 17546 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.127012 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.127396 17546 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.128301 17395 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.092181 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.129032 13648 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:37.130061 13650 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:37.072911 17325 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:36.948053 17536 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.104048 13646 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:36.905561 17559 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.065008 17323 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.102420 17570 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.102845 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:36.923734 17440 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.135043 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.141765 13651 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:37.050441 17562 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.139816 13645 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:37.059442 17568 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.102550 17499 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:37.150110 13643 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:37.152316 13796 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:22:37.153761 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.154356 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.153805 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.156389 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.171535 13648 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:22:37.137351 17576 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:37.176996 13794 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:37.178952 13794 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:37.117538 17411 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.177315 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 544) took 1941 ms. Trace:
I20260709 12:22:37.183892 17559 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.179766 13794 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.134107 17124 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.149191 17499 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.132779 17514 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.185151 13792 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:22:37.137068 17325 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:37.233397 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.257237 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:37.257548 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.257716 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.257898 13793 raft_consensus.cc:2376] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7. Re-sending same reply.
I20260709 12:22:37.260408 13794 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:37.264339 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.255810 13647 rpcz_store.cc:276] 0709 12:22:35.236340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.238367 (+  2027us) service_pool.cc:224] Handling call
0709 12:22:35.238745 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:35.239180 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:35.239223 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:35.239927 (+   704us) log.cc:844] Serialized 26 byte log entry
0709 12:22:35.418277 (+178350us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:36.584560 (+1166283us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b180025fea0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:22:36.584749 (+   189us) log.cc:844] Serialized 14 byte log entry
0709 12:22:36.924249 (+339500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:36.924335 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:36.980062 (+ 55727us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:37.177013 (+196951us) raft_consensus.cc:1685] finished
0709 12:22:37.177047 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:37.177230 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33993,"prepare.queue_time_us":435647,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":7204,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":208,"spinlock_wait_cycles":39577344,"thread_start_us":1034398,"threads_started":3,"wal-append.queue_time_us":811158}
I20260709 12:22:37.269603 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.269726 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.271164 13796 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:37.271139 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.271733 13796 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:37.273073 13796 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.274633 13796 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:37.173647 17482 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:37.252050 17573 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:22:37.284142 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:37.275794 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.284976 14423 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:37.212922 17503 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.279280 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:37.229399 17574 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:22:37.247901 17557 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.281682 13800 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:37.225097 17546 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.200874 17576 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.307778 17325 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.308787 17576 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.309309 17562 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.311015 17514 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.315886 17499 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.316578 17503 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.282078 17190 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.251536 17458 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.283629 17124 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.342182 13796 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:22:37.286695 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.288208 13644 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:37.252261 17572 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:37.293439 13794 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:22:37.348239 13794 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 6
I20260709 12:22:37.348613 13794 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.7
I20260709 12:22:37.236596 17477 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.231292 17568 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.295253 13800 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:37.348886 13794 raft_consensus.cc:2917] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Aborting config change with OpId 8.7: Aborted: Op aborted by new leader
I20260709 12:22:37.297531 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:37.352062 13794 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.146740 17570 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:37.298985 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.300645 17411 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.266515 17403 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.287199 17482 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.360626 17570 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.362434 17482 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.362879 13646 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:22:37.287593 17550 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:37.363478 17550 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:37.256831 17486 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.748s	user 0.012s	sys 0.000s
I20260709 12:22:37.365219 17486 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.366531 17550 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:37.367674 17486 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:37.369688 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 72216.7us
I20260709 12:22:37.265873 17474 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.286495 17577 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.280437 17323 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.283713 17436 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:37.278456 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 153150us
W20260709 12:22:37.338650 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53859.8us
I20260709 12:22:37.173058 17549 consensus_queue.cc:1243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:22:37.350344 17572 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.378418 17436 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.379789 17572 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.351038 17503 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.236254 17536 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.364506 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.371268 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:37.388721 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:37.380633 17577 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:37.375468 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.377004 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:37.393602 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.395251 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:37.377110 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.381764 13797 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:37.397058 13796 pending_rounds.cc:85] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 4
I20260709 12:22:37.397469 13796 pending_rounds.cc:107] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:22:37.397703 13796 raft_consensus.cc:2889] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.398629 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.387830 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 724) took 1112 ms. Trace:
I20260709 12:22:37.402654 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.399452 14419 rpcz_store.cc:276] 0709 12:22:36.275696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.286060 (+ 10364us) service_pool.cc:224] Handling call
0709 12:22:36.296285 (+ 10225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:36.889660 (+593375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.387351 (+497691us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:37.387620 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347788160}
I20260709 12:22:37.410830 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.411962 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:37.412238 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:37.412591 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.412875 13647 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:37.413077 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.374987 17124 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:37.417531 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.416579 13793 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:22:37.418515 13797 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:37.419305 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.415084 13643 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 6: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:37.421715 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.385543 17536 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:37.424818 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.431861 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.431962 13793 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 5.
I20260709 12:22:37.385679 17530 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.434649 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:37.434700 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46057.3us
I20260709 12:22:37.436290 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.411325 17346 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:37.441555 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.442374 13801 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:37.445415 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.445299 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.449604 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.450287 13793 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:37.450486 13647 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:37.401443 17549 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:22:37.451423 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.475553 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.475842 13801 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:37.476331 14419 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:37.451411 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.477353 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.423519 17423 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:37.478605 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.447111 17482 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:22:37.480027 17577 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.481101 13647 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:37.481981 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.423318 17359 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:22:37.487277 17436 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.439661 17124 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.530105 14416 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:37.444427 17572 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.536087 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:37.538729 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.519515 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 722) took 1247 ms. Trace:
I20260709 12:22:37.540365 14421 rpcz_store.cc:276] 0709 12:22:36.272111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.272819 (+   708us) service_pool.cc:224] Handling call
0709 12:22:36.273243 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:36.469196 (+195953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.519209 (+1050013us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b540010f4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:37.519432 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215125760}
I20260709 12:22:37.447372 17190 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:37.447029 17568 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:37.452324 17458 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:37.522403 17579 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
I20260709 12:22:37.546567 17458 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:37.544772 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55740.3us
I20260709 12:22:37.548998 17458 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.543944 17572 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.546115 17559 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:37.553685 13647 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:22:37.554319 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:37.444180 17533 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.567480 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.491489 17423 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.544669 17190 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.569144 17577 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:37.542316 17124 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.560472 17449 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:37.574631 17570 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:22:37.576977 17533 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:37.580683 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 720) took 1313 ms. Trace:
I20260709 12:22:37.546190 17568 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.555015 17559 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.579068 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:37.583798 17434 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:22:37.584102 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:37.584132 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.591959 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.581532 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:37.593466 14422 rpcz_store.cc:276] 0709 12:22:36.267603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.267745 (+   142us) service_pool.cc:224] Handling call
0709 12:22:36.980655 (+712910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:36.980912 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:36.980946 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:36.981001 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:36.981046 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:36.981106 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:37.580449 (+599343us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b54006e0fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:37.580600 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638686464}
I20260709 12:22:37.603896 17559 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:37.610809 17413 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.620036 17190 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.621908 17572 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.624038 17124 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:37.586917 17423 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:37.626645 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.636615 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.637418 14422 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:22:37.638780 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:37.639678 17568 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:37.643169 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69732.8us
I20260709 12:22:37.643277 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:37.644106 13644 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:37.652303 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.659705 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:37.660634 17583 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:37.660804 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.661094 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.661757 14421 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:37.661966 17583 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:37.662127 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.662227 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.661978 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:37.606171 17485 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:37.730894 13650 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: [8.9-10.10]
W20260709 12:22:37.731542 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69647.7us
W20260709 12:22:37.732577 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.736707 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.742957 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.593945 17449 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.650033 17568 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:37.771818 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.730453 17583 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.787281 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.646920 17537 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.123s
I20260709 12:22:37.734177 17485 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.797668 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.808238 13797 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:37.741304 17570 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:37.805143 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:37.811527 13792 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:37.766383 17458 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:37.768222 17419 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:37.733918 17519 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:22:37.821350 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:37.785579 17568 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.799513 17583 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:37.822631 17485 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:37.658463 17559 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.823793 17458 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.833258 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.828111 17419 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.849500 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:37.840996 17568 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:37.868610 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1851) took 2508 ms. Trace:
I20260709 12:22:37.876917 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:37.880654 14418 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:37.830369 17581 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:22:37.847551 17458 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:37.855417 17533 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:37.882777 13799 rpcz_store.cc:276] 0709 12:22:35.360573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.360674 (+   101us) service_pool.cc:224] Handling call
0709 12:22:35.360954 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:35.383263 (+ 22309us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:36.062701 (+679438us) log.cc:844] Serialized 14 byte log entry
0709 12:22:36.369350 (+306649us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:36.369430 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:36.369494 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.868281 (+1498787us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:37.868350 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:37.868527 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":756877,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":54,"raft.queue_time_us":27426,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":144,"spinlock_wait_cycles":50774400,"thread_start_us":968656,"threads_started":2,"wal-append.queue_time_us":437007,"wal-append.run_cpu_time_us":1497,"wal-append.run_wall_time_us":1043710}
I20260709 12:22:37.887854 14417 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:37.871181 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:37.849236 17519 consensus_queue.cc:1243] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:22:37.891172 14425 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 562) took 2120 ms (client timeout 1807 ms). Trace:
I20260709 12:22:37.844823 14421 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:37.858101 17557 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:22:37.898775 14425 rpcz_store.cc:269] 0709 12:22:35.759569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:35.759803 (+   234us) service_pool.cc:224] Handling call
0709 12:22:37.880104 (+2120301us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:37.880318 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398855424}
I20260709 12:22:37.856213 17419 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:37.883319 17530 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:37.901599 17533 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:37.919982 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 152891us
I20260709 12:22:37.883092 17413 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.883503 17474 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:37.880195 17503 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:37.927284 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.905037 17557 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:37.917491 17581 consensus_queue.cc:1243] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:22:37.890421 17482 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:22:37.906347 17537 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:37.975857 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:37.953091 17533 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:37.978948 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:37.980037 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:37.981288 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:37.983718 13792 pending_rounds.cc:85] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 6
I20260709 12:22:37.984548 13792 pending_rounds.cc:107] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:22:37.984877 13792 raft_consensus.cc:2889] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:37.996302 13644 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:37.979678 17537 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:37.967165 17557 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:38.109550 17537 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:38.110219 17449 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:37.986183 17564 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:22:37.961743 17474 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:37.964457 17578 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 12:22:37.958527 17530 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:37.961372 17413 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:38.150094 17593 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:22:37.965478 17503 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:38.203410 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 435559us
W20260709 12:22:38.298417 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 421628us
I20260709 12:22:38.299348 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:38.300518 14416 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:22:38.301323 17503 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:38.301548 17486 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:38.302335 17486 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:38.302845 17486 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:38.303695 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:38.305030 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:38.228693 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:38.308799 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:38.230013 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 358913us
I20260709 12:22:38.353685 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:38.357437 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52454.7us
W20260709 12:22:38.386363 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59630.2us
I20260709 12:22:38.392186 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:38.417551 13692 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.020s	sys 0.023s
W20260709 12:22:38.418550 13692 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.020s	sys 0.023s
W20260709 12:22:38.417472 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 228311us
I20260709 12:22:38.400003 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:38.420209 13726 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:38.420784 13801 raft_consensus.cc:2410] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:22:38.390132 17503 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.425429 17503 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.427882 13794 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:38.428196 17503 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:38.428499 13794 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:38.429554 13794 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.434753 13726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "255f10165e844f2cb50d654996d639cf"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:22:38.435635 13726 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:38.436134 13726 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:38.433243 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.439381 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.391314 17346 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:38.445796 13721 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:38.397176 17323 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.447022 13722 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:38.454212 13721 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:38.455224 17530 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:38.456632 17423 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.832s	user 0.011s	sys 0.000s
I20260709 12:22:38.456907 13724 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:38.457357 17423 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:38.458580 17596 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:22:38.401459 17559 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:38.467226 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 78824.9us
W20260709 12:22:38.468530 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:38.466287 13726 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.392117 17553 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:22:38.398716 17482 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:22:38.479764 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.481974 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 703) took 1966 ms. Trace:
I20260709 12:22:38.446713 17606 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:38.483244 14424 rpcz_store.cc:276] 0709 12:22:36.515222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.515364 (+   142us) service_pool.cc:224] Handling call
0709 12:22:36.534717 (+ 19353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:36.598482 (+ 63765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.481670 (+1883188us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400644bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:38.481878 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55037184}
I20260709 12:22:38.486311 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.489071 13726 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:38.498884 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 189240us
I20260709 12:22:38.506963 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.508090 13726 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:38.518806 13724 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:22:38.467087 17564 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:22:38.488489 17606 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.462759 17323 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:38.526077 17601 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:22:38.522300 17346 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:38.527498 17346 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.529937 13722 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:38.532976 13725 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:22:38.559602 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 167156us
I20260709 12:22:38.561213 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:38.548677 17190 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:38.546428 17606 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:38.504365 17605 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:22:38.569603 17530 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:38.636915 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:38.715984 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2353) took 1609 ms. Trace:
I20260709 12:22:38.716424 13648 rpcz_store.cc:276] 0709 12:22:37.106952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.228722 (+121770us) service_pool.cc:224] Handling call
0709 12:22:37.229214 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.284108 (+ 54894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.284164 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:37.284210 (+    46us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:37.284254 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.715690 (+1431436us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b54001e6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:38.715743 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:38.715915 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125681024}
I20260709 12:22:38.637697 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:38.637969 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:38.689791 13721 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:22:38.564857 17602 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:38.589669 17583 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.735s	user 0.012s	sys 0.000s
I20260709 12:22:38.682657 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:38.715171 17485 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.859s	user 0.016s	sys 0.000s
W20260709 12:22:38.597151 17568 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.003s
W20260709 12:22:38.726625 17533 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.623s	user 0.025s	sys 0.003s
W20260709 12:22:38.752449 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 114634us
I20260709 12:22:38.755462 14421 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
W20260709 12:22:38.601224 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:38.753183 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.756524 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:38.757536 13725 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:38.759002 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:38.754695 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:38.762353 13721 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.570984 17413 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.760526 17618 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:38.787261 17618 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:38.770818 17323 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:38.773573 17458 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.002s
I20260709 12:22:38.789578 17458 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.792322 17458 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.793411 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:38.795135 17458 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:38.795435 14417 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:22:38.797111 17436 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:38.797263 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:38.797609 17436 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:38.797943 14416 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:38.798666 17436 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.758665 17568 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.799770 17458 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:38.760941 17486 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.801302 17568 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.801642 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2358) took 1168 ms. Trace:
I20260709 12:22:38.801748 17486 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:38.765476 17537 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.001s
I20260709 12:22:38.754292 17617 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:38.802726 17537 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.803529 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:38.804559 13795 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:22:38.805064 13795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1882) took 1622 ms (client timeout 1834 ms). Trace:
I20260709 12:22:38.805233 17537 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:38.805608 17419 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.850s	user 0.013s	sys 0.003s
I20260709 12:22:38.806160 17419 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:38.806960 17458 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.810336 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.810868 17606 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.811492 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2351) took 1706 ms. Trace:
I20260709 12:22:38.811878 13646 rpcz_store.cc:276] 0709 12:22:37.104958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.134814 (+ 29856us) service_pool.cc:224] Handling call
0709 12:22:37.145384 (+ 10570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.363264 (+217880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.811144 (+1447880us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400096854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:38.811418 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498827520}
I20260709 12:22:38.799871 17436 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.812961 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.814096 14421 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:22:38.814652 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.802040 13643 rpcz_store.cc:276] 0709 12:22:37.633081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.633342 (+   261us) service_pool.cc:224] Handling call
0709 12:22:37.633677 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.643156 (+  9479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.643196 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:37.643238 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:37.643285 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.801394 (+1158109us) spinlock_profiling.cc:243] Waited 9.26 ms on lock 0x7b5400114c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:38.801443 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:38.801578 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21293312}
W20260709 12:22:38.805490 13795 rpcz_store.cc:269] 0709 12:22:37.183082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.252325 (+ 69243us) service_pool.cc:224] Handling call
0709 12:22:38.803308 (+1550983us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:38.804984 (+  1676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3876608}
I20260709 12:22:38.816181 13794 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:38.735496 17602 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.840657 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:38.840819 13794 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:22:38.817517 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.818432 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.842417 13643 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:38.817941 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.813360 13798 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:38.813036 14416 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:38.778347 13721 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:38.838708 17323 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.846397 17323 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.849471 17323 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:38.596031 17423 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:38.813151 17606 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:38.790126 17557 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.002s
W20260709 12:22:38.855213 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93768.5us
I20260709 12:22:38.853039 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.855007 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:38.858708 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 725) took 2361 ms. Trace:
I20260709 12:22:38.859330 14420 rpcz_store.cc:276] 0709 12:22:36.491828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.491997 (+   169us) service_pool.cc:224] Handling call
0709 12:22:36.492441 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:36.834385 (+341944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:36.834422 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:36.834494 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:36.834539 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.852861 (+2018322us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b54006a72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:38.852922 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:38.853069 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25265,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7903,"spinlock_wait_cycles":2744064,"thread_start_us":333625,"threads_started":1}
I20260709 12:22:38.861591 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 719) took 2587 ms. Trace:
I20260709 12:22:38.862125 14418 rpcz_store.cc:276] 0709 12:22:36.266661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.266815 (+   154us) service_pool.cc:224] Handling call
0709 12:22:37.880435 (+1613620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.885299 (+  4864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.853897 (+968598us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400515ed0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:38.854066 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3710107008}
I20260709 12:22:38.845355 17602 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:38.891801 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:38.865518 17557 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.759665 17485 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.572139 17190 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.760203 17533 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:38.896552 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1812) took 2034 ms (client timeout 1610 ms). Trace:
W20260709 12:22:38.897064 13651 rpcz_store.cc:269] 0709 12:22:36.862609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.881254 (+ 18645us) service_pool.cc:224] Handling call
0709 12:22:38.896336 (+2015082us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:38.896466 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590735104}
I20260709 12:22:38.897842 17540 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:38.898267 17540 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:38.899206 17540 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.900141 14423 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEARNER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:38.900887 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 729) took 1271 ms. Trace:
I20260709 12:22:38.901237 17540 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:38.901206 14423 rpcz_store.cc:276] 0709 12:22:37.629306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.899542 (+1270236us) service_pool.cc:224] Handling call
0709 12:22:38.899892 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:38.900619 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.900827 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:38.904165 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2342) took 1813 ms. Trace:
I20260709 12:22:38.904577 13649 rpcz_store.cc:276] 0709 12:22:37.090521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.092978 (+  2457us) service_pool.cc:224] Handling call
0709 12:22:38.902565 (+1809587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:38.902785 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:38.902812 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:38.902871 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:38.902919 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.902975 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:38.903965 (+   990us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b5400197b50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:38.904099 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4140929920}
I20260709 12:22:38.907464 13800 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.756778 17583 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.891040 17537 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:38.854339 17202 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:22:38.887784 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:38.916759 13648 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:38.916405 17537 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.918756 17537 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.887401 13721 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:22:38.888453 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3871681649998209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2828s for a response
W20260709 12:22:38.925159 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3499961700003951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3008s for a response
I20260709 12:22:38.889370 13725 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:38.890085 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:38.889542 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:38.892459 14425 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:38.895146 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.930212 13720 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:38.915267 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:38.931084 13648 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:38.932014 13724 raft_consensus.cc:2410] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:38.934882 13645 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:38.935359 13798 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:38.936331 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1872) took 1784 ms. Trace:
I20260709 12:22:38.936429 13726 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:38.924291 17474 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.937879 13798 rpcz_store.cc:276] 0709 12:22:37.152160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.152324 (+   164us) service_pool.cc:224] Handling call
0709 12:22:38.813035 (+1660711us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:22:38.813235 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:38.935796 (+122561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.936262 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4575488}
I20260709 12:22:38.948071 17485 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:38.944221 17530 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:38.944334 17606 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:38.950986 17530 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:38.951586 17606 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:38.951982 17606 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:38.952725 17503 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.959981 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:38.960834 14423 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:38.970822 17413 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:22:38.971428 17413 raft_consensus.cc:1068] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:22:38.958379 17474 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.972553 17474 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.975013 17474 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.962055 17485 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:38.958688 17503 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:38.975907 17485 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:38.977025 17595 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.617s	user 0.000s	sys 0.002s
I20260709 12:22:38.978843 17485 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.979527 17583 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.982721 17190 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:38.934902 17419 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:38.977977 17503 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:39.017947 17440 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:39.023281 17553 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:39.023890 17553 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:39.024796 17553 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:39.025517 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 525592us
W20260709 12:22:39.026379 17599 consensus_queue.cc:1175] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dafe08a4f404502822a0732fe570de4" responder_term: 11 status { last_received { term: 6 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: [8.9-10.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:39.048298 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 464) took 1090 ms. Trace:
I20260709 12:22:39.048723 17529 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:22:39.048823 13797 rpcz_store.cc:276] 0709 12:22:37.973711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.974180 (+   469us) service_pool.cc:224] Handling call
0709 12:22:38.096992 (+122812us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:38.097476 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:38.097526 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:38.118708 (+ 21182us) log.cc:844] Serialized 26 byte log entry
0709 12:22:38.423424 (+304716us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:38.465835 (+ 42411us) log.cc:844] Serialized 14 byte log entry
0709 12:22:38.996938 (+531103us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18003d7a40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:22:38.997153 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:38.997237 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:38.997280 (+    43us) raft_consensus.cc:1685] finished
0709 12:22:39.042603 (+ 45323us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.048219 (+  5616us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2782,"prepare.queue_time_us":43120,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":545392,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":3678,"spinlock_wait_cycles":2925952,"thread_start_us":177148,"threads_started":1,"wal-append.queue_time_us":299654}
W20260709 12:22:39.051743 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 209720us
I20260709 12:22:39.052024 14425 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:22:39.058794 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:39.060004 13647 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:39.063573 14423 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:39.067662 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:39.069746 13644 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:39.068938 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.050691 17602 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.071262 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.048398 17553 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:39.073776 13724 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:39.075160 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:39.072549 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.061458 17323 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:39.076238 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:39.108928 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.112016 13721 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
W20260709 12:22:39.112259 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.112787 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.117050 14423 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:22:39.120095 17625 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.150s	user 0.003s	sys 0.000s
W20260709 12:22:39.120436 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.123252 14425 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:39.124081 17537 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:22:39.124588 14418 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
W20260709 12:22:39.124939 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.126143 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.066850 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2536156900000606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:22:39.126672 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0495514539998112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:22:39.126792 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.127015 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0151156900001297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 12:22:39.127303 13693 connection.cc:256] RPC call timeout handler was delayed by -1.01175145399975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 12:22:39.127566 13693 connection.cc:256] RPC call timeout handler was delayed by -1.01175145399975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260709 12:22:39.086886 17589 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.479s	user 0.010s	sys 0.009s
I20260709 12:22:39.128252 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:39.077438 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:39.131028 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:39.132913 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.133472 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:39.134218 13798 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
W20260709 12:22:39.077566 17570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.003s	sys 0.017s
I20260709 12:22:39.085485 17530 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:39.139246 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:39.139770 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:39.140070 17630 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:39.140486 13647 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:39.140784 17630 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.142295 17602 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:39.142448 17569 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.137346 17598 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:39.143019 17569 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.143483 17598 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:39.143900 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:39.145175 17549 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:39.145493 17419 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:39.146250 17419 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:39.148391 17533 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:39.150436 17568 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:39.150990 17568 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.151563 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:39.152443 17630 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:39.153916 17605 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:37.768144 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:39.154820 17605 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:39.155871 17605 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.156175 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:39.156924 17557 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:39.160501 17436 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:39.161232 17436 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:39.162576 17630 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:39.163645 17630 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:39.164574 17630 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.167591 17610 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:22:39.168941 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 91583.2us
I20260709 12:22:39.169555 17449 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:39.169828 17549 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:39.145258 17569 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:39.187052 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.145542 17598 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:39.169697 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:39.086615 17577 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:39.173570 13798 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:22:39.174436 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:39.198498 13798 pending_rounds.cc:85] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 4
I20260709 12:22:39.198837 13798 pending_rounds.cc:107] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:22:39.175999 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.199154 13798 raft_consensus.cc:2889] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:39.199604 13718 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:39.135842 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:39.189112 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:39.200387 14418 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:39.200520 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:39.190752 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:39.201459 13798 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:39.192070 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:39.193109 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:39.161003 13547 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 3 to 4, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 4 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.194801 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:39.153470 17568 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:39.198715 13725 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:39.136209 13645 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:39.203394 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:39.138222 17570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.003s	sys 0.018s
I20260709 12:22:39.076148 17562 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 12:22:39.207777 17503 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:39.403492 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 574) took 1876 ms. Trace:
W20260709 12:22:39.335603 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 180386us
I20260709 12:22:39.223105 13798 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:39.251070 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:39.433979 13547 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 3 to 4, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 4 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:39.134248 17323 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:39.407567 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 207475us
I20260709 12:22:39.295742 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:39.340008 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:39.250352 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.218171 13719 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:22:39.255935 13797 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:22:39.185745 17423 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:39.273774 13644 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:39.347584 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173435us
W20260709 12:22:39.238050 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43752.2us
W20260709 12:22:39.182888 17615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.009s	sys 0.018s
I20260709 12:22:39.148437 17419 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:39.556272 17503 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:39.636147 17503 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.568239 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.639081 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.639878 13724 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:39.641012 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:39.569569 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.569869 13797 pending_rounds.cc:85] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 5
I20260709 12:22:39.644245 13797 pending_rounds.cc:107] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:22:39.559384 13723 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:39.644567 13797 raft_consensus.cc:2889] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:39.656261 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:39.658056 13797 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:39.666648 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.669840 13725 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:39.671727 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:39.674142 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.573593 17423 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:39.574088 17557 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:39.682619 17557 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:39.680110 13725 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:39.686300 17557 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:39.688936 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:39.575521 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.444806 17623 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:22:39.605439 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:39.251464 17482 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:22:39.486408 17633 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
W20260709 12:22:39.381049 17621 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:22:39.556885 14419 rpcz_store.cc:276] 0709 12:22:37.526871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.527090 (+   219us) service_pool.cc:224] Handling call
0709 12:22:37.533644 (+  6554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.534049 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.534074 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:37.534620 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:22:37.587938 (+ 53318us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:37.587978 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:37.588215 (+   237us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.085674 (+1497459us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:39.364657 (+278983us) raft_consensus.cc:1685] finished
0709 12:22:39.364680 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.403371 (+ 38691us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7691,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":13592960,"thread_start_us":52988,"threads_started":1,"wal-append.queue_time_us":73219}
W20260709 12:22:39.624466 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 326236us
W20260709 12:22:39.617856 17553 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.003s
I20260709 12:22:39.573947 17533 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:39.779692 17533 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.730094 17616 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:39.783140 17616 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.783655 17616 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:39.208324 17578 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.532s
I20260709 12:22:39.576714 17540 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:39.576531 17550 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:39.297449 17436 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:39.574584 17634 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
W20260709 12:22:39.741004 17639 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 12:22:39.558452 17606 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.594s	user 0.013s	sys 0.000s
I20260709 12:22:39.297772 17577 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:39.578449 17615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.018s	sys 0.036s
I20260709 12:22:39.508054 17549 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:39.847172 17606 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.863299 17606 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:39.866151 17553 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:39.867082 17553 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:39.868408 17553 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:39.565528 13547 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 1 to 2, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:40.147219 17553 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:22:39.928058 13550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 251) took 23656 ms (client timeout 15000 ms). Trace:
I20260709 12:22:39.800568 17550 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:39.945727 13617 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.006s	sys 0.030s
W20260709 12:22:40.154651 13617 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.006s	sys 0.030s
W20260709 12:22:40.155608 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 888407us
I20260709 12:22:39.566900 13649 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:39.563259 13650 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:39.948287 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 389718us
I20260709 12:22:39.950410 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 569) took 1093 ms. Trace:
I20260709 12:22:39.959661 13643 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:39.526149 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:39.678143 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.970731 13720 raft_consensus.cc:2368] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
I20260709 12:22:39.693069 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.973404 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.556352 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.978181 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:39.680864 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.985806 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:39.610661 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:39.700700 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129233us
W20260709 12:22:39.995432 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 389874us
W20260709 12:22:39.605599 13792 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.039s	sys 0.018s
I20260709 12:22:39.996930 13794 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:22:39.999173 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 475) took 1437 ms. Trace:
W20260709 12:22:40.001695 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.002429 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.006026 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:39.561069 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.354429 13798 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:40.355592 13723 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:40.357285 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:39.569873 13800 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.358506 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2533) took 1083 ms. Trace:
I20260709 12:22:40.358961 13727 rpcz_store.cc:276] 0709 12:22:39.274882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.555999 (+281117us) service_pool.cc:224] Handling call
0709 12:22:40.358288 (+802289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.360781 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.012272 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 463) took 2054 ms. Trace:
I20260709 12:22:40.016453 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.365479 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2656) took 1186 ms. Trace:
I20260709 12:22:40.365523 13719 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:39.442633 17277 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:22:40.366976 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.001s	sys 0.023s
I20260709 12:22:40.366847 17277 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:40.367419 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.001s	sys 0.023s
I20260709 12:22:39.566972 17323 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:40.368031 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.16464e+06us
I20260709 12:22:40.059389 17395 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:39.742645 17423 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.368898 17395 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:39.849143 17577 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.369568 17395 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.063952 17190 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:40.372030 17190 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:40.126514 17440 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.374713 17440 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:40.377557 17440 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:40.378227 17578 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.379063 17578 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:40.379467 17606 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:39.800041 17540 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:40.144150 17503 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.010s	sys 0.001s
W20260709 12:22:40.144649 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.153904 17550 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:40.161628 13646 rpcz_store.cc:276] 0709 12:22:38.856599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.856779 (+   180us) service_pool.cc:224] Handling call
0709 12:22:38.886746 (+ 29967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.049851 (+163105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.049881 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.052916 (+  3035us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.133971 (+ 81055us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:22:39.134013 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.134088 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.949445 (+815357us) raft_consensus.cc:1685] finished
0709 12:22:39.949510 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.949998 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":161996,"raft.queue_time_us":162832,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":149,"thread_start_us":56824,"threads_started":1,"wal-append.queue_time_us":60829}
W20260709 12:22:40.383013 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.564s (SENT) [suppressed 29 similar messages]
W20260709 12:22:40.150671 13550 rpcz_store.cc:269] 0709 12:22:16.271833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:16.273242 (+  1409us) service_pool.cc:224] Handling call
0709 12:22:39.927546 (+23654304us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:32.054704 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:32.055498 (+   794us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:32.055555 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:22:32.055593 (+    38us) tablet.cc:662] Decoding operations
0709 12:22:32.200532 (+144939us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:32.250739 (+ 50207us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:32.250775 (+    36us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:22:32.322639 (+ 71864us) tablet.cc:701] Row locks acquired
0709 12:22:32.368233 (+ 45594us) write_op.cc:260] PREPARE: finished
0709 12:22:32.394378 (+ 26145us) write_op.cc:270] Start()
0709 12:22:32.394566 (+   188us) write_op.cc:276] Timestamp: P: 1783599752381657 usec, L: 0
0709 12:22:32.394590 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:22:32.510232 (+115642us) log.cc:844] Serialized 9359 byte log entry
0709 12:22:32.670132 (+159900us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b
0709 12:22:32.967130 (+296998us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:33.260950 (+293820us) op_driver.cc:464] REPLICATION: finished
0709 12:22:33.469672 (+208722us) write_op.cc:301] APPLY: starting
0709 12:22:33.469819 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:33.470421 (+   602us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:33.470468 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:33.473502 (+  3034us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:33.476239 (+  2737us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=24
0709 12:22:33.476298 (+    59us) write_op.cc:312] APPLY: finished
0709 12:22:33.477131 (+   833us) log.cc:844] Serialized 164 byte log entry
0709 12:22:33.500421 (+ 23290us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:33.562746 (+ 62325us) write_op.cc:454] Released schema lock
0709 12:22:33.675162 (+112416us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:33.729482 (+ 54320us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:33.733378 (+  3896us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":582417,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":175669,"apply.run_cpu_time_us":15974,"apply.run_wall_time_us":268111,"num_ops":24,"prepare.queue_time_us":178666,"prepare.run_cpu_time_us":35401,"prepare.run_wall_time_us":851830,"raft.queue_time_us":278951,"raft.run_cpu_time_us":20792,"raft.run_wall_time_us":147834,"replication_time_us":866130,"spinlock_wait_cycles":138504320,"thread_start_us":826428,"threads_started":4,"wal-append.queue_time_us":426777,"wal-append.run_cpu_time_us":51187,"wal-append.run_wall_time_us":1588065}]]}
I20260709 12:22:40.386677 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2661) took 1204 ms. Trace:
I20260709 12:22:40.286127 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 742) took 1094 ms. Trace:
I20260709 12:22:40.161257 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:40.306236 17600 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:40.306659 17611 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.313457 17533 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:40.314076 17583 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.003s	sys 0.014s
W20260709 12:22:40.314671 13726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.005s	sys 0.011s
I20260709 12:22:40.319444 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2375) took 1253 ms. Trace:
I20260709 12:22:40.399641 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2530) took 1178 ms. Trace:
I20260709 12:22:40.362037 13727 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:40.400218 13643 rpcz_store.cc:276] 0709 12:22:39.066503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.067673 (+  1170us) service_pool.cc:224] Handling call
0709 12:22:39.958598 (+890925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.257165 (+298567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.319157 (+ 61992us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54001487d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:40.319357 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047739648}
I20260709 12:22:40.387606 14420 rpcz_store.cc:276] 0709 12:22:39.191462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.191587 (+   125us) service_pool.cc:224] Handling call
0709 12:22:39.191896 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.192127 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.192152 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.192662 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.621378 (+428716us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:39.621444 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.621562 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:40.285300 (+663738us) raft_consensus.cc:1685] finished
0709 12:22:40.285357 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.285780 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":428320,"threads_started":1,"wal-append.queue_time_us":934751}
W20260709 12:22:40.260927 17449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.021s	sys 0.033s
I20260709 12:22:40.333338 13794 pending_rounds.cc:85] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 9
I20260709 12:22:40.411568 13794 pending_rounds.cc:107] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:22:40.412082 13794 raft_consensus.cc:2889] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:40.338112 13795 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.348779 14419 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:40.350682 13725 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:40.365996 13801 rpcz_store.cc:276] 0709 12:22:39.179622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.015699 (+836077us) service_pool.cc:224] Handling call
0709 12:22:40.365334 (+349635us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.283780 17458 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.088s
W20260709 12:22:40.126176 17605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.056s	sys 0.176s
W20260709 12:22:40.386094 13792 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.300s	user 0.040s	sys 0.018s
I20260709 12:22:40.387116 13796 rpcz_store.cc:276] 0709 12:22:39.181881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.004902 (+823021us) service_pool.cc:224] Handling call
0709 12:22:40.386606 (+381704us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:40.388051 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 227048us
I20260709 12:22:40.402304 13724 rpcz_store.cc:276] 0709 12:22:39.221332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.691981 (+470649us) service_pool.cc:224] Handling call
0709 12:22:40.342889 (+650908us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.405813 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:40.408419 13726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.006s	sys 0.011s
I20260709 12:22:40.410777 13718 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:40.313971 17549 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
W20260709 12:22:40.410058 17583 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.004s	sys 0.014s
I20260709 12:22:40.362527 13799 rpcz_store.cc:276] 0709 12:22:37.957511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.957686 (+   175us) service_pool.cc:224] Handling call
0709 12:22:37.974463 (+ 16777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:37.977018 (+  2555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:37.977046 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:37.977575 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.347916 (+1370341us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b1800187180. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:39.624844 (+276928us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:39.669103 (+ 44259us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.011113 (+342010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.011320 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:40.011442 (+   122us) raft_consensus.cc:1685] finished
0709 12:22:40.011664 (+   222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.012017 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":343272,"prepare.queue_time_us":155952,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":469659,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":61471,"spinlock_wait_cycles":3475840,"thread_start_us":313010,"threads_started":2,"wal-append.queue_time_us":1891711}
W20260709 12:22:40.420241 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 224496us
I20260709 12:22:40.419710 13719 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.426543 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2538) took 1149 ms. Trace:
I20260709 12:22:40.333441 13793 rpcz_store.cc:276] 0709 12:22:38.561288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:38.561517 (+   229us) service_pool.cc:224] Handling call
0709 12:22:38.562046 (+   529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.086058 (+524012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.086090 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.087063 (+   973us) log.cc:844] Serialized 26 byte log entry
0709 12:22:39.235032 (+147969us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:39.235092 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:39.235194 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:39.997076 (+761882us) raft_consensus.cc:1685] finished
0709 12:22:39.997141 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:39.998125 (+   984us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":574374,"raft.queue_time_us":539502,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":2816,"thread_start_us":147572,"threads_started":1,"wal-append.queue_time_us":767894}
W20260709 12:22:40.420409 17605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.057s	sys 0.176s
W20260709 12:22:40.433353 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.435074 17578 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:40.442934 17540 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:40.443189 17190 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:40.444743 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.429827 13725 rpcz_store.cc:276] 0709 12:22:39.276674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.690157 (+413483us) service_pool.cc:224] Handling call
0709 12:22:40.422002 (+731845us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:40.428907 17449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.027s	sys 0.038s
I20260709 12:22:40.451320 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:40.451189 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81756us
I20260709 12:22:40.423470 17458 consensus_queue.cc:1243] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:22:40.454609 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.454757 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.426203 17600 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:40.459946 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 876654us
I20260709 12:22:40.460423 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:22:40.455654 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 710) took 1311 ms (client timeout 1674 ms). Trace:
I20260709 12:22:40.462904 13798 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:40.465534 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:40.462759 13720 rpcz_store.cc:269] 0709 12:22:39.137835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.138985 (+  1150us) service_pool.cc:224] Handling call
0709 12:22:40.424185 (+1285200us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:40.448773 (+ 24588us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1908906752}
W20260709 12:22:40.474907 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2655) took 1291 ms (client timeout 1577 ms). Trace:
I20260709 12:22:40.470430 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 479) took 1337 ms. Trace:
I20260709 12:22:40.476116 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:40.476819 13798 rpcz_store.cc:269] 0709 12:22:39.178749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.560674 (+381925us) service_pool.cc:224] Handling call
0709 12:22:40.464471 (+903797us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.478972 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.476051 13794 rpcz_store.cc:276] 0709 12:22:39.133235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.134235 (+  1000us) service_pool.cc:224] Handling call
0709 12:22:39.134561 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.412890 (+1278329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.462003 (+ 49113us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:40.470360 (+  8357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978256768}
I20260709 12:22:40.481595 13725 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:40.493301 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.499773 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.503708 17423 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.503989 13727 raft_consensus.cc:2368] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:22:40.506958 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.475631 17458 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.507925 13798 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:40.509443 13722 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:40.513695 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:40.460005 17611 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:40.509528 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.517310 13723 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:40.519418 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.520534 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2540) took 1161 ms. Trace:
I20260709 12:22:40.520962 13725 rpcz_store.cc:276] 0709 12:22:39.359348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.511186 (+1151838us) service_pool.cc:224] Handling call
0709 12:22:40.520451 (+  9265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.523178 13801 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:40.497426 17323 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:40.522135 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:40.523509 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2653) took 1339 ms. Trace:
I20260709 12:22:40.515256 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.524120 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2548) took 1002 ms. Trace:
I20260709 12:22:40.530599 13723 rpcz_store.cc:276] 0709 12:22:39.521903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.484123 (+962220us) service_pool.cc:224] Handling call
0709 12:22:40.524053 (+ 39930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.531392 13793 rpcz_store.cc:276] 0709 12:22:39.175798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.464166 (+1288368us) service_pool.cc:224] Handling call
0709 12:22:40.515082 (+ 50916us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.532281 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.532989 13796 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:40.516639 17613 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.533818 17323 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:40.535413 14388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.002s	sys 0.024s
I20260709 12:22:40.539072 13795 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:40.540411 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2654) took 1362 ms. Trace:
I20260709 12:22:40.539390 13794 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:40.542582 13720 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:40.543118 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2529) took 1322 ms. Trace:
I20260709 12:22:40.544421 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 712) took 1343 ms. Trace:
I20260709 12:22:40.543476 13720 rpcz_store.cc:276] 0709 12:22:39.220901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.482210 (+1261309us) service_pool.cc:224] Handling call
0709 12:22:40.543049 (+ 60839us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:40.547672 14388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.003s	sys 0.025s
I20260709 12:22:40.544854 13721 rpcz_store.cc:276] 0709 12:22:39.200708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.202589 (+  1881us) service_pool.cc:224] Handling call
0709 12:22:39.564386 (+361797us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:39.576528 (+ 12142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.576578 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:39.610201 (+ 33623us) log.cc:844] Serialized 52 byte log entry
0709 12:22:40.482874 (+872673us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:40.489071 (+  6197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.494773 (+  5702us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:40.543997 (+ 49224us) raft_consensus.cc:1685] finished
0709 12:22:40.544018 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.544199 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10099,"raft.queue_time_us":10401,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":137,"thread_start_us":433855,"threads_started":1,"wal-append.queue_time_us":521569}
I20260709 12:22:40.539759 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2552) took 1014 ms. Trace:
I20260709 12:22:40.549300 13725 rpcz_store.cc:276] 0709 12:22:39.523481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.521726 (+998245us) service_pool.cc:224] Handling call
0709 12:22:40.537273 (+ 15547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.541656 13795 rpcz_store.cc:276] 0709 12:22:39.177934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.001499 (+823565us) service_pool.cc:224] Handling call
0709 12:22:40.540332 (+538833us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.541357 13718 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:40.552037 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:40.542493 14424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.021s	sys 0.062s
I20260709 12:22:40.548453 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.553874 13798 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:40.554188 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.553110 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2527) took 1333 ms. Trace:
I20260709 12:22:40.556923 13718 rpcz_store.cc:276] 0709 12:22:39.220078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.455542 (+1235464us) service_pool.cc:224] Handling call
0709 12:22:40.553032 (+ 97490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.558169 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.565320 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.561872 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.561012 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:40.572886 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2536) took 1296 ms. Trace:
I20260709 12:22:40.563325 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:40.555168 13618 connection.cc:256] RPC call timeout handler was delayed by -1.1674097869999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 12:22:40.562063 14424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.021s	sys 0.063s
W20260709 12:22:40.570195 13797 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.838s	user 0.009s	sys 0.015s
I20260709 12:22:40.591311 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:40.594099 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 211131us
I20260709 12:22:40.594233 13801 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:40.592365 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.573499 13722 rpcz_store.cc:276] 0709 12:22:39.276083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.529030 (+1252947us) service_pool.cc:224] Handling call
0709 12:22:40.572784 (+ 43754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.588040 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2544) took 1067 ms. Trace:
I20260709 12:22:40.596884 13650 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:40.596872 13725 rpcz_store.cc:276] 0709 12:22:39.520219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.553476 (+1033257us) service_pool.cc:224] Handling call
0709 12:22:40.587970 (+ 34494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.598348 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.600085 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2555) took 1075 ms. Trace:
I20260709 12:22:40.600500 13723 rpcz_store.cc:276] 0709 12:22:39.525092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.531979 (+1006887us) service_pool.cc:224] Handling call
0709 12:22:40.600000 (+ 68021us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.598546 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:40.594414 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2545) took 1073 ms. Trace:
I20260709 12:22:40.602784 13718 rpcz_store.cc:276] 0709 12:22:39.520540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.557750 (+1037210us) service_pool.cc:224] Handling call
0709 12:22:40.594335 (+ 36585us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.611204 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:40.613677 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1832) took 1414 ms (client timeout 1673 ms). Trace:
I20260709 12:22:40.607184 13797 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
W20260709 12:22:40.614104 13650 rpcz_store.cc:269] 0709 12:22:39.192060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.192229 (+   169us) service_pool.cc:224] Handling call
0709 12:22:40.603448 (+1411219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.555670 17458 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.618691 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.595312 17190 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:40.624976 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2658) took 1444 ms (client timeout 1858 ms). Trace:
I20260709 12:22:40.618655 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:40.626757 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2532) took 1404 ms. Trace:
W20260709 12:22:40.625710 13797 rpcz_store.cc:269] 0709 12:22:39.180501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.655663 (+475162us) service_pool.cc:224] Handling call
0709 12:22:40.617093 (+961430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.627635 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:40.624861 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.629968 13793 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:40.630683 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.633349 13799 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9.
I20260709 12:22:40.627020 13727 rpcz_store.cc:276] 0709 12:22:39.222186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.531722 (+1309536us) service_pool.cc:224] Handling call
0709 12:22:40.626703 (+ 94981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.634769 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.631651 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:40.643056 13795 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:40.642654 13724 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:40.648128 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2531) took 1426 ms. Trace:
I20260709 12:22:40.648586 13724 rpcz_store.cc:276] 0709 12:22:39.221815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.553576 (+1331761us) service_pool.cc:224] Handling call
0709 12:22:40.648033 (+ 94457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.647294 13720 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:40.652236 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2528) took 1431 ms. Trace:
I20260709 12:22:40.652671 13720 rpcz_store.cc:276] 0709 12:22:39.220426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.559275 (+1338849us) service_pool.cc:224] Handling call
0709 12:22:40.652151 (+ 92876us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.655309 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.659168 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.669360 17486 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:40.669150 17190 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.670377 17664 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 12:22:40.670935 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130018us
I20260709 12:22:40.672233 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:40.669171 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:40.675633 14416 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:40.658425 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.676962 13797 raft_consensus.cc:2376] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9. Re-sending same reply.
I20260709 12:22:40.686969 13725 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:40.669405 17600 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:40.686043 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 281294us
I20260709 12:22:40.601363 17323 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:40.618204 17613 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:40.688532 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:40.690939 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 717) took 1156 ms. Trace:
I20260709 12:22:40.689554 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:40.698130 13720 rpcz_store.cc:276] 0709 12:22:39.534410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.656658 (+1122248us) service_pool.cc:224] Handling call
0709 12:22:40.690157 (+ 33499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.690518 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.690554 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.690605 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:40.690649 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.690708 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.690856 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.688392 13722 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.705152 17562 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:40.705216 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:40.705384 14418 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:40.706784 13727 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:40.707574 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2539) took 1430 ms. Trace:
I20260709 12:22:40.708035 13727 rpcz_store.cc:276] 0709 12:22:39.276951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.638595 (+1361644us) service_pool.cc:224] Handling call
0709 12:22:40.706306 (+ 67711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.707222 (+   916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.707421 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.709514 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.710628 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.711287 13795 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:40.711951 13795 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:40.713258 13794 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:40.714470 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.715236 13794 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:22:40.716226 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.711478 13797 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
W20260709 12:22:40.711556 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.676105 17611 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:40.696812 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2525) took 1468 ms. Trace:
I20260709 12:22:40.651222 17395 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.689908 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:40.718150 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 467) took 2707 ms. Trace:
I20260709 12:22:40.722126 13725 rpcz_store.cc:276] 0709 12:22:39.219580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.603274 (+1383694us) service_pool.cc:224] Handling call
0709 12:22:40.646778 (+ 43504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.687543 (+ 40765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.687828 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.713251 13795 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:40.721767 13727 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:40.734828 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.654357 17583 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:22:40.735989 13798 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 9.
I20260709 12:22:40.737362 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.738236 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.742527 17148 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEARNER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:40.743232 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 583) took 1549 ms. Trace:
I20260709 12:22:40.744170 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.713308 17436 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:40.745193 17436 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.745568 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.748317 17436 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:40.749651 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2516) took 1576 ms. Trace:
I20260709 12:22:40.743688 14424 rpcz_store.cc:276] 0709 12:22:39.193625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.193750 (+   125us) service_pool.cc:224] Handling call
0709 12:22:39.250397 (+ 56647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:39.701306 (+450909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:39.701344 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:39.701864 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:22:40.655872 (+954008us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:40.681878 (+ 26006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.689691 (+  7813us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:40.742962 (+ 53271us) raft_consensus.cc:1685] finished
0709 12:22:40.742986 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.743154 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":726570,"raft.queue_time_us":463147,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":11993,"thread_start_us":888655,"threads_started":1,"wal-append.queue_time_us":812214}
I20260709 12:22:40.750620 17277 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.751268 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:40.751693 17440 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:40.752197 17440 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:40.752272 14421 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:40.752974 17530 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:40.753458 17440 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.755075 14419 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:22:40.766608 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 94560.9us
W20260709 12:22:40.755558 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 200150us
I20260709 12:22:40.768116 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:40.768438 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:40.769428 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:40.769791 17653 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:22:40.756408 17630 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:40.733440 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:40.728853 17611 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:40.737823 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2537) took 1461 ms. Trace:
I20260709 12:22:40.761096 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.761101 17606 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:40.762712 13795 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:40.727871 13792 rpcz_store.cc:276] 0709 12:22:37.979863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:37.982944 (+  3081us) service_pool.cc:224] Handling call
0709 12:22:37.983419 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:38.065492 (+ 82073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:38.065529 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:38.297401 (+231872us) log.cc:844] Serialized 52 byte log entry
0709 12:22:38.540877 (+243476us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:38.557322 (+ 16445us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.427654 (+1870332us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.654192 (+226538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.657855 (+  3663us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:40.675798 (+ 17943us) raft_consensus.cc:1685] finished
0709 12:22:40.675817 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.686682 (+ 10865us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":533156,"prepare.queue_time_us":706691,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":149,"raft.queue_time_us":536009,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":19129,"thread_start_us":1601664,"threads_started":3,"wal-append.queue_time_us":196849,"wal-append.run_cpu_time_us":2240,"wal-append.run_wall_time_us":1024878}
I20260709 12:22:40.765008 17148 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:40.785529 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.786083 13719 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:40.745647 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:40.786916 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2549) took 1264 ms. Trace:
I20260709 12:22:40.787739 13720 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
I20260709 12:22:40.780704 13721 rpcz_store.cc:276] 0709 12:22:39.276370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.622722 (+1346352us) service_pool.cc:224] Handling call
0709 12:22:40.686644 (+ 63922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.737443 (+ 50799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.737490 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.737546 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:40.737592 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.737652 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.737766 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.736285 17600 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:40.752338 17277 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.789048 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:40.780481 14416 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:40.787357 13719 rpcz_store.cc:276] 0709 12:22:39.522241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.366680 (+844439us) service_pool.cc:224] Handling call
0709 12:22:40.786840 (+420160us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:40.801807 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:40.804474 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 114739us
I20260709 12:22:40.833971 13795 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:40.795567 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.794622 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 718) took 1259 ms. Trace:
I20260709 12:22:40.808964 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2546) took 1288 ms. Trace:
I20260709 12:22:40.794752 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.851528 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:40.875458 13721 rpcz_store.cc:276] 0709 12:22:39.534861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.793438 (+1258577us) service_pool.cc:224] Handling call
0709 12:22:40.793927 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.794171 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.794196 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.794252 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:40.794348 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.794414 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.794525 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.853971 13796 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:40.876940 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2659) took 1696 ms. Trace:
I20260709 12:22:40.877403 13796 rpcz_store.cc:276] 0709 12:22:39.180961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.473446 (+1292485us) service_pool.cc:224] Handling call
0709 12:22:40.876864 (+403418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.802506 17440 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:40.869251 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:40.801368 17553 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.011s
I20260709 12:22:40.750137 13726 rpcz_store.cc:276] 0709 12:22:39.172875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.173039 (+   164us) service_pool.cc:224] Handling call
0709 12:22:39.186231 (+ 13192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:39.186559 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:39.633653 (+447094us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.517326 (+883673us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.536367 (+ 19041us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.569244 (+ 32877us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b180047aa00. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:40.748823 (+179579us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b180047aa00. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:22:40.749123 (+   300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.749201 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:40.749283 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.749429 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.749592 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217718,"prepare.queue_time_us":616280,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":157,"raft.queue_time_us":236683,"raft.run_cpu_time_us":627,"raft.run_wall_time_us":31299,"spinlock_wait_cycles":14929024,"thread_start_us":1171375,"threads_started":2,"wal-append.queue_time_us":826386}
W20260709 12:22:40.793912 17616 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.009s	user 0.016s	sys 0.008s
I20260709 12:22:40.802083 17658 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:40.879091 14420 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:40.920178 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.921288 13797 raft_consensus.cc:2376] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7. Re-sending same reply.
I20260709 12:22:40.875553 13724 rpcz_store.cc:276] 0709 12:22:39.521020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.654669 (+1133649us) service_pool.cc:224] Handling call
0709 12:22:40.726790 (+ 72121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.808359 (+ 81569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.808406 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.808496 (+    90us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:40.808588 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.808683 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.808860 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.868098 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:40.794629 17577 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:40.794410 17530 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:40.803403 17606 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:22:40.881861 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.803805 17669 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:40.883945 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.803958 17613 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:40.850040 17458 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:40.925668 13722 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:40.932580 13794 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 6.
I20260709 12:22:40.932849 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:40.936375 17578 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:40.936761 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2652) took 1761 ms. Trace:
I20260709 12:22:40.930050 17530 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:40.938336 17606 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:40.938580 14425 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:40.946313 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.799860 17611 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:40.937371 13796 rpcz_store.cc:276] 0709 12:22:39.175545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.883459 (+1707914us) service_pool.cc:224] Handling call
0709 12:22:40.935630 (+ 52171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.936214 (+   584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:40.936253 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.936345 (+    92us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:40.936386 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.936452 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.936665 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:40.917204 17658 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:40.851873 17533 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:22:40.937242 17578 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:40.956611 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 198384us
I20260709 12:22:40.945101 17530 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:40.948045 17606 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:40.973816 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 268700us
W20260709 12:22:40.960937 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 191723us
I20260709 12:22:40.957221 13800 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:22:40.948177 17577 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:40.978279 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:40.958756 17658 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:40.989589 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:40.992857 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 713) took 1718 ms. Trace:
W20260709 12:22:40.997560 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 275809us
I20260709 12:22:40.993275 13718 rpcz_store.cc:276] 0709 12:22:39.269265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.603556 (+1334291us) service_pool.cc:224] Handling call
0709 12:22:40.623562 (+ 20006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:40.624000 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:22:40.736197 (+112197us) log.cc:844] Serialized 14 byte log entry
0709 12:22:40.987318 (+251121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:40.987382 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:22:40.987449 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.987520 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:40.987639 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321554,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"thread_start_us":204205,"threads_started":2}
I20260709 12:22:40.999154 13797 pending_rounds.cc:85] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 3
I20260709 12:22:41.011467 13797 pending_rounds.cc:107] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:22:41.011775 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 458) took 4813 ms. Trace:
I20260709 12:22:41.011940 13797 raft_consensus.cc:2889] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:41.019970 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2660) took 1838 ms. Trace:
I20260709 12:22:41.020565 13801 rpcz_store.cc:276] 0709 12:22:39.181454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:40.446249 (+1264795us) service_pool.cc:224] Handling call
0709 12:22:41.019727 (+573478us) spinlock_profiling.cc:243] Waited 67.7 ms on lock 0x7b5400103454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:41.019883 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155759872}
I20260709 12:22:40.929179 17616 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.029443 14416 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:40.992473 17458 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:41.012440 13800 rpcz_store.cc:276] 0709 12:22:36.185558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:36.185736 (+   178us) service_pool.cc:224] Handling call
0709 12:22:36.206755 (+ 21019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:37.281573 (+1074818us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54000a30a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:22:38.906957 (+1625384us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b54000a30a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b7206f 00007f5393beaf60 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62
0709 12:22:38.924096 (+ 17139us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18000390c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393c2f646 00007f5393b6958e 00007f5393beaf9e 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4
0709 12:22:39.350712 (+426616us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b58007340d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b500aa 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4
0709 12:22:40.978284 (+1627572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:40.999098 (+ 20814us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288549504}
I20260709 12:22:41.030047 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:41.024813 14420 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:22:41.043251 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47047.2us
W20260709 12:22:41.048560 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 264100us
I20260709 12:22:41.018875 17395 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:41.048883 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:41.033484 17458 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:41.039855 17533 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:41.041481 17190 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:41.058032 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:41.053443 17666 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 12:22:41.067312 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:41.059170 17533 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:41.080159 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:41.081935 13766 connection.cc:256] RPC call timeout handler was delayed by -1.2143273629999385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28900000000000003s for a response
W20260709 12:22:41.089473 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.098222 14424 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:22:41.100072 17556 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 12:22:41.110525 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124569us
I20260709 12:22:41.097754 17190 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:41.041347 17677 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:22:41.096302 17533 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:41.132145 17539 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.132927 17539 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.134229 17675 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.134974 17675 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:41.136246 17671 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
I20260709 12:22:41.011241 17669 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:41.127375 17670 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
W20260709 12:22:41.159888 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 126415us
W20260709 12:22:41.161482 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 186010us
W20260709 12:22:41.161825 17683 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
I20260709 12:22:41.137596 17395 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.164608 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:41.166925 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.167572 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:41.180029 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67142.4us
I20260709 12:22:41.094973 17578 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.191533 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:41.181480 17677 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:41.227147 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.179917 17611 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:41.232232 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48872.1us
W20260709 12:22:41.231071 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92687.4us
I20260709 12:22:41.190667 17395 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.233736 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.235481 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:41.250531 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69162.8us
I20260709 12:22:41.151419 17577 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.251816 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:41.252934 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:41.254199 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:41.190935 17458 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.265017 13648 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:41.271972 17562 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:41.272328 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.273530 13646 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
I20260709 12:22:41.273427 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:41.256600 17690 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:22:41.259199 17583 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:41.278859 13651 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:41.277400 17616 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.277658 17583 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:41.287993 17458 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:41.280133 17700 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:41.302469 17681 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:41.299055 17692 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:22:41.286695 17323 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:41.324549 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.291548 17395 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:41.335228 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:41.336421 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82340.7us
I20260709 12:22:41.325665 17583 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:41.338213 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.328542 17700 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:41.351058 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.334015 17681 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:41.341578 17677 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:41.348855 17562 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:41.346544 17700 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:41.373231 13651 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 8: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:41.382231 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:41.383054 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.377456 17669 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:41.414768 17695 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:22:41.416395 17562 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:41.418699 17277 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.545s	user 0.012s	sys 0.008s
I20260709 12:22:41.419013 13649 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:22:41.427027 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 80864.5us
I20260709 12:22:41.406316 17323 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:41.409377 17436 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:41.390034 17677 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.466714 17708 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:41.468403 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.469295 13648 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:22:41.441429 17701 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:22:41.470037 17681 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.471097 17530 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.471635 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.473657 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.443742 17700 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.475986 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:41.476859 17700 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.466730 17436 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.479543 17436 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:41.445793 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:41.445407 17323 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:41.481570 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47055.5us
I20260709 12:22:41.482515 17395 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:41.482703 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 222357us
I20260709 12:22:41.449018 17658 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.450721 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.450551 17583 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:41.459291 17698 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:22:41.495894 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:41.498853 13646 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:22:41.501997 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:41.504513 13649 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:41.516228 13651 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
W20260709 12:22:41.521941 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 262575us
I20260709 12:22:41.518151 17530 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.545878 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:41.556762 17486 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:41.557410 17486 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:41.557812 17486 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:41.558560 17630 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:22:41.559122 17630 raft_consensus.cc:1068] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:22:41.560963 17440 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.010s
I20260709 12:22:41.563270 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:41.564073 13793 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:41.568121 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1831) took 2376 ms (client timeout 2184 ms). Trace:
W20260709 12:22:41.569151 13647 rpcz_store.cc:269] 0709 12:22:39.191679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:39.191843 (+   164us) service_pool.cc:224] Handling call
0709 12:22:41.567629 (+2375786us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54001e7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:41.567827 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3397182336}
I20260709 12:22:41.592959 13767 leader_election.cc:400] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
I20260709 12:22:41.593463 13767 leader_election.cc:403] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:22:41.594573 17423 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
I20260709 12:22:41.598990 17583 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.600667 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:41.603819 17440 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:41.603672 17533 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.602141 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:41.602772 17578 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:41.606541 13644 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:41.610487 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:41.608301 17277 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:22:41.615351 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:41.616580 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:41.618458 17436 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:41.618393 17707 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:22:41.620997 17578 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:41.622052 17578 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:41.623019 17578 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:41.555073 17664 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:22:41.589265 17658 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:41.626175 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43689.2us
I20260709 12:22:41.627930 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:41.629508 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:41.648098 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:41.652387 13723 raft_consensus.cc:2368] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
W20260709 12:22:41.657337 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:41.651273 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95417.4us
W20260709 12:22:41.631768 17606 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.638s	user 0.007s	sys 0.006s
I20260709 12:22:41.660751 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:41.658834 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:41.662356 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45864.5us
I20260709 12:22:41.626467 17533 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.716313 14425 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:41.662400 17658 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.723371 17674 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.585s	user 0.000s	sys 0.004s
I20260709 12:22:41.749756 13720 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:41.688232 17707 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:41.724227 17436 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:41.775708 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:41.771816 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:41.778627 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163396us
I20260709 12:22:41.686702 17606 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.766091 17707 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:41.777041 17436 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:41.802431 17703 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:41.803143 17703 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:41.807783 17606 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.805476 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:41.793474 17530 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:41.776582 17658 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.797712 17707 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:41.830462 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:41.809808 17436 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:41.820199 17703 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:41.831203 17678 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.042s	sys 0.049s
W20260709 12:22:41.847055 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 193023us
I20260709 12:22:41.831743 17530 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:41.852491 17678 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.043s	sys 0.049s
I20260709 12:22:41.881816 17530 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:41.849514 17606 raft_consensus.cc:2764] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:22:41.891031 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 213628us
I20260709 12:22:41.890028 17715 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:41.853518 17423 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:41.901243 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:41.910225 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60390.8us
I20260709 12:22:41.912355 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:41.915652 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:41.917645 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63443us
I20260709 12:22:41.924422 17675 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.927201 17677 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:41.927776 17677 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:41.928187 17677 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:41.890585 17728 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:22:41.935160 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:41.909906 17715 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:41.943045 17726 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:22:41.944092 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:41.943090 17715 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:41.945463 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:41.946420 17735 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:41.947144 17735 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:41.947489 17735 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:41.947878 17735 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:41.917166 17423 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:41.951429 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:41.952898 17733 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:41.953158 17423 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:41.954362 13766 leader_election.cc:400] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
I20260709 12:22:41.954828 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:41.955350 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:41.957772 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:41.960741 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45726.9us
I20260709 12:22:41.963162 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:41.963603 17658 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:41.964021 17630 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:41.964444 17630 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:41.964532 17423 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:41.964949 17630 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:41.967589 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:41.968899 13797 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 6.
I20260709 12:22:41.961776 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:42.151131 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 189399us
I20260709 12:22:41.978946 13793 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:42.151950 17600 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:42.071475 17630 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.096083 17677 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.074685 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:41.913666 17700 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:41.941569 17486 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.100419 14390 leader_election.cc:400] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
I20260709 12:22:42.175870 14390 leader_election.cc:403] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:22:42.104764 14419 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEARNER]: Advancing to term 10
I20260709 12:22:42.083276 17530 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.085793 17539 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:42.075139 14424 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:42.085734 17735 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.111135 17190 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:22:42.086122 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 115591us
I20260709 12:22:41.964479 17733 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:42.120556 14420 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:42.123275 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:42.128994 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:42.130636 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:42.138836 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.140439 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 483) took 1103 ms. Trace:
I20260709 12:22:42.084369 17715 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.195626 17605 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7.
I20260709 12:22:42.196252 17190 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:42.196673 17190 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:42.199054 17715 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.128925 17707 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.142900 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.200656 13720 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:42.201958 17707 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:42.197978 17735 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.197978 17190 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.203796 13795 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:42.204283 17190 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:42.194849 13801 rpcz_store.cc:276] 0709 12:22:41.036732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:41.043576 (+  6844us) service_pool.cc:224] Handling call
0709 12:22:41.043967 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:41.199012 (+155045us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:22:41.235933 (+ 36921us) log.cc:844] Serialized 14 byte log entry
0709 12:22:41.491044 (+255111us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:41.491130 (+    86us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:22:41.491206 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.140201 (+648995us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:42.140248 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:42.140381 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5465,"prepare.queue_time_us":9025,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":24205,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":14225,"spinlock_wait_cycles":283725696,"thread_start_us":248931,"threads_started":2,"wal-append.queue_time_us":189282}
I20260709 12:22:42.209307 13720 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:41.975065 17578 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.067605 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:42.152031 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:42.210377 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 202860us
W20260709 12:22:42.210707 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58778.2us
I20260709 12:22:42.211520 17578 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.212165 17578 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:42.213747 17148 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:42.153086 17440 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.214164 17148 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:42.214601 17148 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:42.214556 17440 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.209627 13794 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:42.130336 13538 maintenance_manager.cc:643] P d5e54c6d2c954fbf93d5136276b7b78c: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 5.734s	user 0.196s	sys 0.089s Metrics: {"bytes_written":29148,"cfile_init":1,"dirs.queue_time_us":134545,"dirs.run_cpu_time_us":588,"dirs.run_wall_time_us":11327,"drs_written":1,"lbm_read_time_us":592,"lbm_reads_lt_1ms":4,"lbm_write_time_us":101110,"lbm_writes_1-10_ms":5,"lbm_writes_10-100_ms":3,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"rows_written":63,"thread_start_us":61078,"threads_started":1}
I20260709 12:22:42.097362 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.135201 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:42.174396 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:42.177093 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:42.417430 13798 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.165849 17600 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:42.420846 17600 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:42.181628 17677 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:42.182893 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108282us
I20260709 12:22:42.183647 17700 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.184442 17530 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:42.185367 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 85098.1us
I20260709 12:22:42.186421 17630 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:42.187543 14425 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:42.188522 17486 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:42.194947 13722 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:41.974331 17436 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.209585 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:42.218658 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.218827 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.219079 13795 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:42.436370 17436 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:42.436858 13718 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:42.437925 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.438613 13718 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:42.219201 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:42.219664 13796 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:22:42.220570 13723 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:42.447268 17277 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.447562 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.436431 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.448365 13723 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:42.448184 17277 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.448772 13725 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:22:42.449826 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.221223 13800 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:42.223402 17440 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:42.451645 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:42.452757 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.453434 13725 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:42.225885 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:42.226332 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.459167 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.459563 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:42.460634 13719 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:22:42.227035 14424 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:42.217768 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:42.228006 17664 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:42.466230 13792 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:42.231048 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.469314 13721 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:42.219599 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.234719 14419 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:22:42.470587 14419 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 6
I20260709 12:22:42.470636 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:42.471074 14419 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:22:42.471500 14419 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:42.472064 13727 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:42.236552 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:42.472891 13726 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9. Re-sending same reply.
I20260709 12:22:42.473464 13727 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:42.247692 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.487561 13720 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:42.252938 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.468817 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:42.491321 13724 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:42.493458 13724 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:42.246084 17323 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:22:42.243793 17539 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.494663 17539 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:42.258612 17715 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:42.275180 13799 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:42.274510 17669 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.249847 17737 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:42.287555 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.497437 17669 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.276376 17148 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.497987 17539 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.500186 17669 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:42.500622 17148 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:42.329931 17741 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.052s	user 0.021s	sys 0.024s
W20260709 12:22:42.332656 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.152s (SENT)
I20260709 12:22:42.501516 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:42.502764 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 170153us
I20260709 12:22:42.342150 17578 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:42.502589 13546 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 5 to 7, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 7 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:42.358044 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:42.371228 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:42.374722 17605 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:42.393833 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:42.396972 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:42.397833 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:42.474488 13726 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:22:42.525420 13726 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 4
I20260709 12:22:42.525717 13726 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:22:42.525936 13726 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:42.241427 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:42.476058 17700 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:42.478037 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 380754us
I20260709 12:22:42.479625 13721 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:42.495333 17323 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:42.285277 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:42.532553 13649 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:22:42.504159 17458 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:42.354750 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:42.508253 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:42.514467 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:42.527578 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.528564 13792 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:42.534118 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.532364 17323 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:42.536459 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:42.536787 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:42.537096 13797 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:42.532341 17669 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:42.538141 17669 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.538609 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:42.540990 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:42.547192 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:42.547489 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:42.520390 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.564s (SENT)
I20260709 12:22:42.547392 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:42.550385 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.551252 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:42.552304 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:42.531646 13798 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:42.526942 17277 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.552366 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:42.558337 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:42.557318 17713 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:42.565534 17423 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:42.566139 17423 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:42.566696 17423 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:42.566835 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:42.539361 17600 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.569095 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:42.570171 17733 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:42.570586 17733 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:42.570624 17669 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.571259 17735 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:42.572621 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:42.574540 17503 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:42.575181 17503 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:42.575695 17503 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.575829 17677 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:42.570977 17733 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.575434 17530 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:22:42.577683 17530 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:42.578821 17530 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:42.576391 17677 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:42.536790 17323 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:42.574039 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.571045 17600 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:42.584113 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:22:42.616655 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81422.3us
W20260709 12:22:42.650851 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130434us
W20260709 12:22:42.652444 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.589s (SENT)
I20260709 12:22:42.653061 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:42.597049 17677 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:42.555186 17583 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:22:42.532634 17605 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.579772 17530 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.706471 17530 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:42.663762 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:42.707821 14416 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:42.711987 17605 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:42.682190 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108294us
I20260709 12:22:42.715842 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:42.663287 17542 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:42.588449 17556 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:22:42.649685 17486 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:42.619515 17630 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:42.562604 17700 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 12:22:42.699631 13793 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:42.704044 13797 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:42.705209 13649 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:42.718099 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.721128 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:42.725823 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107176us
I20260709 12:22:42.726558 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:42.731789 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79387us
I20260709 12:22:42.720433 17703 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:42.722798 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 153759us
W20260709 12:22:42.741271 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 168085us
I20260709 12:22:42.748571 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:42.725709 17606 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:42.724521 17486 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:42.733255 17277 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:42.765714 13539 maintenance_manager.cc:419] P d5e54c6d2c954fbf93d5136276b7b78c: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9136 bytes on disk
I20260709 12:22:42.765717 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.766104 17486 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:42.769146 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:42.772655 17542 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:42.772953 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54991.6us
I20260709 12:22:42.773064 17606 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:42.773511 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47044.1us
I20260709 12:22:42.788057 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:42.774578 17148 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:42.764067 17436 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:42.734473 17707 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:42.781752 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 229437us
I20260709 12:22:42.750286 17578 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:22:42.814884 13538 maintenance_manager.cc:643] P d5e54c6d2c954fbf93d5136276b7b78c: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.024s	user 0.003s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":4303,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260709 12:22:42.790220 17277 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:42.825889 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.827121 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.824966 17748 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:42.840163 17606 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:42.834419 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70100.8us
I20260709 12:22:42.838750 17423 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.847589 17436 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:42.848660 17423 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:42.849581 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.850147 17148 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:42.850462 17539 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:42.846501 17745 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:22:42.851176 17148 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:42.856782 17611 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:42.857429 17611 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:42.859256 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:42.859618 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:42.859982 13649 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:42.860236 13650 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:42.863045 13652 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:22:42.863340 17440 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:42.864820 17440 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:42.866472 17713 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:42.867003 17713 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:42.868021 17713 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.866534 17578 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:42.871510 17578 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:42.872792 17578 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.879220 17578 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:42.879071 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:42.865896 17440 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
W20260709 12:22:42.896437 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 160508us
W20260709 12:22:42.902060 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.735s (SENT)
I20260709 12:22:42.902531 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:42.903520 13649 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:42.958468 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 134570us
I20260709 12:22:42.838424 17749 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:42.962150 17703 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:42.962185 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:43.968673 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.00655e+06us
I20260709 12:22:42.893142 17748 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:43.099145 17749 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:43.979533 17530 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.272s	user 0.023s	sys 0.001s
I20260709 12:22:43.980078 17749 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:43.100627 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 199668us
W20260709 12:22:43.110157 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:43.174496 17323 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.526s	user 0.002s	sys 0.014s
W20260709 12:22:43.885409 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.08542e+06us
I20260709 12:22:42.907258 17713 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:43.983675 17703 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:43.984525 17703 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:43.985476 17703 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:22:43.904146 14388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.243s	user 0.037s	sys 0.025s
W20260709 12:22:43.986469 14388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.326s	user 0.038s	sys 0.026s
I20260709 12:22:42.875816 17542 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:43.925140 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:42.883478 17600 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 1.054s
W20260709 12:22:43.965195 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:43.975145 17748 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:43.992775 17530 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:22:43.976609 17677 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.294s	user 0.022s	sys 0.002s
I20260709 12:22:43.977090 17458 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:43.982326 17323 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.005612 17677 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:42.848611 17436 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.021061 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.024828 17530 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.023934 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.021693 17323 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:44.028652 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3211163940000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 12:22:44.029760 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2228944299999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
W20260709 12:22:44.030442 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1722877910001444s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
W20260709 12:22:44.030993 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1623877910001283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 12:22:44.031452 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1533877910001138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 12:22:44.032061 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1301944299998468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 12:22:44.036788 13617 connection.cc:256] RPC call timeout handler was delayed by -1.09509442999979s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 12:22:44.037293 13617 connection.cc:256] RPC call timeout handler was delayed by -1.085194429999774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 12:22:44.037714 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0275944299996809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 12:22:44.034614 17458 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:22:44.038100 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0240383440000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 12:22:44.043043 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:44.045171 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 72777.8us
I20260709 12:22:44.046573 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.053952 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.054829 13648 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.058130 17530 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.059594 17323 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.060267 17323 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.043042 17458 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.062932 17323 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.063071 17458 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.063567 17323 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.064548 17323 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.066011 17677 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:44.069274 17578 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.189s	user 0.011s	sys 0.009s
I20260709 12:22:44.069322 17323 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.071137 17323 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:44.071977 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.072984 17530 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:44.074690 17530 raft_consensus.cc:2764] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 12:22:44.074844 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.21282e+06us
I20260709 12:22:44.076061 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:44.082752 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.087926 13648 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:44.064515 17458 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:44.092232 13673 debug-util.cc:398] Leaking SignalData structure 0x7b080010b380 after lost signal to thread 12472
W20260709 12:22:44.105399 13673 debug-util.cc:398] Leaking SignalData structure 0x7b08001c0660 after lost signal to thread 13584
I20260709 12:22:44.086243 17323 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:44.109544 17323 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:44.139946 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85706.8us
I20260709 12:22:44.110837 17323 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.147315 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:44.149179 17677 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:22:44.161105 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.166790 17677 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.151141 17530 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:22:44.170313 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.178356 17530 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.168262 17677 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:22:44.181989 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:44.186523 17530 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.187146 17530 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:44.187614 17530 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:44.218546 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.556s (SENT)
W20260709 12:22:44.221442 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.567s (SENT)
I20260709 12:22:44.221958 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.228668 17765 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:44.230557 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.215925 17767 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:22:44.162240 17323 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.235720 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.577s (SENT)
W20260709 12:22:44.241122 13617 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.728s (SENT)
I20260709 12:22:44.239810 17677 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:44.244870 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.776s (SENT)
I20260709 12:22:44.239384 17763 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.246426 17764 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.247601 17762 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.247274 17763 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.248330 17762 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.246712 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.247452 17764 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.252122 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.808s (SENT)
W20260709 12:22:44.253069 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.858s (SENT)
W20260709 12:22:44.253875 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.879s (SENT)
I20260709 12:22:44.247195 17767 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.243446 17771 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:44.261461 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.918s (SENT)
I20260709 12:22:44.263598 17764 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.264516 17613 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:44.264964 17613 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.260257 17771 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.265978 17613 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.268697 17767 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:44.269174 17767 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.267524 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.266008 17762 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.270457 17578 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:44.270311 17713 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.271183 17578 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.271760 14424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.424s	user 0.004s	sys 0.016s
I20260709 12:22:44.271570 17713 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.272315 14424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.424s	user 0.006s	sys 0.016s
I20260709 12:22:44.273288 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 596) took 1749 ms. Trace:
I20260709 12:22:44.273718 17578 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.273906 17713 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:44.275300 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 199327us
I20260709 12:22:44.270047 17767 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.257718 17765 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.283978 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.285377 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.273667 14424 rpcz_store.cc:276] 0709 12:22:42.524164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.524727 (+   563us) service_pool.cc:224] Handling call
0709 12:22:42.548510 (+ 23783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.550318 (+  1808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.550356 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.558112 (+  7756us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.847859 (+289747us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b18004298e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:22:44.272770 (+1424911us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:44.272819 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.272907 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.272956 (+    49us) raft_consensus.cc:1685] finished
0709 12:22:44.272969 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.273201 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19214,"raft.queue_time_us":5181,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5280,"spinlock_wait_cycles":14187904,"thread_start_us":1424685,"threads_started":1,"wal-append.queue_time_us":360741,"wal-append.run_cpu_time_us":2401,"wal-append.run_wall_time_us":1002470}
I20260709 12:22:44.267633 17613 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.276921 17440 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:44.275911 17436 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.302052 17436 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.304488 17436 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:44.305424 17436 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.276186 17715 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:44.306473 17436 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:44.306699 17715 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.307101 17630 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:44.307804 17630 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.308854 17715 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.310111 17763 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:44.311271 17766 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.096s	user 0.006s	sys 0.000s
I20260709 12:22:44.310097 17630 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:44.301288 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 154064us
I20260709 12:22:44.313436 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:44.314502 17763 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:44.315152 17763 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.316460 17763 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.294888 17677 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.297806 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.276451 17735 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:44.319393 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57990.5us
I20260709 12:22:44.319162 17735 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.320194 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.929s (SENT)
I20260709 12:22:44.320622 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.321127 17765 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.300558 17440 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:44.297025 17773 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:44.326488 17440 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.321656 17735 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.323632 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.324400 17677 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:44.279553 17323 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.334750 17530 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.267177 17771 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:44.337191 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.993s (SENT)
I20260709 12:22:44.309144 17277 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:44.334841 17762 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.337273 17530 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:44.301240 17767 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.338152 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.339608 17277 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.348162 17277 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.349670 17277 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.343451 17677 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.350520 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:44.351430 13793 raft_consensus.cc:606] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:44.351173 17677 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.351832 13793 raft_consensus.cc:614] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:44.352834 17715 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:44.354446 17669 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:44.354900 17669 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.356220 13693 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9.
I20260709 12:22:44.355824 17669 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.357219 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.357339 17669 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.358438 17630 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.359108 17630 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.360965 17486 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.571s	user 0.017s	sys 0.008s
I20260709 12:22:44.361609 17630 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.361454 17486 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:44.362674 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.362547 17323 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:44.357270 17190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.480s	user 0.013s	sys 0.021s
W20260709 12:22:44.365513 17190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.489s	user 0.014s	sys 0.021s
W20260709 12:22:44.365891 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.996s (SENT)
W20260709 12:22:44.367015 13617 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.064s (SENT)
I20260709 12:22:44.367430 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:42.831184 17503 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.368304 17323 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:44.368628 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.068s (SENT)
I20260709 12:22:44.369109 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.370096 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.364480 13793 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:44.370937 13792 raft_consensus.cc:2368] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:44.321930 17765 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.371901 14416 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:44.346338 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:22:44.375486 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.128s (SENT)
I20260709 12:22:44.376258 13693 leader_election.cc:400] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6.
I20260709 12:22:44.377331 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:44.354081 17677 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.342583 17578 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:22:44.362223 17630 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.381261 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:44.375319 17323 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.380461 17503 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:44.389226 17503 raft_consensus.cc:2764] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:22:44.375670 17530 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.394927 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.396076 17713 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:44.396404 17578 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:44.400066 17578 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.396960 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.398801 13796 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:44.399304 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.389632 17677 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:44.401854 13795 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:22:44.402683 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.157s (SENT)
I20260709 12:22:44.404035 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.405539 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:44.405148 17713 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.406713 17713 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.407024 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:44.407896 17713 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:22:44.408542 17733 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.832s	user 0.019s	sys 0.012s
I20260709 12:22:44.408640 17713 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.409080 17733 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.411074 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.411332 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.411720 13725 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:44.411949 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.409915 17713 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.413270 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.413882 13722 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:44.412983 13793 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:44.411974 13726 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:44.416129 17733 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:44.402711 17677 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:44.417012 17677 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:44.423316 17530 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.426589 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.427621 17503 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.431443 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:44.406219 17578 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.442117 13722 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:44.448899 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.451411 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.452337 13794 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:44.402792 17776 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:44.454515 17775 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:44.458388 17677 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:44.459298 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56643.5us
I20260709 12:22:44.460201 13693 leader_election.cc:400] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 9.
I20260709 12:22:44.461232 17677 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:44.444455 17486 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:44.423926 17530 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.448300 17503 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.463910 17530 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:44.439463 17765 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.464208 17713 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.464972 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.464812 17713 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.465723 13726 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.466917 17440 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.467409 17713 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.467998 13800 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:44.471138 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:44.463570 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:44.471825 13796 raft_consensus.cc:606] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.473158 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:44.474808 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.475108 17735 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:44.475514 17735 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.475828 13649 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:22:44.476207 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:44.475872 17735 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.476984 13651 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:44.477550 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:44.478652 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:44.478817 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:44.477908 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:44.479828 13650 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:44.474005 17539 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:44.480578 17539 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.480780 17703 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:44.481066 13618 leader_election.cc:400] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:44.481454 13618 leader_election.cc:403] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:22:44.481774 17733 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:44.482306 17771 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7.
I20260709 12:22:44.482359 17733 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.483909 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.481002 17539 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.484794 13649 raft_consensus.cc:2410] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:22:44.485368 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.488858 13720 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:44.487003 13721 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:44.488104 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.369s (SENT)
I20260709 12:22:44.488039 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:44.490728 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:44.492964 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.493808 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.494532 13796 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:44.493081 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.495935 17703 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:44.496402 17703 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.497232 17703 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.502046 17762 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:22:44.506074 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.506603 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:44.507043 13800 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:22:44.507171 13796 raft_consensus.cc:606] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.507537 13796 raft_consensus.cc:614] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:44.508260 17440 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:44.502786 17578 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:44.502172 17733 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.511963 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2140) took 2271 ms. Trace:
I20260709 12:22:44.512360 13723 rpcz_store.cc:276] 0709 12:22:42.240638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.460205 (+219567us) service_pool.cc:224] Handling call
0709 12:22:42.460458 (+   253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:42.552599 (+ 92141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.552633 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:42.587632 (+ 34999us) log.cc:844] Serialized 52 byte log entry
0709 12:22:42.791558 (+203926us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:42.801354 (+  9796us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.801804 (+   450us) log.cc:844] Serialized 14 byte log entry
0709 12:22:42.812468 (+ 10664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:42.812567 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:42.858740 (+ 46173us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b180042c4c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:44.216112 (+1357372us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b180042c4c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:44.511720 (+295608us) raft_consensus.cc:1685] finished
0709 12:22:44.511746 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.511900 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10049,"prepare.queue_time_us":1424636,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":2082,"raft.queue_time_us":112290,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":18584,"spinlock_wait_cycles":9768704,"thread_start_us":203453,"threads_started":1,"wal-append.queue_time_us":275836,"wal-append.run_cpu_time_us":3565,"wal-append.run_wall_time_us":1240209}
W20260709 12:22:44.515209 13617 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.527s (SENT)
I20260709 12:22:44.515699 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:44.515884 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.516937 13617 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.562s (SENT)
I20260709 12:22:44.517025 17458 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:44.517408 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.498348 17776 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.498383 17771 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:44.518718 17458 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.518931 17771 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.519450 17771 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:44.520514 13726 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:22:44.522761 17616 raft_consensus.cc:670] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:44.523391 14420 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEARNER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:44.524430 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:22:44.524507 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.586s (SENT)
I20260709 12:22:44.525111 13800 raft_consensus.cc:606] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.525419 13800 raft_consensus.cc:614] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:44.532997 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:44.533497 13799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.716s	user 0.002s	sys 0.034s
I20260709 12:22:44.533874 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.534340 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.535023 13800 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
W20260709 12:22:44.535877 13673 debug-util.cc:398] Leaking SignalData structure 0x7b0800359000 after lost signal to thread 13676
W20260709 12:22:44.536257 13673 debug-util.cc:398] Leaking SignalData structure 0x7b0800261c00 after lost signal to thread 13677
W20260709 12:22:44.533906 13799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.722s	user 0.002s	sys 0.034s
W20260709 12:22:44.547381 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.548700 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.549561 13727 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.551520 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:44.552318 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.705s (SENT)
I20260709 12:22:44.553049 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.555655 17423 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:44.556438 17733 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:44.556798 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 500) took 2027 ms. Trace:
I20260709 12:22:44.557663 17733 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.558133 17733 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.558073 13799 rpcz_store.cc:276] 0709 12:22:42.528938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.529061 (+   123us) service_pool.cc:224] Handling call
0709 12:22:42.529420 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:42.529777 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:42.529805 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:42.534872 (+  5067us) log.cc:844] Serialized 26 byte log entry
0709 12:22:42.786705 (+251833us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:44.547456 (+1760751us) log.cc:844] Serialized 14 byte log entry
0709 12:22:44.555367 (+  7911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:44.556522 (+  1155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:44.556569 (+    47us) raft_consensus.cc:1685] finished
0709 12:22:44.556582 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:44.556738 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18379,"prepare.queue_time_us":171841,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":77,"raft.queue_time_us":958,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":3964,"thread_start_us":1931207,"threads_started":3,"wal-append.queue_time_us":283774,"wal-append.run_cpu_time_us":1756,"wal-append.run_wall_time_us":1001734}
I20260709 12:22:44.561501 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.562371 13799 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:44.563398 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.572345 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.575285 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:44.576308 13799 raft_consensus.cc:606] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.576781 13799 raft_consensus.cc:614] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:44.578436 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:44.586585 17774 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:44.587138 17774 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.588049 17774 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.518744 17776 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:44.595333 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.846s (SENT)
W20260709 12:22:44.596524 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.865s (SENT)
I20260709 12:22:44.596948 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.601433 17764 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.613431 17735 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.619750 17771 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.622443 17773 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:22:44.616460 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.944s (SENT)
I20260709 12:22:44.634717 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.616686 17539 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.621454 17713 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:44.592664 17190 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:44.621562 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:44.616760 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.639765 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:44.640249 13718 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.644183 13800 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 7: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:44.645150 13798 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:44.638345 17764 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:44.647790 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:44.652186 17539 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:44.653103 17715 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:22:44.653767 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.168s (SENT)
I20260709 12:22:44.654109 17715 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.654479 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.655259 17715 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.647188 17764 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.659298 17764 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.660269 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.660524 17764 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.650242 17578 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:44.661533 17578 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.660944 13722 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.662467 17578 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.666019 13718 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:44.666764 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.667758 13799 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:44.668880 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:44.674856 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:44.675416 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.670884 17767 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.674328 17776 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:44.669651 17190 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.669520 17771 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.696718 13799 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:44.698503 13727 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:44.694497 13722 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:22:44.714515 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.717949 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:44.738004 17669 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:44.695839 17190 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.702505 17600 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.703899 17775 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:44.702767 17776 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.716969 17767 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:22:44.746372 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.978s (SENT)
W20260709 12:22:44.750522 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.750443 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:44.752489 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:44.756278 17791 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:22:44.749699 17775 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.759117 17775 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:44.760437 17767 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:44.761019 17767 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.747923 17713 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:44.762667 17713 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:44.764010 17795 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:22:44.764358 17713 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.767130 17190 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.765761 17703 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.772042 17703 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.750051 17733 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.776110 13649 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:44.776503 13647 raft_consensus.cc:1217] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Deduplicated request from leader. Original: 6.5->[8.6-8.6]   Dedup: 8.6->[]
I20260709 12:22:44.777037 13647 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:44.778393 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.778728 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.779984 13651 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:44.780062 13648 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.780447 17611 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.767165 17776 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.785189 17556 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:44.785750 17556 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:44.747201 17600 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.789646 17600 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.761937 17767 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.795553 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.797317 13799 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.802934 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:44.803777 13792 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:44.782047 17611 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:44.806759 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:44.810494 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:44.836225 13648 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:44.775776 17733 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:44.806391 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:44.838922 13546 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 9, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 9 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:44.844880 13652 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:44.845362 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 127543us
I20260709 12:22:44.846168 13649 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:44.847174 13650 raft_consensus.cc:1217] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Deduplicated request from leader. Original: 8.6->[9.7-9.7]   Dedup: 9.7->[]
I20260709 12:22:44.847669 13650 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:22:44.848948 13646 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:22:44.851495 17798 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:44.854159 13647 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:22:44.854592 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108373us
W20260709 12:22:44.855271 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.073s (SENT)
W20260709 12:22:44.855616 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48969.2us
I20260709 12:22:44.857578 17747 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:44.858055 13792 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:44.858767 13796 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:22:44.859982 13651 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:44.865218 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.865967 13722 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:44.867218 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.868170 13649 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:22:44.871037 13649 pending_rounds.cc:85] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 8
I20260709 12:22:44.871326 13649 pending_rounds.cc:107] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:22:44.871423 17797 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:22:44.872656 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.868767 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.872653 13726 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:22:44.871613 13649 raft_consensus.cc:2889] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:44.870193 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:44.875831 13727 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.875890 13648 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.871649 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.878871 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.883373 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.887172 17804 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:44.858086 17747 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:44.888010 17747 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:44.895588 13727 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:44.898413 13722 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:44.906154 13799 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:44.906863 13727 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:44.907662 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.907866 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:44.909138 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.911126 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.911872 13649 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:44.924526 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.929225 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.929634 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.930081 13727 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:44.910483 17630 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:44.931438 13648 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:44.933300 13796 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:44.935900 13792 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:44.938138 13793 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:44.941100 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.942451 13727 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:44.941565 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:44.945761 13649 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:44.880679 17773 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:22:44.946631 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.947184 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.927146 17804 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:44.950416 13648 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:44.951205 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.955163 13796 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:44.932562 17630 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.960227 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.867077 17556 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:44.960697 17799 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:44.973583 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:44.973948 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.972703 13648 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:44.974649 13727 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11.
I20260709 12:22:44.976059 13792 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:22:44.978994 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.956737 17804 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:44.955708 13795 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:44.982998 13799 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:44.986081 13799 raft_consensus.cc:1217] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 8.6->[9.7-9.7]   Dedup: 9.7->[]
I20260709 12:22:44.986649 13799 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:22:44.967420 13796 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:44.990751 13799 raft_consensus.cc:1217] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[6.5-8.6]   Dedup: 8.6->[]
I20260709 12:22:44.990835 13796 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:44.991319 13799 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:22:44.992790 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:44.993481 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:44.995694 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.001461 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.002651 17556 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.002892 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.004347 17630 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:45.004992 17804 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:44.994792 17799 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.007148 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.007892 13723 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10.
I20260709 12:22:45.009425 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.010396 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.010499 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.010607 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.011224 13718 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.011410 13649 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.011232 13723 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:45.012502 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.013113 13652 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.012719 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:45.016976 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.017001 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.017573 13651 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.018723 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.020710 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.021344 13648 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:45.022773 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:45.022902 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:45.023782 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.025712 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.025869 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.031375 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.031868 13725 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.032071 13722 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.025717 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:45.030189 17779 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.450s	user 0.001s	sys 0.000s
I20260709 12:22:45.030539 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.030874 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:45.036161 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.893s (SENT)
I20260709 12:22:45.038625 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.040148 13718 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.037436 13649 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:22:45.045574 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.046095 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.046761 13722 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:45.046825 13723 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11.
I20260709 12:22:45.048033 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.049736 13651 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:45.049933 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.051721 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.052511 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.015964 17799 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:45.053264 13722 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.056455 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.053812 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.055095 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.057937 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:45.030984 17747 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.059439 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.060196 13725 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.060770 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.062031 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.072016 13546 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 4 to 8, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:22:44.873404 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:45.074519 17277 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:45.074995 17277 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:45.075820 13617 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.292s (SENT)
I20260709 12:22:45.077646 13726 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:45.073892 13546 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 6 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:45.079159 13727 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:45.085053 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.088063 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.094828 13727 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.098021 13722 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:45.098487 13722 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.099112 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:45.099262 13722 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.100211 13722 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.100646 13693 leader_election.cc:400] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8.
I20260709 12:22:45.101936 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:45.103621 17737 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:22:45.103941 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:45.102499 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.105724 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.107280 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:45.107920 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.108805 13723 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:22:45.109162 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:45.111271 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.111389 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.112123 13718 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.114634 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.108733 13727 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.114732 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:45.109504 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.346s (SENT)
I20260709 12:22:45.117800 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:45.120618 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:45.121363 13727 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:45.121369 13794 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:45.123611 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.120718 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.124910 13718 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 6: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:45.106618 17737 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:45.075848 17277 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.134742 13726 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:45.135481 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.135792 13726 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.137229 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.106719 17735 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:45.139199 13727 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:22:45.140189 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:45.141755 13718 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:22:45.140803 13720 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:45.139901 13727 pending_rounds.cc:85] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 2
I20260709 12:22:45.144622 13727 pending_rounds.cc:107] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:22:45.145013 13727 raft_consensus.cc:2889] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:45.130623 13722 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:22:45.146241 13722 pending_rounds.cc:85] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:22:45.136968 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.136170 13725 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:45.128549 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:45.155781 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.201856 13718 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:45.202556 13718 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:45.205507 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 76995.5us
I20260709 12:22:45.206352 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:45.206841 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.207230 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:45.148524 13722 pending_rounds.cc:107] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260709 12:22:45.162712 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.23758e+06us
I20260709 12:22:45.162933 13719 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:22:45.168007 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.167773 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.169144 17605 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:22:45.169644 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81892.8us
I20260709 12:22:45.169932 17751 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:45.228914 17751 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.230720 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:45.231745 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.231937 17751 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:45.126112 17323 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:45.232793 17799 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:45.233109 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.181663 13793 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:45.233758 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.183478 17277 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:45.185081 17794 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.419s	user 0.001s	sys 0.001s
I20260709 12:22:45.130558 17737 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:45.190367 17774 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:45.136953 13726 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:45.192055 13723 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:22:45.197176 13794 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:45.210541 13719 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.211249 13652 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:45.250535 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.216517 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:45.182426 13795 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:22:45.251434 14418 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:22:45.239390 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:45.241240 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.251969 13723 pending_rounds.cc:85] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:22:45.252447 13723 pending_rounds.cc:107] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:22:45.252705 13723 raft_consensus.cc:2889] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:45.253374 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.253634 13726 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:45.236739 17630 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:45.250564 14421 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:45.257550 13725 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:45.249491 17616 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:45.258767 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:45.258394 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48307.8us
I20260709 12:22:45.262463 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.263033 14422 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:45.258399 13723 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:45.247602 13720 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:45.219714 13722 raft_consensus.cc:2889] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:45.254688 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:45.255046 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.233686 17799 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:45.254094 14425 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:45.259325 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:45.267771 13673 debug-util.cc:398] Leaking SignalData structure 0x7b08003617a0 after lost signal to thread 13751
W20260709 12:22:45.277498 13673 debug-util.cc:398] Leaking SignalData structure 0x7b080019bd80 after lost signal to thread 13752
I20260709 12:22:45.244050 13718 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.275523 13801 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
W20260709 12:22:45.276379 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.279233 13650 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
W20260709 12:22:45.280349 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63976.9us
I20260709 12:22:45.250012 17737 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:45.258620 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.285754 13652 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:45.287159 13718 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:45.281297 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:45.283093 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.261078 13719 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.290381 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:45.307603 13719 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:45.309830 13725 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:45.316125 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.316850 13650 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:45.319463 13727 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:45.314474 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:45.324154 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.324427 13793 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:45.329804 13720 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:45.261173 17801 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:22:45.332726 17801 consensus_queue.cc:1243] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:22:45.323019 17606 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:45.333904 17606 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.259506 17605 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:45.335448 17606 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.336683 17605 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.285482 17797 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:45.338495 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:45.258905 17616 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.338678 17797 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:45.339814 17797 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.340570 17616 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:45.342716 13726 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:22:45.343341 13726 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:22:45.343621 13726 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:22:45.343937 13726 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:45.338680 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:45.345971 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.278151 17799 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:45.349292 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.353185 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:45.285367 17539 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:45.359797 13793 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:45.353818 13797 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8. Re-sending same reply.
I20260709 12:22:45.354804 13725 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:45.362731 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2685) took 1016 ms. Trace:
I20260709 12:22:45.363139 13797 rpcz_store.cc:276] 0709 12:22:44.346163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.314073 (+967910us) service_pool.cc:224] Handling call
0709 12:22:45.362664 (+ 48591us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:45.372963 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.374857 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:45.308945 17805 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:45.378844 13718 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:22:45.379490 13718 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 4
I20260709 12:22:45.379834 13718 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:22:45.383100 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:45.385070 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 216576us
W20260709 12:22:45.399442 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 192245us
I20260709 12:22:45.352967 17616 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:45.323791 17600 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:22:45.424139 13718 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:45.380499 17811 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:22:45.428195 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:45.431881 13673 debug-util.cc:398] Leaking SignalData structure 0x7b08003739c0 after lost signal to thread 13825
W20260709 12:22:45.441322 13673 debug-util.cc:398] Leaking SignalData structure 0x7b08003f0e80 after lost signal to thread 13826
I20260709 12:22:45.440073 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:45.443575 13767 connection.cc:256] RPC call timeout handler was delayed by -1.2326244639998549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:22:45.444237 13767 connection.cc:256] RPC call timeout handler was delayed by -1.1347463450001669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 12:22:45.444672 13767 connection.cc:256] RPC call timeout handler was delayed by -1.1203463450001436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 12:22:45.445591 13719 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:45.450009 13673 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0360 after lost signal to thread 14449
W20260709 12:22:45.450469 13673 debug-util.cc:398] Leaking SignalData structure 0x7b080027dd40 after lost signal to thread 14450
I20260709 12:22:45.382423 17539 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:45.410158 17798 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:45.405148 17805 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:45.359899 17556 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:22:45.466799 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:45.448338 17669 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:45.455693 17751 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:45.457993 17814 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:45.467487 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:45.439953 17616 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.464586 17486 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:45.476418 13794 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:45.482930 17805 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:45.484313 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 216879us
W20260709 12:22:45.485872 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:45.486315 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.490108 13726 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:45.491189 17812 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:45.491788 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.492355 17812 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:45.492725 17812 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:45.489485 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 200213us
I20260709 12:22:45.496789 17747 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:45.475991 17735 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:45.497998 17735 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:45.498391 17735 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:45.502218 13673 debug-util.cc:398] Leaking SignalData structure 0x7b080032d8c0 after lost signal to thread 17738
W20260709 12:22:45.510782 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.510751 13718 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:45.440312 17799 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:45.519363 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.515975 13793 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:45.517777 17613 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:45.522739 13794 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:45.473032 17669 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.533506 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.535377 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.472978 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:45.538462 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.521554 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:45.541123 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.541814 13646 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:45.541926 13648 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:22:45.493652 17812 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:45.543439 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.546135 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.526369 17539 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.515717 17486 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.547889 17675 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:45.548488 17675 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:45.549589 17458 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:45.526820 17556 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:45.550539 17822 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:45.550735 17556 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.551460 17822 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.552541 17822 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.554205 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.555332 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 736) took 1630 ms. Trace:
I20260709 12:22:45.555753 14423 rpcz_store.cc:276] 0709 12:22:43.924456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.174036 (+249580us) service_pool.cc:224] Handling call
0709 12:22:44.174562 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:45.243298 (+1068736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:45.243344 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:45.251880 (+  8536us) log.cc:844] Serialized 26 byte log entry
0709 12:22:45.463639 (+211759us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:45.463681 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.463775 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:45.555059 (+ 91284us) raft_consensus.cc:1685] finished
0709 12:22:45.555087 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:45.555261 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1031147,"raft.queue_time_us":1021668,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3520,"thread_start_us":278040,"threads_started":2,"wal-append.queue_time_us":288713}
I20260709 12:22:45.541656 17799 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:45.523268 17751 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:22:45.559321 17751 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:45.559718 17751 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:45.560110 17751 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:45.562770 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:45.565594 17616 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:45.526207 17630 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.546633 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.566824 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.523514 17723 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:22:45.577126 13649 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:45.550145 17458 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:45.582232 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:45.577560 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 149460us
I20260709 12:22:45.569521 17556 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:45.594024 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.573880 17616 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:45.595041 13649 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:22:45.602749 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 264154us
I20260709 12:22:45.605620 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:45.614992 13643 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:45.584522 17723 consensus_queue.cc:1243] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:22:45.587908 17812 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:45.587018 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:45.629254 13643 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:45.549783 17675 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.642484 17630 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.647886 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:45.649930 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:45.656402 14423 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:45.657395 14423 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:45.656106 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69174.5us
W20260709 12:22:45.665282 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.663305 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:45.596035 17616 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:45.665535 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 192557us
I20260709 12:22:45.637598 17773 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:45.671406 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:45.675405 13720 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:45.682709 13650 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:45.668076 14423 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.691567 14423 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:45.664436 17774 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:22:45.694731 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173289us
I20260709 12:22:45.674856 17277 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:45.673903 17613 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:45.681774 17817 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:45.666913 17821 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:45.709190 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.711369 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.713029 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.669914 17458 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:45.716493 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:45.716861 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:45.718798 13793 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:45.720106 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.676470 17611 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:22:45.722873 13794 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:45.684021 17713 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 12:22:45.727044 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46346.9us
I20260709 12:22:45.689584 17578 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:22:45.729537 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:45.689502 17776 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:22:45.731916 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:45.705237 17613 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:45.734503 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.719408 17817 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.695303 17606 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
W20260709 12:22:45.738888 17818 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:22:45.738698 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:45.743539 13799 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:45.748904 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1916) took 1208 ms. Trace:
I20260709 12:22:45.747475 17613 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:45.799635 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67816.2us
I20260709 12:22:45.727890 17821 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.807567 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:45.754189 13800 rpcz_store.cc:276] 0709 12:22:44.534640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.556740 (+ 22100us) service_pool.cc:224] Handling call
0709 12:22:45.742712 (+1185972us) spinlock_profiling.cc:243] Waited 70.7 ms on lock 0x7b54003d45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:45.742898 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162585984}
I20260709 12:22:45.808194 13643 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:45.722956 17605 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:22:45.720975 17814 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:45.695693 17773 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:45.838991 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137929us
I20260709 12:22:45.722126 17616 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:45.799391 17820 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:22:45.834600 17813 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:45.870828 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.838624 17837 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:45.851260 17669 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:45.880311 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 150859us
I20260709 12:22:45.858635 17630 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:22:45.891983 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:45.904584 17616 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:45.954725 17832 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 12:22:45.956532 17827 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
W20260709 12:22:45.958936 17838 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
W20260709 12:22:45.960273 17839 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:22:45.962857 17830 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:22:45.849368 17817 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:46.002398 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.035140 13721 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:46.038121 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.038861 13721 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:45.871158 17813 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:46.047096 13690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.654s (SENT) [suppressed 44 similar messages]
I20260709 12:22:45.933461 17630 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:46.050537 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.064249 13723 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:22:46.042500 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 235009us
I20260709 12:22:46.069558 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:46.048264 17737 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.731s	user 0.013s	sys 0.000s
I20260709 12:22:46.072468 13721 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:22:46.065339 17797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.010s	sys 0.014s
I20260709 12:22:46.086980 14423 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:22:46.090327 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:46.049088 17556 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.053313 17578 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:46.087697 17797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.010s	sys 0.015s
I20260709 12:22:46.106122 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:46.108194 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61126us
I20260709 12:22:46.113771 13723 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:45.919185 17814 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:46.135072 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.101956 17735 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.000s
I20260709 12:22:46.151556 13546 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 6 to 8, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:46.146804 13722 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:46.159548 13721 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:45.912583 17821 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:46.115592 17751 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.001s
W20260709 12:22:46.167858 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122800us
W20260709 12:22:46.168968 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62982.7us
I20260709 12:22:46.191903 13721 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:46.090026 17737 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.162704 17773 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.144675 17605 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:22:46.110713 17812 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:22:46.111608 17556 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:46.229491 17798 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.766s	user 0.014s	sys 0.003s
I20260709 12:22:46.233103 17440 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:46.251405 17841 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:46.115216 17578 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.238428 17423 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:46.233888 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.167217 17735 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.271297 13721 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:46.271716 17813 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:46.272872 17735 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:46.273546 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2724) took 1032 ms. Trace:
W20260709 12:22:46.275115 17859 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:22:46.273945 13798 rpcz_store.cc:276] 0709 12:22:45.240958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.241582 (+   624us) service_pool.cc:224] Handling call
0709 12:22:45.251642 (+ 10060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:45.251847 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:22:45.284013 (+ 32166us) log.cc:844] Serialized 14 byte log entry
0709 12:22:45.284330 (+   317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:45.284384 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:45.284441 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:45.284517 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:46.272473 (+987956us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b18003ecd40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4
0709 12:22:46.273329 (+   856us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b5400102cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:46.273480 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8998,"prepare.queue_time_us":29999,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":101,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":22645,"spinlock_wait_cycles":14732544}
W20260709 12:22:46.276904 17840 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:22:46.294499 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1924) took 1423 ms. Trace:
W20260709 12:22:46.295747 17851 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:22:46.300411 17773 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:46.300788 17824 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.301110 17773 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.301532 17824 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.305660 17556 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.307652 13721 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:22:46.308812 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:46.310151 17812 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.252060 13718 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:46.223577 17630 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:46.168433 17277 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
W20260709 12:22:46.260651 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63000.7us
I20260709 12:22:46.322083 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.195050 17600 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:22:46.220568 17817 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:46.192972 17751 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.256632 17440 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:46.268133 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 766) took 3553 ms. Trace:
I20260709 12:22:46.268306 17841 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:46.269711 17853 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:22:46.270000 17578 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.270119 17843 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:22:46.270519 17423 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.270915 17847 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:46.309156 17858 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:22:46.295310 13792 rpcz_store.cc:276] 0709 12:22:44.871026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.941994 (+ 70968us) service_pool.cc:224] Handling call
0709 12:22:44.975682 (+ 33688us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:44.976611 (+   929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.294270 (+1317659us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b54000e4cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:46.294426 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76131584}
W20260709 12:22:46.315630 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.318226 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.319018 17836 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:46.313791 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.314404 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:46.321069 13724 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:22:46.220988 17820 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:46.310720 17812 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:46.328322 14416 rpcz_store.cc:276] 0709 12:22:42.711688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:42.711874 (+   186us) service_pool.cc:224] Handling call
0709 12:22:42.712208 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:44.374183 (+1661975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.260192 (+1886009us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540010f4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:46.265258 (+  5066us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3816397056}
I20260709 12:22:46.257336 17798 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:46.388976 17812 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:46.331348 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.334726 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 223728us
I20260709 12:22:46.379251 17841 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.330094 17813 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:22:46.329675 17605 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:22:46.395215 17277 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:46.380791 17857 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:22:46.397768 17630 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.372382 17855 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 12:22:46.398384 17630 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:46.400627 17630 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.401750 17737 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.338558 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.371035 17606 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:22:46.403409 17737 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:22:46.325208 17833 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.403s	user 0.002s	sys 0.000s
W20260709 12:22:46.327785 17613 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.001s
W20260709 12:22:46.404553 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66065.8us
I20260709 12:22:46.404699 17613 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.405375 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.406276 17606 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:46.407217 17613 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.383340 17847 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.408170 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.409268 17820 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:46.409458 17847 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.409690 17820 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:46.322713 17849 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:46.374380 14425 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:46.325403 17763 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:46.341110 17797 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.385437 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:46.336149 17822 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:22:46.387341 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.379194 17423 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:46.377903 17440 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.340344 17852 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:22:46.421114 13792 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:46.420639 17763 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.431927 17797 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.414570 17606 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.439450 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:46.431265 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46025.8us
I20260709 12:22:46.447928 13719 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:46.449306 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.443142 17440 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:46.447388 17857 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:46.436625 17850 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:22:46.452947 17857 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.453742 17850 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:46.454300 17850 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:46.442385 17852 consensus_queue.cc:1243] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:22:46.456043 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:46.437079 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:46.459349 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.464315 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:46.466674 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:46.467695 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.456050 17817 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.438880 17802 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.478060 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.442307 17763 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.447846 17423 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:46.443619 17837 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:46.442095 17797 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.444320 17606 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:46.448730 17616 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:22:46.455503 17850 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.469383 17852 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:22:46.476902 17822 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:46.476729 17630 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:46.490419 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:46.437966 17849 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.469586 17737 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:46.473533 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:46.480834 17802 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:46.480037 17857 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.470031 17836 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:22:46.470506 17277 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:22:46.477815 17613 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:22:46.486773 17837 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:46.491003 17606 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:46.476239 17817 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.507128 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:46.517655 13792 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:46.499817 17616 consensus_queue.cc:1243] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:22:46.524832 13727 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:46.460081 17820 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.480363 17862 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:22:46.509126 17802 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:46.532497 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.510105 17850 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:46.515081 17828 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:46.490329 17423 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.516144 17773 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.517292 17630 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:46.545567 17849 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.548952 17578 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:46.550915 17825 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:22:46.556559 13799 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:46.559906 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.565824 13801 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:46.531152 17822 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:46.570004 13800 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:46.549569 17578 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.599421 17837 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.629990 17837 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:46.630293 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1927) took 1689 ms. Trace:
I20260709 12:22:46.630944 17578 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:46.599205 17630 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.635562 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:46.637082 17828 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.637638 17828 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.580524 17822 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:46.639982 17828 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.639613 17822 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.642345 17828 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:46.642637 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.642874 17828 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.630708 13795 rpcz_store.cc:276] 0709 12:22:44.941388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:44.984648 (+ 43260us) service_pool.cc:224] Handling call
0709 12:22:45.182119 (+197471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:45.251393 (+ 69274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.630058 (+1378665us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b5400102cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:46.630191 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":453244672}
I20260709 12:22:46.645552 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:46.646598 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.649495 13718 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:22:46.639153 17611 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.582243 17812 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:46.540110 17749 consensus_queue.cc:1175] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 9 status { last_received { term: 9 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:22:46.540776 17820 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:46.586853 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:46.590181 17669 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:46.602636 17819 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:22:46.591059 17870 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:22:46.577963 17486 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:46.604480 17423 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:46.593046 17869 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.107s	user 0.001s	sys 0.001s
W20260709 12:22:46.606922 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.609031 17817 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.609457 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.708180 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.709266 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.612118 17824 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:46.613699 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:46.714583 13798 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:46.678728 17578 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.614683 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:46.620144 13546 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 4 to 5, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:22:46.622392 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.717036 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.624409 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 2438) took 1089 ms. Trace:
I20260709 12:22:46.719009 13650 rpcz_store.cc:276] 0709 12:22:45.535012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.535116 (+   104us) service_pool.cc:224] Handling call
0709 12:22:46.624114 (+1088998us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400199454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:46.624341 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336429312}
I20260709 12:22:46.624863 13649 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:22:46.720623 13649 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 6
I20260709 12:22:46.720942 13649 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:22:46.721185 13649 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:46.594755 13796 pending_rounds.cc:85] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 6
I20260709 12:22:46.722100 13796 pending_rounds.cc:107] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:22:46.722410 13796 raft_consensus.cc:2889] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:46.595153 17843 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:46.724182 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:46.724565 17843 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:46.725033 13645 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:46.596629 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:46.596828 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:46.596998 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.729954 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.597277 17804 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.733990 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.734614 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:46.684892 17611 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:46.597635 17854 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:46.540144 17735 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.598618 17857 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:46.541299 17737 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:46.643304 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:46.580753 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.654310 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:46.768843 13648 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:46.658809 13792 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:22:46.643905 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.685681 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.672911 17828 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:46.771909 13718 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:46.692552 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.773309 13718 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:46.688047 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:46.695012 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108299us
I20260709 12:22:46.698429 14418 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:46.652726 17600 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.698760 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:46.699137 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:46.699604 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:46.775769 13644 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:46.699956 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:46.776530 13652 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:46.701208 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.627666 17751 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:46.736450 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 139891us
I20260709 12:22:46.748086 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:46.774606 14418 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:46.820840 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:46.735117 17423 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:46.750289 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:46.765318 13722 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:46.766026 13727 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:22:46.768693 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62144.8us
W20260709 12:22:46.828788 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.644176 17458 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:46.698493 17801 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:46.614553 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:46.749070 17611 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:46.806186 13648 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:46.835445 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:46.835816 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.836395 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:46.809657 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.739373 17861 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:46.817798 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54514.9us
I20260709 12:22:46.821877 13718 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:46.826736 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.832453 13546 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 6 to 7, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 7 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:46.744490 17797 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.693768 17669 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:46.783015 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:46.761199 17616 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:46.756907 17277 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:46.757205 17857 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:46.817581 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.817272 17737 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:46.854110 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 166604us
I20260709 12:22:46.759824 17763 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:46.867303 13546 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 7 to 9, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:46.841157 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.868733 17797 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.872543 17812 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:46.873350 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.767392 17854 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:46.873955 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.874357 13798 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:22:46.761025 17824 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260709 12:22:46.862984 17616 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:46.848308 13650 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:46.733958 17578 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.725539 17843 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.748812 17804 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.781944 17735 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:46.833547 17600 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:46.964787 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.078073 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:47.080307 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:46.961799 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.082034 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.083385 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:46.973299 13644 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:46.925483 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:46.933580 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:46.882920 14424 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:47.013962 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 265992us
I20260709 12:22:47.090334 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:47.081195 14417 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:47.079035 14422 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:46.833432 17798 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:46.837208 17323 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:46.971138 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:47.101296 14424 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:47.078814 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.087353 13546 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.112174 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:47.119745 14421 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:22:47.124214 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 293391us
I20260709 12:22:47.127404 13648 pending_rounds.cc:85] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 8
I20260709 12:22:47.127810 13648 pending_rounds.cc:107] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:22:47.128185 13648 raft_consensus.cc:2889] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:47.130383 13798 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:22:47.131054 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 160047us
I20260709 12:22:47.094132 17616 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.072647 17539 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:47.097551 17737 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:47.072999 17814 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.170231 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:47.122926 17606 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.002s
I20260709 12:22:46.836691 17423 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.178517 17812 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
W20260709 12:22:47.188021 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 437311us
I20260709 12:22:47.220162 13546 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 8 to 9, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:46.866060 17861 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.137130 17763 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:22:47.255640 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 143570us
I20260709 12:22:47.177145 17436 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:47.269407 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:46.927485 17277 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:47.274915 13546 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.192211 17814 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.128863 17323 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.327718 13721 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:47.138468 17817 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.130s
I20260709 12:22:47.130224 17798 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.101344 17857 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.335007 13546 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.187187 17539 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.278689 17828 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:47.280237 17849 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.273942 17436 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:47.363034 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 229) took 46400 ms (client timeout 15000 ms). Trace:
W20260709 12:22:47.346284 17802 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.805s	user 0.013s	sys 0.000s
I20260709 12:22:47.196264 17606 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.273788 17855 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:22:47.355775 13546 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:22:47.362571 17880 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.487s	user 0.001s	sys 0.001s
I20260709 12:22:47.403746 13546 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 5, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 5 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:47.368399 17857 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:47.330536 17763 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.369632 17436 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:47.368880 17817 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:47.414008 13546 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.391201 17713 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:47.409798 17878 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.449s	user 0.000s	sys 0.001s
I20260709 12:22:47.377485 17849 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.476428 17849 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.429744 17713 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.480863 17713 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.407430 17889 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:22:47.366873 13547 rpcz_store.cc:269] 0709 12:22:00.599846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:00.607916 (+  8070us) service_pool.cc:224] Handling call
0709 12:22:02.532585 (+1924669us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:22:02.754861 (+222276us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393bfcf3a 00007f53999b2e9c 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:47.000114 (+44245253us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:40.646071 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:40.713919 (+ 67848us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:40.713990 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:22:40.714014 (+    24us) tablet.cc:662] Decoding operations
0709 12:22:40.714801 (+   787us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:40.714944 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:40.714977 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:22:40.715526 (+   549us) tablet.cc:701] Row locks acquired
0709 12:22:40.715577 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:22:40.715838 (+   261us) write_op.cc:270] Start()
0709 12:22:40.716004 (+   166us) write_op.cc:276] Timestamp: P: 1783599760715745 usec, L: 0
0709 12:22:40.716028 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:22:40.781875 (+ 65847us) log.cc:844] Serialized 1303 byte log entry
0709 12:22:42.079311 (+1297436us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b180029f740. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:42.093023 (+ 13712us) op_driver.cc:464] REPLICATION: finished
0709 12:22:42.107167 (+ 14144us) write_op.cc:301] APPLY: starting
0709 12:22:42.107275 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:42.107795 (+   520us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:42.107852 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:42.359443 (+251591us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:42.359792 (+   349us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=3,key_file_lookups=3,delta_file_lookups=0,mrs_lookups=6
0709 12:22:42.359845 (+    53us) write_op.cc:312] APPLY: finished
0709 12:22:42.360179 (+   334us) log.cc:844] Serialized 53 byte log entry
0709 12:22:42.360410 (+   231us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:42.360524 (+   114us) write_op.cc:454] Released schema lock
0709 12:22:42.360594 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:42.361120 (+   526us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:42.361154 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":270813,"reported_tablets":33,"spinlock_wait_cycles":91040768,"child_traces":[["op",{"apply.queue_time_us":13838,"apply.run_cpu_time_us":19089,"apply.run_wall_time_us":253898,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4188,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6221,"cfile_init":1,"lbm_read_time_us":109760,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":6,"num_ops":3,"prepare.queue_time_us":258220,"prepare.run_cpu_time_us":14265,"prepare.run_wall_time_us":579801,"raft.queue_time_us":11164,"raft.run_cpu_time_us":1172,"raft.run_wall_time_us":18136,"replication_time_us":1376785,"spinlock_wait_cycles":17307520,"thread_start_us":427269,"threads_started":4,"wal-append.queue_time_us":362564,"wal-append.run_cpu_time_us":13568,"wal-append.run_wall_time_us":2457544}]]}
I20260709 12:22:47.436190 17323 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:47.439324 17539 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.439556 17578 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.324646 17854 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.441460 17814 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.441627 17905 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:47.442811 17895 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:22:47.449628 17763 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:47.450148 17899 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:47.452446 17876 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:22:47.456220 17861 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.457942 17606 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:47.459406 17798 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.460151 17616 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:47.462500 17458 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.624s	user 0.006s	sys 0.005s
W20260709 12:22:47.464246 17850 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.926s	user 0.011s	sys 0.002s
I20260709 12:22:47.466500 17804 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.470952 17423 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.471566 17842 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:47.472816 17797 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.422511 17817 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.484519 13546 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 9 to 10, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:47.416375 17669 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.489774 17323 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:47.501845 17854 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:47.466001 17837 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.836s	user 0.008s	sys 0.002s
I20260709 12:22:47.522898 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:47.615160 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92337.5us
I20260709 12:22:47.390661 17802 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.601121 13645 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:22:47.597607 17878 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:22:47.605160 17458 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:47.502512 17905 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:47.607748 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 476122us
I20260709 12:22:47.609241 13643 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:47.593464 17889 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.645284 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2211) took 1167 ms. Trace:
I20260709 12:22:47.640218 13795 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:47.610870 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 341521us
W20260709 12:22:47.603561 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 294352us
I20260709 12:22:47.600520 17606 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:47.599053 17798 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:47.623214 13546 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 4 to 7, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 7 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.626005 17804 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
W20260709 12:22:47.666826 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.683040 13726 rpcz_store.cc:276] 0709 12:22:46.478394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.478591 (+   197us) service_pool.cc:224] Handling call
0709 12:22:46.479019 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:46.479438 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.479485 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:46.483196 (+  3711us) log.cc:844] Serialized 26 byte log entry
0709 12:22:46.708509 (+225313us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:46.708574 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.708676 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.644980 (+936304us) raft_consensus.cc:1685] finished
0709 12:22:47.645013 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.645201 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":740,"raft.queue_time_us":125,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":4867,"thread_start_us":204834,"threads_started":1,"wal-append.queue_time_us":589496}
I20260709 12:22:47.612102 17842 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:47.626415 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:47.683856 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.003s	sys 0.009s
W20260709 12:22:47.637761 17801 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.016s	sys 0.043s
I20260709 12:22:47.693718 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:47.753113 13643 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
W20260709 12:22:47.695371 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.754536 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.755131 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.756275 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:47.615048 17669 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:22:47.665666 17823 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.006s	sys 0.024s
W20260709 12:22:47.698087 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.009s	sys 0.005s
W20260709 12:22:47.776547 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.009s	sys 0.005s
W20260709 12:22:47.776963 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 651947us
I20260709 12:22:47.704267 13546 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 5, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:47.707916 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:47.701179 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.709527 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:47.709816 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.003s	sys 0.011s
W20260709 12:22:47.780234 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 661436us
I20260709 12:22:47.781569 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:47.711980 17902 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:47.712931 17897 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:22:47.792599 17669 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:47.793790 17669 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.773818 17630 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:47.794453 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2220) took 1015 ms. Trace:
I20260709 12:22:47.794971 17630 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.794921 13718 rpcz_store.cc:276] 0709 12:22:46.779534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.843561 (+ 64027us) service_pool.cc:224] Handling call
0709 12:22:46.843884 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:46.866480 (+ 22596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.866514 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:46.867059 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:22:47.271214 (+404155us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:47.271277 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.271387 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.794158 (+522771us) raft_consensus.cc:1685] finished
0709 12:22:47.794181 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.794379 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22198,"raft.queue_time_us":32241,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":42029,"thread_start_us":272061,"threads_started":1,"wal-append.queue_time_us":632459}
I20260709 12:22:47.797329 17852 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:22:47.798307 17323 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.798766 17323 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:47.799152 17323 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:47.802775 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.713200 17905 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:47.713886 17820 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.024s	user 0.012s	sys 0.004s
I20260709 12:22:47.714704 17616 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.692468 17458 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.692214 17909 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:22:47.716317 17916 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:47.716922 17798 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.687573 17861 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:22:47.689388 17837 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:47.689736 17578 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.845572 17578 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.847785 17578 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:47.848414 17578 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.848963 17578 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.689764 17606 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:47.851019 17606 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.852100 17606 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:47.852569 17877 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.131s	user 0.010s	sys 0.012s
W20260709 12:22:47.718434 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92172.9us
I20260709 12:22:47.854637 17916 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.854876 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:47.731668 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:47.733943 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 781) took 1033 ms. Trace:
I20260709 12:22:47.707371 17889 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.736102 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.857020 17916 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.735603 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.857843 13792 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:47.736735 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 740) took 2029 ms. Trace:
I20260709 12:22:47.817924 17915 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.858944 14425 rpcz_store.cc:276] 0709 12:22:45.707383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.707534 (+   151us) service_pool.cc:224] Handling call
0709 12:22:45.721863 (+ 14329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.412894 (+691031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.736452 (+1323558us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:47.736669 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1499929856}
I20260709 12:22:47.858577 13797 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:47.859735 17889 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.860471 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:47.713251 17882 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.446s	user 0.000s	sys 0.005s
I20260709 12:22:47.736783 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2451) took 2094 ms. Trace:
I20260709 12:22:47.863359 13649 rpcz_store.cc:276] 0709 12:22:45.642202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.650479 (+  8277us) service_pool.cc:224] Handling call
0709 12:22:46.624363 (+973884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.721747 (+ 97384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.736619 (+1014872us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b5400199450. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:47.736730 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2238857472}
I20260709 12:22:47.738400 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:47.864816 14424 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:47.866117 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2458) took 1219 ms. Trace:
I20260709 12:22:47.775413 17842 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.866551 13651 rpcz_store.cc:276] 0709 12:22:46.646642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.648527 (+  1885us) service_pool.cc:224] Handling call
0709 12:22:46.648992 (+   465us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:46.649620 (+   628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.649652 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:46.653858 (+  4206us) log.cc:844] Serialized 52 byte log entry
0709 12:22:46.760255 (+106397us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:46.760315 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.760392 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.481629 (+721237us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:47.865734 (+384105us) raft_consensus.cc:1685] finished
0709 12:22:47.865759 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.866050 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2210,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":55501696,"thread_start_us":77230,"threads_started":1,"wal-append.queue_time_us":291081}
I20260709 12:22:47.867596 17842 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:47.857085 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:47.738986 17556 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:47.868547 17556 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.868618 17862 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.869032 17556 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:47.869149 17862 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.869666 17862 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.739995 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:47.834863 17797 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:22:47.871307 14422 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:22:47.743266 17802 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:47.872085 17848 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:47.873463 17458 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:47.741572 17817 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:47.874819 17817 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:22:47.876502 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:47.876999 17817 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:47.877497 17817 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:47.878371 17539 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:47.857069 14421 rpcz_store.cc:276] 0709 12:22:46.700420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.701867 (+  1447us) service_pool.cc:224] Handling call
0709 12:22:47.089405 (+387538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.120286 (+ 30881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.733678 (+613392us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400498c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:47.733871 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890444032}
W20260709 12:22:47.748528 17885 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.400s	user 0.001s	sys 0.000s
I20260709 12:22:47.880885 17914 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.757272 17841 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.881415 17914 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:47.881945 17914 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:47.757783 17923 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:47.759311 14420 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:47.885509 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:47.613831 17277 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:47.767339 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.888291 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:47.888715 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.755225 17763 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:47.752887 17801 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.018s	sys 0.045s
W20260709 12:22:47.710923 17831 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.796s	user 0.003s	sys 0.002s
I20260709 12:22:47.909533 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:47.782073 13546 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 5 to 7, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 7 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:22:47.783331 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75520.1us
W20260709 12:22:47.774819 17823 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.009s	sys 0.024s
I20260709 12:22:47.789028 17913 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:47.790723 17902 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:47.807278 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97813.6us
I20260709 12:22:47.805655 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:47.782526 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:47.916358 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:47.817533 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2616) took 1134 ms. Trace:
I20260709 12:22:47.918607 13722 rpcz_store.cc:276] 0709 12:22:46.683574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.719923 (+ 36349us) service_pool.cc:224] Handling call
0709 12:22:46.765065 (+ 45142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:46.827960 (+ 62895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.806232 (+978272us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b54001a4850. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:47.817450 (+ 11218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102993152}
I20260709 12:22:47.821558 14419 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:22:47.824956 13794 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:47.841799 13798 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:47.805908 17804 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.834224 17773 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:47.844473 17837 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.847450 17436 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:47.747581 17611 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.884s	user 0.003s	sys 0.011s
I20260709 12:22:47.750711 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2454) took 1122 ms. Trace:
I20260709 12:22:47.881466 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1865) took 1052 ms. Trace:
I20260709 12:22:47.759002 13793 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:22:47.935917 13650 rpcz_store.cc:276] 0709 12:22:46.829159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.831303 (+  2144us) service_pool.cc:224] Handling call
0709 12:22:46.848058 (+ 16755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.085501 (+237443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.881206 (+795705us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b54002229d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:47.881399 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37536512}
I20260709 12:22:47.883898 13718 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:47.887331 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2209) took 1411 ms. Trace:
W20260709 12:22:47.896174 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:47.896570 13726 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:47.908646 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.941340 13799 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:47.935307 13647 rpcz_store.cc:276] 0709 12:22:46.628421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.630357 (+  1936us) service_pool.cc:224] Handling call
0709 12:22:46.630730 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.137822 (+507092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.137882 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.174957 (+ 37075us) log.cc:844] Serialized 26 byte log entry
0709 12:22:47.401316 (+226359us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:47.401362 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.750173 (+348811us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b5400095954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:47.750230 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.750317 (+    87us) raft_consensus.cc:1685] finished
0709 12:22:47.750336 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.750633 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":351757,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":1162836736,"thread_start_us":141310,"threads_started":1,"wal-append.queue_time_us":272631}
I20260709 12:22:47.923852 13546 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.910243 17916 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:47.922703 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 140255us
I20260709 12:22:47.937878 13724 rpcz_store.cc:276] 0709 12:22:46.476188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.476351 (+   163us) service_pool.cc:224] Handling call
0709 12:22:46.486674 (+ 10323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:46.487235 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.487278 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:46.495666 (+  8388us) log.cc:844] Serialized 52 byte log entry
0709 12:22:46.719077 (+223411us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:46.719119 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:46.719187 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.852216 (+1133029us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b2400166aa8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b4609d 00007f539377810e 00007f539378f88c 00007f539378ebb6 00007f539378e7d8 00007f5393798221
0709 12:22:47.886955 (+ 34739us) raft_consensus.cc:1685] finished
0709 12:22:47.886989 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:47.887260 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2207,"raft.queue_time_us":3136,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4253,"spinlock_wait_cycles":51778816,"thread_start_us":204956,"threads_started":1,"wal-append.queue_time_us":217671}
W20260709 12:22:47.945664 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 246977us
I20260709 12:22:47.909531 17801 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:47.886601 17824 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:22:47.890599 17841 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.947202 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:47.932646 17899 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:47.962303 17906 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:47.966341 17630 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:47.945220 13546 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:22:47.973686 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:47.898744 17923 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.940589 17861 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:47.913116 17763 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.939918 17822 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:48.003618 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.003944 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.016265 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:48.016525 13722 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:48.004562 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.018123 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.005256 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:47.968129 17915 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:47.975176 17423 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.007519 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.030536 13792 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:22:47.994097 14418 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.954701 17820 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.005942 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.019191 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.022111 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.024506 13798 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:48.023907 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.006701 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:47.931890 17918 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:22:48.028151 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.031586 14418 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:22:48.032239 17890 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.035s	sys 0.026s
I20260709 12:22:47.988045 17836 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:48.072399 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2214) took 1524 ms. Trace:
I20260709 12:22:48.072849 13719 rpcz_store.cc:276] 0709 12:22:46.546115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.651045 (+104930us) service_pool.cc:224] Handling call
0709 12:22:46.651376 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:46.651766 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.651798 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:46.652398 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 12:22:46.839861 (+187463us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:47.140451 (+300590us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.605877 (+465426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.605956 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.747747 (+141791us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:48.030369 (+282622us) raft_consensus.cc:1685] finished
0709 12:22:48.069972 (+ 39603us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.070205 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":537458,"prepare.run_wall_time_us":6019,"raft.queue_time_us":331463,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":72286,"spinlock_wait_cycles":61307392,"thread_start_us":886936,"threads_started":3,"wal-append.queue_time_us":534346}
I20260709 12:22:48.040771 13652 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:48.048609 13723 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:48.049693 13722 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:48.054973 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:47.939931 17919 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 12:22:48.094348 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.058540 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:47.940198 17611 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.970710 17616 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:22:47.954555 17916 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:47.975642 17889 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:22:48.022907 17906 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:48.055716 17890 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.035s	sys 0.026s
I20260709 12:22:48.103595 13798 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:48.104382 13792 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:22:47.935708 17817 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:47.973659 17901 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 12:22:48.025823 17822 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.010987 17924 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
W20260709 12:22:48.038849 17857 consensus_queue.cc:1175] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 7 status { last_received { term: 7 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 7 index: 8 } }
I20260709 12:22:48.069459 17836 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.028107 17556 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.110478 17906 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.070799 17812 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.094552 17861 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:48.017951 17899 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 12:22:48.061815 17820 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.133276 17824 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:48.162674 17824 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:48.143260 17737 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.904s	user 0.013s	sys 0.006s
I20260709 12:22:48.131450 17713 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.140358 17836 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:48.162189 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.163165 17824 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.049284 17423 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.095400 17675 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 12:22:48.150089 17884 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.603s	user 0.000s	sys 0.006s
I20260709 12:22:48.146294 17817 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:22:48.181968 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 208352us
I20260709 12:22:48.154553 17798 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.127697 17823 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 12:22:48.197417 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.163923 17737 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.170787 17902 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.191893 17932 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.155435 17812 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.125969 17801 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.169474 17713 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.215488 17798 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:48.209911 17917 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:48.211836 17578 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:48.247622 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 304427us
I20260709 12:22:48.249505 13727 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
W20260709 12:22:48.249776 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.256249 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.169718 17922 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.291s
I20260709 12:22:48.231396 17837 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:48.225957 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 281410us
W20260709 12:22:48.273125 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.280179 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.281635 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.239919 17902 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.285390 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:48.289500 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:48.291699 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:48.250784 17436 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.300518 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:48.295626 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.296589 13721 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:48.250185 17798 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:48.298135 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.303927 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:48.315963 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.317677 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:48.319459 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:48.321178 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:48.323796 17846 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:22:48.324234 17939 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:48.285959 17578 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.284008 17837 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:48.324970 17905 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:48.292579 17611 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:48.296154 17938 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.302966 17904 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:48.298804 17916 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.236572 17861 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.325389 17436 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:48.293800 17902 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.347504 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:48.343477 17837 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.368883 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:48.359803 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.337953 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:48.361519 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.326725 17802 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:48.327237 17423 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.334434 17905 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:48.378307 17820 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.372973 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:48.379143 17440 raft_consensus.cc:2764] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:22:48.324800 17939 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.331749 17630 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.380347 14418 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:22:48.397006 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 779) took 1706 ms. Trace:
I20260709 12:22:48.397365 14418 rpcz_store.cc:276] 0709 12:22:46.690151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.690413 (+   262us) service_pool.cc:224] Handling call
0709 12:22:46.698226 (+  7813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.979133 (+1280907us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b5400555b20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b7206f 00007f5393beaf60 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62
0709 12:22:48.396707 (+417574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.396943 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10647936}
I20260709 12:22:48.397532 17630 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.400813 17630 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.267590 17917 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:48.392441 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:48.393496 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93073.5us
I20260709 12:22:48.369910 17846 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.369452 17556 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.334594 17578 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.405331 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:48.365473 17904 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.376434 17916 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:48.463234 17916 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.243333 17932 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.388985 17905 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:48.449337 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.394937 17423 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 12:22:48.451555 17931 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:22:48.448423 17846 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.473217 17953 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:48.474237 17953 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:48.475766 17949 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:48.451952 17941 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:48.452392 17888 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:48.452814 17942 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.478005 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:48.359141 17938 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:48.479382 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.461010 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:48.461238 17933 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:22:48.461865 17556 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:48.462059 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:48.462059 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:48.442744 17855 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.488045 14422 raft_consensus.cc:2376] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7. Re-sending same reply.
I20260709 12:22:48.491889 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:48.454933 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:48.480754 17937 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:22:48.381078 17890 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:22:48.488878 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:48.490653 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.531556 14418 raft_consensus.cc:2368] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 8.
I20260709 12:22:48.538507 14422 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:22:48.539999 17947 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.486930 17737 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.491734 17888 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.550868 17916 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.551290 17951 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:48.551673 17914 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.001s
I20260709 12:22:48.551924 17951 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.530295 17940 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.530910 17713 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:48.537020 17812 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.551584 17837 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:48.561707 17835 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:22:48.562893 17846 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:48.563102 17930 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:22:48.567369 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2738) took 1464 ms. Trace:
I20260709 12:22:48.610798 13801 rpcz_store.cc:276] 0709 12:22:47.103191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.118409 (+ 15218us) service_pool.cc:224] Handling call
0709 12:22:47.118775 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.386701 (+267926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.386766 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:47.386831 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:47.386892 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.567088 (+1180196us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:48.567142 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.567297 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615422848}
I20260709 12:22:48.612499 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.613626 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:48.568830 17862 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.003s
I20260709 12:22:48.556146 17888 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:48.480801 17942 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.571067 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2470) took 1874 ms. Trace:
I20260709 12:22:48.571164 17957 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:48.572028 17934 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:22:48.624944 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.540828 17947 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.530441 17941 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.552248 17914 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:48.591269 17842 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.723s	user 0.017s	sys 0.000s
I20260709 12:22:48.554198 17855 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.587682 17836 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:48.621536 17944 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:22:48.578781 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.643783 14424 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:48.620347 13645 rpcz_store.cc:276] 0709 12:22:46.696568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.766944 (+ 70376us) service_pool.cc:224] Handling call
0709 12:22:46.817126 (+ 50182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.682667 (+865541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.570800 (+888133us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400115154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:48.570991 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801939200}
W20260709 12:22:48.623862 17323 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.002s
I20260709 12:22:48.627085 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.624543 17846 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.648034 17846 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.648561 17938 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.648780 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.624395 17862 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.636287 17735 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:48.637071 17904 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.638058 17939 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.638523 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 747) took 2715 ms. Trace:
I20260709 12:22:48.638782 17578 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.640134 17932 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.632447 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.654160 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:48.668787 13722 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.658835 13720 rpcz_store.cc:276] 0709 12:22:45.922909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:45.923254 (+   345us) service_pool.cc:224] Handling call
0709 12:22:45.923789 (+   535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:46.091439 (+167650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.091471 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:46.613713 (+522242us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:47.772068 (+1158355us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:47.774611 (+  2543us) log.cc:844] Serialized 52 byte log entry
0709 12:22:48.129781 (+355170us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:48.129823 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.139139 (+  9316us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:48.638210 (+499071us) raft_consensus.cc:1685] finished
0709 12:22:48.638234 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.638432 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18189,"raft.queue_time_us":165824,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":46202,"spinlock_wait_cycles":71521024,"thread_start_us":504791,"threads_started":2,"wal-append.queue_time_us":448970}
I20260709 12:22:48.650841 17846 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.661815 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:48.670809 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:48.671638 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:48.630934 17947 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:48.631043 17843 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:48.673789 13720 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.677363 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.659861 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:48.682928 13643 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:48.691479 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:48.689649 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.700944 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.701303 17941 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.701777 13793 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
W20260709 12:22:48.705415 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43668.4us
I20260709 12:22:48.705293 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:48.705217 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 332344us
I20260709 12:22:48.708997 13643 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:48.713446 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.708142 13645 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:22:48.719166 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:48.674021 17939 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:48.655642 17323 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.724185 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.606243 17940 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.724726 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.728986 13651 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.728322 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:48.736495 13722 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:22:48.739898 13651 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:48.737556 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:48.703110 17917 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.719818 17947 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:22:48.724220 17906 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.006s
I20260709 12:22:48.753854 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.706759 17950 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.754760 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:48.757347 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.759944 13645 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:48.770715 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.774083 13649 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:22:48.774415 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:48.774644 13646 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:48.762316 13651 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:48.771816 13650 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:22:48.775342 13793 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:48.754006 17824 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.001s
W20260709 12:22:48.754213 17945 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.215s	user 0.001s	sys 0.001s
I20260709 12:22:48.777271 13651 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:48.739396 17836 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:48.778891 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 373595us
W20260709 12:22:48.758720 17606 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.906s	user 0.015s	sys 0.000s
I20260709 12:22:48.756213 17735 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:48.742830 17855 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.784247 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1869) took 1954 ms. Trace:
I20260709 12:22:48.756572 17947 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.783691 17539 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:48.790354 17914 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.794634 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.827363 13651 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:48.783531 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:48.828894 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45602.2us
I20260709 12:22:48.734808 17939 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.792806 13648 rpcz_store.cc:276] 0709 12:22:46.830285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.856971 (+ 26686us) service_pool.cc:224] Handling call
0709 12:22:46.857292 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:47.328781 (+471489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.328822 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:47.329535 (+   713us) log.cc:844] Serialized 52 byte log entry
0709 12:22:47.645321 (+315786us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:47.909547 (+264226us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.910091 (+   544us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.921786 (+ 11695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.778733 (+856947us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:48.778792 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:48.783996 (+  5204us) raft_consensus.cc:1685] finished
0709 12:22:48.784032 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:48.784185 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":682121,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":102,"raft.queue_time_us":346374,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":4246,"spinlock_wait_cycles":644353408,"thread_start_us":619394,"threads_started":3,"wal-append.queue_time_us":448719}
I20260709 12:22:48.820515 14420 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.772444 17906 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.783782 17776 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:48.779415 17862 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:48.824213 14422 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:48.818831 17947 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:48.834059 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 523) took 1605 ms. Trace:
I20260709 12:22:48.786994 17735 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.838936 17735 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:48.839462 17942 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.837411 13795 rpcz_store.cc:276] 0709 12:22:47.228286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.228440 (+   154us) service_pool.cc:224] Handling call
0709 12:22:47.324540 (+ 96100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.683312 (+358772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.833748 (+1150436us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54002cb5d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:48.833975 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725293440}
I20260709 12:22:48.786218 17824 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.787243 17950 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.846206 13647 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 4.
I20260709 12:22:48.850080 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.848567 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.852396 13648 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:48.852537 13644 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:22:48.850867 13799 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:48.850875 13649 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:48.853574 13648 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
W20260709 12:22:48.845331 17817 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.003s
I20260709 12:22:48.835395 17776 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:48.842165 17600 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:48.875993 17940 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.857980 13795 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:22:48.878559 13795 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:48.880661 13795 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.856364 13793 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:48.856559 13651 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:48.873906 13643 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:48.826166 17905 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.844975 17964 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:48.895911 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 102982us
I20260709 12:22:48.896243 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:48.897217 13651 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:48.874423 17776 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:48.902073 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.900655 13649 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:48.902974 13648 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:48.868901 17914 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:22:48.871769 17899 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:22:48.909282 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:48.910873 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.895264 17836 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.915885 17804 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:48.916469 17804 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:48.916020 17801 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:48.917275 17804 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:48.918470 17804 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:48.918603 17801 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:48.920461 17801 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:48.921360 17964 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:48.921974 17964 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:48.923652 13651 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:22:48.934434 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:48.935350 13651 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:48.941138 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:48.960167 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.935783 17846 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:48.940117 17323 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:48.983844 17961 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:22:48.922958 17964 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.985186 17824 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.925122 17836 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:48.986403 17971 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:22:48.987517 17836 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:48.975790 17962 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:22:48.977352 17846 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:48.981204 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:48.992741 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.000092 14416 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:49.000818 17675 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.001327 17675 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:48.985213 17964 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.002635 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.002943 17939 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.007877 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.009716 17842 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.010344 17842 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:49.012177 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.012847 17842 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.002775 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:49.015731 17846 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:48.995460 17906 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:49.016173 13615 leader_election.cc:400] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 8.
I20260709 12:22:49.020236 17913 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.023495 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:49.024220 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.025142 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.025754 13648 raft_consensus.cc:2376] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13. Re-sending same reply.
I20260709 12:22:49.026631 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.032302 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.033295 13649 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:49.037649 17950 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:49.040762 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.042168 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.042879 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:22:49.046080 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.046741 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.047812 17828 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.048317 17828 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.048756 17905 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:49.049795 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1937) took 2493 ms. Trace:
I20260709 12:22:49.051158 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.050173 13796 rpcz_store.cc:276] 0709 12:22:46.556647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.556803 (+   156us) service_pool.cc:224] Handling call
0709 12:22:46.594528 (+ 37725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:46.723223 (+128695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:46.723254 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:46.723968 (+   714us) log.cc:844] Serialized 26 byte log entry
0709 12:22:47.071964 (+347996us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:47.350095 (+278131us) log.cc:844] Serialized 14 byte log entry
0709 12:22:47.690011 (+339916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:47.690125 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:47.877625 (+187500us) spinlock_profiling.cc:243] Waited 59.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:48.260117 (+382492us) raft_consensus.cc:1685] finished
0709 12:22:48.267088 (+  6971us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.049618 (+782530us) spinlock_profiling.cc:243] Waited 35.9 ms on lock 0x7b54002cbfd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.049735 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":270375,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":90944,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"spinlock_wait_cycles":220322560,"thread_start_us":912129,"threads_started":3,"wal-append.queue_time_us":449798}
I20260709 12:22:49.052062 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:49.053745 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.053973 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:49.054052 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.054852 17901 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:49.055269 17901 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.056867 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:49.057600 13725 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.055763 17901 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.060268 13649 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:49.068609 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:49.068850 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 347985us
I20260709 12:22:49.076871 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:49.077319 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.078541 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.079694 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.080729 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.081876 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.082445 13648 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:49.086447 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:49.083284 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.064330 17905 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:49.088084 13648 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:49.091830 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.092453 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.093325 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.093977 14425 raft_consensus.cc:2410] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:22:49.095572 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:49.095701 13725 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:49.064536 17915 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:49.097602 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:49.067708 17828 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.098686 13725 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:49.099388 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:49.064615 17846 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:22:49.106855 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:49.108909 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:49.111047 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:49.111212 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.111897 13648 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:49.112071 17855 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:49.091114 17888 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.112788 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.113574 17932 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:49.113837 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.096079 17423 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:49.091534 17905 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:49.114502 17888 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:49.076500 17539 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 12:22:49.125620 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.127444 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.130224 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:49.136400 14421 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:22:49.136804 14421 pending_rounds.cc:85] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 9
I20260709 12:22:49.137110 14421 pending_rounds.cc:107] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:22:49.137377 14421 raft_consensus.cc:2889] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:49.138568 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.139101 17941 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:49.123741 17423 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:49.129029 17905 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:49.138525 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:49.142822 13725 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.131659 17855 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:49.137269 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.140008 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.146776 13648 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:49.134732 17917 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:49.149672 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:49.150321 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.132462 17901 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.142163 17423 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:49.151449 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.140226 17953 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:22:49.152305 17828 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:49.150223 17917 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:49.131942 17947 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.201593 13725 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:49.157536 17854 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:49.213088 17735 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.213546 17854 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:49.214375 17902 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.214568 17854 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.215925 14420 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:49.153074 17950 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:22:49.219337 14419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 776) took 2577 ms (client timeout 2121 ms). Trace:
W20260709 12:22:49.219842 14419 rpcz_store.cc:269] 0709 12:22:46.641681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.645282 (+  3601us) service_pool.cc:224] Handling call
0709 12:22:49.219060 (+2573778us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.219225 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2702409472}
I20260709 12:22:49.225222 14417 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:22:49.225842 14417 pending_rounds.cc:85] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 7
I20260709 12:22:49.226317 14417 pending_rounds.cc:107] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:22:49.226892 14417 raft_consensus.cc:2889] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:49.157821 17963 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:49.162547 17932 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:49.268975 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:49.269860 14421 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.161967 17606 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:22:49.212786 17947 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:49.198843 17917 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:49.200737 17828 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:49.231557 17953 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:22:49.275920 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126205us
I20260709 12:22:49.210204 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:49.277647 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 185623us
I20260709 12:22:49.267908 17323 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:49.300573 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:49.308406 13721 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:22:49.310016 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:49.312074 13721 pending_rounds.cc:85] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 4
I20260709 12:22:49.312369 13721 pending_rounds.cc:107] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:22:49.312291 13722 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.312763 13721 raft_consensus.cc:2889] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:49.268476 17963 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:49.272517 17932 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:49.329609 13648 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:49.352182 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.290302 17828 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:49.368072 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 157921us
I20260709 12:22:49.369532 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:49.351336 17930 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:22:49.307013 17953 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:49.383184 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.387818 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87318.2us
W20260709 12:22:49.388756 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.392097 13722 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:49.354063 17932 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.392658 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.354740 17963 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:49.354430 17323 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.394327 17805 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:49.399556 17862 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:49.418833 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.422322 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.425788 14421 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:49.426594 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.427269 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:49.430135 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.431239 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.432307 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.435482 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.407965 17991 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:22:49.437565 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.439383 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.439464 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.441783 13795 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:49.439381 17901 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:49.450521 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.451289 13801 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:49.454421 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:49.460829 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:49.470211 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.576589 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.429253 17953 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.464023 17991 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:49.590453 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.467329 17904 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:22:49.566149 17923 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.600593 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:49.501521 17854 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:49.606953 14418 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.469494 17805 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:49.526531 17813 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:49.595271 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.523377 17763 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.605427 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 525) took 2336 ms. Trace:
I20260709 12:22:49.473121 17735 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:49.481026 17886 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:22:49.600593 17964 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.003s
I20260709 12:22:49.523325 17951 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.614666 13648 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:22:49.614229 13794 rpcz_store.cc:276] 0709 12:22:47.269035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.779348 (+510313us) service_pool.cc:224] Handling call
0709 12:22:47.824641 (+ 45293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:47.920546 (+ 95905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.605094 (+1684548us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b54000d2fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.605356 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103166848}
W20260709 12:22:49.616585 17990 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:22:49.604086 17953 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.650976 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.651871 13648 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:22:49.604036 17983 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:49.604867 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:49.604918 17904 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:49.654722 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:49.613832 17813 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:49.655929 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.604453 17906 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:22:49.662773 13648 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:22:49.649747 17886 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:49.616143 17854 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:49.666018 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60765.2us
I20260709 12:22:49.666424 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.661573 17904 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:49.671734 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.674469 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.675433 13648 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:22:49.666177 17323 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:49.677316 17997 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:22:49.682533 17813 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.676254 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.667873 17987 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:49.685900 17935 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.380s	user 0.007s	sys 0.006s
W20260709 12:22:49.675879 17776 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.004s
I20260709 12:22:49.669672 17983 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:49.699939 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.700253 17886 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.703503 17852 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:49.704486 17776 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:49.710507 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1870) took 2879 ms. Trace:
I20260709 12:22:49.647863 17991 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:49.711342 13652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 2474) took 2115 ms (client timeout 2091 ms). Trace:
I20260709 12:22:49.712385 17813 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:49.728982 13652 rpcz_store.cc:269] 0709 12:22:47.596189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.731382 (+135193us) service_pool.cc:224] Handling call
0709 12:22:49.711100 (+1979718us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.711245 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423323392}
W20260709 12:22:49.713058 17984 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.321s	user 0.000s	sys 0.002s
I20260709 12:22:49.735309 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.714443 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.736780 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.727241 13644 rpcz_store.cc:276] 0709 12:22:46.830549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.099829 (+269280us) service_pool.cc:224] Handling call
0709 12:22:47.100462 (+   633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.852923 (+1752461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.710149 (+857226us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54001482d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:49.710396 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4028682112}
I20260709 12:22:49.728953 17852 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:49.737432 13648 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:49.746140 17852 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.751323 17852 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.751950 17776 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:49.754076 17850 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:49.754578 17850 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:49.755497 17850 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.756495 17850 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.725214 17987 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:49.769876 14422 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:49.771386 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:49.784746 14419 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEARNER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:49.795006 17904 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.796830 17964 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:49.797495 17904 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:49.798058 17964 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:49.798525 17964 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:49.799778 18003 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:22:49.799484 17964 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.800767 17964 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:49.798525 17822 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:49.804356 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.804966 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.805446 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.806553 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.807750 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:49.816629 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:49.820715 14416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 615) took 3104 ms (client timeout 1605 ms). Trace:
W20260709 12:22:49.821228 14416 rpcz_store.cc:269] 0709 12:22:46.716873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:46.826215 (+109342us) service_pool.cc:224] Handling call
0709 12:22:49.820523 (+2994308us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54005965d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.820637 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4957596416}
I20260709 12:22:49.823802 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.827891 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:49.828423 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.831012 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.807834 17862 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:49.783815 17919 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:22:49.836887 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46913.8us
I20260709 12:22:49.838002 14419 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:49.840299 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:49.844933 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.845639 14419 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:22:49.853708 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:49.854899 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:49.857968 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:49.859203 14422 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:49.862924 17798 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.864820 17798 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:49.864907 17854 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.860355 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:22:49.877601 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:49.860370 17423 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.012s
I20260709 12:22:49.850608 17919 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:49.868423 17813 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:49.890188 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 799) took 1418 ms. Trace:
I20260709 12:22:49.893663 14424 rpcz_store.cc:276] 0709 12:22:48.461418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.461695 (+   277us) service_pool.cc:224] Handling call
0709 12:22:49.880050 (+1418355us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.880178 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353608576}
I20260709 12:22:49.890795 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2746) took 1982 ms. Trace:
I20260709 12:22:49.869259 17836 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:22:49.892190 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.894586 13797 rpcz_store.cc:276] 0709 12:22:47.908344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.908480 (+   136us) service_pool.cc:224] Handling call
0709 12:22:47.920800 (+ 12320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:47.971764 (+ 50964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:47.971796 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:47.972392 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.197899 (+225507us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:48.197944 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:48.198017 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:49.685262 (+1487245us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:49.708630 (+ 23368us) raft_consensus.cc:1685] finished
0709 12:22:49.708655 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:49.890615 (+181960us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b54000d2fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:49.890728 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64949,"raft.queue_time_us":51364,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":25180,"spinlock_wait_cycles":58759808,"thread_start_us":225178,"threads_started":1,"wal-append.queue_time_us":290418}
I20260709 12:22:49.847065 17923 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:22:49.903560 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:49.907426 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.909788 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.866991 17917 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:49.917207 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.880968 17843 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 12:22:49.908468 17828 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.002s
W20260709 12:22:49.922569 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115219us
I20260709 12:22:49.890043 17423 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:49.922703 17963 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 12:22:49.924204 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.932916 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
W20260709 12:22:49.924360 17905 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.775s	user 0.014s	sys 0.000s
I20260709 12:22:49.931068 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:49.926872 17913 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.906s	user 0.005s	sys 0.011s
I20260709 12:22:49.941597 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:49.874900 17983 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.945881 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:49.948688 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:49.946586 13652 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:49.954999 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.955725 13652 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:49.880620 17987 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.956718 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:49.943511 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:49.954110 18005 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:22:49.954528 17913 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:49.938760 17919 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:49.938978 17963 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.976239 17963 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.982577 17841 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:49.984673 17917 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:49.998440 17423 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:49.930074 17828 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.004258 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 1949) took 1998 ms. Trace:
I20260709 12:22:50.005200 13792 rpcz_store.cc:276] 0709 12:22:48.005751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.095871 (+ 90120us) service_pool.cc:224] Handling call
0709 12:22:48.096280 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.118186 (+ 21906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.003369 (+1885183us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b54000d25d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:50.003756 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17982336}
I20260709 12:22:49.941874 17905 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:50.012956 17932 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.000s
I20260709 12:22:50.013532 17905 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:50.013495 17932 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:49.966756 17854 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.030501 13723 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:50.031744 17932 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.027145 17828 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.037941 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 792) took 1754 ms. Trace:
I20260709 12:22:50.038450 14417 rpcz_store.cc:276] 0709 12:22:48.283470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.283641 (+   171us) service_pool.cc:224] Handling call
0709 12:22:49.221952 (+938311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:49.227486 (+  5534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.037678 (+810192us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b5400018950. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.037866 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2115299200}
I20260709 12:22:50.040306 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1878) took 1324 ms. Trace:
I20260709 12:22:50.040747 13643 rpcz_store.cc:276] 0709 12:22:48.716059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.716610 (+   551us) service_pool.cc:224] Handling call
0709 12:22:48.717041 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.883971 (+166930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.040036 (+1156065us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400096d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:50.040236 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356801280}
I20260709 12:22:50.043843 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:50.048034 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.048616 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 761) took 2164 ms. Trace:
I20260709 12:22:50.049100 13727 rpcz_store.cc:276] 0709 12:22:47.884118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:47.884836 (+   718us) service_pool.cc:224] Handling call
0709 12:22:47.885090 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.266859 (+381769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.048362 (+1781503us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54001a5254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:50.048547 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837550848}
I20260709 12:22:50.050446 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:50.053279 17966 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.079s	user 0.003s	sys 0.000s
I20260709 12:22:50.053149 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.054737 17423 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.055276 17904 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.055810 17905 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:50.057493 17905 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.057509 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2748) took 1872 ms. Trace:
I20260709 12:22:50.058804 17905 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.059082 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:50.059111 13652 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
W20260709 12:22:50.062574 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1879) took 1346 ms (client timeout 1782 ms). Trace:
I20260709 12:22:50.059284 13800 rpcz_store.cc:276] 0709 12:22:48.185651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.378146 (+192495us) service_pool.cc:224] Handling call
0709 12:22:48.378600 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:48.471700 (+ 93100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.471733 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:48.482338 (+ 10605us) log.cc:844] Serialized 26 byte log entry
0709 12:22:48.875557 (+393219us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:48.875688 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.051664 (+1175976us) spinlock_profiling.cc:243] Waited 92.7 ms on lock 0x7b54003c0354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:50.051719 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.052633 (+   914us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:50.055354 (+  2721us) raft_consensus.cc:1685] finished
0709 12:22:50.055375 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.057436 (+  2061us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5640,"raft.queue_time_us":100,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":140,"spinlock_wait_cycles":218198144,"thread_start_us":333937,"threads_started":1,"wal-append.queue_time_us":413566}
W20260709 12:22:50.063025 13647 rpcz_store.cc:269] 0709 12:22:48.716436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.723864 (+  7428us) service_pool.cc:224] Handling call
0709 12:22:50.062381 (+1338517us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400114c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.062511 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270739328}
I20260709 12:22:50.064008 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.062928 13643 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:50.067340 13649 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:50.068640 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:50.071836 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.154536 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:50.154860 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83132.1us
I20260709 12:22:50.155685 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:50.157198 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2491) took 1113 ms. Trace:
I20260709 12:22:50.157691 13651 rpcz_store.cc:276] 0709 12:22:49.043605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.737704 (+694099us) service_pool.cc:224] Handling call
0709 12:22:49.737961 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:49.738151 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:49.832635 (+ 94484us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.156535 (+323900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.156599 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:50.156661 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.156951 (+   290us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.157123 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":154109,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3124,"thread_start_us":387279,"threads_started":2}
I20260709 12:22:50.180694 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2499) took 1134 ms. Trace:
I20260709 12:22:50.181378 13649 rpcz_store.cc:276] 0709 12:22:49.046077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.179481 (+1133404us) service_pool.cc:224] Handling call
0709 12:22:50.179988 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.180267 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.180307 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.180363 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:50.180405 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.180463 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.180604 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:50.155400 13647 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:50.182541 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.187322 13647 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:22:50.188374 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2504) took 1140 ms. Trace:
I20260709 12:22:50.189579 13647 rpcz_store.cc:276] 0709 12:22:49.047541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.186309 (+1138768us) service_pool.cc:224] Handling call
0709 12:22:50.186822 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.187998 (+  1176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.188295 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:50.192668 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.208415 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:50.232264 13643 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
W20260709 12:22:50.289930 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.156306 17850 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.307588 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2501) took 1260 ms. Trace:
I20260709 12:22:50.308269 13651 rpcz_store.cc:276] 0709 12:22:49.046654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.184396 (+1137742us) service_pool.cc:224] Handling call
0709 12:22:50.306771 (+122375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.307046 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.307113 (+    67us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.307181 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:50.307242 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.307323 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.307498 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:50.303054 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:50.329566 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121232us
I20260709 12:22:50.331343 13767 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
W20260709 12:22:50.334517 14418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.015s	sys 0.038s
W20260709 12:22:50.335130 14418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.015s	sys 0.038s
I20260709 12:22:50.184682 17669 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.207336 17861 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.184676 17423 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:22:50.154615 17957 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:50.337648 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:50.225035 17886 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:50.243782 17909 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:50.305188 17850 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:50.374297 17850 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.343824 17539 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:50.373109 17556 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:22:50.380532 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.354719 17991 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:50.340952 17669 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.347839 17861 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:50.382221 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.382648 14424 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:22:50.396461 14424 pending_rounds.cc:85] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 7
W20260709 12:22:50.396322 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 431271us
I20260709 12:22:50.396713 14424 pending_rounds.cc:107] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:22:50.396955 14424 raft_consensus.cc:2889] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:50.391651 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.399071 17423 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.399461 17423 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.382225 17675 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:50.399894 17423 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.400313 17886 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:50.401635 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:50.402549 17983 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:22:50.382949 17862 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:22:50.403118 17983 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:50.361533 17578 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:50.394405 17669 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.397878 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:50.395080 17861 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.383260 17539 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:50.368572 17957 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.404186 17983 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.408457 17987 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.409371 17556 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:50.406885 17578 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.531597 17539 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.539228 17578 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:50.501940 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.547360 17841 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:50.514669 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.548646 17841 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.553653 17841 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:50.444766 17862 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
I20260709 12:22:50.557368 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:50.528412 17556 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.568589 17556 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:50.570974 17957 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.434775 17987 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.385840 17991 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.653494 17905 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:50.446465 18008 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.429s	user 0.005s	sys 0.011s
I20260709 12:22:50.661134 17756 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:22:50.661490 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 263544us
I20260709 12:22:50.662364 17756 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.663079 17756 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.663508 17991 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:50.663744 17905 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:50.632125 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 221762us
I20260709 12:22:50.665762 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:50.447013 18007 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.515s	user 0.006s	sys 0.020s
I20260709 12:22:50.672793 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2494) took 1628 ms. Trace:
I20260709 12:22:50.635452 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1901) took 1351 ms. Trace:
I20260709 12:22:50.649785 17669 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.655978 17987 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:50.672782 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2263) took 1372 ms. Trace:
I20260709 12:22:50.682641 13723 rpcz_store.cc:276] 0709 12:22:49.299680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.300209 (+   529us) service_pool.cc:224] Handling call
0709 12:22:49.300577 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.031063 (+730486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.672103 (+641040us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400175054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:50.672469 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1677969280}
I20260709 12:22:50.674626 13649 rpcz_store.cc:276] 0709 12:22:49.283368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.304902 (+1021534us) service_pool.cc:224] Handling call
0709 12:22:50.305902 (+  1000us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.351751 (+ 45849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.351845 (+    94us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:50.365865 (+ 14020us) log.cc:844] Serialized 26 byte log entry
0709 12:22:50.628053 (+262188us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:50.628179 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.628325 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.634736 (+  6411us) raft_consensus.cc:1685] finished
0709 12:22:50.634766 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.635156 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45081,"raft.queue_time_us":49341,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":7852,"spinlock_wait_cycles":187904,"thread_start_us":127617,"threads_started":1,"wal-append.queue_time_us":129437}
I20260709 12:22:50.699517 14424 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:22:50.703560 18030 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:50.704064 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:50.704406 18030 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:50.707465 18031 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:50.707985 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:50.709250 14419 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:50.709128 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:50.710227 14421 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:50.710456 17423 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.674557 17756 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.719918 17889 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:50.720980 17889 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.724730 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1880) took 1999 ms. Trace:
I20260709 12:22:50.725255 13650 rpcz_store.cc:276] 0709 12:22:48.725304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.726373 (+  1069us) service_pool.cc:224] Handling call
0709 12:22:48.726762 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:48.775488 (+ 48726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.724074 (+1948586us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b54002e45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:50.724474 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83175296}
I20260709 12:22:50.713501 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:50.732842 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:50.706068 18030 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.736058 17824 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:50.737355 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:50.673275 13648 rpcz_store.cc:276] 0709 12:22:49.044685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.742048 (+697363us) service_pool.cc:224] Handling call
0709 12:22:49.744021 (+  1973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:49.744375 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:49.744409 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:49.744766 (+   357us) log.cc:844] Serialized 26 byte log entry
0709 12:22:49.835875 (+ 91109us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:49.835932 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:49.844739 (+  8807us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.392134 (+547395us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:50.672233 (+280099us) raft_consensus.cc:1685] finished
0709 12:22:50.672277 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.672562 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14134,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":89342720,"thread_start_us":90551,"threads_started":1,"wal-append.queue_time_us":164671}
I20260709 12:22:50.738226 13652 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:50.752476 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:50.766130 13767 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:50.761179 17835 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.004s	sys 0.021s
I20260709 12:22:50.779599 17756 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:22:50.779011 17835 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.004s	sys 0.022s
I20260709 12:22:50.786334 17756 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:22:50.803872 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45139us
I20260709 12:22:50.810514 17982 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:50.811050 17982 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.811405 17982 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.817610 17850 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.823019 14422 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:50.795787 17889 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.806093 17756 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.789709 17964 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:50.835448 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52286.1us
I20260709 12:22:50.835789 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2515) took 1027 ms. Trace:
I20260709 12:22:50.836184 13649 rpcz_store.cc:276] 0709 12:22:49.805557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.761949 (+956392us) service_pool.cc:224] Handling call
0709 12:22:50.781500 (+ 19551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.832289 (+ 50789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.832319 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.832369 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:50.832403 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.832461 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.832586 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:50.837350 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:50.839072 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.843184 17964 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.844168 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 1900) took 1561 ms. Trace:
I20260709 12:22:50.844741 13643 rpcz_store.cc:276] 0709 12:22:49.282572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.303292 (+1020720us) service_pool.cc:224] Handling call
0709 12:22:50.303927 (+   635us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.327777 (+ 23850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.327817 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:50.328523 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 12:22:50.703168 (+374645us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b480033c558. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:22:50.745154 (+ 41986us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:50.748643 (+  3489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.748753 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.843568 (+ 94815us) raft_consensus.cc:1685] finished
0709 12:22:50.843585 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.843943 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35516,"raft.queue_time_us":35283,"raft.run_cpu_time_us":1896,"raft.run_wall_time_us":359381,"spinlock_wait_cycles":10990464,"thread_start_us":181886,"threads_started":1,"wal-append.queue_time_us":406475}
I20260709 12:22:50.854429 17423 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:50.823923 17669 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:50.856186 17964 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:50.833536 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2514) took 1025 ms. Trace:
I20260709 12:22:50.862501 13652 rpcz_store.cc:276] 0709 12:22:49.804787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.749083 (+944296us) service_pool.cc:224] Handling call
0709 12:22:50.774211 (+ 25128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.829405 (+ 55194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.829459 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.829564 (+   105us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:50.829674 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.829784 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.829968 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:50.863932 14421 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:50.864516 13652 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:50.865206 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 632) took 1039 ms. Trace:
I20260709 12:22:50.865542 13652 rpcz_store.cc:276] 0709 12:22:49.825483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.863604 (+1038121us) service_pool.cc:224] Handling call
0709 12:22:50.864110 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.864921 (+   811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.865109 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:50.868551 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2505) took 1820 ms. Trace:
I20260709 12:22:50.871294 17857 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:50.871975 17857 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:50.872536 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:50.873214 17857 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.847182 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:50.875299 17904 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:50.875875 17904 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:50.876310 17904 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:50.875381 13648 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:50.848994 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 142930us
I20260709 12:22:50.884133 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:50.887008 17850 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.887315 18021 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:50.888052 18021 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:50.888209 17857 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.885957 17861 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:50.889593 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1943) took 2886 ms (client timeout 1790 ms). Trace:
W20260709 12:22:50.890050 13799 rpcz_store.cc:269] 0709 12:22:48.003236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.005016 (+  1780us) service_pool.cc:224] Handling call
0709 12:22:50.889039 (+2884023us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.889305 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911850240}
I20260709 12:22:50.891674 17861 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:50.892038 13798 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:50.893518 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2747) took 2709 ms. Trace:
I20260709 12:22:50.897128 17861 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:50.869045 13647 rpcz_store.cc:276] 0709 12:22:49.047860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.193066 (+1145206us) service_pool.cc:224] Handling call
0709 12:22:50.194502 (+  1436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.196944 (+  2442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.196996 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:50.197729 (+   733us) log.cc:844] Serialized 26 byte log entry
0709 12:22:50.867707 (+669978us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54003181a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393b6b6b8 00007f5393bf492b 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c
0709 12:22:50.867820 (+   113us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:50.867908 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.868019 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.868072 (+    53us) raft_consensus.cc:1685] finished
0709 12:22:50.868085 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.868359 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29011,"raft.queue_time_us":811,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":36696,"spinlock_wait_cycles":345634176,"thread_start_us":301937,"threads_started":1,"wal-append.queue_time_us":256403}
W20260709 12:22:50.904398 13645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1877) took 2188 ms (client timeout 1580 ms). Trace:
I20260709 12:22:50.904748 13646 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:50.905071 13645 rpcz_store.cc:269] 0709 12:22:48.715677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.718870 (+  3193us) service_pool.cc:224] Handling call
0709 12:22:50.903788 (+2184918us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54001b6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.904094 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91240192}
I20260709 12:22:50.906996 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 622) took 1913 ms. Trace:
I20260709 12:22:50.907421 13646 rpcz_store.cc:276] 0709 12:22:48.993647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.993775 (+   128us) service_pool.cc:224] Handling call
0709 12:22:50.904299 (+1910524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.905320 (+  1021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.906701 (+  1381us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54001b6a50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.906915 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4387298560}
I20260709 12:22:50.911224 17940 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:22:50.911759 17940 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:50.855332 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:50.915098 13643 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.915555 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2257) took 1786 ms. Trace:
I20260709 12:22:50.916049 13718 rpcz_store.cc:276] 0709 12:22:49.128878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.129011 (+   133us) service_pool.cc:224] Handling call
0709 12:22:49.129471 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:49.129750 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:49.130130 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.914756 (+1784626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.914910 (+   154us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:50.915006 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.915150 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.915359 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1630070,"prepare.queue_time_us":1367,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":153874,"threads_started":1,"wal-append.queue_time_us":242364,"wal-append.run_cpu_time_us":1600,"wal-append.run_wall_time_us":1066954}
I20260709 12:22:50.919239 13647 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:50.919739 13647 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:50.919076 17940 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:50.920177 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.920663 17940 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:50.920754 13647 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:50.922049 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.888459 18021 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.924620 13652 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:50.927629 13652 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:50.929241 17828 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:50.929008 13652 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.930207 13652 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:50.930751 17822 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:50.931550 17822 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.934587 17964 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.934785 17630 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.934965 17821 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:50.935415 17964 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.893884 13798 rpcz_store.cc:276] 0709 12:22:48.183928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.184176 (+   248us) service_pool.cc:224] Handling call
0709 12:22:50.891510 (+2707334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.892521 (+  1011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.893248 (+   727us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b54000d2ad0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.893420 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6187086208}
I20260709 12:22:50.935817 17821 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.937860 17964 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:50.938005 17821 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.930049 17828 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:50.949352 17828 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:50.935809 17630 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:50.953900 17630 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:50.954718 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 150893us
W20260709 12:22:50.957721 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.959437 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:50.959860 17423 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:50.960475 17423 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:50.960597 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:50.962541 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:50.964038 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:50.964356 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2492) took 1920 ms. Trace:
I20260709 12:22:50.964743 13644 rpcz_store.cc:276] 0709 12:22:49.044009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.738755 (+694746us) service_pool.cc:224] Handling call
0709 12:22:50.062522 (+323767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.062740 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.062771 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.062846 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:22:50.062881 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.062946 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:50.963972 (+901026us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400114c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:50.964167 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677761280}
I20260709 12:22:50.960351 14421 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
W20260709 12:22:50.988422 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.991614 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:50.992028 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:50.949945 17756 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:50.993875 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:50.994670 13648 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:50.995697 13647 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:50.992657 13798 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:50.961802 17423 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:50.976542 17822 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:51.042905 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.045465 13649 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:22:51.045986 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.002900 17828 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:51.016058 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2512) took 1583 ms. Trace:
I20260709 12:22:51.051317 13650 rpcz_store.cc:276] 0709 12:22:49.433176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.745014 (+1311838us) service_pool.cc:224] Handling call
0709 12:22:50.774129 (+ 29115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:50.812600 (+ 38471us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:22:50.812964 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.999430 (+186466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:50.999488 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:51.015770 (+ 16282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.015849 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.015988 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":759,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":4659,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":41555,"thread_start_us":182370,"threads_started":1}
I20260709 12:22:51.063544 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:51.059873 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.744s (SENT) [suppressed 20 similar messages]
I20260709 12:22:51.051088 17828 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.094185 17828 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.095175 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:51.056218 17904 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.103876 17904 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.000631 17982 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.106674 17982 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:51.061662 17423 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:51.107200 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47367.2us
W20260709 12:22:51.110612 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:51.072697 18045 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:22:51.080582 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:51.082211 18037 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.380s	user 0.006s	sys 0.000s
I20260709 12:22:51.083973 17756 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:51.126242 13652 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:51.116272 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:51.126173 13643 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:51.134713 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:51.145639 17982 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:51.178437 17797 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:51.179247 17797 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.179741 17797 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:51.194831 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 138154us
I20260709 12:22:51.196267 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:51.199354 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:51.200038 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.200990 13798 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:51.201927 13793 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:22:51.202378 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.200310 13799 raft_consensus.cc:606] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:51.203788 13792 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 6. Current term is 7. Ops: []
I20260709 12:22:51.203799 13799 raft_consensus.cc:614] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:51.205767 13797 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:51.206661 17843 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.206951 17982 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:51.207975 17843 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:51.208362 17982 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.218922 18021 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.220553 14423 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:22:51.221480 18021 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:51.241214 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.249553 13801 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
W20260709 12:22:51.258081 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.245059 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:51.260186 13799 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:51.266696 17947 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:51.267612 17947 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.271613 17947 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.272388 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:51.273551 18021 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:51.274071 18021 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:51.274583 18021 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:51.276176 18040 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:51.277683 18029 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:51.278110 18029 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:51.279119 18029 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.280511 18029 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:51.281904 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:51.282063 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2510) took 1891 ms. Trace:
I20260709 12:22:51.282796 13651 rpcz_store.cc:276] 0709 12:22:49.390251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:50.364119 (+973868us) service_pool.cc:224] Handling call
0709 12:22:50.501599 (+137480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:50.502692 (+  1093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:50.502756 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:50.504001 (+  1245us) log.cc:844] Serialized 26 byte log entry
0709 12:22:50.682210 (+178209us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:50.727122 (+ 44912us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.731284 (+  4162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:50.731475 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:50.914283 (+182808us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b18003b1b80. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5397ac858f 00007f5390b07571 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd
0709 12:22:51.050666 (+136383us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:51.281568 (+230902us) raft_consensus.cc:1685] finished
0709 12:22:51.281592 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.281801 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2738,"prepare.queue_time_us":45932,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":36356,"raft.run_cpu_time_us":549,"raft.run_wall_time_us":1790,"spinlock_wait_cycles":60069504,"thread_start_us":171364,"threads_started":1,"wal-append.queue_time_us":197428}
I20260709 12:22:51.285272 17964 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.286033 17936 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.286736 17936 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.288621 13793 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:22:51.289173 17936 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:51.273490 13546 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 152) took 18924 ms (client timeout 15000 ms). Trace:
W20260709 12:22:51.290997 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.244522 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.298596 13647 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:51.302739 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:51.307236 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:51.326442 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.326982 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:51.322018 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:51.328490 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:51.330132 13792 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:51.303349 17756 raft_consensus.cc:2764] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:22:51.340620 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:51.341213 13643 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:51.335652 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:51.344676 13801 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.346155 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:51.347102 13649 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.348973 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.350797 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:51.351583 13644 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:22:51.352681 13799 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:51.353924 18040 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.354024 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:51.355134 13650 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:51.344372 18034 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:22:51.286034 17964 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:51.367030 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.374979 13649 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:51.380823 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:51.387475 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45287.8us
W20260709 12:22:51.391108 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 109347us
I20260709 12:22:51.400159 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:51.401660 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.403472 18054 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:51.413971 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:51.415313 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:51.415421 17843 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:51.416765 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.417657 13723 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:51.417723 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.418336 18051 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.418761 13727 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:51.418902 17821 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:51.419103 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.419562 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.419934 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.420684 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.421458 13718 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:22:51.422415 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.423357 13722 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:51.424814 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:51.354856 18040 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.426585 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:51.427318 13644 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.428031 18040 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:51.428790 18055 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.429492 18055 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.431527 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.431775 18053 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.432473 18053 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.434468 18052 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.419651 17821 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.435141 18052 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.437453 17821 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.437698 18052 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.440205 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:51.440488 18021 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.442052 17613 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:51.404322 18054 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.441833 17539 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:22:51.442660 17613 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:51.443801 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.443773 17613 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.444516 18054 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.446358 17964 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:51.448011 17828 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.412569 17909 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:51.450469 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 780) took 2237 ms. Trace:
I20260709 12:22:51.450945 17909 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:51.421589 13725 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.451995 17909 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.450974 13719 rpcz_store.cc:276] 0709 12:22:49.168032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.168214 (+   182us) service_pool.cc:224] Handling call
0709 12:22:49.199512 (+ 31298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:49.199805 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:22:49.223501 (+ 23696us) log.cc:844] Serialized 14 byte log entry
0709 12:22:50.913226 (+1689725us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b18003b81e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:22:51.055464 (+142238us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.324521 (+269057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:51.335227 (+ 10706us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:51.356782 (+ 21555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.381259 (+ 24477us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.405735 (+ 24476us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78917,"prepare.queue_time_us":264323,"prepare.run_cpu_time_us":151,"prepare.run_wall_time_us":21548,"raft.queue_time_us":1712924,"raft.run_cpu_time_us":468,"raft.run_wall_time_us":25341,"spinlock_wait_cycles":5762944,"thread_start_us":313415,"threads_started":2,"wal-append.queue_time_us":322739}
W20260709 12:22:51.291203 13546 rpcz_store.cc:269] 0709 12:22:32.349238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:32.349507 (+   269us) service_pool.cc:224] Handling call
0709 12:22:51.273139 (+18923632us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:48.364929 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:48.379290 (+ 14361us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:48.379342 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:22:48.381745 (+  2403us) tablet.cc:662] Decoding operations
0709 12:22:48.638794 (+257049us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:48.638867 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:48.638896 (+    29us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:22:48.641453 (+  2557us) tablet.cc:701] Row locks acquired
0709 12:22:48.641608 (+   155us) write_op.cc:260] PREPARE: finished
0709 12:22:48.641800 (+   192us) write_op.cc:270] Start()
0709 12:22:48.641955 (+   155us) write_op.cc:276] Timestamp: P: 1783599768641735 usec, L: 0
0709 12:22:48.641974 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:22:48.673544 (+ 31570us) log.cc:844] Serialized 8192 byte log entry
0709 12:22:49.007105 (+333561us) op_driver.cc:464] REPLICATION: finished
0709 12:22:49.080459 (+ 73354us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b180032ad60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:49.080619 (+   160us) write_op.cc:301] APPLY: starting
0709 12:22:49.080724 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:49.469823 (+389099us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:49.472493 (+  2670us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:50.383658 (+911165us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:50.383954 (+   296us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 12:22:50.383994 (+    40us) write_op.cc:312] APPLY: finished
0709 12:22:50.384328 (+   334us) log.cc:844] Serialized 188 byte log entry
0709 12:22:50.452661 (+ 68333us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:50.457460 (+  4799us) write_op.cc:454] Released schema lock
0709 12:22:50.563105 (+105645us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:50.564373 (+  1268us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:50.564463 (+    90us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4029023,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":34718,"apply.run_cpu_time_us":56553,"apply.run_wall_time_us":1483499,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92075,"num_ops":21,"prepare.queue_time_us":166115,"prepare.run_cpu_time_us":30026,"prepare.run_wall_time_us":551795,"raft.queue_time_us":85339,"raft.run_cpu_time_us":7521,"raft.run_wall_time_us":33994,"replication_time_us":364971,"spinlock_wait_cycles":55604608,"thread_start_us":411460,"threads_started":5,"wal-append.queue_time_us":514699,"wal-append.run_cpu_time_us":35902,"wal-append.run_wall_time_us":1119755}]]}
I20260709 12:22:51.420014 13723 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.471827 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:51.481209 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:51.481532 18034 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:51.482465 18021 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:51.483031 17423 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:22:51.484388 13767 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:51.485076 17828 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.488260 17735 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:51.490916 13793 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:51.492081 13801 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:51.492444 13797 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:51.466198 17920 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:51.425817 13646 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.494803 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.557175 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.495242 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:51.501253 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.504225 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:51.561345 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
W20260709 12:22:51.496696 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 77929.4us
I20260709 12:22:51.562248 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.532388 13722 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:22:51.563620 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.532531 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:51.564038 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:51.510092 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:51.535094 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50813.2us
I20260709 12:22:51.528378 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:51.535717 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.568203 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:51.567986 17797 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.569252 13799 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:51.569727 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.570705 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.571522 17936 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:51.572443 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.573866 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.533507 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.574841 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:51.572707 17936 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:51.500240 14422 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:51.579490 13796 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:22:51.609128 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:51.494239 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:51.611605 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.592567 13724 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:51.593312 13792 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:51.614634 13719 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.540510 13800 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:51.628544 13646 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:51.630656 13650 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:51.614794 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.621363 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:51.635547 14423 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.554987 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:51.539176 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.534766 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:51.637183 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.620880 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:51.548991 13548 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:51.549489 17763 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:22:51.646538 13645 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:51.647269 17763 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:22:51.534591 17423 consensus_queue.cc:1243] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:22:51.649547 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:51.651134 13651 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:22:51.651648 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:51.606328 18029 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:51.513288 18053 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:51.647928 17763 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:51.546599 17669 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:51.687624 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 104223us
I20260709 12:22:51.627266 17936 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:51.510886 18055 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.692128 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.544166 17735 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:51.648123 17797 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.695046 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.696806 13722 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:51.696981 17936 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.699249 18033 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.699752 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.699864 18033 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.684330 18029 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.701529 13718 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:51.701808 18029 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:51.699259 17936 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:51.703325 17763 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.704965 17763 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:51.549753 17940 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.157s
I20260709 12:22:51.692663 17828 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.708719 17828 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.711786 17889 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:51.691954 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.719292 13727 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:51.719718 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:51.720990 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:51.722746 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.419106 18051 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.726974 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.728953 13724 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:51.727700 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.725181 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:51.727692 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:51.693786 17756 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.736770 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.736920 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.749917 13719 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:51.719931 17828 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:51.757320 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.730054 18021 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:51.722990 18053 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.727147 17669 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:51.755223 14421 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:51.759027 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.751664 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:51.754376 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.758445 17857 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:51.734664 17797 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.775657 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54361b2742ab4613b55749cbb699113b"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:51.776615 13794 raft_consensus.cc:606] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:51.770862 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.772346 13724 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.762300 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:51.774153 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.779270 13718 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:22:51.733891 17843 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:51.747014 17822 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.805881 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:51.748744 13548 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 7, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:51.785449 18053 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.786662 18051 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:51.790692 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47975.6us
I20260709 12:22:51.794484 17797 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.795854 17669 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.796665 17964 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:51.798834 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:51.807016 13794 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:51.812965 14416 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.813972 14423 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
W20260709 12:22:51.799479 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.798880 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.769281 17423 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:51.768926 17857 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:51.808827 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:51.817867 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:51.821537 13727 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.822547 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:51.823010 17797 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:51.823734 17797 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:51.823860 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.825124 13727 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.824645 17423 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.818725 18040 raft_consensus.cc:2764] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:22:51.826304 17857 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.827543 17423 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:51.829248 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.830132 13727 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:51.824290 17797 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:51.833297 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.834474 13718 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:51.752853 17756 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.838961 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.843989 13548 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 5, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 5 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:51.840997 17841 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:22:51.860237 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.813803 17964 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:51.867357 13718 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:51.861343 13719 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:51.859520 17889 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:51.895475 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.842309 17822 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.899158 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.897184 13719 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 4.
I20260709 12:22:51.862104 17940 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:22:51.904477 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2264) took 2462 ms. Trace:
I20260709 12:22:51.875985 17841 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:51.905365 13548 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:22:51.904598 17889 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.911839 13718 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:51.811589 17735 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.912792 13724 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:22:51.916550 17797 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.912499 17841 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:51.920724 18029 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:51.928117 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.935350 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.934410 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2249) took 3328 ms. Trace:
I20260709 12:22:51.934434 13548 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:51.942619 13720 rpcz_store.cc:276] 0709 12:22:48.604545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.692230 (+ 87685us) service_pool.cc:224] Handling call
0709 12:22:48.692704 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:49.107971 (+415267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:49.108003 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:49.108619 (+   616us) log.cc:844] Serialized 52 byte log entry
0709 12:22:49.273539 (+164920us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:49.356684 (+ 83145us) log.cc:844] Serialized 14 byte log entry
0709 12:22:49.564259 (+207575us) spinlock_profiling.cc:243] Waited 987 us on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:22:51.498022 (+1933763us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.898988 (+400966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.915212 (+ 16224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:51.930708 (+ 15496us) raft_consensus.cc:1685] finished
0709 12:22:51.930752 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.933309 (+  2557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3352271,"prepare.queue_time_us":107694,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":100,"raft.queue_time_us":2000846,"raft.run_cpu_time_us":2424,"raft.run_wall_time_us":1273991,"spinlock_wait_cycles":2269440,"thread_start_us":908569,"threads_started":5,"wal-append.queue_time_us":305018,"wal-append.run_cpu_time_us":5198,"wal-append.run_wall_time_us":1272939}
I20260709 12:22:51.945586 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.910638 13721 rpcz_store.cc:276] 0709 12:22:49.299971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:49.345921 (+ 45950us) service_pool.cc:224] Handling call
0709 12:22:49.346451 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:49.346793 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:22:49.374910 (+ 28117us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.152832 (+1777922us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.648758 (+495926us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.689925 (+ 41167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:51.719930 (+ 30005us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:51.731207 (+ 11277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:51.746445 (+ 15238us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:51.762052 (+ 15607us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1542994,"prepare.queue_time_us":363781,"prepare.run_cpu_time_us":581,"prepare.run_wall_time_us":2400,"raft.queue_time_us":1593876,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":14105,"spinlock_wait_cycles":53979008,"thread_start_us":708082,"threads_started":2}
I20260709 12:22:51.947309 13718 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:51.952608 13720 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.915280 17630 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:51.964838 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:51.970537 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.972224 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.971427 13719 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:51.974210 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.960697 17735 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:51.975891 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:51.970408 17613 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.013s	sys 0.016s
I20260709 12:22:51.971555 17763 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:51.990638 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:51.993940 13548 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:51.990512 17613 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.013s	sys 0.016s
I20260709 12:22:52.006171 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.956286 17841 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.008432 13718 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:22:51.964805 17964 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:52.014916 13766 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.042s
W20260709 12:22:52.030900 13766 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.006s	sys 0.043s
W20260709 12:22:52.031275 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 739045us
I20260709 12:22:52.015785 13795 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:52.024096 13720 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:51.969869 17857 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.028017 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.033109 13795 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:52.037560 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:51.973615 17630 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:51.995631 17763 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:52.057008 13548 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:52.058619 13548 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:52.047063 13795 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:52.062722 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.038664 18053 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:52.065798 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.066658 13721 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:52.069392 13548 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:52.070521 13795 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:52.079136 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 329476us
I20260709 12:22:52.081390 13720 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:22:52.083930 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.085414 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.050439 17756 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.090843 13719 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:52.107545 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.089071 17841 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:52.086990 18029 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.113829 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.105791 17843 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:52.118016 13719 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:52.104571 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:52.120252 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.122963 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.123245 13720 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:52.125224 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:52.140844 13801 raft_consensus.cc:606] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:52.111164 13548 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:52.126085 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.152326 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 327928us
I20260709 12:22:52.153944 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:52.154943 14416 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:52.126893 17964 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:52.104135 17756 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:22:52.163136 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.164206 13724 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:52.169384 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:52.173360 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.174750 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:52.178903 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 441589us
I20260709 12:22:52.181778 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.187711 13727 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:22:52.190631 14416 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:52.192350 13719 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:52.198426 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.199627 14421 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:22:52.169210 17904 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:52.202479 13720 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:52.206846 17843 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:52.208954 18069 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:22:52.150709 17936 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:22:52.209988 18070 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:22:52.147809 17613 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:22:52.126047 17763 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:52.209192 17949 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:52.176671 18021 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:52.225075 17889 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.224975 18053 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:52.225823 17889 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:52.229561 18072 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:22:52.218103 13548 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:52.232076 13548 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:22:52.234861 18067 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:22:52.235476 17983 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:52.236008 17983 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:52.239030 18074 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:22:52.239274 18075 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:22:52.230099 18033 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:52.240234 18066 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:22:52.244555 17822 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.247157 13721 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:22:52.248903 18071 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.002s
I20260709 12:22:52.254174 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2237) took 4245 ms. Trace:
I20260709 12:22:52.256042 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:52.254673 13726 rpcz_store.cc:276] 0709 12:22:48.008915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:48.046955 (+ 38040us) service_pool.cc:224] Handling call
0709 12:22:48.047670 (+   715us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:22:48.143442 (+ 95772us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:48.143479 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:22:49.575074 (+1431595us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:49.576026 (+   952us) log.cc:844] Serialized 78 byte log entry
0709 12:22:49.664841 (+ 88815us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:49.671832 (+  6991us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.376059 (+1704227us) log.cc:844] Serialized 14 byte log entry
0709 12:22:51.896404 (+520345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.253575 (+357171us) spinlock_profiling.cc:243] Waited 85.5 ms on lock 0x7b54001a4d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:52.253666 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:52.253754 (+    88us) raft_consensus.cc:1685] finished
0709 12:22:52.253772 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:52.253971 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1258306,"prepare.queue_time_us":201050,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":82,"raft.queue_time_us":1307796,"raft.run_cpu_time_us":429,"raft.run_wall_time_us":28447,"spinlock_wait_cycles":206449664,"thread_start_us":980639,"threads_started":4,"wal-append.queue_time_us":810723,"wal-append.run_cpu_time_us":4204,"wal-append.run_wall_time_us":1262681}
I20260709 12:22:52.257133 17822 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:22:52.257596 13726 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
W20260709 12:22:52.261235 18073 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:22:52.262102 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:52.200699 18055 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:52.265494 18055 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:52.273056 17613 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:52.273660 17613 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:52.274087 17630 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.274852 17613 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:52.276324 17613 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:52.278798 17920 raft_consensus.cc:993] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:22:52.278947 18068 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:22:52.279452 17920 raft_consensus.cc:1081] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:22:52.280382 17763 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:52.282095 17857 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:52.282439 17982 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:52.282902 17982 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:52.283275 17982 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.295760 18083 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.257915 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:52.222687 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.233657 13548 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 6, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:52.315974 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.316777 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.318821 13718 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:52.322969 13727 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 10. Ops: []
I20260709 12:22:52.321377 13724 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:52.330927 13646 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:52.336066 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:52.336076 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:52.336154 13651 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:22:52.341740 13650 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:22:52.330852 13548 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:52.341781 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:52.303635 17949 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:52.314697 18073 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.003s	sys 0.001s
I20260709 12:22:52.336778 13793 raft_consensus.cc:606] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:52.349120 13720 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:52.356482 13645 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:52.363945 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:52.381421 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.369508 13548 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 8, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:52.386652 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 128853us
I20260709 12:22:52.392758 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:52.393712 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:22:52.406417 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.407653 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 344250us
W20260709 12:22:52.412462 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.412748 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:52.415585 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 242348us
W20260709 12:22:52.413136 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.368578 17630 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:52.424750 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:52.425493 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.422546 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:52.423601 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:52.431221 14422 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:22:52.430052 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:52.410956 13548 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 8 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:52.447574 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.429268 17843 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:22:52.450695 13548 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:22:52.453629 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 189019us
I20260709 12:22:52.452353 13548 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 7, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 7 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:22:52.459708 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.451625 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:22:52.467317 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.433393 17822 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:52.479148 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.488544 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.490196 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.434316 18021 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:52.515874 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.571s (SENT)
I20260709 12:22:52.516496 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:52.515316 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 178027us
I20260709 12:22:52.475247 17630 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:52.480798 17822 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:52.479948 18054 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:52.511431 17983 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:52.467669 18053 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:22:52.546859 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:52.548142 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.548785 13725 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:52.550345 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 95891.7us
I20260709 12:22:52.553947 18085 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:52.554621 18085 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:52.555775 18085 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.530170 17862 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:22:52.530362 17423 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:22:52.522149 17889 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:52.559130 17889 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.559726 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:52.560360 13644 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.553754 17848 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 12:22:52.570782 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.526652 17630 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:52.574941 13644 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:52.529541 17822 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:52.577112 13795 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:22:52.577803 13795 pending_rounds.cc:85] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 9
I20260709 12:22:52.578138 13795 pending_rounds.cc:107] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:22:52.593365 13795 raft_consensus.cc:2889] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:52.594821 18092 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:52.597342 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:22:52.599082 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83266.8us
W20260709 12:22:52.600135 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.571s (SENT)
I20260709 12:22:52.600744 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:52.603255 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.584088 17630 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:52.586414 18079 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:52.619001 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.598s (SENT)
I20260709 12:22:52.619511 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:52.620693 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2773) took 1124 ms. Trace:
I20260709 12:22:52.621258 13720 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:22:52.619622 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.621133 13795 rpcz_store.cc:276] 0709 12:22:51.492075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:51.495513 (+  3438us) service_pool.cc:224] Handling call
0709 12:22:51.495960 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:52.015498 (+519538us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400376ca0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:22:52.616409 (+600911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:52.616658 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378031488}
W20260709 12:22:52.626549 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 80664.3us
I20260709 12:22:52.637220 13725 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:52.639917 17841 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.004s
I20260709 12:22:52.640353 17940 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:52.640800 17940 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:52.641060 18053 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:52.641454 18053 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:52.641871 18053 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:52.646986 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:52.647724 13651 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:52.650830 17920 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:22:52.651328 17920 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:52.652221 17920 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.653573 17920 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:52.625550 18087 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:52.641713 17940 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.654529 18087 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.655596 17940 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:22:52.623008 17861 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:22:52.663826 13651 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:22:52.613837 18055 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.507575 17949 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:52.618242 18092 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:22:52.676530 18091 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:22:52.679553 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.682660 13720 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:52.681614 18096 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.686404 17949 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:52.686566 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67604.4us
I20260709 12:22:52.691037 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.691154 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:52.691982 18055 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:52.692198 18077 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:52.692561 18055 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:52.692862 18077 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:52.693195 18055 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:52.692016 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:52.697206 17857 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:22:52.697754 17857 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:52.698792 17857 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.714781 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:52.714821 13720 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:22:52.717775 13720 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:52.677812 17613 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.733194 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:52.734130 13796 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:52.629467 18079 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.752884 13720 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:52.814579 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.736096 18092 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:52.724730 17861 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:52.693928 18077 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:52.830892 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.855748 14416 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:52.855505 18110 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:52.857933 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:52.830693 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.859831 14423 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:22:52.860160 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.822467 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 271105us
W20260709 12:22:52.822069 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.860836 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.822979 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.824435 13720 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:22:52.825971 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.826668 18087 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:52.862596 13725 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:52.827476 13724 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:52.863178 13724 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:52.748107 18108 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:22:52.820302 17982 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.003s
I20260709 12:22:52.831827 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:52.746215 17613 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:52.874135 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46644.6us
I20260709 12:22:52.870877 13724 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:52.877751 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46060.7us
I20260709 12:22:52.858464 17861 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.828302 18054 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:52.886534 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:52.874930 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:52.885495 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:52.880177 18101 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:22:52.883688 17613 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:52.870401 17763 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.590s	user 0.016s	sys 0.003s
I20260709 12:22:52.880895 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:52.892704 13644 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:22:52.893536 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:52.894214 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:52.894676 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.894925 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:52.895395 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.895742 13793 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:52.857906 18110 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:22:52.864773 17936 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:52.883363 18053 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.907814 13801 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:52.917268 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.920032 13652 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:52.922191 18103 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:52.922791 18103 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:52.923733 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 436974us
I20260709 12:22:52.829174 18092 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:52.929093 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:52.923209 18103 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.930156 13724 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:52.931450 13650 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:52.934636 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.935264 13718 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11. Re-sending same reply.
I20260709 12:22:52.932092 13643 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:22:52.906919 17964 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:52.877454 17982 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.925872 13719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54361b2742ab4613b55749cbb699113b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
W20260709 12:22:52.939214 18105 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:22:52.943300 18083 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.647s	user 0.006s	sys 0.008s
I20260709 12:22:52.946964 18113 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:52.948534 17630 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.950764 18054 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:52.952124 18110 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:22:52.916610 17889 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:22:52.954726 18053 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:52.960974 13719 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:52.966374 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:52.968077 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.979797 13719 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:52.952970 17861 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:52.916867 17936 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:52.977792 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.959137 17964 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:52.988705 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:52.989521 13723 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:52.993398 13649 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:52.996977 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:52.998517 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:52.969092 18083 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:52.997998 13721 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:52.999257 13719 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:53.011759 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.014443 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:53.017746 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45413.2us
I20260709 12:22:53.015434 13645 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:22:53.019245 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.021907 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:53.023144 18053 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:22:53.023536 13726 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:53.023656 18053 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.024134 18053 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:22:53.025923 17630 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:52.973589 18113 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.026613 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:53.027902 18113 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.028200 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:52.927103 18108 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:22:53.031911 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:53.032146 17936 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.033367 18108 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:53.035444 18079 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:53.039561 18092 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.041453 13719 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:52.990376 17964 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:53.026777 13725 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:53.043123 13725 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:53.045959 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:53.047355 17909 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:53.052062 17909 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.052464 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:53.053123 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:53.049263 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:52.989842 17822 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.064564 17857 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:53.066042 17857 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.066541 17857 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.066671 17822 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:53.067493 14419 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:22:53.070292 13727 raft_consensus.cc:2368] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:22:53.070940 13650 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:22:53.070886 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.065840 17982 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:53.043972 18096 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:53.074059 13645 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:53.058734 13725 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.076231 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:53.081568 13718 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:53.083212 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:53.071668 17861 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:53.091094 13796 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:22:53.091399 17861 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:22:53.091931 17861 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.092164 17857 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.094663 17857 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.080588 18055 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.078192 17669 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:22:53.091946 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:53.096772 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:53.096091 13726 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:53.091588 13796 pending_rounds.cc:85] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 7
I20260709 12:22:53.098332 13796 pending_rounds.cc:107] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:22:53.098611 13796 raft_consensus.cc:2889] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.097571 14419 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:53.126802 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:53.149932 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65433.6us
I20260709 12:22:53.145803 13718 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:22:53.157001 14423 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:53.159197 13799 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:22:53.160797 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75777.5us
I20260709 12:22:53.157892 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:53.159245 13725 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.171326 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:53.126689 18092 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:53.174372 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95485us
I20260709 12:22:53.176271 13722 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:22:53.183414 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.081182 17613 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
W20260709 12:22:53.217696 17920 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.564s	user 0.015s	sys 0.000s
W20260709 12:22:53.210487 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:53.221031 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.217578 13723 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:53.211314 18055 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:53.211944 18100 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.214457 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:53.214900 17982 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:22:53.183037 17822 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:53.223526 13720 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:22:53.225437 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:22:53.233770 13720 pending_rounds.cc:85] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 4
I20260709 12:22:53.226930 13725 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:22:53.234458 13725 pending_rounds.cc:85] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:22:53.234776 13725 pending_rounds.cc:107] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:22:53.235059 13725 raft_consensus.cc:2889] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.227219 14418 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.226061 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.237438 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.238420 13651 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:22:53.244006 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.244556 13720 pending_rounds.cc:107] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:22:53.244740 13644 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:53.244998 13720 raft_consensus.cc:2889] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.236406 13718 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:53.249191 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.250106 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:53.237139 17822 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:53.252111 17822 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.222218 18092 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.229270 17909 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:53.228538 18089 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.228508 17982 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.232044 18100 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.289687 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:53.290716 13724 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:53.291738 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102883us
I20260709 12:22:53.295488 13644 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:53.297632 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.241351 17983 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:22:53.272202 17982 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:22:53.300086 18115 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.225s	user 0.000s	sys 0.002s
I20260709 12:22:53.278182 18089 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:53.332098 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:53.338075 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.308436 17909 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:22:53.353295 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.356864 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:53.358436 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.358903 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:53.359191 13720 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:53.365202 13724 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:53.366922 14419 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:53.367512 18080 raft_consensus.cc:993] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
W20260709 12:22:53.373270 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.376629 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:53.385782 18112 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:22:53.386332 13693 connection.cc:256] RPC call timeout handler was delayed by -3.8052768969996578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 12:22:53.386842 18112 consensus_queue.cc:1243] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:22:53.387177 18108 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:53.387428 18112 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:53.387745 18108 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:53.388055 18112 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:53.388965 18108 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.389140 18112 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.390429 18054 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.390975 18054 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.393066 18054 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.393131 17630 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:53.393795 17630 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.394711 18053 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.396590 17630 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:53.397181 13693 connection.cc:256] RPC call timeout handler was delayed by -3.6234768969998186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
I20260709 12:22:53.397541 18053 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.398469 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:53.399139 13720 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.399587 17964 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.400885 13645 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:53.403110 17861 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.404723 18112 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.405441 17861 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:53.407706 18103 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.409876 18103 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:53.387770 13793 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:22:53.414716 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:53.415119 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:53.415596 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.416874 13795 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
W20260709 12:22:53.418080 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.424141 13720 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
W20260709 12:22:53.425951 18069 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.433s	user 0.006s	sys 0.000s
I20260709 12:22:53.426367 18113 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:53.397571 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4971039789998031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 12:22:53.427013 18113 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:53.427544 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4764039789997696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
I20260709 12:22:53.427770 18089 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:53.429824 18092 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:53.430447 18118 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 12:22:53.430680 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:53.432502 17947 raft_consensus.cc:1081] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:22:53.426296 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:53.436597 13644 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:53.437763 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:53.440255 18079 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:53.443209 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:53.444753 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:53.447407 18085 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:53.450613 14418 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:22:53.427862 13693 connection.cc:256] RPC call timeout handler was delayed by -2.358503978999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 12:22:53.451809 13693 connection.cc:256] RPC call timeout handler was delayed by -2.121803978999651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 12:22:53.452168 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0435039789999792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20350000000000001s for a response
W20260709 12:22:53.452555 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9652039789998526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 12:22:53.452898 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9085039789997609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
W20260709 12:22:53.453284 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8833039789997201s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
I20260709 12:22:53.402019 17964 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:53.451967 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:53.461943 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:53.459339 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:53.466584 13649 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:53.468638 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:53.469120 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:53.469795 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.470281 18113 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:53.470454 13650 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:22:53.470897 17669 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:53.471349 17669 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:53.471779 17669 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:53.473510 13651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:22:53.474077 13651 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:53.474474 13651 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:53.470516 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:53.476186 13801 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:22:53.480379 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.481556 13646 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:22:53.487043 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:53.456130 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 260251us
I20260709 12:22:53.497125 13644 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:53.499990 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:53.453876 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8779039789997114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
W20260709 12:22:53.502980 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8725039789997027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
W20260709 12:22:53.504554 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1021039789998213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30810000000000004s for a response
I20260709 12:22:53.432535 18100 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:53.568907 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 131256us
W20260709 12:22:53.568872 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 69008.3us
I20260709 12:22:53.565248 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:53.567498 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.572556 13723 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:53.455408 18085 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:22:53.569013 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 118106us
I20260709 12:22:53.570719 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:53.566200 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:53.578455 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:53.573798 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:53.458916 17756 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:22:53.575520 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.569412 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:53.481002 18083 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.582756 13724 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:22:53.583849 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101620us
I20260709 12:22:53.588343 14422 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:22:53.651808 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 206032us
I20260709 12:22:53.646083 13651 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.661767 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:53.648196 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:53.689960 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124918us
I20260709 12:22:53.585115 17756 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.583550 18085 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:53.704268 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:53.712980 13801 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:22:53.705551 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 127211us
W20260709 12:22:53.706108 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49889.2us
I20260709 12:22:53.717923 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:53.718509 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:53.739090 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:53.740458 13796 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:22:53.577605 17735 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:22:53.755519 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 186181us
I20260709 12:22:53.661245 17613 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:53.688970 18021 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.769754 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:53.584518 17949 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
W20260709 12:22:53.775484 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55206.8us
I20260709 12:22:53.780830 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:53.564046 18089 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 12:22:53.782091 13795 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:53.783140 13651 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:53.697248 18055 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:53.784448 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.789072 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.711194 17964 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:53.723804 17756 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:53.792573 13646 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:53.723085 18100 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:22:53.733551 17947 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:22:53.796108 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 412637us
I20260709 12:22:53.671943 18083 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.693163 18079 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:22:53.803422 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:53.693621 18081 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:22:53.772456 17843 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:53.862898 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80636.5us
I20260709 12:22:53.864145 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.787364 17613 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.787623 18021 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.787087 18094 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 12:22:53.819186 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 248527us
I20260709 12:22:53.820870 13648 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:53.853562 13801 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:53.885308 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:53.854794 18100 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.800591 18096 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.881500 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:53.855108 17947 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:53.904301 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:53.906989 13650 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:22:53.868242 18053 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:53.868094 17904 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:53.785701 13649 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:22:53.913146 14419 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:53.920951 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:53.922029 13649 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:53.922741 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 207432us
I20260709 12:22:53.924142 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:53.924984 17889 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:53.925395 17889 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:22:53.908182 17822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.020s	sys 0.027s
W20260709 12:22:53.934995 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49782.4us
I20260709 12:22:53.904294 18100 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:22:53.910391 18108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.015s	sys 0.034s
I20260709 12:22:53.905248 17947 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:22:53.944774 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76351.7us
I20260709 12:22:53.866185 17843 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:53.912106 17797 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:22:53.925755 17889 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:54.003785 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:53.944638 17822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.021s	sys 0.027s
W20260709 12:22:54.019464 18125 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.252s	user 0.001s	sys 0.001s
I20260709 12:22:53.915333 17904 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:54.068267 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 302114us
I20260709 12:22:53.909158 18096 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:53.911219 17964 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:54.119022 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.120373 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 339291us
I20260709 12:22:54.108980 18129 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:54.164746 13796 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 12:22:54.165167 13796 pending_rounds.cc:85] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 12
I20260709 12:22:54.165381 13796 pending_rounds.cc:107] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 12:22:54.165588 13796 raft_consensus.cc:2889] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:53.927704 18055 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:54.122979 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.125010 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.127694 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.134364 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.137569 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.139205 13795 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:54.175100 13795 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:54.087013 17735 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:54.123720 18108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.018s	sys 0.034s
W20260709 12:22:54.127573 18080 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.004s	sys 0.017s
W20260709 12:22:54.187784 18080 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.005s	sys 0.017s
I20260709 12:22:54.187796 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.204716 17904 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:54.210186 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.211763 17669 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.003s
I20260709 12:22:54.212312 17669 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:54.220286 18112 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.815s	user 0.014s	sys 0.008s
I20260709 12:22:54.173591 18129 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.231186 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:54.232987 18079 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:22:54.234488 18079 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:54.235041 18079 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:54.236049 18079 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.237300 17843 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:54.240883 17949 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.241740 17949 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:54.245580 18092 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:22:54.246809 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124486us
W20260709 12:22:54.135690 17982 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.814s	user 0.014s	sys 0.005s
I20260709 12:22:54.156456 13719 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:22:54.248482 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:54.248971 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111486us
I20260709 12:22:54.249305 18092 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:54.250120 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:54.250695 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:54.251811 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 790) took 1216 ms. Trace:
I20260709 12:22:54.252221 13719 rpcz_store.cc:276] 0709 12:22:53.035253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.090529 (+ 55276us) service_pool.cc:224] Handling call
0709 12:22:53.372806 (+282277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:54.248471 (+875665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:54.250911 (+  2440us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54000276d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:54.251602 (+   691us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54000276d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:54.251744 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450130048}
I20260709 12:22:54.252744 13795 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.253647 17822 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:54.254292 13795 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:22:54.254426 17822 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:54.154762 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:54.255602 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100948us
I20260709 12:22:54.256074 17669 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.166018 14419 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
W20260709 12:22:54.175942 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 291027us
I20260709 12:22:54.256965 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:54.257354 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.258872 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.259327 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.260252 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.260550 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.261416 17920 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.261699 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.261945 17920 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:54.262439 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43894.8us
I20260709 12:22:54.263335 18053 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:54.264384 17920 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.264946 17920 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:22:54.266810 17920 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:54.267628 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:54.269615 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.270042 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.271060 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.274134 13800 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:22:54.276458 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:54.251813 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.291208 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.284667 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.292725 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.296882 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.297258 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.297560 13797 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:22:54.298032 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:54.298928 14419 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:22:54.247694 17949 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:54.302359 13794 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:54.269207 17920 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:54.282974 18092 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:54.303324 18092 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:54.303745 18092 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:54.304070 17947 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:54.304908 17423 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.305409 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.305544 17423 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.283461 17669 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:54.291438 18094 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:54.172204 18021 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:54.292372 17904 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.310901 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.312261 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.311662 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.324090 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:54.332985 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59802.9us
I20260709 12:22:54.343194 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.337116 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.337940 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.334854 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.351821 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.348704 13645 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:54.306248 17947 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:54.308393 17669 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:54.357039 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.359582 13794 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:54.319783 18055 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:54.362620 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:54.364066 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.376034 14418 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:54.364763 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:54.333241 17904 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.366758 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:54.383443 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107081us
I20260709 12:22:54.367909 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.368217 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.384907 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:54.368387 18094 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.391438 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:54.368811 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.310804 17423 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.368837 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.317871 13649 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
W20260709 12:22:54.374724 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 464975us
I20260709 12:22:54.376832 14419 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:22:54.365600 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.287603 17982 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.378274 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.383512 13646 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:54.396191 13795 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:22:54.367322 18129 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.402467 13793 raft_consensus.cc:1217] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Deduplicated request from leader. Original: 8.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 12:22:54.402976 13793 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:22:54.337352 17949 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:54.389678 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:54.392932 14416 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:54.397187 13645 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:22:54.371953 18083 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.404870 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.406507 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:54.411722 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:54.427935 17982 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.428961 18146 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:22:54.343428 18131 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:22:54.432381 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 201258us
I20260709 12:22:54.435659 14419 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:22:54.436802 13794 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:22:54.437383 14419 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:54.440950 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:54.400578 17797 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:54.454670 13644 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:54.456171 13646 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:22:54.412665 18147 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:54.491284 13800 raft_consensus.cc:1217] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Deduplicated request from leader. Original: 5.4->[6.5-6.5]   Dedup: 6.5->[]
I20260709 12:22:54.491933 13800 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:22:54.492937 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.496398 18055 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:22:54.497447 18055 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:54.497777 18055 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:54.498721 18055 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.405393 17949 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.502370 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:54.508405 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.500020 18055 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:54.518966 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78095.2us
I20260709 12:22:54.520082 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.508216 18149 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:54.487208 18134 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:54.487406 18083 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:54.399724 18094 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:54.544849 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.552728 13794 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:22:54.398097 17735 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.454586 17843 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 12:22:54.513329 17949 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:54.404083 18100 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:54.487087 18079 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:22:54.492069 18077 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:22:54.589290 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.533331 18149 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:54.612915 13650 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:54.612182 13795 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:22:54.616178 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 82158.8us
I20260709 12:22:54.538347 18146 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:54.613605 13643 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:22:54.617190 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:54.555997 18147 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:54.558441 18112 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:54.622119 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.587122 18053 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:54.628294 13645 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:54.637809 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:54.544104 18021 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:22:54.561630 18096 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:54.645681 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.654188 18147 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:54.655403 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:54.646462 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.657289 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:54.625981 18132 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.309s	user 0.001s	sys 0.000s
W20260709 12:22:54.626916 18085 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.858s	user 0.005s	sys 0.009s
I20260709 12:22:54.663039 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:54.663733 13718 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:54.648453 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:54.624272 18092 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:54.641844 18152 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 12:22:54.678175 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.706700 13724 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:22:54.707257 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.661985 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:54.681502 18157 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:22:54.682404 18158 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:22:54.682299 18092 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:54.556473 18134 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.683035 13719 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:22:54.683534 17964 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.001s
I20260709 12:22:54.587258 17669 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:54.687793 18100 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:54.688452 17756 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.826s	user 0.014s	sys 0.000s
I20260709 12:22:54.651440 18053 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.654749 18147 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:54.694375 13646 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:22:54.694543 18085 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.696673 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.699239 18173 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:54.701887 18094 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.702894 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:54.703758 18149 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:54.720253 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71939.9us
W20260709 12:22:54.726207 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.727494 13793 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:22:54.727910 13766 connection.cc:256] RPC call timeout handler was delayed by -1.1373736529999405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3194s for a response
I20260709 12:22:54.725279 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:54.809307 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:54.729198 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:54.809775 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:54.810464 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:54.731668 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:54.810791 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:54.810904 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:54.811412 13725 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:54.811043 13643 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:22:54.742949 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 222971us
I20260709 12:22:54.783309 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.785895 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.791200 13795 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:22:54.812868 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:54.794100 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2582) took 1478 ms. Trace:
I20260709 12:22:54.667866 18112 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:54.790572 18168 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.140s	user 0.006s	sys 0.002s
I20260709 12:22:54.733708 18150 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:22:54.803753 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.885838 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:54.737211 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.813619 13652 rpcz_store.cc:276] 0709 12:22:53.315558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:53.326509 (+ 10951us) service_pool.cc:224] Handling call
0709 12:22:53.352581 (+ 26072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:53.378598 (+ 26017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:53.378629 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:53.425135 (+ 46506us) log.cc:844] Serialized 26 byte log entry
0709 12:22:53.705199 (+280064us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:53.705246 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:53.705332 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:54.793861 (+1088529us) raft_consensus.cc:1685] finished
0709 12:22:54.793882 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:54.794037 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5278,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":279599,"threads_started":1,"wal-append.queue_time_us":726224}
W20260709 12:22:54.887019 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.815321 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:54.808352 13766 connection.cc:256] RPC call timeout handler was delayed by -1.1165172099999836s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 12:22:54.887671 13727 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:54.816880 13796 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:54.764952 17964 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:54.806924 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.820631 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:54.791417 18147 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:54.821580 13726 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:54.802207 18173 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:54.761152 18160 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:54.849963 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:54.864696 13724 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:54.867226 18085 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:54.873865 18134 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:54.877821 18154 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:54.878598 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:54.878674 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:54.880653 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:54.881240 18159 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
W20260709 12:22:54.880640 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67868us
W20260709 12:22:54.850046 18161 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
W20260709 12:22:54.820808 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95628.6us
I20260709 12:22:54.793130 17904 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.944397 13719 raft_consensus.cc:2368] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
W20260709 12:22:54.891253 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.050s	sys 0.099s
I20260709 12:22:54.908424 13718 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:54.945155 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.050s	sys 0.099s
I20260709 12:22:54.904175 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:54.946003 13726 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:54.909739 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102908us
I20260709 12:22:54.789535 17756 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.957829 17756 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:54.958398 13792 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15.
I20260709 12:22:54.949177 18079 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:54.959707 18079 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:54.920723 13725 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:22:54.962617 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.963402 14423 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:54.963769 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.964162 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:54.964432 14416 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:22:54.964819 14421 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:22:54.965760 18156 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:22:54.966663 14418 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 14. Ops: []
I20260709 12:22:54.967921 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.962024 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:54.928519 13797 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:22:54.926882 13648 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:54.793262 18146 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:54.846832 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.974962 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.976202 17843 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:54.976662 17843 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:54.977499 17843 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:54.950476 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:54.934950 18154 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:54.952183 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 700416us
I20260709 12:22:54.952675 17613 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:54.953167 17857 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:54.955552 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:55.020851 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65385.3us
I20260709 12:22:54.961995 18079 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:54.892031 18077 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:54.835712 18094 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:54.989580 13726 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:22:54.987419 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:55.009171 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64960.4us
I20260709 12:22:54.939213 18129 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:54.963810 18053 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.011616 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:54.983760 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:54.907373 18172 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:55.025265 13719 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
W20260709 12:22:54.935133 17889 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.778s	user 0.014s	sys 0.000s
I20260709 12:22:54.897188 18150 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.033327 17613 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:22:55.035567 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 48292.1us
I20260709 12:22:55.071491 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:54.905315 18149 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:54.895531 18096 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:55.052572 18172 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.029824 18077 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:55.090247 18077 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:55.052490 18103 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:22:55.091881 18083 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
I20260709 12:22:55.092084 18103 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:55.092434 18083 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:55.092578 18103 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:55.081456 18151 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:55.095602 18151 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:55.096035 18151 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:55.081166 18092 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:55.094897 18096 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.099031 18092 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.012202 17964 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:54.911290 18173 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:55.076889 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:55.070502 18100 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.063707 17889 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.070894 18150 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:55.070760 18053 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.070694 17613 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.129657 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:55.130383 13726 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.089696 18172 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:54.949832 13720 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
W20260709 12:22:55.121690 18055 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.001s
I20260709 12:22:55.072266 17843 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:55.146409 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 268372us
I20260709 12:22:54.927196 18160 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.071326 18129 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:55.128639 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163877us
I20260709 12:22:55.110059 13643 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:22:55.141851 18053 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:55.154947 18177 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:22:55.156332 18053 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:55.159152 18170 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:22:55.160359 17909 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:22:55.162577 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.162202 18129 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:55.163096 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.163475 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.163908 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.165104 13720 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:55.165310 17904 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:55.165772 17904 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:55.109797 18154 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.166663 17904 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:55.169386 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.169922 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:55.172793 18021 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:22:55.173573 18151 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.176230 14391 leader_election.cc:400] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15.
I20260709 12:22:55.176395 18151 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.113749 18094 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:55.180169 18094 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.182421 18094 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.184332 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:55.145876 13726 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:55.177060 18021 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:55.176728 14391 leader_election.cc:403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:22:55.189352 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:55.197160 18021 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:55.200650 13799 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:22:55.141324 17613 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:22:55.216866 18156 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.221653 13797 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 13: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:55.230646 18096 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:55.233939 17797 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.732s	user 0.003s	sys 0.012s
I20260709 12:22:55.234694 17797 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:55.236770 18092 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.130666 18083 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.244822 17889 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.246491 13723 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:22:55.247364 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:55.248027 13794 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:55.249799 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:55.250424 14424 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:55.265863 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:55.285727 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.290376 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:22:55.292384 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:55.126662 18100 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.249155 18176 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:55.302030 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:55.247404 18160 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.217724 18156 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.293944 18021 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15.
I20260709 12:22:55.300966 18131 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:22:55.344003 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.349365 17949 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.735s	user 0.008s	sys 0.003s
I20260709 12:22:55.304435 18176 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:55.370981 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 359757us
I20260709 12:22:55.330083 18160 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:55.382696 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:55.305294 18077 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.329087 18166 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:22:55.397012 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 238562us
W20260709 12:22:55.400954 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 467914us
I20260709 12:22:55.367174 18176 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:55.368108 18131 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:55.349901 17949 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.383396 18183 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:55.433004 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 411187us
I20260709 12:22:55.434221 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:55.450743 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:55.455543 14420 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:22:55.397605 18160 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.473417 13794 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:22:55.475406 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:55.476125 13643 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:55.477145 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:55.478570 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.475407 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:55.465436 18147 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.001s
I20260709 12:22:55.490029 13652 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:55.479270 18149 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:55.510640 18182 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.112s
W20260709 12:22:55.534543 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.543007 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 70671.1us
I20260709 12:22:55.488255 18180 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:55.546330 13794 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:22:55.499578 18080 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:22:55.501500 18188 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:55.511209 17423 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:22:55.444113 18183 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.571137 14424 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:55.512799 18149 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:55.500233 17947 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:22:55.548727 18180 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:55.579675 18100 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.580510 18183 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.593900 17947 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:22:55.595037 17947 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:55.596022 17947 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.597469 17947 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:55.491683 18147 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.594154 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:55.629740 13794 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:55.631575 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:55.557209 18146 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:22:55.585117 18149 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:55.576200 18188 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:55.640460 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 468475us
I20260709 12:22:55.587568 18180 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:55.636365 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48355.8us
I20260709 12:22:55.586325 18156 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:55.631748 18157 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:22:55.575708 17982 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:22:55.598215 18112 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.587652 18151 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:22:55.546391 18131 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.639731 18100 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:22:55.659917 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.660686 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:55.661087 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.651275 18077 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:55.668008 18150 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.523s	user 0.005s	sys 0.006s
I20260709 12:22:55.668531 18150 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.659415 17949 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.670702 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:55.671329 13800 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.671865 18147 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.659886 18100 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:55.674765 18100 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:55.665373 18195 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:22:55.678694 17613 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.681072 17613 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.682863 13792 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:22:55.701041 18150 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:55.707597 13800 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:55.709196 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:55.711730 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.714072 18160 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.721591 18147 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:22:55.722325 18147 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:55.723187 18147 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:55.742908 13617 connection.cc:256] RPC call timeout handler was delayed by -1.7918305019998115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 12:22:55.743489 13617 connection.cc:256] RPC call timeout handler was delayed by -1.4687578469997788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 12:22:55.744021 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3629100949997337s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 12:22:55.747015 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1448239229998762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:22:55.747838 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0889578470000743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 12:22:55.749899 13643 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:22:55.746461 18103 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.005s
I20260709 12:22:55.744728 17797 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.753553 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:55.757331 13644 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
W20260709 12:22:55.758917 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 196867us
W20260709 12:22:55.754840 18172 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.605s	user 0.000s	sys 0.014s
I20260709 12:22:55.767298 13651 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:22:55.768539 18173 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.000s
I20260709 12:22:55.755553 18103 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.769055 18173 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.735864 18188 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.770653 13799 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:55.771281 13643 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:55.758831 17797 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:55.770872 18083 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.000s
W20260709 12:22:55.794430 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:55.797210 18182 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:55.798693 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:55.800599 17843 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 12:22:55.801908 14418 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:22:55.803181 18198 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
I20260709 12:22:55.797359 18083 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:55.797899 18182 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:55.805538 18194 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.295s	user 0.002s	sys 0.000s
I20260709 12:22:55.806839 18173 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.786216 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:55.807899 13646 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:55.775856 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:55.825251 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:55.826643 13800 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:22:55.831794 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:55.832193 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:55.832507 14418 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:55.783859 18103 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:55.764370 17613 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:22:55.766338 18172 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.856907 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:55.865482 13647 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:55.876641 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:55.822614 18100 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.895951 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:55.898918 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123179us
I20260709 12:22:55.901652 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:55.792049 18176 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.903072 18150 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:22:55.905786 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:55.906764 13646 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15.
I20260709 12:22:55.907737 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:55.866714 18151 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:55.852989 18211 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:22:55.928947 14418 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:55.882879 18147 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:55.934695 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:55.936389 14419 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:55.936980 14420 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:55.953385 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:55.962302 18218 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:22:55.963044 18218 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:55.964016 18218 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:55.966432 18083 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:55.967906 18188 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:55.968029 17857 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:22:55.968217 18154 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:55.968353 18188 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:22:55.971671 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 171459us
I20260709 12:22:55.937152 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:55.938271 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:55.938691 18173 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:55.940160 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:55.932107 18212 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:55.974886 13650 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:55.941278 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:55.942741 13617 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.656s (SENT)
I20260709 12:22:55.943435 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:55.944105 18207 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:55.944856 18172 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.895270 18182 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:55.947520 18100 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:55.947824 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46284.1us
I20260709 12:22:55.992950 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:55.881107 18181 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:55.972981 14422 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:55.981988 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:55.982224 13646 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
W20260709 12:22:55.989848 18053 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.016s	sys 0.005s
I20260709 12:22:55.953564 18151 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:55.995473 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52777.3us
W20260709 12:22:56.003261 18053 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.016s	sys 0.005s
I20260709 12:22:55.985925 18212 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:55.955369 18147 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:55.999600 18173 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:56.021641 18209 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:22:55.999284 18207 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:55.991858 17949 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
W20260709 12:22:56.023108 18221 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:22:56.038142 17949 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:22:56.038635 17949 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:56.039661 17949 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.040989 17949 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:56.023561 18151 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.017463 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:56.043541 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:56.044929 18197 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.493s	user 0.002s	sys 0.000s
I20260709 12:22:56.045415 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:56.031455 14422 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:56.009023 17613 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:22:55.969372 18188 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.010383 18100 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:56.054832 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:56.055568 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:56.030324 18212 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.052279 18210 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:22:56.059274 18210 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:56.059623 18176 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.057057 17920 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:56.060388 18210 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:56.045416 18207 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:56.062155 17920 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:56.064049 17920 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:22:56.058444 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56895us
I20260709 12:22:56.067976 18077 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:56.068789 18173 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.068969 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:22:56.069749 18209 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:56.070150 18209 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:56.011924 18182 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:22:56.059234 18203 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:56.015945 18079 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:22:56.073354 14391 leader_election.cc:400] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15.
W20260709 12:22:56.083235 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78831.1us
W20260709 12:22:56.084012 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.836s (SENT)
I20260709 12:22:56.083878 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:56.069352 18173 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.094076 18204 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:56.094561 18204 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.095571 18173 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.072342 18100 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.098492 18100 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:56.098810 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 725454us
I20260709 12:22:56.099849 18186 raft_consensus.cc:2764] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:22:56.057646 17797 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.081554 17669 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:56.102933 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 807) took 1085 ms. Trace:
I20260709 12:22:56.103341 13725 rpcz_store.cc:276] 0709 12:22:55.018000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.018189 (+   189us) service_pool.cc:224] Handling call
0709 12:22:55.018588 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:55.284926 (+266338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:55.284957 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:55.293907 (+  8950us) log.cc:844] Serialized 26 byte log entry
0709 12:22:55.407468 (+113561us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:22:55.498453 (+ 90985us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.044324 (+545871us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:56.102499 (+ 58175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:56.102581 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:56.102639 (+    58us) raft_consensus.cc:1685] finished
0709 12:22:56.102723 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:56.102864 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":642678,"prepare.queue_time_us":102798,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":123562,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":92535,"spinlock_wait_cycles":27532544,"thread_start_us":386567,"threads_started":3,"wal-append.queue_time_us":242530}
I20260709 12:22:56.085996 13766 leader_election.cc:400] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
I20260709 12:22:56.006363 18181 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.106760 13766 leader_election.cc:403] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:22:56.086230 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.906s (SENT) [suppressed 40 similar messages]
I20260709 12:22:56.091574 17920 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:22:56.091512 18180 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.092221 18209 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.093559 18147 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.100426 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:56.090816 18103 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:22:56.108079 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.058s (SENT)
I20260709 12:22:56.132535 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:56.114806 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 310769us
I20260709 12:22:56.136813 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:56.137501 13644 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:56.138958 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.078s (SENT)
I20260709 12:22:56.139606 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:56.140964 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:56.116777 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:56.095542 18204 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.147684 18204 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.150098 13652 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:22:56.150895 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:56.151515 13652 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:56.151700 13649 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:22:56.152029 18180 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.152879 13652 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:22:56.153610 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.229s (SENT)
I20260709 12:22:56.155265 13648 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:56.154078 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:56.159441 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.161356 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.161468 18055 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:56.161897 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.163558 18187 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:56.163995 18187 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:56.164326 18187 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.166651 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:56.166916 18033 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:22:56.167323 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50634.2us
I20260709 12:22:56.168016 18186 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.168639 18033 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:56.168756 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 810) took 1048 ms. Trace:
I20260709 12:22:56.168550 18186 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.169237 18033 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.145673 17797 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:56.170725 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70393.5us
I20260709 12:22:56.170816 17797 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.173148 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:56.173451 17797 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.174106 18183 raft_consensus.cc:2764] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:22:56.174300 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:56.174990 13726 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.162041 18055 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.177232 18055 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.147684 18209 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.169286 13719 rpcz_store.cc:276] 0709 12:22:55.123200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.123379 (+   179us) service_pool.cc:224] Handling call
0709 12:22:55.140271 (+ 16892us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:55.248482 (+108211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:55.248518 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:55.808284 (+559766us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:55.809259 (+   975us) log.cc:844] Serialized 52 byte log entry
0709 12:22:55.900790 (+ 91531us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:55.988696 (+ 87906us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.159911 (+171215us) log.cc:844] Serialized 14 byte log entry
0709 12:22:56.160179 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:56.160278 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:56.168535 (+  8257us) raft_consensus.cc:1685] finished
0709 12:22:56.168558 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:56.168701 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170699,"prepare.queue_time_us":78168,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":3898,"raft.queue_time_us":312010,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":10108,"spinlock_wait_cycles":22558592,"thread_start_us":91001,"threads_started":1,"wal-append.queue_time_us":145046}
I20260709 12:22:56.180435 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:56.181501 13723 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:22:56.181849 18147 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:56.179955 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:56.183914 18181 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.185245 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:56.185582 18183 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:56.186689 18183 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:22:56.187052 18183 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:56.187388 18183 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:22:56.189205 18211 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:22:56.190435 18160 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:56.190855 18160 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:56.191676 18160 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.192857 18160 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:56.184026 18186 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.216449 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:56.222473 18228 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:56.113338 18130 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:22:56.225117 13652 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:56.133211 18203 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10.
W20260709 12:22:56.227110 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73530.1us
I20260709 12:22:56.131980 18188 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:56.159360 18172 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:56.229465 18172 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.229740 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:56.229918 18172 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.231976 18228 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:56.232343 18228 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.232676 18228 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:56.234089 17797 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:56.156858 18146 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:56.238062 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.238564 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59686us
I20260709 12:22:56.145643 18176 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.238332 18146 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.239519 18176 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.240396 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:56.241225 18146 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.241704 18176 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.242354 17423 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:22:56.244045 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:56.247236 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.879s (SENT)
I20260709 12:22:56.252477 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:56.253213 13727 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.253825 13649 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:22:56.255292 13644 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:56.255924 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:56.263978 13726 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:22:56.268606 18228 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:56.244896 18149 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.000s
I20260709 12:22:56.273015 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:56.157927 18103 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:22:56.276119 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.045s (SENT)
I20260709 12:22:56.276654 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:56.277400 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:56.280187 13727 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:56.280920 13724 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:22:56.281391 13724 pending_rounds.cc:85] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:22:56.281682 13724 pending_rounds.cc:107] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:22:56.281981 13724 raft_consensus.cc:2889] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:56.282935 13727 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:22:56.289039 13722 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
W20260709 12:22:56.289309 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.288067 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:56.250850 17669 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:56.250437 18172 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.297732 13650 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:22:56.297789 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.171s (SENT)
W20260709 12:22:56.298908 13801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 1973) took 2032 ms (client timeout 1727 ms). Trace:
I20260709 12:22:56.257354 18055 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:22:56.303788 13723 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:22:56.311090 13723 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:22:56.311479 13723 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:22:56.311787 13723 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:56.313457 18182 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.278514 18103 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.245085 18176 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 12:22:56.306095 13801 rpcz_store.cc:269] 0709 12:22:54.259847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.270408 (+ 10561us) service_pool.cc:224] Handling call
0709 12:22:56.291858 (+2021450us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003e17d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:56.292010 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2590002432}
W20260709 12:22:56.318653 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.291996 17756 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:56.326061 13721 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:22:56.326444 13721 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:22:56.326772 13721 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:56.328613 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:56.329420 13649 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:56.256781 18210 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.250486 18033 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.290638 18149 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.316962 18207 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:56.335160 13652 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:22:56.315012 17613 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:56.344878 14391 leader_election.cc:400] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:22:56.351868 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:56.352793 13649 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
W20260709 12:22:56.370692 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 130363us
I20260709 12:22:56.332623 17756 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:56.392474 17756 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.375209 18187 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.397281 18187 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.321983 18182 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.399082 14422 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:22:56.400151 18182 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.401031 18146 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.361187 17613 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.377794 18210 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.381047 18183 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.382354 18055 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:56.369866 18021 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:22:56.307451 18190 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:22:56.383858 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 569) took 1184 ms. Trace:
W20260709 12:22:56.387916 13793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 567) took 1510 ms (client timeout 1773 ms). Trace:
I20260709 12:22:56.373521 18103 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.390322 18149 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.406968 13726 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
W20260709 12:22:56.412400 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.406919 14421 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:56.409821 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:56.454208 18147 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.455470 18210 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:56.454970 18147 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.456138 18210 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.458418 18147 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.457546 18210 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.460934 18130 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:22:56.369602 18207 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.464856 17920 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:22:56.466969 18131 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:22:56.465795 18207 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.467648 18131 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.468837 18131 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.474959 18131 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:22:56.454749 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 64350.4us
I20260709 12:22:56.483515 14422 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:22:56.496345 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:56.499083 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:22:56.505260 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.510958 18033 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.511967 18033 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.517642 18033 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:56.523533 13793 rpcz_store.cc:269] 0709 12:22:54.877644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.880413 (+  2769us) service_pool.cc:224] Handling call
0709 12:22:56.387710 (+1507297us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:56.387851 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083579648}
I20260709 12:22:56.430414 13792 rpcz_store.cc:276] 0709 12:22:55.199236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.199381 (+   145us) service_pool.cc:224] Handling call
0709 12:22:55.199773 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:55.694591 (+494818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:56.383517 (+688926us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:56.383771 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110413696}
I20260709 12:22:56.401990 18228 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:56.528199 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 96526.6us
I20260709 12:22:56.538276 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:56.452459 18055 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.541646 18055 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.401573 18146 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.422477 18166 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.124s
W20260709 12:22:56.542869 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.544905 18146 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.545782 18212 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.547183 17613 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.549690 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:56.551429 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:56.552464 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.554644 13793 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:56.557569 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:22:56.568887 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 281444us
I20260709 12:22:56.575786 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:56.578181 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:56.578655 13643 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:56.579355 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:56.555176 18103 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:56.597491 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:56.555063 18055 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:56.592007 13795 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:56.591902 13793 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:56.580240 18238 raft_consensus.cc:2764] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:22:56.555470 18146 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.594542 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:56.594625 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:56.560874 18112 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:56.595844 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.623260 13801 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:56.567641 18181 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:56.574015 18021 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.593742 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:56.606637 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:56.631335 14422 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:22:56.615095 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.588447 18129 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:22:56.587313 18212 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:56.645928 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:56.648555 13643 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:56.700528 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103117us
W20260709 12:22:56.703863 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 96478.7us
I20260709 12:22:56.709416 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:56.636296 18184 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:56.642941 18245 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:22:56.622153 18204 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:22:56.703307 17797 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:56.631505 18146 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:56.632635 18209 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 12:22:56.730765 14419 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:22:56.723199 18129 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:22:56.751425 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.763777 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:56.643679 18021 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.769820 13643 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:22:56.778872 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66428.4us
W20260709 12:22:56.751283 18236 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:22:56.715982 18134 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:22:56.787886 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:56.734700 18203 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:56.751674 18204 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:22:56.802764 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:56.803741 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:56.804620 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:56.814690 13800 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:56.813997 13794 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:56.811815 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.840677 13793 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
W20260709 12:22:56.841172 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.847479 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59698.8us
I20260709 12:22:56.853463 13794 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:56.890374 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.891311 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:56.893450 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.734230 17797 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:56.755676 18151 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:22:56.799167 18203 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.812741 18204 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:56.902884 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.905529 18146 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.907500 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.847716 18183 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.910935 18134 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:56.914144 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:56.854605 13719 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.013s	sys 0.014s
I20260709 12:22:56.790320 18254 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 12:22:56.798277 18184 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 12:22:56.924806 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.924129 13800 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:56.933565 13798 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:22:56.936849 13719 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.014s	sys 0.015s
I20260709 12:22:56.749846 18129 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:56.930881 18185 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.443s	user 0.002s	sys 0.005s
I20260709 12:22:56.925398 18203 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:56.948316 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:56.957643 14419 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:22:56.958134 13792 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:22:56.944342 18234 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.341s	user 0.001s	sys 0.001s
I20260709 12:22:56.964385 14422 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:56.936453 18134 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:56.992832 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:56.967000 18246 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:22:57.020028 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:56.994537 18188 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.766s	user 0.006s	sys 0.007s
W20260709 12:22:57.027946 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 207735us
I20260709 12:22:57.048604 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255f10165e844f2cb50d654996d639cf"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:57.049366 13792 raft_consensus.cc:606] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:57.050148 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:57.059237 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:56.994479 18134 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:57.064738 13647 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:22:57.067962 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:57.069391 18021 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:56.968631 18033 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 12:22:57.019217 17756 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:57.090742 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.095968 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:57.099105 13792 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:57.100924 13794 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:22:57.093181 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:57.089253 18188 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:57.118039 18172 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:57.119526 13652 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:57.120070 13652 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:57.122296 13652 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.124285 13652 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:57.125705 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.089207 17756 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:22:57.126967 13693 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50270 dropped due to backpressure: service queue is full with 50 items
I20260709 12:22:57.158985 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:57.157266 18146 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:22:57.163475 13801 raft_consensus.cc:606] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:57.164112 13801 raft_consensus.cc:614] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:22:57.166644 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.124593 18268 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:22:57.189252 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83121us
I20260709 12:22:57.190940 13652 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:22:57.192282 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:57.200433 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2806) took 1991 ms. Trace:
I20260709 12:22:57.201066 13799 rpcz_store.cc:276] 0709 12:22:55.209511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.209675 (+   164us) service_pool.cc:224] Handling call
0709 12:22:55.209979 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:55.772360 (+562381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.200156 (+1427796us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:57.200364 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288951680}
I20260709 12:22:57.162714 13693 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50270 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2688)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2700)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2691)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2350)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2704)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2707)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2690)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2684)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2347)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2694)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2702)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2710)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2715)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2693)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2345)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2686)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 828)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 829)
W20260709 12:22:57.193614 18103 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.588s	user 0.013s	sys 0.000s
I20260709 12:22:57.207638 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:57.189055 18172 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:57.208330 13799 raft_consensus.cc:606] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:57.208943 13799 raft_consensus.cc:614] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:22:57.211573 13794 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:22:57.212188 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:57.210994 13797 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:57.215241 13652 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:22:57.182161 18268 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:57.295400 18276 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:22:57.297181 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 823) took 1082 ms. Trace:
I20260709 12:22:57.297695 13719 rpcz_store.cc:276] 0709 12:22:56.214567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.215746 (+  1179us) service_pool.cc:224] Handling call
0709 12:22:56.259993 (+ 44247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:56.943201 (+683208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.943239 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:56.943834 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 12:22:57.065740 (+121906us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:57.065813 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.086219 (+ 20406us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:57.296872 (+210653us) raft_consensus.cc:1685] finished
0709 12:22:57.296904 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.297104 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97287,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":11861,"thread_start_us":740509,"threads_started":2,"wal-append.queue_time_us":124761}
I20260709 12:22:57.300324 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:57.301375 13719 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:57.301858 18085 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:22:57.302417 18085 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:57.303635 18085 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.305017 18085 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:22:57.309031 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:57.309830 13799 raft_consensus.cc:606] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:57.318686 18280 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:57.319290 18280 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:57.319671 18280 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:57.321880 18172 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.332209 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:22:57.333181 13799 raft_consensus.cc:606] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:57.335250 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:57.208863 17756 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.336524 17756 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:57.300457 18248 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.694s	user 0.001s	sys 0.001s
I20260709 12:22:57.344014 17797 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:57.324700 13693 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34450 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2691)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2350)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2704)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2707)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2690)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2684)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2347)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2694)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2702)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2710)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2717)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2715)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2693)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2345)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2686)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 828)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 829)
I20260709 12:22:57.376806 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:57.205247 18188 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:57.380110 18129 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.398167 18129 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.399710 18188 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.401007 18129 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:57.403669 14422 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:22:57.363519 13693 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34450 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2691)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 826)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2685)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2678)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2350)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2704)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2707)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2690)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2684)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2347)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2687)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2694)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2708)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2713)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2680)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2712)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2702)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2716)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2710)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2717)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2679)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2681)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2682)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2715)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2693)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2343)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2344)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2345)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2686)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2346)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2348)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2349)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 825)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 827)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 828)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 829)
I20260709 12:22:57.380067 18103 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:57.441952 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.472000 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:57.473201 13643 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:22:57.473368 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.474308 13646 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:22:57.474787 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 2635) took 1329 ms. Trace:
I20260709 12:22:57.475193 13650 rpcz_store.cc:276] 0709 12:22:56.139003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.140644 (+  1641us) service_pool.cc:224] Handling call
0709 12:22:57.468560 (+1327916us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:57.468720 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316052224}
I20260709 12:22:57.477027 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:57.487413 13719 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:57.493247 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.496266 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.520131 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:57.524780 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:57.528041 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:34450 dropped due to backpressure: service queue is full with 50 items
I20260709 12:22:57.525550 13722 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:57.532207 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.529623 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:57.449239 18103 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.541074 18283 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:57.545297 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:57.535959 18203 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.000s
I20260709 12:22:57.549027 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:57.549612 13719 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:57.533831 17797 raft_consensus.cc:476] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 12:22:57.555892 18268 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:57.565466 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:57.578647 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.580278 13719 raft_consensus.cc:2368] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11.
I20260709 12:22:57.581203 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2685) took 1006 ms. Trace:
I20260709 12:22:57.581630 13719 rpcz_store.cc:276] 0709 12:22:56.575151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.565057 (+989906us) service_pool.cc:224] Handling call
0709 12:22:57.581109 (+ 16052us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:57.590204 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.596196 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:57.597530 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2678) took 1025 ms. Trace:
I20260709 12:22:57.598080 13719 rpcz_store.cc:276] 0709 12:22:56.572100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.595816 (+1023716us) service_pool.cc:224] Handling call
0709 12:22:57.597451 (+  1635us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.599655 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:57.600145 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:57.605275 18200 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 2.054s	user 0.000s	sys 0.010s
I20260709 12:22:57.606504 18207 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.607044 18207 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.609440 18207 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:57.562795 18203 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.621745 18203 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:57.626159 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:22:57.636333 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 675) took 1701 ms (client timeout 1748 ms). Trace:
W20260709 12:22:57.637218 13647 rpcz_store.cc:269] 0709 12:22:55.934531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.940926 (+  6395us) service_pool.cc:224] Handling call
0709 12:22:57.636027 (+1695101us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:57.636239 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501664128}
W20260709 12:22:57.644730 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.615772 18103 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:57.653617 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 685) took 1040 ms. Trace:
I20260709 12:22:57.654917 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:57.657056 13643 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 5: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:57.657779 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.655287 14418 rpcz_store.cc:276] 0709 12:22:56.612916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.613102 (+   186us) service_pool.cc:224] Handling call
0709 12:22:56.619770 (+  6668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.018104 (+398334us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:22:57.018136 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:57.018669 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:22:57.195248 (+176579us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:57.195294 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.638993 (+443699us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54005302d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:57.639067 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:57.639143 (+    76us) raft_consensus.cc:1685] finished
0709 12:22:57.639164 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.653497 (+ 14333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":117756,"raft.queue_time_us":149559,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":9201,"spinlock_wait_cycles":409591168,"thread_start_us":303479,"threads_started":2,"wal-append.queue_time_us":199155}
I20260709 12:22:57.569178 18134 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.675926 13643 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:22:57.671283 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:57.661034 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:57.694412 13719 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:22:57.706516 13643 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 11. Ops: []
I20260709 12:22:57.670298 18134 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:57.703763 18285 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:57.667456 18280 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.722661 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:57.723989 13719 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:57.759315 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.736972 18188 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:57.791255 18287 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:22:57.717800 18285 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.764312 17797 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:57.762522 18188 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.842018 17797 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.866925 17797 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:57.895519 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2633) took 1758 ms. Trace:
I20260709 12:22:57.860143 18285 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:57.872129 18203 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.908058 13645 rpcz_store.cc:276] 0709 12:22:56.136966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.144965 (+  7999us) service_pool.cc:224] Handling call
0709 12:22:56.145284 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:56.173501 (+ 28217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.173536 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:56.173579 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:56.173614 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:57.895252 (+1721638us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:57.895303 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:57.895457 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64417024}
I20260709 12:22:57.909031 18203 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:57.896276 14421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 675) took 2596 ms (client timeout 1570 ms). Trace:
W20260709 12:22:57.910648 14421 rpcz_store.cc:269] 0709 12:22:55.300398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.301627 (+  1229us) service_pool.cc:224] Handling call
0709 12:22:57.896054 (+2594427us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004998d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:57.896163 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2532624640}
I20260709 12:22:57.911849 18203 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:57.873483 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:57.887001 18281 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:57.913116 13799 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:57.913419 18281 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.879828 18188 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:57.856099 18280 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:57.891170 18083 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:22:57.892639 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:57.893534 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:57.927870 18279 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.525s	user 0.004s	sys 0.006s
I20260709 12:22:57.894328 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:57.895463 17423 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.930896 17423 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.931612 13719 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:57.932147 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2704) took 1087 ms. Trace:
I20260709 12:22:57.872678 18151 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.912076 17857 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:57.898525 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:57.966454 17857 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:57.967417 18285 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:57.968211 18285 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:57.903695 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 2820) took 1103 ms. Trace:
I20260709 12:22:57.886179 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:22:57.998859 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 2651) took 1388 ms (client timeout 1775 ms). Trace:
W20260709 12:22:57.999470 13651 rpcz_store.cc:269] 0709 12:22:56.610227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.885735 (+1275508us) service_pool.cc:224] Handling call
0709 12:22:57.998762 (+113027us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:57.919453 13800 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:22:57.923097 18281 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:57.895258 18215 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.991s	user 0.002s	sys 0.017s
I20260709 12:22:57.924486 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:57.925237 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:57.932466 13719 rpcz_store.cc:276] 0709 12:22:56.845193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.720069 (+874876us) service_pool.cc:224] Handling call
0709 12:22:57.932087 (+212018us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.007647 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:57.998165 13797 rpcz_store.cc:276] 0709 12:22:56.800757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.810591 (+  9834us) service_pool.cc:224] Handling call
0709 12:22:57.903478 (+1092887us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:57.903616 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671413632}
I20260709 12:22:58.027187 14421 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:58.030758 13796 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:58.029536 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:58.054544 18273 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:58.055523 18273 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.069630 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:58.080266 18280 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:22:58.093724 18280 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.110128 18280 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:58.080683 17797 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:22:58.111052 18283 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:58.066893 18281 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:22:58.111763 18283 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.112715 18281 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:58.025431 18151 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.115006 18281 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:22:58.144677 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119855us
W20260709 12:22:58.146798 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.147343 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.147612 18151 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:58.177731 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.167110 18154 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.182696 18154 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.191967 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.192796 13719 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.196290 13796 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:22:58.200814 17423 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:58.192162 13799 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:58.208118 18294 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:22:58.206514 18146 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.215719 18146 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.219590 18146 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:58.219801 18282 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.821s	user 0.013s	sys 0.000s
I20260709 12:22:58.220286 18146 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:58.220875 18146 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.223599 18146 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:58.224627 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:22:58.224381 18204 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.228026 18204 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:22:58.229619 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.226240 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:58.191618 18154 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:22:58.230935 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2705) took 1382 ms (client timeout 1747 ms). Trace:
W20260709 12:22:58.232983 13719 rpcz_store.cc:269] 0709 12:22:56.848896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.030738 (+1181842us) service_pool.cc:224] Handling call
0709 12:22:58.230866 (+200128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.232473 14419 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:22:58.236241 13799 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:58.236997 14421 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:22:58.237251 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.239494 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:58.240191 13799 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:58.240259 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:58.241096 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2707) took 1314 ms (client timeout 1684 ms). Trace:
W20260709 12:22:58.241541 13719 rpcz_store.cc:269] 0709 12:22:56.938968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.240025 (+1301057us) service_pool.cc:224] Handling call
0709 12:22:58.241037 (+  1012us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.234593 17843 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:58.237991 18154 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.246327 18154 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.246629 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:58.247473 14424 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:58.248772 18154 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:58.249874 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:58.246738 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:58.245774 17843 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.251530 14416 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:22:58.253134 14417 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:22:58.256305 14419 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:22:58.238989 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:58.260912 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53049.8us
I20260709 12:22:58.261708 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:58.263732 13799 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:22:58.263881 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.234120 17857 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:58.266623 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.268266 14416 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:22:58.268540 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.267791 18147 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:58.269897 18154 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:58.270367 18147 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:58.256242 18273 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:58.271435 18147 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.274503 18151 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:58.269306 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.260718 18181 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:58.278539 18181 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.278978 18181 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:58.283475 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.284262 13718 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:58.284772 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2690) took 1707 ms (client timeout 2058 ms). Trace:
I20260709 12:22:58.284945 18280 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:58.285211 13718 rpcz_store.cc:269] 0709 12:22:56.577556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.283162 (+1705606us) service_pool.cc:224] Handling call
0709 12:22:58.284708 (+  1546us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.287411 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.288378 13718 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
W20260709 12:22:58.289093 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2695) took 1573 ms (client timeout 1925 ms). Trace:
W20260709 12:22:58.289619 13718 rpcz_store.cc:269] 0709 12:22:56.715395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.286173 (+1570778us) service_pool.cc:224] Handling call
0709 12:22:58.289015 (+  2842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.290961 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.294656 13795 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 12:22:58.296206 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.294672 13718 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:22:58.293637 14424 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:22:58.298089 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2684) took 1724 ms (client timeout 2078 ms). Trace:
W20260709 12:22:58.299078 13718 rpcz_store.cc:269] 0709 12:22:56.574029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.290621 (+1716592us) service_pool.cc:224] Handling call
0709 12:22:58.298029 (+  7408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.300345 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.301090 13718 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.303391 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.269345 18285 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:58.304140 13719 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:22:58.303493 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:58.305745 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2621) took 2828 ms. Trace:
I20260709 12:22:58.306823 13648 rpcz_store.cc:276] 0709 12:22:55.475342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.475490 (+   148us) service_pool.cc:224] Handling call
0709 12:22:55.475785 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:56.156459 (+680674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.303239 (+2146780us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54002224d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:58.303411 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562196864}
I20260709 12:22:58.311404 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.311599 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.312089 13800 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:58.313663 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.315304 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:58.320643 13719 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:22:58.321499 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2703) took 1504 ms (client timeout 1866 ms). Trace:
I20260709 12:22:58.304582 17843 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:58.322028 13719 rpcz_store.cc:269] 0709 12:22:56.817638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.302182 (+1484544us) service_pool.cc:224] Handling call
0709 12:22:58.321415 (+ 19233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.305374 18296 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:22:58.327443 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.316721 18273 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:58.305632 17857 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 12:22:58.333564 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2687) took 1757 ms (client timeout 2121 ms). Trace:
W20260709 12:22:58.334007 13719 rpcz_store.cc:269] 0709 12:22:56.575901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.324348 (+1748447us) service_pool.cc:224] Handling call
0709 12:22:58.333333 (+  8985us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.325224 18156 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:58.336265 13718 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:22:58.336881 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2347) took 1572 ms (client timeout 1898 ms). Trace:
W20260709 12:22:58.337069 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.337487 13718 rpcz_store.cc:269] 0709 12:22:56.764616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.299993 (+1535377us) service_pool.cc:224] Handling call
0709 12:22:58.336811 (+ 36818us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.344668 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.335908 18273 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:58.356680 18273 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.358732 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2331) took 2104 ms. Trace:
I20260709 12:22:58.359200 13726 rpcz_store.cc:276] 0709 12:22:56.254045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.267001 (+ 12956us) service_pool.cc:224] Handling call
0709 12:22:56.267391 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:56.452836 (+185445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.358495 (+1905659us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54001a5c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:58.358666 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320323840}
I20260709 12:22:58.360740 13648 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:22:58.360437 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:58.364012 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:58.364737 13648 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:22:58.365763 18218 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:58.366190 18218 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:58.367211 18218 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.368098 17669 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:22:58.368579 18218 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:58.368877 17669 consensus_queue.cc:1243] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:22:58.369415 17669 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:58.369876 17669 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:58.370724 18280 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:58.370873 18211 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:22:58.371027 18203 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:58.371274 18211 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:58.371557 18203 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:58.339771 18156 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:58.371994 18203 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:58.371965 18281 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.374516 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:58.364046 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.375357 13800 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:58.383114 13800 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:22:58.384758 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:22:58.386695 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:58.385749 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2709) took 1413 ms (client timeout 1752 ms). Trace:
I20260709 12:22:58.387393 13718 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:22:58.387897 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2699) took 1610 ms (client timeout 1970 ms). Trace:
W20260709 12:22:58.388208 13718 rpcz_store.cc:269] 0709 12:22:56.777864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.386433 (+1608569us) service_pool.cc:224] Handling call
0709 12:22:58.387845 (+  1412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.389091 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.389770 13718 raft_consensus.cc:2376] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8. Re-sending same reply.
W20260709 12:22:58.390900 14391 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.632s (SENT)
I20260709 12:22:58.372378 18211 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:22:58.398865 18211 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:22:58.403962 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.404050 18283 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:58.403745 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:58.404794 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:58.404995 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:58.406333 13645 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:58.408926 17669 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.409528 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:58.411466 18296 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:22:58.412202 18296 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:58.391520 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:58.406128 13644 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:22:58.415799 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:58.390542 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2694) took 1681 ms (client timeout 2045 ms). Trace:
W20260709 12:22:58.417052 13718 rpcz_store.cc:269] 0709 12:22:56.709217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.388864 (+1679647us) service_pool.cc:224] Handling call
0709 12:22:58.390470 (+  1606us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.417795 13719 rpcz_store.cc:269] 0709 12:22:56.971434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.336225 (+1364791us) service_pool.cc:224] Handling call
0709 12:22:58.384518 (+ 48293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.419481 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.420452 13718 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:22:58.421129 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2692) took 1822 ms (client timeout 2158 ms). Trace:
W20260709 12:22:58.421633 13718 rpcz_store.cc:269] 0709 12:22:56.602321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.419154 (+1816833us) service_pool.cc:224] Handling call
0709 12:22:58.421057 (+  1903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.423228 18296 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.427104 13649 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:58.372298 18156 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:58.428620 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:58.429602 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:58.436693 13800 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:58.441469 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.444346 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:58.445050 13648 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:22:58.446535 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:58.446542 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:58.423391 18209 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:22:58.449776 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.450444 13718 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 5: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:58.448652 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:58.427021 18129 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.408967 18281 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:58.447871 18209 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:22:58.453367 18209 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.454955 18209 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:58.455497 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:58.455700 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.456307 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 1921) took 1027 ms. Trace:
W20260709 12:22:58.456655 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2713) took 1392 ms (client timeout 1719 ms). Trace:
I20260709 12:22:58.457286 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2341) took 2186 ms. Trace:
I20260709 12:22:58.439021 18228 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:22:58.457149 13722 rpcz_store.cc:269] 0709 12:22:57.064051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.451737 (+1387686us) service_pool.cc:224] Handling call
0709 12:22:58.456574 (+  4837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.456655 13652 rpcz_store.cc:276] 0709 12:22:57.429148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.429411 (+   263us) service_pool.cc:224] Handling call
0709 12:22:58.455324 (+1025913us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:22:58.456242 (+   918us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9533824}
I20260709 12:22:58.461186 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:58.474994 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2696) took 1743 ms (client timeout 2038 ms). Trace:
W20260709 12:22:58.476730 13718 rpcz_store.cc:269] 0709 12:22:56.731919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.441109 (+1709190us) service_pool.cc:224] Handling call
0709 12:22:58.474876 (+ 33767us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.438555 18154 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:58.443500 18149 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:22:58.506307 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45193.6us
I20260709 12:22:58.457651 13723 rpcz_store.cc:276] 0709 12:22:56.270974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.322718 (+ 51744us) service_pool.cc:224] Handling call
0709 12:22:56.338210 (+ 15492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:56.338805 (+   595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.338851 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:56.339464 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 12:22:56.554336 (+214872us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:56.713964 (+159628us) log.cc:844] Serialized 14 byte log entry
0709 12:22:57.299619 (+585655us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:22:58.456863 (+1157244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.456951 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.456995 (+    44us) raft_consensus.cc:1685] finished
0709 12:22:58.457093 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.457231 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1768731,"prepare.queue_time_us":1700110,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":189695,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5088,"spinlock_wait_cycles":9041280,"thread_start_us":299723,"threads_started":2,"wal-append.queue_time_us":210588,"wal-append.run_cpu_time_us":7037,"wal-append.run_wall_time_us":1766124}
I20260709 12:22:58.515398 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.516443 13723 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:58.516935 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2714) took 1432 ms (client timeout 1736 ms). Trace:
W20260709 12:22:58.517306 13723 rpcz_store.cc:269] 0709 12:22:57.085025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.515023 (+1429998us) service_pool.cc:224] Handling call
0709 12:22:58.516880 (+  1857us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.507846 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2340) took 2238 ms. Trace:
I20260709 12:22:58.518894 13721 rpcz_store.cc:276] 0709 12:22:56.269513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.308895 (+ 39382us) service_pool.cc:224] Handling call
0709 12:22:56.325849 (+ 16954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:56.333274 (+  7425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.333324 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:56.333997 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 12:22:56.554586 (+220589us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:56.932613 (+378027us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.507359 (+1574746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.507477 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.507541 (+    64us) raft_consensus.cc:1685] finished
0709 12:22:58.507560 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.507753 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1627164,"prepare.queue_time_us":1700149,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"thread_start_us":493076,"threads_started":2,"wal-append.queue_time_us":256228,"wal-append.run_cpu_time_us":3194,"wal-append.run_wall_time_us":1417902}
I20260709 12:22:58.508030 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.523005 13718 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:22:58.523699 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2680) took 1951 ms (client timeout 2229 ms). Trace:
W20260709 12:22:58.524130 13718 rpcz_store.cc:269] 0709 12:22:56.572740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.507660 (+1934920us) service_pool.cc:224] Handling call
0709 12:22:58.523613 (+ 15953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.525276 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.521936 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.526985 13721 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:22:58.526064 13718 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:58.534688 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.535683 13722 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:22:58.537487 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2697) took 1762 ms (client timeout 2020 ms). Trace:
W20260709 12:22:58.538148 13722 rpcz_store.cc:269] 0709 12:22:56.775239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.463673 (+1688434us) service_pool.cc:224] Handling call
0709 12:22:58.537416 (+ 73743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.508764 18080 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:22:58.569993 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.462543 18245 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:58.576485 13718 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:58.578562 13723 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:58.534515 18281 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.582722 18281 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.586721 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.587291 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:58.587724 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2702) took 1790 ms (client timeout 2183 ms). Trace:
W20260709 12:22:58.588191 13722 rpcz_store.cc:269] 0709 12:22:56.797513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.539057 (+1741544us) service_pool.cc:224] Handling call
0709 12:22:58.587654 (+ 48597us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.536980 18181 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.589367 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.591424 18181 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:58.592303 13647 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:22:58.593863 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2326) took 2345 ms. Trace:
I20260709 12:22:58.594449 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.595078 18237 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:58.595386 13726 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:22:58.595841 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2716) took 1272 ms. Trace:
I20260709 12:22:58.595635 18237 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.596180 13726 rpcz_store.cc:276] 0709 12:22:57.323263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.594142 (+1270879us) service_pool.cc:224] Handling call
0709 12:22:58.595776 (+  1634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.518389 18204 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:58.598770 18242 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:58.599509 18242 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.594105 18245 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.597138 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:58.603191 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.603422 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2710) took 1599 ms (client timeout 2031 ms). Trace:
W20260709 12:22:58.603876 13726 rpcz_store.cc:269] 0709 12:22:57.004381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.596890 (+1592509us) service_pool.cc:224] Handling call
0709 12:22:58.603358 (+  6468us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.523015 18160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.919s	user 0.018s	sys 0.018s
I20260709 12:22:58.504707 18129 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.517652 18154 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:58.911700 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.580816 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2706) took 1721 ms (client timeout 2110 ms). Trace:
I20260709 12:22:58.594394 13720 rpcz_store.cc:276] 0709 12:22:56.248922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.249118 (+   196us) service_pool.cc:224] Handling call
0709 12:22:56.249548 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:56.250044 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.250087 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:56.250906 (+   819us) log.cc:844] Serialized 52 byte log entry
0709 12:22:56.552377 (+301471us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:56.580705 (+ 28328us) log.cc:844] Serialized 14 byte log entry
0709 12:22:57.319502 (+738797us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b18000d22e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1
0709 12:22:57.849040 (+529538us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18000d22e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:58.253547 (+404507us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b18000d22e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:22:58.253794 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.453974 (+200180us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b18000d22e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5397ac858f 00007f5390b07571 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd
0709 12:22:58.593485 (+139511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.593577 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.593621 (+    44us) raft_consensus.cc:1685] finished
0709 12:22:58.593632 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.593796 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1950741,"prepare.queue_time_us":1767895,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3768,"raft.queue_time_us":494304,"raft.run_cpu_time_us":424,"raft.run_wall_time_us":51184,"spinlock_wait_cycles":184171520,"thread_start_us":245374,"threads_started":1,"wal-append.queue_time_us":399375}
I20260709 12:22:58.604926 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:58.605237 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2712) took 1524 ms (client timeout 1817 ms). Trace:
W20260709 12:22:58.612186 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.925434 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2717) took 1601 ms (client timeout 1906 ms). Trace:
W20260709 12:22:58.923349 13718 rpcz_store.cc:269] 0709 12:22:56.859332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.524961 (+1665629us) service_pool.cc:224] Handling call
0709 12:22:58.580706 (+ 55745us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.925837 13726 rpcz_store.cc:269] 0709 12:22:57.323562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.604562 (+1281000us) service_pool.cc:224] Handling call
0709 12:22:58.925367 (+320805us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.927253 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.927546 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:58.924460 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.930732 13718 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 10 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:22:58.936312 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2681) took 2363 ms (client timeout 3045 ms). Trace:
W20260709 12:22:58.931299 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2679) took 2358 ms (client timeout 2879 ms). Trace:
I20260709 12:22:58.931618 13726 raft_consensus.cc:2410] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:22:58.938992 13720 rpcz_store.cc:269] 0709 12:22:56.572406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.924107 (+2351701us) service_pool.cc:224] Handling call
0709 12:22:58.931236 (+  7129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.940541 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.941468 13720 raft_consensus.cc:2410] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:22:58.941970 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2715) took 1856 ms. Trace:
I20260709 12:22:58.942360 13720 rpcz_store.cc:276] 0709 12:22:57.085393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.940226 (+1854833us) service_pool.cc:224] Handling call
0709 12:22:58.941904 (+  1678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.943356 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:58.944154 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2693) took 2305 ms. Trace:
I20260709 12:22:58.944522 13720 rpcz_store.cc:276] 0709 12:22:56.638488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.943096 (+2304608us) service_pool.cc:224] Handling call
0709 12:22:58.944095 (+   999us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.925453 13723 rpcz_store.cc:269] 0709 12:22:57.057457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.518133 (+1460676us) service_pool.cc:224] Handling call
0709 12:22:58.581683 (+ 63550us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.939291 13718 rpcz_store.cc:269] 0709 12:22:56.573057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.926979 (+2353922us) service_pool.cc:224] Handling call
0709 12:22:58.936208 (+  9229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.697322 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2338) took 2428 ms. Trace:
W20260709 12:22:58.822463 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2708) took 1876 ms (client timeout 1823 ms). Trace:
W20260709 12:22:58.690441 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.955693 13693 connection.cc:256] RPC call timeout handler was delayed by -2.981718099000318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:22:58.743628 14417 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:22:58.753119 13721 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
W20260709 12:22:58.957472 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2682) took 2384 ms (client timeout 3171 ms). Trace:
W20260709 12:22:58.605603 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2711) took 1579 ms (client timeout 1997 ms). Trace:
W20260709 12:22:58.957930 13726 rpcz_store.cc:269] 0709 12:22:56.573362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.927335 (+2353973us) service_pool.cc:224] Handling call
0709 12:22:58.957380 (+ 30045us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.608022 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 1.828e+06us
I20260709 12:22:58.609004 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:58.960181 14424 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:58.697638 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 690) took 1308 ms. Trace:
I20260709 12:22:58.631546 17423 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.330s
W20260709 12:22:58.698611 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:58.962203 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:58.954453 13719 rpcz_store.cc:269] 0709 12:22:56.946113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.822135 (+1876022us) service_pool.cc:211] Skipping call since client already timed out
0709 12:22:58.822351 (+   216us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:22:58.783730 13725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2683) took 2210 ms (client timeout 1656 ms). Trace:
W20260709 12:22:58.963647 13725 rpcz_store.cc:269] 0709 12:22:56.573689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.783406 (+2209717us) service_pool.cc:211] Skipping call since client already timed out
0709 12:22:58.783620 (+   214us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:22:58.675143 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 587) took 1570 ms. Trace:
W20260709 12:22:58.956137 13693 connection.cc:256] RPC call timeout handler was delayed by -2.981718099000318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 12:22:58.958441 13722 rpcz_store.cc:269] 0709 12:22:57.010635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.589063 (+1578428us) service_pool.cc:224] Handling call
0709 12:22:58.590229 (+  1166us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:22:58.957768 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2701) took 2168 ms (client timeout 2093 ms). Trace:
I20260709 12:22:58.964869 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:58.967386 13720 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:22:58.967808 13720 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:58.965858 13693 connection.cc:256] RPC call timeout handler was delayed by -2.9754180990003078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15560000000000002s for a response
I20260709 12:22:58.598016 18237 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:58.974548 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.968430 13693 connection.cc:256] RPC call timeout handler was delayed by -2.9232180990002234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 12:22:58.975307 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:58.975450 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8647180990001289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 12:22:58.976159 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8287180990005254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 12:22:58.976718 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8215180990005138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 12:22:58.961357 14422 rpcz_store.cc:276] 0709 12:22:57.367207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.368361 (+  1154us) service_pool.cc:224] Handling call
0709 12:22:58.675728 (+1307367us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:58.675859 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59275520}
I20260709 12:22:58.964779 13794 rpcz_store.cc:276] 0709 12:22:57.104433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.104648 (+   215us) service_pool.cc:224] Handling call
0709 12:22:57.104935 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:57.211988 (+107053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.674815 (+1462827us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:58.675071 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244571392}
W20260709 12:22:58.966581 13721 rpcz_store.cc:269] 0709 12:22:56.789872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.521565 (+1731693us) service_pool.cc:224] Handling call
0709 12:22:58.957687 (+436122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.972314 13720 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:58.980001 13720 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:58.974284 13726 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
W20260709 12:22:58.977098 13693 connection.cc:256] RPC call timeout handler was delayed by -2.8071180990004905s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 12:22:58.980636 13726 pending_rounds.cc:85] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:22:58.981036 13726 pending_rounds.cc:107] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260709 12:22:58.980739 13693 connection.cc:256] RPC call timeout handler was delayed by -2.7324180990003697s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 12:22:58.981457 13726 raft_consensus.cc:2889] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:22:58.981666 13693 connection.cc:256] RPC call timeout handler was delayed by -2.6775180990002809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:22:58.982247 13693 connection.cc:256] RPC call timeout handler was delayed by -2.6649180990002606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 12:22:58.982735 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5974180990001514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 12:22:58.983124 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5731180990001121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 12:22:58.983587 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4615180990003864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 12:22:58.901625 18301 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:22:58.984000 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4129180990003078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21810000000000002s for a response
I20260709 12:22:58.985658 13719 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:22:58.992164 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2689) took 2415 ms. Trace:
W20260709 12:22:58.768656 18295 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 12:22:58.826035 18308 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.300s	user 0.001s	sys 0.003s
I20260709 12:22:58.987980 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2349) took 2217 ms. Trace:
I20260709 12:22:58.991118 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2686) took 2415 ms. Trace:
I20260709 12:22:58.998291 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:58.998625 13726 rpcz_store.cc:276] 0709 12:22:56.575556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.973436 (+2397880us) service_pool.cc:224] Handling call
0709 12:22:58.973844 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.990846 (+ 17002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.991059 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:59.000196 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 825) took 2229 ms. Trace:
I20260709 12:22:59.000392 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.000540 13726 rpcz_store.cc:276] 0709 12:22:56.771115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.999423 (+2228308us) service_pool.cc:224] Handling call
0709 12:22:58.999692 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.999885 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.999914 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.999957 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:22:58.999993 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.000042 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.000144 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.992656 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:59.003046 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 827) took 2231 ms. Trace:
I20260709 12:22:59.003213 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:58.567339 18150 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.423s
W20260709 12:22:58.993527 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4003180990002875s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
I20260709 12:22:59.008411 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:59.008764 13693 connection.cc:256] RPC call timeout handler was delayed by -2.3886180990002686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
W20260709 12:22:58.968585 18160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.369s	user 0.018s	sys 0.037s
I20260709 12:22:58.995671 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2344) took 2720 ms. Trace:
I20260709 12:22:59.014511 13723 rpcz_store.cc:276] 0709 12:22:56.275086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.968447 (+2693361us) service_pool.cc:224] Handling call
0709 12:22:58.995132 (+ 26685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.995352 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.995380 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.995423 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:22:58.995466 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.995518 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.995613 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.609846 18281 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:58.992514 13719 rpcz_store.cc:276] 0709 12:22:56.576985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.972328 (+2395343us) service_pool.cc:224] Handling call
0709 12:22:58.985479 (+ 13151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.991922 (+  6443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.992108 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.608008 18080 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:59.002388 13795 raft_consensus.cc:2368] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:22:59.003427 13726 rpcz_store.cc:276] 0709 12:22:56.771427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.002173 (+2230746us) service_pool.cc:224] Handling call
0709 12:22:59.002526 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.002706 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:59.002730 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:59.002770 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:22:59.002839 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.002896 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.002992 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:59.005968 13721 rpcz_store.cc:276] 0709 12:22:56.770718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.982712 (+2211994us) service_pool.cc:224] Handling call
0709 12:22:58.983386 (+   674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:58.983675 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.983712 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:58.983773 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:58.983825 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.983889 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.987894 (+  4005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:58.972052 18083 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:22:59.009390 13693 connection.cc:256] RPC call timeout handler was delayed by -2.3841180990002613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22130000000000002s for a response
W20260709 12:22:59.028877 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6029180990003624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30810000000000004s for a response
W20260709 12:22:59.029285 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5471180990002722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3143s for a response
W20260709 12:22:59.029577 13693 connection.cc:256] RPC call timeout handler was delayed by -1.385118099000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33230000000000004s for a response
I20260709 12:22:58.953037 13727 rpcz_store.cc:276] 0709 12:22:56.268856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.286150 (+ 17294us) service_pool.cc:224] Handling call
0709 12:22:56.286543 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:22:56.533905 (+247362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.533957 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:22:56.535442 (+  1485us) log.cc:844] Serialized 52 byte log entry
0709 12:22:56.630525 (+ 95083us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:22:58.254545 (+1624020us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.661206 (+406661us) log.cc:844] Serialized 14 byte log entry
0709 12:22:58.696815 (+ 35609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:58.696932 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:58.697001 (+    69us) raft_consensus.cc:1685] finished
0709 12:22:58.697019 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:58.697234 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2018868,"prepare.queue_time_us":1639751,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":537359,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":20602,"spinlock_wait_cycles":63130496,"thread_start_us":373001,"threads_started":3,"wal-append.queue_time_us":271148,"wal-append.run_cpu_time_us":2505,"wal-append.run_wall_time_us":1021207}
I20260709 12:22:58.996553 18237 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:58.992863 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:59.045593 13794 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:22:59.045354 18237 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:58.609181 18151 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.392s
I20260709 12:22:59.020129 14424 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:22:59.021600 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2342) took 2750 ms. Trace:
I20260709 12:22:58.984386 18183 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:59.047838 18183 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.049162 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:58.996223 17756 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.052158 17756 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.052958 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:59.053556 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.027253 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:22:59.005336 18218 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.006s
I20260709 12:22:59.054488 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.030195 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:59.054852 13801 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:22:59.010574 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.055680 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:59.055704 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:22:59.056200 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.056849 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.057468 14422 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:22:59.031368 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.057857 14422 pending_rounds.cc:85] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 12
I20260709 12:22:59.058120 14422 pending_rounds.cc:107] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 14.13
W20260709 12:22:59.058470 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:59.033478 18209 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.003s
I20260709 12:22:59.055003 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.035363 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.059038 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 828) took 2287 ms. Trace:
I20260709 12:22:59.036913 18083 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.036885 18080 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.059321 13724 rpcz_store.cc:276] 0709 12:22:56.271305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:56.555469 (+284164us) service_pool.cc:224] Handling call
0709 12:22:56.555816 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:56.568007 (+ 12191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:56.568041 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:57.907340 (+1339299us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:57.907974 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:22:58.031207 (+123233us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:58.270055 (+238848us) log.cc:844] Serialized 14 byte log entry
0709 12:22:59.005211 (+735156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.005347 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.005413 (+    66us) raft_consensus.cc:1685] finished
0709 12:22:59.021252 (+ 15839us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.021506 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":581559,"prepare.queue_time_us":239468,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":376842,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":51138,"spinlock_wait_cycles":14512384,"thread_start_us":533352,"threads_started":2,"wal-append.queue_time_us":301000}
W20260709 12:22:59.027945 18203 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.000s
I20260709 12:22:59.042052 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.068526 13720 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:22:59.042834 18312 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:22:59.044590 18129 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:58.998677 18242 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:59.047149 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.047648 18237 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:59.060032 13723 rpcz_store.cc:276] 0709 12:22:56.771585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.015973 (+2244388us) service_pool.cc:224] Handling call
0709 12:22:59.058556 (+ 42583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.058727 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:59.058753 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:59.058798 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:22:59.058837 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.058888 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.058981 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:22:59.039386 18150 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:59.069478 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2343) took 2797 ms. Trace:
I20260709 12:22:59.071122 13800 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 5.
I20260709 12:22:59.058772 14422 raft_consensus.cc:2889] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:22:59.074795 13794 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:22:59.079937 13720 rpcz_store.cc:276] 0709 12:22:56.272579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.966992 (+2694413us) service_pool.cc:224] Handling call
0709 12:22:58.967269 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.069119 (+101850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.069391 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19840}
I20260709 12:22:59.080101 18238 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.079569 18129 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:22:59.091624 18150 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:22:59.092576 18150 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.071062 18312 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:59.077198 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.072494 18203 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:59.139276 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.748s (SENT)
I20260709 12:22:59.143178 18238 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:22:59.143436 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66338.1us
I20260709 12:22:59.144428 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.144595 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.145385 18209 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:22:59.106593 17756 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:59.110349 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:59.109006 18183 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:59.110864 18083 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:59.114393 18312 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.131794 18150 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:59.146234 18238 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:59.150846 18203 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:59.153667 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.021s	sys 0.036s
I20260709 12:22:59.146198 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:59.166850 18312 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:22:59.191332 18238 raft_consensus.cc:2764] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:22:59.196676 14417 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:22:59.197431 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.197885 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:22:59.198336 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:22:59.199545 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:22:59.202358 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139389us
I20260709 12:22:59.203806 17904 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:22:59.204252 18129 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:22:59.182648 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.207401 18129 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:22:59.207916 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:59.190168 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.021s	sys 0.036s
W20260709 12:22:59.209326 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 670504us
I20260709 12:22:59.209467 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:22:59.209933 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.185209 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:59.172756 18154 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:59.212508 18296 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:22:59.212508 18146 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:22:59.213145 18281 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:22:59.161180 18156 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.712s	user 0.006s	sys 0.005s
I20260709 12:22:59.213641 18281 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.214232 18281 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:22:59.215567 18083 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.216991 18083 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:59.182648 17669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.035s	sys 0.047s
W20260709 12:22:59.218583 17669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.036s	sys 0.047s
I20260709 12:22:59.219228 13793 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:59.221529 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:22:59.191574 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45511.7us
I20260709 12:22:59.229561 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.235965 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:59.237407 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45095.1us
W20260709 12:22:59.243041 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.29461e+06us
I20260709 12:22:59.244179 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.246986 13725 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:59.238582 18083 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:22:59.253846 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.258496 18055 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:59.259392 18146 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:59.260520 18146 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.257709 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.262285 18149 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:22:59.264228 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.233023 17947 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:22:59.233624 17756 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.275405 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:22:59.254364 18313 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:22:59.276288 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.276753 18055 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:59.252549 18296 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:22:59.256765 18212 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:22:59.282511 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:22:59.283420 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44295.2us
I20260709 12:22:59.256196 18238 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.256050 18245 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:22:59.283268 18311 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.275300 18150 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:22:59.276836 17947 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:22:59.243738 18156 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.287194 18212 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.294582 18212 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:22:59.298549 13692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 830) took 2104 ms. Trace:
I20260709 12:22:59.298648 18296 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.299086 13692 rpcz_store.cc:276] 0709 12:22:57.194479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.298458 (+2103979us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:22:59.299793 13692 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50270 dropped due to backpressure: service queue is full with 50 items [suppressed 2 similar messages]
I20260709 12:22:59.300060 13692 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:50270 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
I20260709 12:22:59.284000 18129 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:22:59.307262 18129 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.307551 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2348) took 2537 ms. Trace:
I20260709 12:22:59.307744 18129 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:22:59.308007 18304 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.788s	user 0.007s	sys 0.008s
I20260709 12:22:59.308941 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2345) took 3033 ms. Trace:
I20260709 12:22:59.307982 13722 rpcz_store.cc:276] 0709 12:22:56.770410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.966643 (+2196233us) service_pool.cc:224] Handling call
0709 12:22:59.000244 (+ 33601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.000620 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:59.000647 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:59.001119 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:22:59.199687 (+198568us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:22:59.199728 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.199813 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.307300 (+107487us) raft_consensus.cc:1685] finished
0709 12:22:59.307328 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.307481 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":793,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":8436,"thread_start_us":165230,"threads_started":1,"wal-append.queue_time_us":252249}
I20260709 12:22:59.309293 13718 rpcz_store.cc:276] 0709 12:22:56.275419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.966794 (+2691375us) service_pool.cc:224] Handling call
0709 12:22:58.967269 (+   475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:58.973350 (+  6081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.973384 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:58.997668 (+ 24284us) log.cc:844] Serialized 26 byte log entry
0709 12:22:59.155666 (+157998us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:22:59.156120 (+   454us) log.cc:844] Serialized 14 byte log entry
0709 12:22:59.156388 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.156495 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.308709 (+152214us) raft_consensus.cc:1685] finished
0709 12:22:59.308731 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.308883 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5646,"prepare.queue_time_us":764,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":22527,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":6557,"spinlock_wait_cycles":33792,"thread_start_us":114817,"threads_started":1,"wal-append.queue_time_us":243883}
I20260709 12:22:59.293398 17669 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:22:59.289145 18055 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:22:59.306206 13651 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:22:59.281772 18209 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:22:59.282399 17756 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.317065 17756 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:59.320566 17947 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:59.321843 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.322168 17947 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:22:59.322494 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:59.323017 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:59.323382 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.323169 18149 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:22:59.325695 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:22:59.326439 18245 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.327579 13793 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:22:59.328331 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.328611 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.328980 13724 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:22:59.329216 13720 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 5.
I20260709 12:22:59.330999 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:22:59.331741 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2185051289998228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:22:59.332257 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2117553009998119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 12:22:59.332728 13617 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.708s (SENT)
W20260709 12:22:59.333299 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1879051289997733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 12:22:59.334014 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1168051290001131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.211s for a response
I20260709 12:22:59.335016 13724 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:22:59.335433 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.336035 13727 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:22:59.336407 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.336989 13720 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:59.337256 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.337832 13727 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:22:59.348261 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2346) took 2584 ms. Trace:
I20260709 12:22:59.348732 13725 rpcz_store.cc:276] 0709 12:22:56.764355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.964346 (+2199991us) service_pool.cc:224] Handling call
0709 12:22:58.970565 (+  6219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.247445 (+276880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.347936 (+100491us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400174154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:22:59.348176 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635192064}
I20260709 12:22:59.292187 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.352865 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.357033 13651 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:22:59.357692 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.357841 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:59.358355 13647 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:59.358649 13720 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.360047 14416 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.361769 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.362654 13725 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:22:59.353557 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.355856 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.357533 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.365417 13649 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:59.366397 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:22:59.367610 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 76999us
I20260709 12:22:59.369412 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:59.370517 13726 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:22:59.389647 13720 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:59.392609 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.370294 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:22:59.368479 13723 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:22:59.393646 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.393615 13720 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:59.394309 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.394364 13649 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:22:59.394979 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 7.
I20260709 12:22:59.368251 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.395850 13651 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:22:59.387948 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.396903 18183 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.402769 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.403169 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.403606 13725 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:22:59.403823 13726 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:22:59.404606 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.403599 13718 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:22:59.390115 13722 raft_consensus.cc:2376] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12. Re-sending same reply.
I20260709 12:22:59.407200 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:59.407394 13723 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
I20260709 12:22:59.407816 13721 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:22:59.408027 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.416492 18237 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:22:59.418536 18242 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:22:59.420064 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.421094 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.419343 18242 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:22:59.423281 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 166134us
I20260709 12:22:59.359127 18204 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:59.406320 13649 raft_consensus.cc:2410] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:22:59.382589 18238 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:22:59.430223 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.434804 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:22:59.440435 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:59.347396 18211 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:22:59.343448 18245 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.425438 18183 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:22:59.371044 18281 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.441437 13721 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:22:59.442994 13693 leader_election.cc:400] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:22:59.461073 13693 leader_election.cc:403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:22:59.454468 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.461323 13723 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:22:59.461134 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56251.4us
W20260709 12:22:59.463526 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.454064 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:22:59.450039 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:59.448623 18311 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:22:59.466991 18311 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:22:59.467367 18311 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:59.468259 18311 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.469507 18311 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:22:59.470312 18156 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:22:59.427970 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84386.1us
I20260709 12:22:59.525225 13727 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:22:59.529173 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86390.6us
I20260709 12:22:59.531095 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.532413 13693 leader_election.cc:400] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
W20260709 12:22:59.532567 13617 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.906s (SENT)
I20260709 12:22:59.534451 18187 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:22:59.534327 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:22:59.536413 18326 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.456713 18242 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.541072 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.540432 18242 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:22:59.549878 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95517.5us
I20260709 12:22:59.550951 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:22:59.429499 17669 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:22:59.556109 13652 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:22:59.429849 18204 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.559610 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.561867 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:22:59.569149 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:22:59.572072 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:22:59.566677 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94460.9us
I20260709 12:22:59.525259 18156 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:22:59.591013 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 870) took 1356 ms. Trace:
I20260709 12:22:59.448402 18160 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:22:59.592080 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.591024 18218 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:22:59.473013 18273 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 12:22:59.548676 18238 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:22:59.589200 18156 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.605585 13651 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:22:59.528746 18325 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:22:59.599506 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:22:59.458626 18134 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.139s
W20260709 12:22:59.598016 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43862.8us
I20260709 12:22:59.591699 14420 rpcz_store.cc:276] 0709 12:22:58.231625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:58.233951 (+  2326us) service_pool.cc:224] Handling call
0709 12:22:58.675919 (+441968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:58.676516 (+   597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:58.676559 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:59.202358 (+525799us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b2c000d57a8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:22:59.202812 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 12:22:59.252823 (+ 50011us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:22:59.252871 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:22:59.252950 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:22:59.472195 (+219245us) raft_consensus.cc:1685] finished
0709 12:22:59.472223 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:22:59.587862 (+115639us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b5400584b50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:22:59.588009 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":20397,"spinlock_wait_cycles":1655514112,"thread_start_us":49573,"threads_started":1,"wal-append.queue_time_us":89420}
I20260709 12:22:59.598589 18218 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:59.591845 18329 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:22:59.631563 18156 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:22:59.648554 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 198609us
I20260709 12:22:59.637841 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:22:59.637566 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 310971us
I20260709 12:22:59.650481 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.638899 18238 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:22:59.660962 14390 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
I20260709 12:22:59.660902 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.642616 18218 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:22:59.642200 18329 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:59.679193 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:22:59.680455 13721 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
W20260709 12:22:59.683013 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:22:59.641880 17756 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:22:59.696214 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:22:59.648130 17904 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:22:59.645777 18160 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 12:22:59.712966 18339 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:22:59.743098 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.744098 13646 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:22:59.748307 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.748986 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.749265 13646 raft_consensus.cc:2410] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:22:59.752031 13645 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:22:59.673596 18238 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:22:59.761246 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:22:59.762173 13644 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:22:59.673245 18329 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:22:59.694821 18055 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.696883 18326 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:22:59.788878 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:22:59.760490 18344 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:22:59.703594 18342 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:22:59.695235 18211 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:22:59.801440 18273 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:22:59.848066 18238 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:22:59.849294 18330 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:22:59.849895 17735 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:22:59.852262 18352 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:22:59.808104 18281 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:22:59.868136 18212 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.000s
I20260709 12:22:59.853114 18273 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:22:59.901551 18273 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:22:59.872570 18212 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.907786 18212 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:59.867519 17735 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:22:59.908970 17735 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:22:59.911405 17947 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.001s
W20260709 12:22:59.924197 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 263363us
W20260709 12:22:59.944300 18129 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.629s	user 0.013s	sys 0.001s
I20260709 12:22:59.945107 18129 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:22:59.913179 18329 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:22:59.955397 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 251954us
I20260709 12:22:59.956085 18329 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:22:59.965960 18156 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:22:59.968815 18156 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:59.971737 18129 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:22:59.981000 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.986510 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:00.007400 13644 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:00.028961 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:22:59.961668 18218 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.032342 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:00.033228 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:00.032974 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 336833us
I20260709 12:23:00.039561 13648 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:00.034142 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:23:00.044378 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:00.046702 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:00.004725 18187 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:00.068360 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:00.103709 13650 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.004618 18156 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:00.058666 18365 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.110417 18365 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.112773 18365 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.117388 13648 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:00.127736 13650 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:00.044486 18358 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:00.133379 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 829) took 3011 ms. Trace:
I20260709 12:23:00.133812 13719 rpcz_store.cc:276] 0709 12:22:57.122084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.016574 (+1894490us) service_pool.cc:224] Handling call
0709 12:22:59.022973 (+  6399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:22:59.050551 (+ 27578us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:22:59.050963 (+   412us) log.cc:844] Serialized 14 byte log entry
0709 12:22:59.200747 (+149784us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:22:59.200821 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:22:59.200882 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:00.133120 (+932238us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b54002f5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:00.133174 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:00.133310 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":763,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":402,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":62807424,"thread_start_us":148775,"threads_started":1,"wal-append.queue_time_us":191498}
I20260709 12:23:00.134933 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:00.136385 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92103.5us
I20260709 12:23:00.104787 18342 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:00.031041 18357 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:00.129710 18306 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:00.147161 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.147873 13722 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:00.142001 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:00.136961 18360 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:23:00.148641 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.140512 18342 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.067041 18160 consensus_queue.cc:1243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:00.141757 18356 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:23:00.159389 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 112785us
I20260709 12:23:00.127737 18156 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.075726 18281 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:23:00.148212 18372 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:00.168282 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.151162 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:00.173616 13724 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:23:00.157300 18342 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:00.174885 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.169286 14390 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
I20260709 12:23:00.176076 13726 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:00.171414 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.151427 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.177027 13723 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.177030 13718 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:00.157045 18238 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.170841 13721 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:00.171169 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.166867 13727 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:00.176129 14390 leader_election.cc:403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:23:00.178773 13725 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:00.170701 18380 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:00.160348 18356 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:00.163143 18360 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:00.191025 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.220453 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.181996 18367 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9.
I20260709 12:23:00.209470 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.212886 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.215678 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.216961 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.222610 13718 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10. Re-sending same reply.
I20260709 12:23:00.217752 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.218983 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:00.221927 13727 raft_consensus.cc:2376] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8. Re-sending same reply.
I20260709 12:23:00.223501 13648 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.224053 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.218991 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.225068 13725 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12. Re-sending same reply.
I20260709 12:23:00.214365 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:00.227396 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.227998 13725 raft_consensus.cc:2376] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8. Re-sending same reply.
I20260709 12:23:00.228549 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.229183 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.230022 13721 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12. Re-sending same reply.
I20260709 12:23:00.231190 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.232247 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.233469 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.229188 13727 raft_consensus.cc:2376] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13. Re-sending same reply.
I20260709 12:23:00.234103 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.234617 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.236940 13725 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.234104 13720 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.238688 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.240916 13718 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:00.243870 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.244343 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:00.247829 13648 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:23:00.182566 18372 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:00.255230 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:00.249588 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:00.257838 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.258623 13718 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:00.219424 18362 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:00.269662 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.220522 18380 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:00.266072 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.219749 18356 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.268110 13720 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:23:00.243285 18373 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:23:00.273197 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.280026 13727 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:23:00.284132 13724 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:23:00.284638 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55379.3us
I20260709 12:23:00.285054 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.222064 18358 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:23:00.292476 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.183187 18238 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.295403 13720 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:00.302449 13725 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:00.306102 13794 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:00.314461 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.315505 13726 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:00.317111 13725 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:00.318887 13794 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:00.271072 18218 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:00.287545 18357 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:00.353256 18357 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:00.355217 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 887) took 1158 ms. Trace:
I20260709 12:23:00.355675 14416 rpcz_store.cc:276] 0709 12:22:59.196449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.198015 (+  1566us) service_pool.cc:224] Handling call
0709 12:23:00.355028 (+1157013us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400498c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:00.355150 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370433792}
I20260709 12:23:00.356695 18379 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:23:00.359615 18352 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.360213 18352 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.360764 18209 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.361333 18209 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.361599 18372 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.362602 18352 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.282375 18380 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:00.386660 14418 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:23:00.333387 18156 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:00.333863 18366 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:23:00.363732 18209 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.334949 18362 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:00.336310 17735 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.339962 18245 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.303740 18281 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.341873 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:00.341708 18360 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.370500 13726 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:23:00.371833 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.372576 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:00.428900 13720 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:00.375588 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:00.387566 13723 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:00.388330 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.392505 14420 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:23:00.318717 18377 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:00.430373 13725 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:23:00.385835 18335 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:00.408772 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.392254 17669 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.331286 18238 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.430339 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:00.392011 18334 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.429003 13792 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:23:00.407132 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152011us
I20260709 12:23:00.438225 13796 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:00.448051 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:00.449774 13722 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:23:00.451457 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.452175 13726 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:00.452634 13726 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:00.452122 13721 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:00.453008 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:00.459184 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.460071 13721 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:00.462005 13548 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 252) took 25952 ms (client timeout 15000 ms). Trace:
I20260709 12:23:00.413640 18325 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:23:00.469681 13726 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.479559 13726 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:00.462477 13548 rpcz_store.cc:269] 0709 12:22:34.501521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:34.501733 (+   212us) service_pool.cc:224] Handling call
0709 12:22:35.052858 (+551125us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390ad0640 00007f53999a7846 00007f5399ab190e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:00.454032 (+25401174us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:22:52.980943 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:22:52.981669 (+   726us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:22:52.981729 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:22:52.981750 (+    21us) tablet.cc:662] Decoding operations
0709 12:22:52.983447 (+  1697us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:22:52.983574 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:22:52.983601 (+    27us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:22:52.984934 (+  1333us) tablet.cc:701] Row locks acquired
0709 12:22:52.984972 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:22:52.985220 (+   248us) write_op.cc:270] Start()
0709 12:22:52.985384 (+   164us) write_op.cc:276] Timestamp: P: 1783599772985146 usec, L: 0
0709 12:22:52.985422 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:22:52.986156 (+   734us) log.cc:844] Serialized 7070 byte log entry
0709 12:22:54.203538 (+1217382us) op_driver.cc:464] REPLICATION: finished
0709 12:22:54.521841 (+318303us) write_op.cc:301] APPLY: starting
0709 12:22:54.543798 (+ 21957us) tablet.cc:1367] starting BulkCheckPresence
0709 12:22:54.873951 (+330153us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b1800116560. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:22:55.554422 (+680471us) tablet.cc:1370] finished BulkCheckPresence
0709 12:22:55.563778 (+  9356us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:22:55.741503 (+177725us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:22:55.742027 (+   524us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:22:55.742102 (+    75us) write_op.cc:312] APPLY: finished
0709 12:22:55.829234 (+ 87132us) log.cc:844] Serialized 164 byte log entry
0709 12:22:55.986894 (+157660us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:55.987195 (+   301us) write_op.cc:454] Released schema lock
0709 12:22:55.987305 (+   110us) write_op.cc:341] FINISH: Updating metrics
0709 12:22:55.987942 (+   637us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:22:55.987986 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2510309,"reported_tablets":50,"spinlock_wait_cycles":10316032,"child_traces":[["op",{"apply.queue_time_us":241653,"apply.run_cpu_time_us":59547,"apply.run_wall_time_us":1479889,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79511,"num_ops":18,"prepare.queue_time_us":41544,"prepare.run_cpu_time_us":17782,"prepare.run_wall_time_us":335350,"raft.queue_time_us":805103,"raft.run_cpu_time_us":10813,"raft.run_wall_time_us":143937,"replication_time_us":1217930,"spinlock_wait_cycles":7242112,"thread_start_us":978810,"threads_started":5,"wal-append.queue_time_us":655302,"wal-append.run_cpu_time_us":32214,"wal-append.run_wall_time_us":2408880}]]}
I20260709 12:23:00.454191 18209 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:00.455931 18156 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:00.485179 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.354661 18357 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.487892 13721 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 7.
I20260709 12:23:00.435891 18281 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.437750 17669 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.466420 18349 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:23:00.482110 18273 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.000s
I20260709 12:23:00.451653 18365 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:23:00.509686 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:00.495487 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.519317 13722 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:00.447932 18335 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.485607 18209 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.534682 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:00.539599 13723 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10. Re-sending same reply.
I20260709 12:23:00.456037 18334 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.487409 18156 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:00.540848 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.528875 18349 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:00.538486 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.546487 13723 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:00.535048 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.549714 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.534175 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.544631 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.551558 13721 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:00.530799 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:00.540683 18209 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:00.563535 13644 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:00.535979 18281 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:00.566318 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.548781 18156 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:00.537443 18204 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:00.570416 18149 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:00.525884 18273 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.572995 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:00.576658 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 234877us
I20260709 12:23:00.568670 18245 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:00.573722 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 128736us
W20260709 12:23:00.582836 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 183050us
I20260709 12:23:00.582449 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.572717 18187 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:00.597275 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 201916us
I20260709 12:23:00.586038 18238 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:00.610677 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:00.614022 13797 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:00.602981 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.622601 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.629509 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:00.573870 18149 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:00.651340 18187 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:00.636488 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:00.638320 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:00.593444 17735 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:00.655143 13643 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:00.655310 17735 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.606647 18377 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:00.656060 17735 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.656767 18377 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.598146 18160 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:00.637441 18245 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:23:00.670356 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:00.698356 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80596.2us
W20260709 12:23:00.703336 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:00.704924 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82414us
I20260709 12:23:00.708376 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.712646 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.668294 18281 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:00.676623 18382 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:00.677042 18384 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.704087 18371 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:00.676872 18363 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:00.704456 18403 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.720345 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:00.716617 18238 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.625842 18349 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.719596 18281 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.732321 18363 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.733186 18392 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:23:00.728539 18384 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.722724 18325 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:00.725782 17669 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:00.751695 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:00.732668 18406 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:00.743645 18281 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:00.738828 18371 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:00.741866 18335 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.737639 18413 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:00.741544 18334 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:00.758848 18325 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:00.789958 17669 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 12:23:00.793586 18390 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:23:00.735497 18403 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.807907 18273 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.801345 18325 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:00.801738 18402 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:00.813656 18391 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:23:00.778885 18396 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:23:00.767256 18406 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.878129 18402 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:00.884968 18342 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.000s
W20260709 12:23:00.878679 18407 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:23:00.891352 18406 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:00.892688 18403 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:00.893509 18363 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:00.894469 18356 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.007s
I20260709 12:23:00.895198 18356 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:00.895419 18380 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.897545 18356 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:00.885743 18342 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:00.849833 18382 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:00.897545 18380 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:00.941393 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 223854us
I20260709 12:23:01.033422 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:00.943677 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 307129us
W20260709 12:23:00.944612 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.570s (SENT)
W20260709 12:23:00.959043 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 236951us
I20260709 12:23:00.903735 18402 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:01.057359 18336 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.058370 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.058645 18336 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:23:01.060636 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 116409us
I20260709 12:23:01.030751 18384 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:00.779883 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.056984 18281 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.045266 18365 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:01.065837 18365 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.067209 13798 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:01.069624 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.071137 13792 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:23:01.072721 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.075769 13797 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:01.076789 18363 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:01.079449 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.076977 18281 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.081115 18342 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.075564 18416 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:01.096006 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:01.096102 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63086.2us
I20260709 12:23:01.107023 18335 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:01.119390 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:01.130651 18363 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:01.131142 18156 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.554s	user 0.014s	sys 0.013s
I20260709 12:23:01.130334 18382 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.132402 18156 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.133610 18237 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:01.134539 18237 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:01.136488 18237 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:01.138229 18307 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.476s	user 0.011s	sys 0.004s
I20260709 12:23:01.141916 18363 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.144942 18363 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.147946 18325 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.149247 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.149655 18416 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.152697 18363 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.156873 18365 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.184836 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 690) took 1829 ms. Trace:
I20260709 12:23:01.171824 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:01.165671 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:01.187176 13647 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:23:01.187654 13648 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:01.189379 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:01.189478 18372 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:01.190367 14422 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:01.190519 18372 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.191226 14424 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:01.185662 13652 rpcz_store.cc:276] 0709 12:22:59.355375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.355535 (+   160us) service_pool.cc:224] Handling call
0709 12:23:01.184228 (+1828693us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400112954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:01.184616 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441471360}
I20260709 12:23:01.189733 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2678) took 1806 ms. Trace:
I20260709 12:23:01.193681 18372 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.193730 13647 rpcz_store.cc:276] 0709 12:22:59.383187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.383357 (+   170us) service_pool.cc:224] Handling call
0709 12:23:01.186480 (+1803123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:01.187814 (+  1334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.189455 (+  1641us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b5400112950. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:01.189652 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4145749248}
W20260709 12:23:01.084832 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.684s (SENT)
I20260709 12:23:01.153708 18416 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.178177 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:01.166441 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:01.219202 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47636.4us
W20260709 12:23:01.170879 18209 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.586s	user 0.016s	sys 0.012s
I20260709 12:23:01.219223 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:01.220496 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.220836 14416 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:01.219296 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:01.220295 18209 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.222225 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:01.223932 18209 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.222375 18363 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:01.173897 18387 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.683s	user 0.001s	sys 0.000s
I20260709 12:23:01.149677 18156 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.118232 18149 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.247042 14421 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:23:01.238703 18362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.003s	sys 0.014s
W20260709 12:23:01.256283 18362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.004s	sys 0.015s
I20260709 12:23:01.260587 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.260974 18362 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:01.266690 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:01.268949 18149 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:01.269524 18325 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:01.268971 14416 raft_consensus.cc:2410] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:23:01.247282 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.282730 18430 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.283730 18430 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.286453 18363 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.287380 18363 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.289430 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.291320 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.310250 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.310981 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.270179 18336 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:01.311815 13799 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.312049 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:01.300709 18418 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:01.314857 18418 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:01.314985 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.315135 13794 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:01.315354 18418 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:01.315646 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.303124 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:01.316561 14417 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:01.324419 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:01.293926 18416 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:23:01.290206 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2687) took 1333 ms. Trace:
W20260709 12:23:01.328743 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81825.4us
I20260709 12:23:01.330416 18160 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:01.330526 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50686.4us
I20260709 12:23:01.329136 13649 rpcz_store.cc:276] 0709 12:22:59.957331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:59.957548 (+   217us) service_pool.cc:224] Handling call
0709 12:22:59.958043 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:22:59.960347 (+  2304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:22:59.960428 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:22:59.961437 (+  1009us) log.cc:844] Serialized 26 byte log entry
0709 12:23:01.289319 (+1327882us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b5400128ca0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393b6b6b8 00007f5393bf492b 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c
0709 12:23:01.289419 (+   100us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:01.289541 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.289674 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:01.289735 (+    61us) raft_consensus.cc:1685] finished
0709 12:23:01.289749 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.290005 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":257,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":135,"spinlock_wait_cycles":173341952,"thread_start_us":70742,"threads_started":1,"wal-append.queue_time_us":57086,"wal-append.run_cpu_time_us":3419,"wal-append.run_wall_time_us":1099824}
I20260709 12:23:01.332016 18183 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:01.332687 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:01.332964 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.331259 18160 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.337226 14422 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:23:01.290084 18363 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.329910 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.300218 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:01.307600 18430 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:01.363067 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:01.368824 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 306647us
I20260709 12:23:01.367331 13792 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:01.374931 14419 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:01.384089 13794 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:23:01.384296 18055 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:01.384959 18416 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:01.385303 18055 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.384660 13794 pending_rounds.cc:85] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 10
I20260709 12:23:01.390273 13794 pending_rounds.cc:107] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:01.390753 13794 raft_consensus.cc:2889] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:01.392117 18363 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:01.402449 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102477us
I20260709 12:23:01.409154 18055 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.412602 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2868) took 1090 ms. Trace:
W20260709 12:23:01.419937 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87104.6us
I20260709 12:23:01.419534 13796 rpcz_store.cc:276] 0709 12:23:00.322167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.322410 (+   243us) service_pool.cc:224] Handling call
0709 12:23:00.322719 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.445245 (+122526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.412026 (+966781us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:01.412350 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265059712}
W20260709 12:23:01.432161 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 3.133s (SENT) [suppressed 35 similar messages]
I20260709 12:23:01.449431 14417 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:01.451179 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:01.452001 14420 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
W20260709 12:23:01.453603 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 123871us
I20260709 12:23:01.455579 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.455509 14419 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:01.463212 17735 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:01.466382 13718 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:01.466382 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.469321 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2753) took 1098 ms. Trace:
I20260709 12:23:01.470038 13726 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:23:01.469235 13719 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.469902 13727 rpcz_store.cc:276] 0709 12:23:00.371190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.371504 (+   314us) service_pool.cc:224] Handling call
0709 12:23:01.311767 (+940263us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:23:01.461917 (+150150us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b54001f0c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:01.469172 (+  7255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51193472}
I20260709 12:23:01.391507 18362 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.497356 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:01.498710 13799 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
W20260709 12:23:01.500909 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125349us
W20260709 12:23:01.509075 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53657.2us
I20260709 12:23:01.512825 13792 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:01.510247 18160 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:01.523662 18398 raft_consensus.cc:2764] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:01.525130 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:01.526199 18337 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:23:01.535316 18365 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:01.536145 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 702) took 1402 ms. Trace:
I20260709 12:23:01.537708 18337 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:01.538311 18337 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:01.537094 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:01.508136 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.552371 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:01.561327 18209 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:23:01.563532 14417 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:01.564304 18209 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:01.564746 18209 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:01.566649 18325 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:01.567406 18325 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:01.568518 18325 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.570513 18352 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:01.570828 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.571122 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2864) took 1289 ms. Trace:
I20260709 12:23:01.572844 18281 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:01.571537 13800 rpcz_store.cc:276] 0709 12:23:00.281890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.282079 (+   189us) service_pool.cc:224] Handling call
0709 12:23:00.294906 (+ 12827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:00.295161 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:00.318497 (+ 23336us) log.cc:844] Serialized 14 byte log entry
0709 12:23:00.605759 (+287262us) log.cc:844] Serialized 14 byte log entry
0709 12:23:01.170046 (+564287us) spinlock_profiling.cc:243] Waited 6.79 ms on lock 0x7b1800377d40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:01.564240 (+394194us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b1800377d40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f53937aad70 00007f539379e293 00007f5393793cea 00007f539379423a 00007f5393bfea2a 00007f5393c0e0bf 00007f5393c0e038
0709 12:23:01.570489 (+  6249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:01.570581 (+    92us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:01.570649 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.570763 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.570930 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9259,"prepare.queue_time_us":27628,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":108,"raft.queue_time_us":26110,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":4315,"spinlock_wait_cycles":26933248,"thread_start_us":269216,"threads_started":1,"wal-append.queue_time_us":245703}
I20260709 12:23:01.573444 18281 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:01.574648 18281 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.576805 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.577739 13721 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.584185 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.585325 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.587129 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.588706 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.566047 18209 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.590893 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.591540 18209 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.592430 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:01.593691 18245 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:01.594165 18245 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:01.594704 18245 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.597229 18334 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:01.599160 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:01.606794 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.607765 13727 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:01.608738 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:01.610334 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.616638 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:01.536805 13650 rpcz_store.cc:276] 0709 12:23:00.133985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:00.137623 (+  3638us) service_pool.cc:224] Handling call
0709 12:23:00.138165 (+   542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:00.138579 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:00.138616 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:01.061572 (+922956us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:23:01.063373 (+  1801us) log.cc:844] Serialized 26 byte log entry
0709 12:23:01.241429 (+178056us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:01.249904 (+  8475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:01.257206 (+  7302us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:01.508616 (+251410us) raft_consensus.cc:1685] finished
0709 12:23:01.508641 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:01.535900 (+ 27259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2275,"spinlock_wait_cycles":3707904,"thread_start_us":174050,"threads_started":1,"wal-append.queue_time_us":318607}
I20260709 12:23:01.514755 18416 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
W20260709 12:23:01.617884 18434 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.198s	user 0.003s	sys 0.000s
I20260709 12:23:01.619181 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:01.619539 18149 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:01.620432 18375 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:01.620818 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:01.651587 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:01.652534 13792 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.639423 14417 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:01.658468 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:01.660153 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:01.666396 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45799.2us
I20260709 12:23:01.668787 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:01.573979 18296 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:23:01.681720 18237 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:01.558863 18337 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.680220 18334 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:01.705641 18394 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:23:01.713466 13724 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:23:01.720568 18241 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:01.721357 18241 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:01.721868 18241 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:01.682845 18418 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.734512 18312 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:01.737914 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:01.741505 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.743453 13723 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:01.743705 18398 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:01.744139 18398 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:01.744722 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:01.745534 13723 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:01.746944 13719 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:01.734602 13722 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:01.750913 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 117020us
W20260709 12:23:01.751580 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46590us
I20260709 12:23:01.751727 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:01.753065 18212 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:01.753486 18212 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:01.753852 18212 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:01.754442 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.760514 14422 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:01.679939 18149 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:23:01.761660 18149 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:01.761808 18337 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:01.784459 13792 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:23:01.778640 17735 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:23:01.788158 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:01.762095 18149 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:01.793457 13721 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:01.794900 18312 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:01.794741 18212 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.795522 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.796281 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:01.734097 18418 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:01.796839 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:01.704779 18334 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:01.804466 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:01.805712 13645 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:23:01.811658 13727 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:01.831568 13725 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:01.824016 18245 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:01.847673 18359 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:01.844774 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:01.848976 13718 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:01.849135 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:01.826936 18312 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:01.860074 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:01.864250 18212 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:01.867446 13721 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:01.869061 18372 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:23:01.873920 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 268913us
I20260709 12:23:01.877337 13724 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 12:23:01.883373 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 100752us
I20260709 12:23:01.845850 18209 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:01.848394 18359 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.847609 18334 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:01.943215 13721 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:01.899938 18245 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:01.948184 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54962.3us
I20260709 12:23:02.014951 13645 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:01.906854 18149 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:02.019169 13617 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.719s (SENT)
I20260709 12:23:02.034372 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.017998 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:02.052904 13647 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:01.988109 18340 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:02.068672 13643 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:23:02.097033 18335 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.140421 18372 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:02.141801 18149 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:02.143622 18390 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:23:02.142323 18372 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:02.098433 18340 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.167840 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:02.169082 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:02.170450 18237 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.175738 18237 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.179062 13796 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:02.179493 13796 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:02.178122 18237 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:02.180613 13796 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.181731 13796 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:02.181838 18359 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:02.115641 18312 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.192066 18312 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:02.196544 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 197083us
W20260709 12:23:02.197885 13617 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.734s (SENT)
I20260709 12:23:02.198724 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:02.203477 13720 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:23:02.204388 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:02.205453 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.747s (SENT)
I20260709 12:23:02.205634 13727 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:02.163424 18245 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:02.215920 18245 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.205978 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.231398 13647 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:23:02.234648 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.239603 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:02.244644 13726 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:23:02.245335 13726 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:02.235675 18335 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:02.257980 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:02.259740 13723 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:02.270274 18450 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:23:02.249903 18325 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:02.294894 13725 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:02.294679 18418 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:02.302932 18418 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.305508 18418 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:02.306154 18352 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:02.344255 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:02.345257 13721 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:02.342742 13726 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.349452 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:02.330482 18359 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.354115 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:02.362022 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 156619us
I20260709 12:23:02.364477 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:02.354176 18241 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.004s
I20260709 12:23:02.352092 18245 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:02.362085 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:02.370146 14421 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:23:02.371317 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:02.374751 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.791s (SENT)
I20260709 12:23:02.372798 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:02.382534 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:02.394763 18325 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:02.369511 18241 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:02.406217 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.804s (SENT)
W20260709 12:23:02.408317 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.866s (SENT)
W20260709 12:23:02.403092 18349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.117s	sys 0.190s
W20260709 12:23:02.414708 18349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.117s	sys 0.190s
I20260709 12:23:02.415570 14416 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:02.420037 18241 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:02.420578 13800 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:02.423133 18237 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.423727 18237 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.426137 18237 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:02.422981 18325 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:02.428733 18325 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.370457 18359 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:02.434425 18337 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.672s	user 0.014s	sys 0.002s
I20260709 12:23:02.464684 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:02.465787 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:02.467352 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:02.471210 13727 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11. Re-sending same reply.
W20260709 12:23:02.467664 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0057367660001546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24140000000000003s for a response
I20260709 12:23:02.479094 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:02.481261 18325 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:02.487344 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 185934us
I20260709 12:23:02.490150 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:23:02.597970 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 101049us
W20260709 12:23:02.598186 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.736s (SENT)
I20260709 12:23:02.598934 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.600631 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.606554 18349 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:02.582999 18209 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:23:02.626269 18281 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:02.626412 18359 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:02.658612 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79456.2us
W20260709 12:23:02.659499 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.752s (SENT)
I20260709 12:23:02.659886 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:02.661091 13617 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.817s (SENT)
I20260709 12:23:02.661484 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.662437 18325 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.662390 18337 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:02.669378 13723 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:02.668118 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:02.672147 14421 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
W20260709 12:23:02.673174 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.823s (SENT)
I20260709 12:23:02.674510 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:02.674708 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:02.676355 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.925s (SENT)
W20260709 12:23:02.677459 13617 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.970s (SENT)
I20260709 12:23:02.677865 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:02.687129 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84698.7us
I20260709 12:23:02.684727 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 895) took 1361 ms. Trace:
I20260709 12:23:02.701686 14416 rpcz_store.cc:276] 0709 12:23:01.322063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:01.330441 (+  8378us) service_pool.cc:224] Handling call
0709 12:23:01.330904 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:02.416061 (+1085157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:02.682952 (+266891us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:02.683190 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493856000}
W20260709 12:23:02.683276 18375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.021s	sys 0.014s
I20260709 12:23:02.691241 18418 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:02.716317 18375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.024s	sys 0.014s
I20260709 12:23:02.683573 18337 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.673497 18398 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:02.720450 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.997s (SENT)
I20260709 12:23:02.728454 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.703069 18209 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:02.731995 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:02.710767 18340 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:02.735679 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:02.686903 18281 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:02.740626 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.738596 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:02.745316 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:02.734207 18416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.013s	sys 0.012s
W20260709 12:23:02.751132 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71182.1us
I20260709 12:23:02.743480 18335 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.751901 18415 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.762442 18406 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:02.748695 18462 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 12:23:02.759487 18372 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.008s
W20260709 12:23:02.754310 18416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.013s	sys 0.013s
I20260709 12:23:02.753948 18463 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:02.748031 18209 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:02.787556 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67182.1us
I20260709 12:23:02.791412 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.798652 18209 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:02.774231 18398 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:02.804669 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.766530 18335 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.778630 18463 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.812657 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.814579 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.815769 18334 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:02.818399 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.818718 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.819139 13645 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:02.820714 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.821331 13645 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:23:02.822767 18335 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:02.828552 13726 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:23:02.774911 18372 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.832428 18372 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:02.834776 13617 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.020s (SENT)
W20260709 12:23:02.836238 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 731182us
W20260709 12:23:02.837579 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 500586us
W20260709 12:23:02.837106 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91845.2us
I20260709 12:23:02.839933 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:02.842628 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.817556 18463 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.844131 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:02.846351 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:02.847075 13799 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:02.850159 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:02.850977 13649 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:02.851357 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.852794 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.853509 13647 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:02.857995 13726 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:23:02.858451 13726 pending_rounds.cc:85] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 3
I20260709 12:23:02.858680 13726 pending_rounds.cc:107] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:23:02.858916 13726 raft_consensus.cc:2889] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:02.861115 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:02.861824 13645 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:02.863703 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:02.865921 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:02.866580 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:02.867935 18209 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:02.868618 18335 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:02.874140 18385 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:02.879961 17920 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.880594 17920 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.890569 18470 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.891605 18470 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:02.895709 18406 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:02.896091 18335 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:02.896358 18406 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:02.898396 18406 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.899900 18406 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:02.900290 18418 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.900879 18418 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:02.903348 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:02.904124 18461 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.225s	user 0.002s	sys 0.002s
I20260709 12:23:02.905296 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:02.907083 17423 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:02.908228 17423 raft_consensus.cc:2764] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:02.890467 13799 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:02.920928 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:02.894032 18470 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:02.928439 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83017.1us
I20260709 12:23:02.930137 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:02.913108 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.031s (SENT)
I20260709 12:23:02.932555 14417 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:02.931599 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:02.935974 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87076.9us
I20260709 12:23:02.936375 13651 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:23:02.940820 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:02.941864 13792 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:02.944056 14420 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:02.947460 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:02.948895 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:02.950369 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.038s (SENT)
I20260709 12:23:02.950876 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:02.951884 18245 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:02.952389 18245 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:02.952775 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.045s (SENT)
I20260709 12:23:02.954597 18340 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:02.956066 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:02.958591 18349 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:02.959053 18349 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:02.958957 18312 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:02.959520 18349 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:02.959784 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:02.960997 18385 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:02.961882 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:02.962868 17735 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:02.965314 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:02.965936 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:02.966480 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:02.938163 13645 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:02.904373 18418 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:02.958037 18340 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:02.990710 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:02.961043 18355 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.983187 17735 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:03.003451 18472 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:03.004619 17735 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.009310 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:02.952787 18245 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:02.997334 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:03.019043 13617 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.093s (SENT)
I20260709 12:23:02.994791 18359 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:03.020998 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:02.993376 18335 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:23:02.995080 18237 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:03.018079 18241 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:03.079577 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82321.7us
W20260709 12:23:03.053053 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 94825.9us
I20260709 12:23:03.054764 18209 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:02.999869 18470 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:23:03.082310 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:03.054847 18359 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.068177 18325 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:03.067930 18237 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:03.101715 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49066.6us
I20260709 12:23:03.105501 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:03.100006 18359 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:03.129777 13550 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.104874 18325 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:03.134331 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144377us
I20260709 12:23:03.135474 18473 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:03.135936 18484 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:23:03.141180 18209 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.087998 18398 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:03.065593 18355 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.074481 18463 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.109685 18237 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:03.132546 18476 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.073s	user 0.003s	sys 0.000s
I20260709 12:23:03.117578 17920 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:03.091006 18485 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:03.169253 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.171568 13796 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:03.165836 18325 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:03.155215 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49813.3us
I20260709 12:23:03.206753 18485 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.169840 18484 consensus_queue.cc:1243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:03.210816 18483 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:03.211481 17947 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:03.165122 18473 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:03.217681 13550 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.172046 18398 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:03.234951 18398 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.236400 18398 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.177639 18418 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:03.238432 18418 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:03.238699 18486 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.239044 18365 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:03.239588 18418 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.242023 18237 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.243283 13718 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:23:03.245105 18355 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:03.220008 18485 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.239548 18486 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.250531 17947 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:03.239563 18365 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:03.251672 17947 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.252080 18486 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.252890 18486 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.253594 18486 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.255398 18473 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.232501 18483 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.256834 18473 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:03.235440 18463 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:03.258790 18483 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.258944 18237 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:03.255921 18486 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:03.313180 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.134s (SENT)
W20260709 12:23:03.313205 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79677.4us
I20260709 12:23:03.322521 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:03.324119 18334 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:03.324707 18334 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.325551 18334 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.344568 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:03.339032 13550 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 9 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:03.373649 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.374221 13550 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:03.340771 18472 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.349393 18465 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.395239 17423 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:03.395650 13550 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.402134 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:03.410200 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 177077us
I20260709 12:23:03.395871 17423 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.414466 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.383641 18470 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:03.418437 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:03.412088 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:03.424485 18495 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 12:23:03.439107 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.422780 13648 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:03.445303 14416 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:03.383157 18149 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:23:03.450376 13550 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.454666 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.464905 13550 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 11 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.392704 18472 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:03.467988 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 385793us
I20260709 12:23:03.432566 18394 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:03.469064 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:03.478653 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:03.481868 13617 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.158s (SENT)
I20260709 12:23:03.449654 18416 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.400753 18465 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.524714 18465 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:03.526959 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.529448 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.535663 18362 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.536312 18362 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:03.541944 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123588us
W20260709 12:23:03.546635 18487 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.323s	user 0.001s	sys 0.003s
I20260709 12:23:03.547326 18468 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.547969 18468 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.549278 18472 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.503494 18463 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.550551 18463 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.553172 18463 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.557425 18241 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:03.501538 18237 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:03.559032 18237 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:03.560182 18334 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.561780 13652 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:03.473138 18486 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.564651 18486 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.566992 18486 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:03.572652 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90814us
W20260709 12:23:03.573997 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.183s (SENT)
W20260709 12:23:03.574985 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.575866 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:03.503017 18416 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.579771 18416 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.579959 18337 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:03.462222 18462 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:23:03.585151 18462 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:03.585565 18462 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:03.499197 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:03.587317 18488 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:23:03.512041 17920 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:03.588048 17920 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:03.589004 17920 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.498631 18496 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:03.590553 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.590785 18496 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.592147 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:03.592826 13726 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:03.593083 18496 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.489880 13550 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.516665 18418 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:03.502103 18349 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.005s
I20260709 12:23:03.604519 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.515877 18245 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.600242 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:03.604302 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.613313 13799 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:03.614768 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.620898 13550 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.621800 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.622673 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.623203 13793 raft_consensus.cc:1217] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Deduplicated request from leader. Original: 10.11->[12.12-12.12]   Dedup: 12.12->[]
I20260709 12:23:03.623864 13793 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 12:23:03.624537 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.625212 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.626533 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.610872 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:03.622558 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.626678 18468 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:03.631839 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.632519 13792 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:03.627669 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.625967 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.642107 17423 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:03.622607 13800 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:03.648041 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:03.647085 13550 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 11, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 11 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.627748 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.654933 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:03.658725 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:03.660614 13550 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.628945 18463 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.628726 18483 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:03.594274 18418 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.681169 18463 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.684177 18463 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.684473 18416 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:03.685349 18502 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.685917 18490 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.686137 18502 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.686609 18490 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.688385 18486 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.688834 18502 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.688944 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:03.689175 18463 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:03.689556 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.689654 13648 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.696744 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.697738 13794 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:23:03.697937 18245 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.689725 18463 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.629843 18491 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:23:03.674659 18355 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:03.660087 18335 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.691126 18483 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.560205 18237 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.710400 18418 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.629565 18349 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.713697 18325 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.704742 13550 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 8, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 8 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:03.545917 18365 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:03.701218 18359 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.000s
I20260709 12:23:03.688943 18490 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.728269 18485 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.629145 18470 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:23:03.757069 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.720016 18355 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.726226 18483 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:03.769702 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173559us
I20260709 12:23:03.733781 18334 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.688989 18486 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:23:03.772388 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.771620 13550 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 9, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.783327 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:03.785538 13801 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:03.757685 18491 consensus_queue.cc:1243] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:03.786545 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:03.765570 18355 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:03.818293 18083 consensus_queue.cc:1175] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 10 status { last_received { term: 10 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 10 index: 9 } }
I20260709 12:23:03.818831 18465 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:03.793241 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.801403 13550 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 8 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:03.760823 18490 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:03.773110 18470 consensus_queue.cc:1243] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:23:03.753387 18359 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.834654 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.835120 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.835693 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:03.836522 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:03.793195 18398 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.002s
I20260709 12:23:03.679793 18462 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.851030 13550 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:03.851981 13718 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:03.856205 13550 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 10, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 10 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.863729 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.751724 18335 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.830776 18362 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:03.877228 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:03.878369 14416 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:03.878854 13800 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:03.879243 13800 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:03.886291 13647 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:03.889246 18486 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:03.890666 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 239282us
I20260709 12:23:03.892674 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:03.898993 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:03.840492 18490 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:03.905704 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:03.911545 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:03.871734 18465 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:23:03.896260 18462 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:03.924525 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:03.900251 13800 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:03.940780 13800 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:03.945303 18375 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:03.945848 18375 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.946204 18375 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:03.910300 18334 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:03.949304 18504 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:23:03.950852 18365 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:03.939698 18513 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:23:03.955196 18485 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:03.957933 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:03.962698 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:03.974484 18473 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.008s
W20260709 12:23:03.992547 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.002923 13650 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:04.004485 18325 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:04.002501 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.006707 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112514us
I20260709 12:23:04.007799 18490 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.009497 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.011216 18349 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.012247 18335 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.012914 18463 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.014235 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.014562 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.018004 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:04.019276 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.020371 18463 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.020934 18463 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.021271 18355 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.024531 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:03.901461 13550 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:04.026851 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.029124 13723 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:04.031173 13643 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:04.032024 18359 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.035794 18463 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.038900 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.040661 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.040824 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.041471 18335 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.042330 13726 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:04.087604 13723 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:23:04.080143 13550 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:03.908479 13801 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:04.049201 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.053067 18481 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:04.056059 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:04.061023 13724 pending_rounds.cc:85] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 11
I20260709 12:23:04.090018 13724 pending_rounds.cc:107] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:23:04.078675 14416 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:23:04.086549 13725 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:04.080139 17423 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:04.095584 13800 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:23:04.098170 13800 pending_rounds.cc:85] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 10
I20260709 12:23:04.090471 13724 raft_consensus.cc:2889] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:04.099380 13718 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:04.100637 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.101332 13800 pending_rounds.cc:107] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:23:04.101379 13723 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:04.103132 13720 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:04.104159 13727 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:04.101950 13800 raft_consensus.cc:2889] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:04.111665 13794 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:04.114666 18312 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.117136 18398 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:04.117813 18398 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.117720 18312 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:04.119108 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78424us
I20260709 12:23:04.121004 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.120913 18509 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:23:04.122562 18492 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:23:04.123018 18083 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:04.123750 18083 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.102877 18325 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:04.124322 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:04.124884 18325 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.127267 18398 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.127534 13550 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 8, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:04.129177 13550 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:04.129981 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:04.131268 18522 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:23:04.133387 18480 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.134059 18480 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.136583 18480 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:04.139014 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.142450 18241 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.143311 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.124322 18083 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:04.099150 18470 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.149953 13647 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:04.155213 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 464610us
I20260709 12:23:04.156646 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:04.158556 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.160909 13725 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:23:04.161160 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.162248 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.161863 13644 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:04.166136 13550 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 11, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 11 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:04.168359 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.169673 13644 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:04.171557 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.172308 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.177094 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.178522 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:04.179014 13550 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 8 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.181380 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.168355 18241 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:23:04.184055 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:04.184973 13795 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:04.186060 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.187019 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.184247 18241 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:04.187804 18241 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:04.189006 18241 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.190603 18473 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.193383 18312 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.163458 18359 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:04.149570 18349 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:23:04.197466 13647 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 14 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:23:04.203178 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.209169 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:04.210407 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.210716 13644 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:04.211547 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.212275 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.213006 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.222044 13651 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:23:04.213730 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:04.214102 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62765.6us
I20260709 12:23:04.185679 18375 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.168480 18513 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:04.215634 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.224321 13550 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:04.226576 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:04.227963 13796 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:04.228101 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.230365 13801 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:04.231508 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:04.231570 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:04.235627 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:04.245064 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.252997 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.253796 13727 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:23:04.263897 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.256954 18337 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:04.265023 13794 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:04.266387 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.255985 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.255251 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:04.312474 13652 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:04.324805 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69676.2us
I20260709 12:23:04.267166 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:04.326725 14421 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:04.306669 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.313722 18490 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.314409 18337 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:04.314791 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.315634 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.314410 18237 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.298579 18463 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.327548 13801 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:04.299304 18241 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:04.328475 18513 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:04.366000 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111319us
I20260709 12:23:04.354231 18490 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.265460 18375 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.254654 18470 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:04.370913 13652 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:04.370703 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.239223 18359 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:04.372254 13651 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:04.374568 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.216698 18473 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.379037 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:04.254182 18083 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.453651 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:04.392304 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:04.395514 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:04.396262 18492 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:04.428087 13647 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:04.375391 18485 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.390019 18325 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.479643 14416 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:23:04.480656 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85251.5us
I20260709 12:23:04.390180 17947 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:04.452667 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63119.6us
I20260709 12:23:04.389261 18463 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:04.390867 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 75250.9us
I20260709 12:23:04.433470 18355 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:04.395531 18511 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.386865 18237 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:04.461932 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:04.474648 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 104082us
I20260709 12:23:04.484436 13794 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:04.491134 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.492396 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.493664 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.488262 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.521667 18485 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.522819 18485 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:04.523558 18485 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:04.527101 18529 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:23:04.488132 18416 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:04.485611 18418 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:04.488641 18375 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:04.476452 18337 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.617201 18237 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.618036 18490 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:04.619103 18528 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:23:04.570396 18418 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.620699 18418 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:04.620823 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.617275 18359 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.620189 18375 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:04.641808 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 275642us
I20260709 12:23:04.643141 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.536557 18470 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.498041 18511 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.568275 18365 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:04.494921 17947 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.608608 18509 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.679185 18083 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.695215 18473 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:04.695928 18511 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.697264 18375 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.699842 18463 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:04.701969 17947 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.664705 18513 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:04.702929 18513 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.703507 18513 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:04.708364 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65346us
I20260709 12:23:04.712659 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.713987 13648 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:23:04.717056 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 267552us
I20260709 12:23:04.724475 13648 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:04.730659 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 232031us
W20260709 12:23:04.732100 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 3.133s (SENT)
I20260709 12:23:04.733641 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:04.734486 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 247662us
I20260709 12:23:04.662921 18470 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:04.740557 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:04.742918 13647 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:04.737205 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 243563us
I20260709 12:23:04.751098 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.743422 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:04.710628 18509 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.753737 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:04.755937 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.756944 13651 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:04.756853 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:04.735018 18483 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.948s	user 0.011s	sys 0.008s
I20260709 12:23:04.756225 18245 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:04.754648 14419 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:04.766599 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.767683 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.767756 13794 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:04.770660 18359 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.762598 13651 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:04.769840 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.773942 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:04.777614 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45837us
I20260709 12:23:04.780929 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.781541 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.782169 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.744318 18463 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:23:04.764891 18468 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.784607 18468 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.785563 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.786113 18535 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:04.786902 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.765053 18511 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:04.788322 13645 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:04.788631 13652 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:23:04.791395 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.792608 13650 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17.
I20260709 12:23:04.779856 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:04.794597 18349 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:04.795058 18349 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.795428 18349 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:04.759426 18490 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.748595 18491 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:23:04.799844 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.801121 13649 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:23:04.805883 13645 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:04.777163 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:04.819092 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75747.5us
I20260709 12:23:04.820844 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:04.824443 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47461.7us
I20260709 12:23:04.825697 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.824561 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.828958 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:04.829766 13651 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:04.846623 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 358432us
I20260709 12:23:04.850416 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.851387 13652 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:04.777076 18359 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.855389 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:04.748728 18518 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 12:23:04.865661 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:04.885107 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.883119 14418 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:04.887321 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.887401 14421 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:04.877696 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:04.887796 13650 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:23:04.893136 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.893890 13792 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:04.842341 18535 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.894683 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.895196 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.905890 13644 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:04.772900 18483 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:04.844556 18541 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:04.853241 18463 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:04.921613 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100904us
I20260709 12:23:04.922654 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.921475 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:04.883256 18245 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:04.843073 18509 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.926565 13648 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:04.930245 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:04.930413 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.942864 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.951717 18241 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:04.910991 18398 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.962991 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:04.962868 18359 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:04.933892 18538 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.934499 18463 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:04.931103 17735 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:04.941129 18490 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:04.962880 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:04.989028 13797 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:04.969859 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.989950 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:04.998448 13799 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:05.016780 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.972886 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:04.927387 18541 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:05.007568 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:04.939337 18462 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.112531 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:05.009318 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.113482 18398 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:23:04.965368 18337 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.155s
I20260709 12:23:05.120711 18365 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:05.121279 18365 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.965279 18237 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.157s
I20260709 12:23:05.119297 13652 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13. Re-sending same reply.
I20260709 12:23:04.964453 18539 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:05.119748 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 112344us
I20260709 12:23:05.121712 18365 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.117566 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:05.117231 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86772.3us
I20260709 12:23:05.127300 13795 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:23:05.130179 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 206038us
I20260709 12:23:04.966425 18481 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:05.137238 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.138139 13727 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:05.130087 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:05.027325 18534 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:23:04.966642 18492 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:04.976965 17735 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:04.966243 18468 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:04.965569 18359 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:04.999851 18486 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:05.178335 18486 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:05.178915 18486 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:05.150905 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:04.970041 18542 raft_consensus.cc:2764] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:05.155586 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.184064 18552 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:05.184964 18552 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:05.185111 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.186486 18209 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:05.187002 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.189087 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:05.142046 13720 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:23:04.988363 18490 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:05.129405 18462 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.191009 18490 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:05.160902 17735 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:05.192713 18462 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:05.192402 18490 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.193419 18462 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:04.976504 18463 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.134475 18398 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:04.966452 18470 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:05.195667 18483 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.113237 18541 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.199925 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:04.973260 18538 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.169934 18237 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.175499 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:05.114455 18245 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:23:04.980408 18418 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.144541 18492 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:05.221613 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:05.222616 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 294411us
I20260709 12:23:05.201128 18359 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:05.229760 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54371us
I20260709 12:23:05.233728 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.222389 13643 raft_consensus.cc:2376] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12. Re-sending same reply.
I20260709 12:23:05.238685 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.239359 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:05.240098 13645 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:04.987496 18349 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.244604 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:05.244877 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:05.257169 13793 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.259147 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.260102 13725 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:23:05.259872 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 329663us
I20260709 12:23:05.265132 13643 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:05.269702 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:05.270493 13643 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:23:05.244577 18513 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.000s
I20260709 12:23:05.238684 18491 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:05.274328 13722 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:05.286551 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.219602 18237 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.225071 18539 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:23:05.305814 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.277760 18491 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.316089 13727 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:05.313948 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.311810 18418 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:05.295033 18561 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:05.280649 18513 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.313326 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.325321 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.325390 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.295459 18483 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:05.330150 13652 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:05.326166 13722 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:05.328475 13718 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:05.333212 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.338325 13726 pending_rounds.cc:85] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 3
I20260709 12:23:05.339941 13726 pending_rounds.cc:107] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:23:05.340202 13718 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 7
I20260709 12:23:05.340293 13726 raft_consensus.cc:2889] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:05.340905 13718 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:23:05.341370 13718 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:05.320080 18491 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:05.344049 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.355830 18237 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.309556 17947 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:23:05.359301 18349 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.326766 18553 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:05.363654 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137640us
W20260709 12:23:05.373057 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134483us
I20260709 12:23:05.367897 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.334340 18561 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:05.378641 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:05.380748 18365 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.384462 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:05.387099 13650 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:05.386899 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:05.387753 13645 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:05.388283 13722 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:05.418522 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:05.419986 14423 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:05.424778 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.426066 13722 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.435997 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.390866 18553 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:05.442749 18553 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:05.446986 18560 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:23:05.451354 18365 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:05.400660 18538 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.460436 18359 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.462204 18513 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.462858 18209 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.464565 18561 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.470297 13793 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:05.474798 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:05.482712 18473 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:05.442178 18555 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:23:05.440377 18237 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:23:05.504271 18511 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.488772 18473 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:05.469938 18541 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.513410 18511 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.524709 18541 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:05.525074 18511 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.525471 13794 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:05.500062 18312 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:05.519392 18513 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:05.487111 18209 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.484860 17735 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.541376 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:05.554033 13795 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:05.555367 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:05.570962 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:05.572703 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:05.576946 13724 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:05.576586 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.577943 13727 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:05.589119 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:05.590294 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 315103us
I20260709 12:23:05.595292 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.528240 18312 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:05.612859 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:05.613906 13645 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:05.621402 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:05.622927 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:05.624455 14423 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:23:05.624923 14423 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 9
I20260709 12:23:05.625213 14423 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:23:05.625581 14423 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:05.600903 13792 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:05.618470 18149 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:05.617539 18567 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:05.619036 18568 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.589421 18541 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:23:05.630707 18511 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:05.593349 18513 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:05.670073 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50282.3us
I20260709 12:23:05.633044 18083 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:05.601040 18509 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:23:05.659494 18149 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.661432 18567 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.638516 18480 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:23:05.694593 13645 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:05.672165 18511 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:05.686075 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86944.1us
I20260709 12:23:05.701376 18511 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:05.702391 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:05.704948 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.705637 13724 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:05.717283 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:05.700506 13727 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:05.662475 18486 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.720629 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:05.693841 18149 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:05.680011 18540 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.732975 14416 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:05.658823 18473 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.673501 18542 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:05.736922 18486 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.693475 18567 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.820986 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:05.840461 18483 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:05.841125 18483 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:05.680539 18312 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:05.855791 18513 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:05.866416 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123725us
I20260709 12:23:05.739038 18535 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:05.872406 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:05.873140 13793 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:05.873445 13799 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:23:05.889384 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 294615us
I20260709 12:23:05.891876 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:05.898641 18535 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.778684 18540 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.909418 18535 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:05.913962 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:05.915073 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:05.890450 18209 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:05.862841 18237 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:23:05.892707 17735 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:05.897670 18312 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:05.923499 13722 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:05.922431 13793 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:05.937681 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.952560 13727 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:05.954373 13727 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:05.959728 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:05.961967 18576 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:23:05.962801 18579 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:23:05.967334 18492 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:05.971309 13720 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:05.980245 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.988569 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.989926 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:05.997373 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:06.000070 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:06.004827 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:05.971410 18570 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:06.010651 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.010710 13722 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:06.007786 13727 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:23:06.014588 18237 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:06.019150 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:06.041145 13793 raft_consensus.cc:606] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:06.041666 13793 raft_consensus.cc:614] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:05.986505 18418 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:23:06.021741 18581 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:23:06.046300 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.006734 18539 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:06.035383 18570 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:06.058233 18589 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:23:06.075817 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:06.055984 18535 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:06.057123 18237 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:06.061302 18492 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:06.073523 18570 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.093154 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.093801 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.094599 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.095479 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.112351 13795 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:06.096971 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:06.105566 18491 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.736s	user 0.014s	sys 0.000s
I20260709 12:23:06.106689 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.107538 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.117375 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.118373 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.136662 18604 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.139672 18604 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.142926 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:06.145093 18604 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:06.172722 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:06.175215 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99572.3us
I20260709 12:23:06.176591 13799 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:06.178339 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.178488 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.179656 13793 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:06.179991 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:06.129428 18418 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:06.184314 13799 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:06.191385 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.192131 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:06.192829 13796 raft_consensus.cc:606] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.193183 13796 raft_consensus.cc:614] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:06.170203 18540 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:06.195183 18603 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.175791 18483 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.208283 18603 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.208859 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:06.209514 13796 raft_consensus.cc:606] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.209854 13796 raft_consensus.cc:614] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:06.210703 18603 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:06.212385 18480 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:06.213559 18312 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:23:06.214617 18573 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.364s	user 0.003s	sys 0.000s
I20260709 12:23:06.215309 18556 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:06.215734 18556 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:06.216632 18556 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.217406 18567 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.217875 18556 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:06.195133 18518 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:06.195309 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.222231 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.224041 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.200497 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:06.202562 18237 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.225323 18237 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:06.225736 18568 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.002s
I20260709 12:23:06.226078 18237 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.188890 18604 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.228116 18604 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.186538 18418 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.231192 18418 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.186728 18511 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.234238 18511 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:06.234598 18603 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.235623 13801 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:06.236431 18604 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:06.226799 13725 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:06.238852 18149 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:23:06.240114 18575 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.356s	user 0.002s	sys 0.001s
I20260709 12:23:06.241515 18149 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:06.134847 18491 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.248134 18491 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:06.250137 18567 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:06.250626 13797 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:06.226217 18568 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.213961 18312 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:06.267719 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:06.269661 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:06.272190 13726 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.272429 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.280952 14420 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:06.285113 14416 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:23:06.285988 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:06.289738 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:06.279465 18492 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.263450 18570 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:06.298856 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.303700 18570 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:06.282976 18511 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.305562 18511 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.310417 18511 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:06.338155 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:06.310941 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 608655us
I20260709 12:23:06.313184 18544 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.316313 18312 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.317356 18483 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:06.317837 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:06.318612 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:06.343974 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:06.344221 13645 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.346460 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:06.347164 13719 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:06.319451 18542 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.320457 13651 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:06.348742 13652 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:06.323776 18365 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:06.325093 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:06.325337 18375 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:06.326879 18539 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:06.328191 18359 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:06.330935 13723 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:06.362679 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:06.333074 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:06.363395 13649 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:06.363584 13801 raft_consensus.cc:606] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.364933 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:06.333163 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.292827 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:06.366711 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73958.1us
I20260709 12:23:06.332482 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:06.338295 13726 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:06.369180 13650 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:06.315862 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2798) took 1039 ms. Trace:
W20260709 12:23:06.323148 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 393956us
I20260709 12:23:06.344094 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:06.359650 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 135503us
I20260709 12:23:06.371773 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.373533 13726 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:06.357667 18375 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:06.375087 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:06.375914 13650 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.379668 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.379563 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.383211 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.387099 18572 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.389680 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.391506 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:06.392386 13726 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.394084 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.397869 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.409901 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:06.412250 13645 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:06.381453 18483 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:06.388671 18604 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:06.370222 13643 rpcz_store.cc:276] 0709 12:23:05.276064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.276482 (+   418us) service_pool.cc:224] Handling call
0709 12:23:05.277419 (+   937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.277700 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:23:05.568772 (+291072us) log.cc:844] Serialized 14 byte log entry
0709 12:23:06.315470 (+746698us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:06.315530 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:06.315575 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.315657 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:06.315789 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516219,"prepare.queue_time_us":185829,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":514923,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":520493,"threads_started":2,"wal-append.queue_time_us":399419}
I20260709 12:23:06.379866 18484 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:06.426745 13652 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:23:06.399250 18513 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:06.443116 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:06.459674 13649 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:23:06.405835 18553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.014s	sys 0.024s
I20260709 12:23:06.358640 18542 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.398984 18359 raft_consensus.cc:2764] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:23:06.456776 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:06.483528 13645 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:06.484903 18542 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:06.463025 18553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.015s	sys 0.024s
I20260709 12:23:06.411409 18539 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.486130 18375 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.450240 18334 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:23:06.466137 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:06.466439 18609 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:23:06.466741 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:06.467049 18602 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:06.459123 18484 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:06.498466 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110061us
I20260709 12:23:06.491636 13800 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:06.500648 13800 raft_consensus.cc:614] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:06.447621 18245 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:06.492937 18602 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:06.458504 18483 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.463968 18544 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:06.510927 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.418081 18418 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.518318 13645 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:06.502573 18484 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:06.514154 18557 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:06.521395 18602 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.511793 18483 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.491024 18513 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.560392 13719 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:06.560576 14420 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:06.540241 18418 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:06.525663 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:06.570549 13650 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:06.571466 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:06.576463 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.578462 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:06.580888 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54765.5us
I20260709 12:23:06.572650 18083 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:06.588593 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:06.585239 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 201915us
I20260709 12:23:06.569487 18418 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
W20260709 12:23:06.591228 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 214265us
I20260709 12:23:06.600927 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.599743 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:06.581171 18480 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:06.609733 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:06.611076 13800 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:06.616370 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:06.619968 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.624251 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.624513 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:06.625165 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 245717us
I20260709 12:23:06.626349 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.584172 17947 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:23:06.622586 18480 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.665077 18480 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:06.614836 18334 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:23:06.630982 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:06.667364 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:06.670912 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51047.7us
I20260709 12:23:06.652328 18518 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:06.609613 18463 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.610334 18557 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:23:06.651299 18544 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:06.671604 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:06.673565 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 638) took 1285 ms (client timeout 1568 ms). Trace:
I20260709 12:23:06.687328 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.674793 18468 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:23:06.687422 13797 rpcz_store.cc:269] 0709 12:23:05.383849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:05.384058 (+   209us) service_pool.cc:224] Handling call
0709 12:23:06.669331 (+1285273us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:06.669471 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973801216}
I20260709 12:23:06.677801 18470 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:06.689487 18604 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:06.658049 18245 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:06.731048 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 2928) took 1745 ms. Trace:
I20260709 12:23:06.686167 18539 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:06.713774 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.731537 13792 rpcz_store.cc:276] 0709 12:23:04.985615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:04.985789 (+   174us) service_pool.cc:224] Handling call
0709 12:23:04.986113 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:05.636865 (+650752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:06.730741 (+1093876us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:06.730972 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413286784}
I20260709 12:23:06.693485 18359 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:23:06.740839 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 114654us
I20260709 12:23:06.691871 18468 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:06.707270 18535 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:06.669520 18609 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 12:23:06.746959 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.750038 13727 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:23:06.750213 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 126047us
I20260709 12:23:06.750792 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.711849 18518 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:06.732774 18542 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:06.769981 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:06.732237 18616 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:06.738687 18604 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:06.773084 14416 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:06.776120 14418 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:06.745497 18418 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.773692 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:06.757578 18359 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:06.772280 18542 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:06.826627 18542 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.828342 18542 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:06.795996 18604 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:06.801123 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44289.1us
W20260709 12:23:06.801990 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58283.1us
I20260709 12:23:06.830737 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.831426 18365 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:06.832365 18365 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:06.832708 18365 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:06.833511 18365 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.834710 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.835969 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.836639 13792 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:06.839115 13793 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:23:06.839287 18538 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:06.839651 18538 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:06.840539 18538 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.841646 18538 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:06.842291 18312 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:06.848050 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.848963 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.802198 18470 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:06.805060 18484 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.807098 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:06.806710 18518 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:23:06.817418 18483 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:06.818843 18616 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:06.882422 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:06.819675 18359 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.820618 18418 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:06.782145 18468 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.825448 18513 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:06.859560 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 145934us
I20260709 12:23:06.861099 13797 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.852156 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:06.862211 14419 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:23:06.863873 18365 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.885105 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:06.860472 18083 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:06.922870 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70791.6us
I20260709 12:23:06.922832 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:06.928544 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.918591 18237 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:06.880688 18570 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:23:06.886656 18609 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:06.898595 18553 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:06.935001 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50042.5us
I20260709 12:23:06.948311 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:06.950289 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:06.950608 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.935606 18334 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:06.939507 14420 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:06.895318 13647 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:06.861923 18620 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:06.942144 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.862601 18535 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:06.946579 13797 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
W20260709 12:23:06.947402 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.568s (SENT) [suppressed 19 similar messages]
I20260709 12:23:06.922086 18418 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.886521 18603 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:06.969691 13797 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:06.941462 18624 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.962620 18535 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.957321 18544 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:06.957892 18620 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.948578 18468 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:06.980489 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:06.954051 18606 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:06.949472 18513 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:23:06.946771 18237 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:07.010660 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.972591 18613 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:06.986217 18535 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:06.981346 18624 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:06.957113 18609 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.960145 18553 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.006636 18620 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:07.035861 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:06.977356 18603 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:06.986601 18334 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:23:07.014703 18462 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:07.023900 18509 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:07.068959 18602 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:07.099493 18553 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.006047 18616 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.003336 18588 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:23:07.075997 18492 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:07.103813 18492 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:07.027266 18624 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.105086 18492 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.086194 18483 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.088251 18237 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.089304 18484 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:07.090205 13801 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:07.092139 18603 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.094522 18462 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:07.148729 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.118868 18398 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.078315 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.153055 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.146890 18463 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:07.178804 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:07.179045 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 100876us
W20260709 12:23:07.155763 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 175357us
I20260709 12:23:07.180805 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:07.136955 18611 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:23:07.156638 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:07.164196 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 216812us
I20260709 12:23:07.182981 13792 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:07.167706 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.022006 18613 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:07.173995 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.021600 18511 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.181711 13724 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:07.186695 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.193717 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:07.184386 18492 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:07.185814 18588 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:07.182722 18398 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.236176 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:07.225884 18375 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:07.185142 18463 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.245383 18462 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:07.236181 18631 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:23:07.246122 18462 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:07.246935 18462 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.249594 18462 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.250015 18418 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.250299 18462 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:07.251298 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.253748 18462 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.250976 18567 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:07.259660 18567 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:07.260823 18567 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.261284 18462 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.191045 18613 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:07.274140 18462 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:07.275382 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:07.276458 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62943.3us
I20260709 12:23:07.283684 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:07.277664 18641 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:07.285609 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:07.288996 18588 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:07.289463 18588 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.289816 18588 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:07.300159 18588 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.196033 18511 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.303758 18511 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.304028 18609 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.308156 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.313262 18609 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:07.316499 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.318323 13724 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:07.317226 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:07.325771 13646 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:07.325788 18509 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:07.327486 18509 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.332757 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.332881 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.339092 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.340000 13798 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11.
I20260709 12:23:07.334218 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:07.343422 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122023us
I20260709 12:23:07.346328 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.352308 18535 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.359990 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.361431 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.361770 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.361626 13723 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:23:07.364142 13727 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:07.384733 14391 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11.
I20260709 12:23:07.385432 14391 leader_election.cc:403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:07.387256 18483 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:23:07.388149 18620 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.362275 18470 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:07.390990 18470 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.392045 18535 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.393333 18470 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.393923 18470 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:07.394291 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.398545 18334 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:07.396373 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:07.409214 18334 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:07.412021 18542 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.006s
I20260709 12:23:07.396504 18588 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.397305 13651 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:07.397899 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:07.362426 18619 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:07.398952 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.400048 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.400287 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:07.421387 13652 raft_consensus.cc:2376] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17. Re-sending same reply.
I20260709 12:23:07.401773 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:07.401711 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54213.9us
I20260709 12:23:07.422796 13644 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:07.423398 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.423708 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.424898 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:07.425503 13651 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:07.415663 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.426724 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.431041 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:07.431825 18083 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:07.432297 18083 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:07.433651 18375 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:07.439299 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.419543 18470 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.441310 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:07.354542 18641 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:23:07.378647 18624 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.453521 18624 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.454097 18619 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.454458 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:07.455862 13727 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:07.456620 18619 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.438625 18620 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.433169 18083 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.459681 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.460536 18083 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:07.462702 18627 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.282s	user 0.000s	sys 0.003s
I20260709 12:23:07.465528 18463 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.471966 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.477459 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.479403 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.497905 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.499014 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.499943 13649 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:07.480285 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.437112 18375 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.481886 18620 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.496265 18592 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:07.501075 18149 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:07.481662 18312 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:07.523056 18398 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:07.529498 18628 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.433s	user 0.002s	sys 0.000s
I20260709 12:23:07.482350 18470 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.542701 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:07.547163 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:07.548834 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:07.497948 18463 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:07.479501 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.551743 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.487653 13651 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
W20260709 12:23:07.555817 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76378.3us
I20260709 12:23:07.557116 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:07.561012 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173884us
I20260709 12:23:07.564653 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.567405 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.569375 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:07.569904 13727 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:07.571955 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 293732us
W20260709 12:23:07.577646 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:07.578429 13793 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:07.579361 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.580101 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.580392 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.574069 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.573330 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.583628 13796 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:07.585068 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.588946 13799 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:07.593298 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.557932 18619 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:07.554941 18463 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:07.561031 18149 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:07.565853 18620 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.583444 18542 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:07.576968 18535 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:07.585740 18616 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:07.630605 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.638059 18398 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.623653 18463 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:07.595605 18491 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.561461 18564 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
W20260709 12:23:07.696193 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66263.3us
I20260709 12:23:07.600095 18567 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.619771 13792 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
W20260709 12:23:07.629292 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 82147.2us
I20260709 12:23:07.664541 18398 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.670433 18491 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.676229 18483 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11.
I20260709 12:23:07.676942 18149 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.704608 13724 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: [10.7-13.9]
I20260709 12:23:07.689369 18334 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.695604 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.708616 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:07.649212 18629 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:07.697405 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:07.699378 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.649528 18641 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:07.702164 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108958us
I20260709 12:23:07.711520 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.638736 18538 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:23:07.717305 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.718516 18561 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:07.715839 18486 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:07.722824 18486 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:07.723217 18486 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:07.704006 18647 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:23:07.716372 18535 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:07.718569 18375 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.746481 18591 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:07.747327 18591 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.748085 18491 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:07.750154 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:07.752017 18655 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:07.754637 18591 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.761199 18398 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.769455 18561 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:07.769773 18647 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:07.770709 18483 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:07.771173 18483 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:23:07.769644 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58220us
I20260709 12:23:07.772331 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.778096 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:07.779867 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.754535 18613 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.782008 13724 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:07.783352 17735 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:07.788800 17735 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:07.789784 13792 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:07.786578 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.783077 18538 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.772128 18483 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.794935 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.798776 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.808997 13651 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:23:07.810102 18542 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:07.789711 17735 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.819741 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.775125 18647 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.818452 14420 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:07.823664 14424 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:07.809024 18083 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:07.852749 13652 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:07.770763 18561 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.861049 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.865846 14422 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:07.867178 18655 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:07.869257 18591 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:07.870852 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.878086 13727 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:07.836051 18617 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:07.871887 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:07.835794 18647 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:07.887785 13792 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 12:23:07.880122 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 107896us
I20260709 12:23:07.836340 18624 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:07.889181 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:07.892930 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.894448 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:07.895692 18542 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:07.815608 18538 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.864326 18483 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:07.825345 18486 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.856280 18609 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:23:07.858110 18619 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:23:07.919952 18557 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:07.920504 18557 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:07.921332 18557 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.925899 18486 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.927546 14424 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:23:07.930554 18657 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:23:07.811203 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:07.892153 18617 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:07.935511 18513 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.960625 18617 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.960801 18513 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:07.963109 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.963449 18513 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.904626 18640 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:07.904541 18553 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:07.904196 18511 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:07.943035 18624 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:07.965880 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:07.949406 18538 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:07.955406 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.968482 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:07.955686 18564 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:07.970683 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:07.971335 13796 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:07.972781 18564 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:07.973035 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:07.973574 18564 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:07.958348 18613 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:07.979583 18557 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:07.944658 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:07.957823 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.959054 18602 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:07.957186 13644 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:07.982560 13651 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:07.982712 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:07.983974 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:07.983974 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:07.986830 18513 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:07.987306 18513 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.987716 18513 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:07.961184 14450 heartbeater.cc:646] Failed to heartbeat to 127.12.45.126:44051 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.45.126:44051 timed out after 15.000s (SENT)
I20260709 12:23:07.956197 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:07.991403 13650 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:07.956349 18463 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:07.996513 18538 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:08.035313 13647 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:08.027699 13721 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:23:08.073601 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:08.076614 13796 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:08.076944 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:08.077811 14424 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:08.080127 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.088349 13793 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:23:08.089098 14419 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:08.077039 18556 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.125324 18613 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:08.150007 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:08.126573 18564 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.130575 14416 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:23:08.154783 14416 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 8
I20260709 12:23:08.155087 14416 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:23:08.155316 14416 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:08.069844 18538 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:08.026997 18625 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:08.076759 18486 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:08.141408 18463 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.070137 18602 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.153846 18624 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:23:08.184612 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93176.5us
I20260709 12:23:08.176815 18592 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:08.184748 18245 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:08.194983 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99166.7us
I20260709 12:23:08.153620 18556 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.097905 18492 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:23:08.156741 18613 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.201710 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:08.144019 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55486.6us
I20260709 12:23:08.172573 18486 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.205928 18556 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:08.206780 18615 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:08.188453 18624 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.207674 18513 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:08.260123 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62804.9us
I20260709 12:23:08.263345 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:08.263836 13792 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:23:08.265918 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:08.269181 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:08.270102 13720 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:08.270382 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:08.271117 13724 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:08.209766 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:08.210536 13647 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:08.216655 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:08.275470 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:08.217247 18602 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.191960 18535 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:08.222615 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:08.201807 18245 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.285190 18245 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.235154 18613 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.239986 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.290323 18513 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.247713 18624 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.252274 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.291007 14421 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:08.291738 14420 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:08.293267 18564 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:08.276556 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66937.3us
I20260709 12:23:08.300303 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:08.276765 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.277940 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.196460 18538 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.280308 13797 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:08.279582 18481 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:08.281618 18486 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.310405 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:08.311554 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:08.314633 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:08.315375 13801 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:08.316800 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:08.317634 18614 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:08.320024 18592 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:08.320724 18359 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:08.321354 18359 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:08.182052 18625 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.301867 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:08.305956 18588 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:08.323220 18588 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:08.323633 18588 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:08.324199 18625 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.324895 18625 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:08.325379 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:08.326498 18245 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:08.326938 18245 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.327265 18245 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:08.325372 18625 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.332093 18418 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:08.332230 18625 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.324204 18602 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:23:08.333693 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:08.333829 18588 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:08.340479 18674 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:08.282492 18535 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.344198 18518 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:08.344736 18518 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:08.345700 18535 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.322628 18603 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:08.346524 18603 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.346937 18603 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:08.348839 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:08.320617 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:08.350622 13797 raft_consensus.cc:606] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:08.351159 13797 raft_consensus.cc:614] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:08.354288 18334 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:08.361091 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:08.362084 13643 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:08.363560 18647 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.345620 18518 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.366148 18518 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:08.366564 14423 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16.
I20260709 12:23:08.370661 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:08.371707 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:08.372395 13801 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:08.375236 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:08.375784 13801 raft_consensus.cc:606] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:08.376081 13801 raft_consensus.cc:614] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:08.377802 13764 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16.
I20260709 12:23:08.364511 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:08.378803 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69198.2us
I20260709 12:23:08.378966 13643 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:08.382074 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:08.386188 13646 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:08.388916 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:08.366019 18647 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.411051 13719 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:08.412364 13721 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:08.433947 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:08.436578 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:08.437283 13800 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:08.439296 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:08.442056 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.320650 18592 consensus_queue.cc:1243] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:08.461735 18602 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:08.466483 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84095.9us
I20260709 12:23:08.467435 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.451489 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:08.472167 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:08.398756 18677 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:08.383307 18359 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.487478 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.489296 18613 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:08.489740 18613 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:08.490115 18613 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:08.490680 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.494318 18591 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:23:08.490767 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:08.501336 18657 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:23:08.432200 18609 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:08.475425 18625 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:08.448302 18418 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:08.494881 18535 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:08.560286 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 110516us
I20260709 12:23:08.561931 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:08.546562 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 107353us
I20260709 12:23:08.563308 13797 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:08.557449 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138326us
I20260709 12:23:08.467371 18588 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.557909 13800 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:23:08.547715 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95289.8us
I20260709 12:23:08.441370 18660 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 12:23:08.574339 13793 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:08.575556 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86726.9us
I20260709 12:23:08.552852 18591 consensus_queue.cc:1243] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:08.580202 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.582165 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:08.559062 18636 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:08.587318 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:08.587952 13649 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:08.589232 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.590374 14416 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16.
I20260709 12:23:08.560806 18535 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:08.566305 18245 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.571858 18355 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:08.597419 13764 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16.
I20260709 12:23:08.597810 13764 leader_election.cc:403] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:23:08.598165 13797 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:08.565692 18609 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.504501 18677 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.600309 18535 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:08.571276 18418 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:08.649288 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65208.5us
I20260709 12:23:08.588838 18561 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:23:08.598663 18625 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:08.592131 18636 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:08.663704 13801 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:08.665504 18603 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.667879 18603 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.674124 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:08.623952 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
W20260709 12:23:08.631074 18671 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:23:08.632535 18673 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16.
I20260709 12:23:08.600135 18355 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.633185 18613 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.600198 18486 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:08.651738 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:08.589476 18480 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:23:08.680079 13727 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:08.682197 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:08.660112 18636 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:08.685710 14416 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:23:08.679580 18470 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.693159 18470 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.688215 14422 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 7
I20260709 12:23:08.694485 14422 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.8
I20260709 12:23:08.694907 14422 raft_consensus.cc:2917] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Aborting config change with OpId 10.8: Aborted: Op aborted by new leader
I20260709 12:23:08.695302 14422 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:08.695428 18470 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.690062 13797 raft_consensus.cc:606] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:08.697633 13797 raft_consensus.cc:614] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:08.709339 13647 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 7
I20260709 12:23:08.709750 13647 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.8
I20260709 12:23:08.710043 13647 raft_consensus.cc:2917] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Aborting config change with OpId 10.8: Aborted: Op aborted by new leader
I20260709 12:23:08.710305 18677 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.710536 18613 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.710609 13647 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:08.720391 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110793us
I20260709 12:23:08.731752 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:08.731956 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.733312 18689 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:08.734508 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138710us
I20260709 12:23:08.680070 18355 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:08.738432 18513 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.739151 18513 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.741405 18513 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:08.744518 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62493.7us
I20260709 12:23:08.748785 18617 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:08.749331 18617 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:08.749770 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:08.733964 18689 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:08.750566 13799 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:08.750228 18617 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.751494 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:08.751207 18689 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.752091 13792 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:08.752944 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:08.753486 13800 raft_consensus.cc:606] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:08.753805 13800 raft_consensus.cc:614] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:08.755781 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99107.1us
I20260709 12:23:08.757022 18418 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.757872 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:08.756407 18470 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:08.759024 18673 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:08.759701 18470 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:08.759918 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:08.760327 18481 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:08.761888 18616 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:08.762318 18616 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.762667 18616 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:08.766075 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:08.766713 13792 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:08.767027 13792 raft_consensus.cc:614] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:08.767083 18470 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.767602 18470 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:08.768045 13550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 253) took 13669 ms (client timeout 15000 ms). Trace:
I20260709 12:23:08.770900 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:08.772998 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:08.772938 13549 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 5 to 8, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 8 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.754060 18245 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.696223 18676 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.774142 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:08.797098 18673 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:08.803519 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:08.777880 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46251.7us
I20260709 12:23:08.806670 18557 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.825488 18557 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.809056 18535 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.824319 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:08.831146 18535 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.831605 14420 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:08.852512 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:08.768437 13550 rpcz_store.cc:269] 0709 12:22:55.099086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:55.099380 (+   294us) service_pool.cc:224] Handling call
0709 12:23:08.767925 (+13668545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:04.376482 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:04.377417 (+   935us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:04.377481 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:23:04.377520 (+    39us) tablet.cc:662] Decoding operations
0709 12:23:04.783168 (+405648us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:04.783301 (+   133us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:04.783328 (+    27us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:23:04.785136 (+  1808us) tablet.cc:701] Row locks acquired
0709 12:23:04.785197 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:23:04.785388 (+   191us) write_op.cc:270] Start()
0709 12:23:04.785552 (+   164us) write_op.cc:276] Timestamp: P: 1783599784785318 usec, L: 0
0709 12:23:04.785592 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:23:04.798540 (+ 12948us) log.cc:844] Serialized 8969 byte log entry
0709 12:23:05.398952 (+600412us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b18002cfb60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:05.872950 (+473998us) op_driver.cc:464] REPLICATION: finished
0709 12:23:06.095981 (+223031us) write_op.cc:301] APPLY: starting
0709 12:23:06.106151 (+ 10170us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:06.467053 (+360902us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:06.467105 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:06.681472 (+214367us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:06.682478 (+  1006us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 12:23:06.731501 (+ 49023us) write_op.cc:312] APPLY: finished
0709 12:23:06.806798 (+ 75297us) log.cc:844] Serialized 204 byte log entry
0709 12:23:06.965506 (+158708us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:06.986832 (+ 21326us) write_op.cc:454] Released schema lock
0709 12:23:07.019893 (+ 33061us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:07.079535 (+ 59642us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b5400328128. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5399fb09e1 00007f5399faf64c 00007f53976403fd 00007f539762c389 00007f539762bb53 00007f539762fd61 00007f539762fd19 00007f539762fca9 00007f539762fc71 00007f539762ee9d 0000000000388634
0709 12:23:07.086400 (+  6865us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:07.086436 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3916,"reported_tablets":50,"spinlock_wait_cycles":263552,"child_traces":[["op",{"apply.queue_time_us":187371,"apply.run_cpu_time_us":62471,"apply.run_wall_time_us":1007103,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100451,"num_ops":23,"prepare.queue_time_us":60425,"prepare.run_cpu_time_us":21697,"prepare.run_wall_time_us":556924,"raft.queue_time_us":404384,"raft.run_cpu_time_us":14734,"raft.run_wall_time_us":119161,"replication_time_us":1087081,"spinlock_wait_cycles":53028480,"thread_start_us":648654,"threads_started":5,"wal-append.queue_time_us":384006,"wal-append.run_cpu_time_us":43275,"wal-append.run_wall_time_us":2417619}]]}
I20260709 12:23:08.878922 18609 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.885061 18470 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.886785 18538 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.801637 18676 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.807104 18636 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.887324 18538 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:08.896796 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.931830 18538 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.933231 18636 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:08.934773 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:08.935257 18676 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:08.935927 14419 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:08.936164 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:08.937253 18470 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:08.938860 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:08.939466 13650 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:08.941303 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:08.941857 13800 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:08.942566 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:08.944252 18673 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:08.944614 18673 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.944941 18673 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:08.922032 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:08.948347 14420 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:08.978215 18355 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:08.984416 18557 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:09.003439 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:09.013648 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:08.996946 18616 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.014396 13646 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.015946 18616 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:09.019511 18673 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.023402 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:09.024712 13643 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:09.025751 18673 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:09.027513 18355 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:09.028919 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:09.029579 13797 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:09.033205 13646 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:09.034770 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:09.035387 18692 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:09.036234 18557 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:09.036409 18692 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:09.036859 18692 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:09.037236 18557 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:09.038846 18557 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.037796 18692 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:09.040903 18692 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:09.052175 13800 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:09.054314 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:09.056406 13643 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:09.058136 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:09.067652 13723 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.001s	sys 0.005s
W20260709 12:23:09.068095 13723 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.001s	sys 0.005s
I20260709 12:23:09.079232 13797 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:09.088042 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:09.066505 18624 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:09.093808 18624 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:09.095021 18624 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.070595 18480 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:09.098666 18509 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:09.098946 18480 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:09.099308 18509 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:09.099974 18480 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.100220 18509 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.117076 14420 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:09.118963 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:09.124152 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:09.134627 13652 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:09.192396 18702 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 12:23:09.129312 18603 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:09.086778 18677 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:09.203467 18677 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.205714 18677 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:09.219538 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88858.4us
I20260709 12:23:09.233635 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:09.234742 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:09.235546 13643 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:09.240351 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:09.241780 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:09.242655 13800 raft_consensus.cc:2368] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12.
I20260709 12:23:09.247298 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:09.254817 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:09.256985 13800 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:09.263301 13643 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:09.231845 18603 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:09.281105 13652 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:09.350490 18698 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:23:09.387749 18706 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.196s	user 0.002s	sys 0.003s
I20260709 12:23:09.319268 18624 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.115s
I20260709 12:23:09.428949 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:09.436144 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 154814us
I20260709 12:23:09.442837 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:09.445302 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:09.447409 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:09.445169 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:09.456378 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:09.445729 18677 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:09.468462 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:09.468909 18677 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:09.472368 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:09.425450 18625 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:09.518656 18625 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:09.531062 18711 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:23:09.507448 18614 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.533288 18614 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:09.539433 18701 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:09.542711 18689 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:09.543434 18689 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:09.543982 18705 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:09.545403 18705 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.545885 18689 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:09.547154 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2506) took 1137 ms. Trace:
I20260709 12:23:09.548247 13723 rpcz_store.cc:276] 0709 12:23:08.409255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.413946 (+  4691us) service_pool.cc:224] Handling call
0709 12:23:08.414330 (+   384us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:23:08.450101 (+ 35771us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:08.481222 (+ 31121us) log.cc:844] Serialized 14 byte log entry
0709 12:23:09.104462 (+623240us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b18003d2a60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:23:09.120451 (+ 15989us) log.cc:844] Serialized 14 byte log entry
0709 12:23:09.120697 (+   246us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:23:09.134728 (+ 14031us) log.cc:844] Serialized 78 byte log entry
0709 12:23:09.134921 (+   193us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:09.135120 (+   199us) log.cc:844] Serialized 14 byte log entry
0709 12:23:09.138671 (+  3551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.198030 (+ 59359us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:23:09.198070 (+    40us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b18003d2a60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:09.206142 (+  8072us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:09.209093 (+  2951us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:09.546849 (+337756us) raft_consensus.cc:1685] finished
0709 12:23:09.546870 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.547045 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73825,"prepare.queue_time_us":109346,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":105,"raft.queue_time_us":105707,"raft.run_cpu_time_us":2563,"raft.run_wall_time_us":98918,"spinlock_wait_cycles":47219840,"thread_start_us":507788,"threads_started":1,"wal-append.queue_time_us":572328}
I20260709 12:23:09.547120 18603 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:09.542896 18614 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:09.517841 18677 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:09.521883 18625 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:09.573271 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47771.1us
I20260709 12:23:09.578384 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:09.581466 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:09.582422 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:09.571718 18692 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.530s	user 0.003s	sys 0.015s
I20260709 12:23:09.583639 13648 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:09.576177 18481 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:23:09.589273 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:09.597266 14416 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:09.672786 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:09.620041 18481 raft_consensus.cc:1068] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Unable to promote non-voter b708d8f5f6ac4aabb1eb5cb821ace195: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:23:09.707145 18673 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:09.742141 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2877) took 1042 ms. Trace:
I20260709 12:23:09.742736 13647 rpcz_store.cc:276] 0709 12:23:08.700133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:08.702151 (+  2018us) service_pool.cc:224] Handling call
0709 12:23:08.702471 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:08.772773 (+ 70302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:08.772804 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:08.773761 (+   957us) log.cc:844] Serialized 52 byte log entry
0709 12:23:09.255366 (+481605us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:09.255407 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:09.255477 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:09.741851 (+486374us) raft_consensus.cc:1685] finished
0709 12:23:09.741877 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:09.742060 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":480819,"threads_started":1,"wal-append.queue_time_us":813821}
I20260709 12:23:09.810681 18613 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.811887 18613 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.815270 18613 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:09.816781 18705 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:09.841248 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 368916us
I20260709 12:23:09.842280 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:09.887867 14416 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:23:09.888556 14416 pending_rounds.cc:85] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 13
I20260709 12:23:09.888849 14416 pending_rounds.cc:107] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:23:09.889163 14416 raft_consensus.cc:2889] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:09.908152 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 422010us
I20260709 12:23:09.910451 18613 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:09.911209 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:09.911471 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:09.912686 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:09.914050 13693 leader_election.cc:400] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12.
I20260709 12:23:09.914362 13693 leader_election.cc:403] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:09.857776 18602 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:09.923851 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:09.936412 13792 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:23:09.931203 13647 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:09.856678 18705 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:09.906523 18677 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:23:09.990443 18721 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:09.990653 18603 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:09.991335 18603 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.991743 13797 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:23:09.993474 18700 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.860s	user 0.004s	sys 0.000s
I20260709 12:23:09.993680 18481 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.028321 13797 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:09.965878 18705 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:09.991358 18721 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:09.984418 18727 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:09.994176 18481 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.038285 18603 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.041261 18723 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:10.052306 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.064636 13797 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:23:10.068995 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 438939us
W20260709 12:23:10.072293 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 399597us
I20260709 12:23:10.053169 18723 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:10.078150 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:10.081108 13796 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:10.079814 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:10.049412 18716 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:23:10.086856 18481 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:10.087214 18719 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:10.087882 18719 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.090885 18719 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.092609 14422 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:10.094219 18603 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.094997 18603 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.097822 18603 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:10.075193 18083 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:23:10.109303 18718 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:10.120428 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 189941us
I20260709 12:23:10.145457 18715 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:10.155040 18723 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.118172 18726 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:10.157915 18726 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.110107 18718 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.160805 18726 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:10.083483 18616 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:23:10.146113 18715 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:10.172742 18715 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.130738 18617 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:10.174108 18617 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:10.197683 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 84824us
I20260709 12:23:10.178743 18705 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.157212 18723 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:10.131834 18724 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:10.209956 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84543.9us
I20260709 12:23:10.178790 18617 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:10.214027 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 300110us
I20260709 12:23:10.230521 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.235890 13648 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:10.241477 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.242339 13647 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:10.246814 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:10.250607 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.256808 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:10.261160 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:10.264956 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:10.265821 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:10.245766 18570 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:23:10.260555 18647 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:23:10.227022 18625 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:10.279052 18625 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.285197 18624 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:10.288156 18716 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:10.285841 18624 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.288890 18716 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:23:10.290782 18624 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:10.291266 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.291942 13720 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:10.267405 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.260627 18719 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:10.298308 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:10.298955 13719 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:10.256326 18736 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:23:10.247354 18718 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:10.227964 18724 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.302109 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.302938 13798 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:10.303340 18724 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.303643 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.258445 18701 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.304103 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.304786 13794 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:10.306396 13652 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:10.308399 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.308642 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.309065 13798 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:10.309082 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.309486 13652 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:10.313187 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.317631 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.318864 13722 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:10.322404 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.323729 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:10.204965 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 75591.4us
I20260709 12:23:10.325696 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.347671 13645 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:10.337080 13643 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:23:10.338828 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.344918 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.352099 13720 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:10.354749 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.354830 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.355338 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.349341 13727 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:10.356482 13646 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:10.352564 13720 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:10.357733 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 235692us
I20260709 12:23:10.358816 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.361431 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:10.362377 14420 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:10.363134 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:10.365321 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:10.369787 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.379045 18716 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.338816 18705 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:10.380311 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:10.380787 18705 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.383383 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:10.384651 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:10.383289 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.386786 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.367754 13720 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.388935 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.332845 18689 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
W20260709 12:23:10.412562 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49531.5us
I20260709 12:23:10.413759 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:10.428277 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:10.403890 18470 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:10.389827 18602 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:10.389632 18716 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:23:10.470410 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105146us
I20260709 12:23:10.410045 18736 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:10.407943 18603 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:10.407882 18688 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.433287 18718 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:10.389976 18701 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 12:23:10.467543 18149 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:10.507153 18677 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:10.550591 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122423us
I20260709 12:23:10.532871 18470 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:10.529886 18736 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:10.537752 18645 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:10.540052 18625 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:10.533685 18603 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:10.543454 18718 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.550030 18677 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.555022 18701 consensus_queue.cc:1243] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:23:10.558355 18747 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:10.560206 18736 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:10.613677 18705 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.613499 18470 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.570076 18603 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:10.574743 18718 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:10.552347 18149 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.603420 18677 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:10.623023 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 217977us
W20260709 12:23:10.623349 18754 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:23:10.629920 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135412us
I20260709 12:23:10.634346 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:10.608969 18747 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.644649 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:10.648845 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 208844us
I20260709 12:23:10.611515 18538 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:23:10.644711 18747 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:10.658308 18749 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 12:23:10.674209 18149 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:10.678395 18757 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:10.688807 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:10.701035 18753 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:23:10.707286 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:10.718434 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 151934us
W20260709 12:23:10.703893 18750 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:23:10.727496 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:10.744206 18770 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:10.748775 13727 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:23:10.764108 18757 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:10.770539 18734 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:23:10.778687 18617 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.520s	user 0.006s	sys 0.005s
I20260709 12:23:10.779307 18617 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.781064 13648 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:10.783326 18747 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.784252 18480 raft_consensus.cc:1064] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:23:10.784354 18718 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:10.797837 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 145973us
I20260709 12:23:10.798818 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:10.812250 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.786392 18715 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:10.814459 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74655.9us
I20260709 12:23:10.820434 18718 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:10.821439 18355 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:10.822000 18355 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.824417 18355 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:10.825603 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:10.813890 18726 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.004s
I20260709 12:23:10.826802 18726 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.828523 18747 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:10.796211 18617 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.834967 18603 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.829833 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:10.836249 18720 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.772s	user 0.007s	sys 0.003s
I20260709 12:23:10.837010 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:10.837234 18603 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.838446 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:10.839056 14420 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:10.839775 18723 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.612s	user 0.013s	sys 0.000s
I20260709 12:23:10.840610 13719 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:10.840977 13719 pending_rounds.cc:85] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 5
I20260709 12:23:10.841200 13719 pending_rounds.cc:107] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 11.6
W20260709 12:23:10.841329 18721 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.004s
I20260709 12:23:10.841457 13719 raft_consensus.cc:2889] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:10.841790 18721 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12.
I20260709 12:23:10.812039 18774 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:10.847024 18736 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:10.829480 18726 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:10.853282 13726 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:10.854816 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.856331 14420 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
W20260709 12:23:10.858628 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45397.3us
I20260709 12:23:10.857357 13723 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:10.861893 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:10.862851 13796 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.863338 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:10.865178 18691 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:10.865746 18691 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:10.870613 18691 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:10.875587 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:10.876942 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.889384 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:10.878274 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:10.883680 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:10.895624 13648 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:10.886018 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:10.896266 13796 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:10.896601 13719 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:10.897652 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:10.904003 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.908943 13719 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:10.910360 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:10.911190 13796 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:10.911507 13726 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:10.874361 18736 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:10.924176 18726 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:10.874717 18677 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.874189 18480 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:10.933310 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:10.944972 18613 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:10.945626 18613 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:10.949564 18677 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:10.954324 13724 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:10.959491 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:10.981328 13693 leader_election.cc:400] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:10.990792 13693 leader_election.cc:403] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:10.999948 13648 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:11.006652 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:11.010808 13801 raft_consensus.cc:2368] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
I20260709 12:23:11.021380 18785 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:11.026010 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:10.991590 18719 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:11.028038 13693 leader_election.cc:400] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
I20260709 12:23:11.028496 13693 leader_election.cc:403] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:23:11.010443 18736 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:11.025226 18613 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:11.022614 18778 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:11.046291 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:11.030387 18721 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:11.050096 13693 leader_election.cc:400] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:11.036589 18774 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:11.036629 18619 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:11.028650 18785 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:11.051169 18778 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:11.069339 18778 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:11.053730 18736 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:11.070997 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:11.074668 13725 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:23:11.078152 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 176594us
I20260709 12:23:11.113340 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:11.115779 14422 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:11.117861 14423 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:11.107169 18785 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:11.129323 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 170985us
W20260709 12:23:11.135756 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.144842 17735 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:11.168458 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.061141 18619 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:11.060632 18774 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:11.211170 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 185259us
W20260709 12:23:11.235801 18782 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:23:11.272104 18774 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:11.274966 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 765) took 1353 ms. Trace:
I20260709 12:23:11.308782 18619 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:11.317337 18721 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
W20260709 12:23:11.372735 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.380177 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 693) took 1101 ms. Trace:
I20260709 12:23:11.381278 13800 rpcz_store.cc:276] 0709 12:23:10.278551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.282880 (+  4329us) service_pool.cc:224] Handling call
0709 12:23:10.283240 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:10.283615 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:10.283644 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:10.970210 (+686566us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:23:10.970720 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:23:11.378949 (+408229us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:11.379029 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.379167 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:11.379351 (+   184us) raft_consensus.cc:1685] finished
0709 12:23:11.379366 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:11.379772 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9674,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":90,"spinlock_wait_cycles":25122560,"thread_start_us":407461,"threads_started":1,"wal-append.queue_time_us":316060}
I20260709 12:23:11.275641 14416 rpcz_store.cc:276] 0709 12:23:09.921613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:09.922715 (+  1102us) service_pool.cc:224] Handling call
0709 12:23:09.937148 (+ 14433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:09.937785 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:23:10.103850 (+166065us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b18003343c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:23:10.366349 (+262499us) log.cc:844] Serialized 14 byte log entry
0709 12:23:10.559092 (+192743us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:11.238716 (+679624us) spinlock_profiling.cc:243] Waited 67.2 ms on lock 0x7b2c000d57a8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:23:11.239158 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:23:11.239343 (+   185us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:11.239378 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.239448 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:11.274624 (+ 35176us) raft_consensus.cc:1685] finished
0709 12:23:11.274657 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:11.274875 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11357,"prepare.queue_time_us":481403,"prepare.run_cpu_time_us":183,"prepare.run_wall_time_us":180,"raft.queue_time_us":66465,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":16034,"spinlock_wait_cycles":160966656,"thread_start_us":453501,"threads_started":2,"wal-append.queue_time_us":248192}
I20260709 12:23:11.388777 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:11.453980 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:11.480978 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2832) took 1181 ms. Trace:
I20260709 12:23:11.459038 18717 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:11.489447 18717 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:11.492703 13720 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:11.493352 18717 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:11.494458 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:11.487928 13725 rpcz_store.cc:276] 0709 12:23:10.289988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.290137 (+   149us) service_pool.cc:224] Handling call
0709 12:23:11.074242 (+784105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:11.075143 (+   901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:11.460977 (+385834us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54001f0c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:11.470919 (+  9942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1802344192}
I20260709 12:23:11.499987 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:11.501387 18603 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:11.512836 18747 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:11.514179 18747 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:11.522923 18721 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:11.524156 18721 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:11.535890 18747 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:11.591320 18721 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:11.598078 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:11.595052 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56061.6us
W20260709 12:23:12.291049 13766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.522s (SENT) [suppressed 1 similar messages]
W20260709 12:23:13.111686 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080024e560 after lost signal to thread 12472
W20260709 12:23:13.211620 13766 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.003s	sys 0.006s
W20260709 12:23:13.212209 13766 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.003s	sys 0.006s
W20260709 12:23:13.212567 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 921696us
I20260709 12:23:13.214545 18491 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.215401 18491 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.218500 18491 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.370797 18602 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.370563 18491 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.371838 18602 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.372421 18491 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:13.375352 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:13.376888 18701 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.377876 18491 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.378382 18701 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.379755 18491 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.380546 18491 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.381129 18701 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.375442 18602 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.384691 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.386590 18602 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.387735 18602 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.388441 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.389760 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:13.390813 18602 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.391044 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:13.391661 18602 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.392012 13723 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:13.392866 18654 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.392576 18602 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.393637 18654 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.385946 18701 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.395350 18701 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.395900 18602 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.397054 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.397827 18654 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.397934 18701 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.398437 18654 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.399082 18654 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.391315 18491 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.400796 18491 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:13.401541 18491 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.401731 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.404068 18491 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.404274 13726 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:13.404726 18491 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.405555 18491 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:13.397104 12474 debug-util.cc:398] Leaking SignalData structure 0x7b080000ffe0 after lost signal to thread 13618
I20260709 12:23:13.397975 13727 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:13.402657 18654 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:13.408391 12474 kernel_stack_watchdog.cc:198] Thread 13618 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 1259ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:23:13.408179 18654 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.411391 18654 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.415347 13727 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:13.416226 18701 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.420770 18701 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.421550 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.423283 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.423764 18701 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.424515 18701 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.425287 18701 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.427358 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:13.429281 13726 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:23:13.429888 18701 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.430742 18770 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.431524 18701 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.431748 18770 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.432612 18701 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.434705 17735 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.435765 18701 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.435540 17735 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.436362 18701 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.436995 18701 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:23:13.437894 17735 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.446882 17735 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:13.451161 18770 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.453481 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2933) took 2585 ms. Trace:
I20260709 12:23:13.455379 13643 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:13.457949 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.461227 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.466878 18774 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.468032 18774 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.454058 13649 rpcz_store.cc:276] 0709 12:23:10.868196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.870780 (+  2584us) service_pool.cc:224] Handling call
0709 12:23:10.871077 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:10.871312 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:23:10.927374 (+ 56062us) log.cc:844] Serialized 14 byte log entry
0709 12:23:11.124578 (+197204us) log.cc:844] Serialized 14 byte log entry
0709 12:23:11.245264 (+120686us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180035bba0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:13.452280 (+2207016us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180035bba0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:23:13.452669 (+   389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:13.452755 (+    86us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:13.452917 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.453044 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.453246 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2360,"prepare.queue_time_us":74679,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":76,"raft.queue_time_us":2411,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6454,"spinlock_wait_cycles":50601728,"thread_start_us":252077,"threads_started":2,"wal-append.queue_time_us":292697}
I20260709 12:23:13.475394 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.476392 13799 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:13.491539 18774 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.492446 18774 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.417527 18491 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.441701 18701 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.495127 18701 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.495994 18701 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.496882 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:13.443939 18602 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.443998 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:13.498647 18602 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:13.499671 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55835.5us
I20260709 12:23:13.500774 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:13.501729 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:13.502524 18491 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:13.503350 18701 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.503945 18491 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:13.504594 18491 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.505064 18491 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:13.493070 18721 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.498121 18717 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:13.418704 18654 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.507434 18557 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:13.511410 18557 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.511785 18557 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:13.506917 18721 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.514163 18721 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.514792 18721 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.515350 18721 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.517443 18721 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.517927 18721 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.524293 17735 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.526245 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:13.505754 18701 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.525537 17735 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:13.531766 13720 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 2.038s	user 0.019s	sys 0.021s
I20260709 12:23:13.508873 18717 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.506740 18602 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.524678 18774 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.537921 18518 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.537946 18732 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.538522 17735 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.539109 18732 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.538916 18518 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.538108 18774 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.535109 18717 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:13.545702 18774 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.544924 17735 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.546233 18800 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.547698 18732 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.554404 13720 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:23:13.555929 17735 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.560917 13649 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:23:13.623890 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800343740 after lost signal to thread 13584
I20260709 12:23:13.569365 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.547724 18518 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.542150 18602 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.563999 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:13.625541 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:13.633026 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.574719 18718 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:13.558454 18701 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.564664 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:13.622452 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2138769910002338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:23:13.567608 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.566598 18745 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.568259 18774 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.580744 18513 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:13.586618 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.578449 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 703) took 1956 ms. Trace:
W20260709 12:23:13.643714 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133509us
I20260709 12:23:13.564445 18654 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.555380 13720 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:23:13.587132 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:13.638725 13721 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:13.649561 13796 rpcz_store.cc:276] 0709 12:23:11.622384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:13.577068 (+1954684us) service_pool.cc:224] Handling call
0709 12:23:13.577553 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:13.577837 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:13.577874 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:13.577933 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:13.577978 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.578050 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.578194 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:13.634102 17735 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.555667 18721 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.656426 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.657135 13727 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:13.577711 18491 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.659674 18625 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.659835 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:13.660740 13651 raft_consensus.cc:2368] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
I20260709 12:23:13.660550 18625 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.661866 18721 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.557384 18673 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:23:13.661978 18745 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.662334 18602 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:13.664636 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75647.2us
I20260709 12:23:13.664809 18718 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.664654 18602 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.667263 18718 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.668385 18769 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:13.669001 18769 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.670135 18701 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.652467 13720 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:23:13.675314 13720 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:13.676285 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2543) took 3324 ms. Trace:
I20260709 12:23:13.676887 18808 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.677075 18802 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.676620 13720 rpcz_store.cc:276] 0709 12:23:10.351555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.351676 (+   121us) service_pool.cc:224] Handling call
0709 12:23:10.351985 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:11.489163 (+1137178us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540030e1a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b69a1d 00007f5393b501a4 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419
0709 12:23:13.675937 (+2186774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.676193 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506836736}
I20260709 12:23:13.688292 18654 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.689512 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.691594 18654 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.692564 18491 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.696560 13723 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:13.671267 18769 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.568948 18800 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.679572 18603 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.707166 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.664990 18745 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.718698 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:13.720371 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.721082 13721 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:23:13.722146 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48418.1us
I20260709 12:23:13.724152 13799 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:13.734055 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.735033 13800 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:13.704069 18808 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.739089 18557 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.740603 18808 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.741343 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:13.741647 18557 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:13.741884 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54340.7us
I20260709 12:23:13.743019 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:13.743297 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:11.935287 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:13.750689 18747 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:13.699736 18802 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.751189 18747 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.751677 18747 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:13.751904 18491 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:13.752741 18625 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.753640 18491 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.754355 18491 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.756222 18752 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.756964 18752 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.757786 18751 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:13.758157 17735 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.758405 18751 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.758742 17735 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.759500 18752 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.751619 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:13.760140 18614 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.760999 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:13.760826 18614 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:13.756376 13767 connection.cc:256] RPC call timeout handler was delayed by -1.03736519199947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
I20260709 12:23:13.763444 18518 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:13.763862 18518 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.764254 18518 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:13.764946 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:13.765650 18491 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.765887 13725 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:13.766458 18732 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:13.766878 18732 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.767351 18732 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:13.767185 18491 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:13.769969 18751 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.721778 18603 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:13.774362 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800326d20 after lost signal to thread 13618
I20260709 12:23:13.774618 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:13.775215 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08002f4ae0 after lost signal to thread 13646
W20260709 12:23:13.776021 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800281f00 after lost signal to thread 13676
W20260709 12:23:13.776343 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800298280 after lost signal to thread 13677
I20260709 12:23:13.777647 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.777194 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:13.778780 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.783564 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.784498 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.785032 18769 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:13.785380 13647 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:23:13.785571 18769 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.786034 18769 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:13.786732 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.787560 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 6.
I20260709 12:23:13.788470 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.788813 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.756851 18673 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.790071 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.790117 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.790952 13799 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:13.790773 18673 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.785933 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:13.791918 18802 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.792884 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.793372 18673 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.793529 13800 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:13.795264 13648 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:13.795881 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:13.796450 13725 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:13.725977 18745 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.791164 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
W20260709 12:23:13.797305 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:13.777680 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5e40 after lost signal to thread 13751
W20260709 12:23:13.802022 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800364b60 after lost signal to thread 13825
W20260709 12:23:13.804625 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800394c40 after lost signal to thread 14449
W20260709 12:23:13.804919 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080023bb00 after lost signal to thread 14450
W20260709 12:23:13.805238 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800100220 after lost signal to thread 18334
W20260709 12:23:13.805538 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003547e0 after lost signal to thread 18470
I20260709 12:23:13.805899 13649 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
W20260709 12:23:13.806702 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080037a460 after lost signal to thread 18509
W20260709 12:23:13.807250 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003523a0 after lost signal to thread 18613
W20260709 12:23:13.807554 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003263e0 after lost signal to thread 18616
W20260709 12:23:13.807833 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800343f00 after lost signal to thread 18617
W20260709 12:23:13.809319 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800321040 after lost signal to thread 18619
W20260709 12:23:13.810333 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800336e40 after lost signal to thread 18636
W20260709 12:23:13.811308 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003c3840 after lost signal to thread 18715
W20260709 12:23:13.811623 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08002fc060 after lost signal to thread 18716
I20260709 12:23:13.810101 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:13.776639 18752 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:13.821767 17735 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.829640 18809 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.830667 18809 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.833588 18809 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.835588 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:13.841805 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60793.9us
I20260709 12:23:13.843235 18809 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.843932 18809 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.846222 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.847327 13649 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:13.848619 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2949) took 2478 ms. Trace:
I20260709 12:23:13.849041 13646 rpcz_store.cc:276] 0709 12:23:11.369786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.370607 (+   821us) service_pool.cc:224] Handling call
0709 12:23:11.371354 (+   747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:11.372037 (+   683us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:11.372714 (+   677us) log.cc:844] Serialized 14 byte log entry
0709 12:23:13.848098 (+2475384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:13.848168 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:13.848215 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.848294 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:13.848413 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2475039,"prepare.queue_time_us":21655,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":2559}
I20260709 12:23:13.850978 18470 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:13.851696 18470 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.858747 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.859091 18470 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.859776 13801 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:13.854045 18509 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:13.860636 18509 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:13.863340 18509 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.866138 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 2958) took 2431 ms. Trace:
I20260709 12:23:13.866621 13643 rpcz_store.cc:276] 0709 12:23:11.435218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.435504 (+   286us) service_pool.cc:224] Handling call
0709 12:23:13.454643 (+2019139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:13.456077 (+  1434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:13.865903 (+409826us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b5400096850. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:13.866066 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4642209536}
W20260709 12:23:13.867157 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 2.60172e+06us
W20260709 12:23:13.878523 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68892us
I20260709 12:23:13.877533 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.879810 13799 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:13.877084 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:13.879379 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:13.876309 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 2.73926e+06us
I20260709 12:23:13.900497 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:13.875115 18083 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:13.886619 18800 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.902753 18083 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.903887 18645 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:13.904332 18645 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.904776 18645 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:13.906032 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:13.908828 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65862.7us
I20260709 12:23:13.927101 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.846246 18809 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.927968 13801 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:13.877581 18774 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.815834 18745 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.890503 18808 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:13.931769 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.942238 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.939241 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:13.945075 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.945499 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:13.945894 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45054.8us
I20260709 12:23:13.952569 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:13.953118 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47919.6us
I20260709 12:23:13.954282 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:13.955054 14417 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:13.957955 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.912024 18717 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:13.959290 18083 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:13.959597 18688 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:13.960312 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.960532 18717 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:13.961578 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.962023 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.919662 18718 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:13.963714 18808 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:13.964183 18808 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:13.961498 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.965219 13795 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:13.963335 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.967065 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:13.950037 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:13.890412 18614 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:13.980268 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:13.985756 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:13.987815 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.996403 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:13.999739 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.001776 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:14.001914 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97875.6us
I20260709 12:23:13.984030 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:13.999967 13794 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:13.963735 18718 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.007989 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.008262 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.012025 14423 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:14.012269 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:13.958096 18732 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.007776 18718 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:13.986685 18752 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:14.031570 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92209.1us
I20260709 12:23:13.959326 18791 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:23:14.033018 18715 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:14.034368 18791 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.034970 18791 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.035821 18509 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.036513 18509 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.037148 18791 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.042366 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.044925 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.045902 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.046625 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.047202 13650 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:14.047919 18334 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.048082 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.048548 18334 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.049391 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.050117 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.050855 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.047230 17735 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.052384 18617 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.052980 17735 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.053496 17735 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:14.053419 18617 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.055226 18769 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.056102 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:14.070636 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 192295us
I20260709 12:23:14.070812 13723 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:14.056443 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.046593 13649 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:14.052414 18636 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.052444 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.058849 18334 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.080085 13795 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:14.065341 18616 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.053057 18810 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:13.988893 18518 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.049477 18470 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.067233 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:14.030979 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.066370 18513 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.053321 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.068634 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.078799 17735 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.060776 18747 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.080835 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.074381 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:14.082234 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52023.2us
I20260709 12:23:14.065694 18603 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.093114 13722 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:14.052703 18745 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.081209 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:14.033800 18715 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.068670 18083 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.091486 18570 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:14.069633 18619 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.097224 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.097394 18716 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:14.101732 18509 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.084261 18636 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.093789 13646 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:14.103148 18513 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.104651 18083 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.105603 18518 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.106956 18715 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:14.107438 18470 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.145277 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.107944 18617 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.108840 13799 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:14.081666 18688 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.110055 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.110500 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.111722 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.111809 18645 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.113306 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.116851 13718 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:14.124646 18619 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.126129 18770 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:14.128680 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.129407 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.129536 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.101610 18616 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.130522 13797 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:14.126010 18732 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.131871 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.132421 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.134989 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.169518 13643 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
W20260709 12:23:14.169866 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51085.6us
I20260709 12:23:14.137583 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:14.140636 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59513.1us
I20260709 12:23:14.171239 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.171897 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.172663 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.152979 18557 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:14.146190 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.155447 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.156349 13794 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:14.157348 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.162634 18636 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.133487 18752 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.175642 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.163092 18769 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.163717 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.164656 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.168252 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.137454 17735 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.173133 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.178954 13647 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:14.176527 13646 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:14.141309 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.174965 13650 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:23:14.189291 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.189544 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.189810 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.188570 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.186120 13644 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
W20260709 12:23:14.189612 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48396.6us
I20260709 12:23:14.179517 13648 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:23:14.197481 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.199766 13645 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:14.201318 14424 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:14.202687 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.198976 18654 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:14.196549 18817 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:14.207005 18747 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.193276 18732 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.178421 18688 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.185598 18816 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:14.210671 18816 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.211154 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.212505 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.212630 18616 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.213676 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.215101 18470 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.215930 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.216784 13650 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:23:14.217495 18645 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.217669 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.175609 18557 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.218797 18557 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.219908 18083 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.222471 18491 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:14.223063 18688 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.223956 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.225116 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.225359 14420 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:14.226568 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.226750 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.228212 18513 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.186358 18518 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:14.229127 18813 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:23:14.232926 18730 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:14.233925 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.175428 18570 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:14.236799 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.238037 18603 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.238485 18603 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.238878 18603 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:14.155189 18716 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.241267 18716 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:14.178988 18673 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:14.242457 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.243078 13727 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:14.244954 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.244072 18817 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:14.247671 18817 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.244431 13650 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:14.211068 18816 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.251852 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.252596 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.255506 18619 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.255708 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.256944 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.261812 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.262917 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.263960 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.264638 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.267285 13801 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:14.267697 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.268462 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.269670 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.270020 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.272887 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.273299 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.274320 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.274590 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.275454 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.278156 18808 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.280473 18808 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.281140 18718 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.283803 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:14.286245 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50265.3us
I20260709 12:23:14.264947 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.297593 13644 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:14.304713 18718 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.308068 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:14.310566 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.312273 14416 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:14.313400 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.306022 13649 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:14.316319 13793 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:14.319984 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67897us
I20260709 12:23:14.322600 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.325935 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.331051 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.333268 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.338850 13792 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:14.333153 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.343657 18673 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:14.344185 18673 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.344564 18673 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.325255 18718 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:14.350889 18718 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.351284 18718 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.356074 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:14.358795 18616 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:14.360066 18616 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:14.360420 18747 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 4 to 10, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:14.362037 18518 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:14.362458 18806 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:14.363023 18806 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.332751 18470 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:23:14.363509 18806 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.365201 18470 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:14.365592 18470 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.370894 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.371650 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.373076 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.373679 13796 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.355343 18557 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.362530 18518 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.390995 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.393071 18603 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.360455 18616 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.388909 14424 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 4 to 10, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:14.395274 18616 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.396575 18616 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:14.378973 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.379135 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.389930 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:14.398835 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62187.2us
I20260709 12:23:14.382556 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:14.400434 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69469.7us
I20260709 12:23:14.400590 18570 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:14.401177 18570 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.401508 18570 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.401856 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.339993 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:14.406073 18470 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.407032 18724 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:14.407481 18724 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.407902 18724 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:14.409147 12474 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8b40 after lost signal to thread 13618
I20260709 12:23:14.376664 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:14.410393 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70485.6us
I20260709 12:23:14.411108 18716 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:14.411626 18716 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:14.412029 18716 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.382952 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.414498 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.378021 18619 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:23:14.349855 18816 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.424032 13798 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:14.423678 13544 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:14.424860 18603 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.426023 18816 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.422425 13727 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:14.427896 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.428861 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:14.407824 18470 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:14.432113 17735 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:14.387562 18730 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:14.435765 18688 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:14.433668 18513 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:23:14.439250 13727 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:14.473109 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.433794 18614 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:14.453269 18724 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.454509 17735 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:14.455315 18557 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:14.456976 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:14.507783 17735 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:14.481791 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125778us
I20260709 12:23:14.488844 18614 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:14.510457 18614 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:14.510085 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.495690 18570 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.516829 18570 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.496626 18718 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.504189 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.518978 18806 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.520267 18770 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:14.521075 18806 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.504745 18619 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:14.508878 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.521636 18619 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:14.523269 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 123470us
I20260709 12:23:14.525002 14422 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:14.526348 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:14.525317 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65846.6us
I20260709 12:23:14.527992 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:14.528110 17735 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.533483 18717 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:14.534025 18717 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:14.534405 18717 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:14.509042 18716 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.536933 18716 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:14.539822 14420 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:14.541028 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:14.541561 18614 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.542598 18619 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.543628 18636 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:14.544109 18614 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.544224 18636 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.547026 14416 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:14.548511 14420 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.550575 13724 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:23:14.551771 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.548933 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.543709 18619 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:14.550930 13544 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 9 to 11, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 11 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:14.531353 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:14.560674 13647 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:14.567838 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:14.568629 13544 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 4 to 9, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 9 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:14.570879 18817 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:14.571532 18817 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:14.518990 18718 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.576068 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.588577 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.553464 18654 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:14.527029 18806 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.598395 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.592149 13720 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:14.599391 13794 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:14.593720 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.599905 13645 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.594358 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.548895 18673 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.598616 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.595630 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.604434 13722 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.601867 13725 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:14.602023 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.597035 18654 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:14.613350 13724 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:14.616721 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.617700 13793 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
W20260709 12:23:14.618199 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62842.2us
I20260709 12:23:14.618553 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.619165 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.619525 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.617799 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.621201 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:14.627238 13726 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:14.628664 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:14.623637 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.632877 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.572554 18817 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.630244 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.635067 13650 raft_consensus.cc:2368] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
I20260709 12:23:14.634035 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.638190 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:14.638391 13794 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:14.622013 13721 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 4 to 10, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:14.648542 18724 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.612761 18654 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:14.641801 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:14.639389 17735 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:14.642416 13798 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.655308 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.656059 13725 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:14.660411 13801 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:14.665143 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.666798 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:14.669108 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45536.3us
I20260709 12:23:14.675011 14424 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:14.677057 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.679944 13793 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:14.679682 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.679736 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:14.683511 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.690857 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:14.706575 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:14.695063 13724 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:14.627913 18688 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:23:14.635824 18617 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:23:14.678148 18645 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:14.679023 18673 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.738902 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.742316 13725 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:14.763809 18745 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:14.748370 18811 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:23:14.678561 18715 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:23:14.749904 18470 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:14.705654 18769 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:23:14.707144 18616 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:23:14.776578 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.793258 13718 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:14.781894 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.735368 18518 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.797652 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:14.791040 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 100269us
I20260709 12:23:14.786579 18470 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:14.802409 18834 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:14.803185 18834 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.803769 18844 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:14.780896 18716 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:23:14.808502 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.796130 18717 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.822247 18636 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:14.835904 18636 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:14.838917 14424 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:14.819787 18518 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:14.844413 18717 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:14.850126 14420 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:14.850978 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:14.817566 18654 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.801636 18470 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:14.859985 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.862493 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:14.806347 18688 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:14.896505 13793 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.874387 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:14.850890 18716 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:23:14.877470 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 252338us
I20260709 12:23:14.900622 13720 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.900734 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:14.807343 18844 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.804267 18834 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:14.903867 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.906852 13796 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:14.899279 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.903019 13800 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.904433 13799 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:14.893078 18747 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:14.905470 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.892750 18688 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.891780 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:14.919163 13645 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:14.894789 18842 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:23:14.912355 18614 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:14.923589 18654 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:14.900195 18716 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:14.928223 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.930227 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.935679 13646 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:14.927842 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.936789 13719 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:14.936834 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.926676 18837 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:14.927366 13726 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:14.928467 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.935617 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.939363 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.940043 13721 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:14.935708 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:14.939460 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.926834 13794 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:14.931490 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:14.946348 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.933662 18715 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:23:14.902911 18619 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:23:14.950764 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.952184 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:14.952734 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.954149 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:14.917868 18083 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.959404 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:14.960971 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:14.967633 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:14.949328 18614 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:14.955896 18839 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:14.969379 18513 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:14.974622 18808 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:14.985598 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:14.989171 14416 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:15.105283 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130688us
W20260709 12:23:15.110533 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125031us
I20260709 12:23:14.969542 18717 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:15.111305 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.131459 18839 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:15.104518 18617 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.145s
W20260709 12:23:15.149150 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 171149us
I20260709 12:23:15.118682 18808 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:15.153265 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:15.156381 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49429.8us
I20260709 12:23:15.134616 18603 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:23:15.155823 18839 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:15.172690 18838 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:15.146979 18716 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.175402 18840 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:15.195269 18840 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:15.170512 18614 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.196301 18840 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.175963 18808 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.202095 18688 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.203508 18688 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.179880 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:15.212425 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:15.214663 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.187282 18843 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:15.216758 18843 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:15.188149 18083 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:23:15.174657 18850 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:23:15.217826 18843 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.219455 18843 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.184916 18839 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.223611 18851 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:15.224196 18851 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:15.223071 13647 raft_consensus.cc:2955] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 4 to 10, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:15.225176 18851 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.183411 18829 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:15.227183 18829 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.228224 18829 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.158649 18715 consensus_queue.cc:1243] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:23:15.229629 18829 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:15.238375 18513 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:23:15.176105 18848 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:23:15.243803 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123145us
I20260709 12:23:15.250519 18849 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:15.256143 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:15.257725 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.264076 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.261945 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:15.263281 18513 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:23:15.262411 18843 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:15.271795 18848 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:15.264907 18715 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:23:15.276333 18819 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:15.276868 18819 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:15.279206 18614 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:15.279779 18614 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.280951 18614 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.277755 18819 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.287091 13544 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 9 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 11 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:15.293699 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:15.301735 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.303618 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.306306 18688 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:15.306563 18513 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:15.307623 18848 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:15.308583 13549 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 230) took 44823 ms (client timeout 15000 ms). Trace:
I20260709 12:23:15.308981 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.309882 18688 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:15.309576 13544 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 9, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 9 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:15.312039 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:15.309336 13549 rpcz_store.cc:269] 0709 12:22:30.485312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.485634 (+   322us) service_pool.cc:224] Handling call
0709 12:23:15.308311 (+44822677us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:08.903200 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:08.903779 (+   579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:08.903818 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:23:08.903834 (+    16us) tablet.cc:662] Decoding operations
0709 12:23:08.904213 (+   379us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:08.904269 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:08.904296 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:08.904483 (+   187us) tablet.cc:701] Row locks acquired
0709 12:23:08.904494 (+    11us) write_op.cc:260] PREPARE: finished
0709 12:23:08.904639 (+   145us) write_op.cc:270] Start()
0709 12:23:08.904747 (+   108us) write_op.cc:276] Timestamp: P: 1783599788904589 usec, L: 0
0709 12:23:08.904762 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:23:08.905544 (+   782us) log.cc:844] Serialized 533 byte log entry
0709 12:23:09.477506 (+571962us) op_driver.cc:464] REPLICATION: finished
0709 12:23:10.044553 (+567047us) write_op.cc:301] APPLY: starting
0709 12:23:10.044674 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:10.071353 (+ 26679us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:10.075795 (+  4442us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:10.095916 (+ 20121us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:10.096151 (+   235us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:23:10.096197 (+    46us) write_op.cc:312] APPLY: finished
0709 12:23:10.096446 (+   249us) log.cc:844] Serialized 25 byte log entry
0709 12:23:10.096647 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:10.096748 (+   101us) write_op.cc:454] Released schema lock
0709 12:23:10.096813 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:10.097519 (+   706us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:10.097553 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":546757,"reported_tablets":50,"spinlock_wait_cycles":4480,"child_traces":[["op",{"apply.queue_time_us":557159,"apply.run_cpu_time_us":5071,"apply.run_wall_time_us":52954,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8315,"num_ops":1,"prepare.queue_time_us":82942,"prepare.run_cpu_time_us":8805,"prepare.run_wall_time_us":168756,"raft.queue_time_us":206485,"raft.run_cpu_time_us":52642,"raft.run_wall_time_us":628250,"replication_time_us":529874,"thread_start_us":755275,"threads_started":4,"wal-append.queue_time_us":156158,"wal-append.run_cpu_time_us":20669,"wal-append.run_wall_time_us":2040324}]]}
I20260709 12:23:15.313964 18688 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:15.313133 13544 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 6 to 9, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:23:15.315032 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.316231 18834 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:15.319831 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 377460us
I20260709 12:23:15.256274 18470 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:15.341696 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:15.342690 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75507.4us
I20260709 12:23:15.314178 18616 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:15.369375 18616 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:15.369580 18513 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:15.370559 18715 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:15.370427 18616 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.371510 18715 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:15.371865 18715 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:15.372912 14420 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:15.370572 18843 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:15.373588 18843 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:15.367093 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.374475 18843 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.375870 18843 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.376873 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:15.377637 18849 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:23:15.361244 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.378923 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:15.372936 18715 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.379554 13651 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.380086 18715 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.371924 18616 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:15.385529 13544 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 4 to 5, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 5 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:15.389784 18848 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:15.391053 18848 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:15.391408 18848 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:15.392269 18848 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.393036 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.403761 18570 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:15.404955 18491 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:15.405354 18491 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.406232 18491 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.407596 18491 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.411494 18844 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:15.413662 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44669.8us
W20260709 12:23:15.415068 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.358793 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:15.419039 13651 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:15.393710 18848 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:15.426602 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:15.430338 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87847.9us
I20260709 12:23:15.430603 17735 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:15.394019 18806 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:15.413182 18844 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:15.457998 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:15.441800 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:15.439119 18616 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.452152 18617 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.456094 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.480767 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.483098 17735 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:15.456624 18470 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:15.388602 13544 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 6 to 7, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:15.487062 13724 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:23:15.493089 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90147.4us
W20260709 12:23:15.504464 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.452240 18849 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.510471 13645 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:23:15.512279 18836 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:15.511929 18616 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.514750 18716 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:15.491531 18617 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.523645 17735 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.524662 17735 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.524120 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:15.526762 17735 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:15.526974 18815 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:15.528970 18730 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:15.529376 18616 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.531899 18617 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:15.524935 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 358076us
I20260709 12:23:15.535861 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:15.537916 13720 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:15.539812 18819 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:15.539992 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:15.540752 13651 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:23:15.545480 18845 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:15.546056 18845 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:15.547708 13643 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:15.547328 18845 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.551721 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.554705 13794 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:15.551941 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:15.552539 18850 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:15.556171 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:15.556252 18850 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:15.556926 18850 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:15.556967 13799 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:15.557714 18851 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.519205 13544 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 4 to 6, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 6 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:15.560810 18843 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.561393 18843 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.532953 18836 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.561805 13544 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 10, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 10 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:15.563482 18836 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.553534 18617 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:15.564934 18617 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.566095 18864 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:23:15.522573 18849 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.566540 18864 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:15.566962 18864 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:15.552809 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:15.564695 13544 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:15.570152 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:15.572105 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 168649us
I20260709 12:23:15.572659 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:15.572736 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:15.571875 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:15.597018 13794 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:15.573254 18849 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:15.575949 18149 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:23:15.599455 18849 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.600018 18849 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.600498 18849 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.606854 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:15.607322 18724 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.574222 18852 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:15.607788 13796 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:15.608562 18852 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.609038 18852 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:15.549075 18845 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.611181 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:15.612334 18844 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.612810 18844 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:15.613245 18844 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.613859 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:15.607820 18724 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:15.614941 18724 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.617743 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:15.618485 13794 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:15.619585 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:15.619611 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:15.620846 18616 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:15.621217 18616 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.621594 18616 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.622372 13651 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 12:23:15.623494 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.644593 18617 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.648705 18616 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:15.651151 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81077.4us
I20260709 12:23:15.652091 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:15.652434 18616 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.653157 18859 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:15.648710 18850 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.653621 18859 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:15.654028 18859 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.656277 18850 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.659629 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.661727 13649 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:15.664577 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:15.668126 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:15.607115 13544 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 9, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 9 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:15.668017 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:15.676539 13719 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:15.679347 13799 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:15.676569 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:15.666373 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52699.6us
I20260709 12:23:15.683364 13794 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:15.672943 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:15.666181 18843 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.669647 18810 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:15.688742 18810 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:15.689842 18557 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:15.689672 18810 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.684572 14416 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:15.676153 18603 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:15.692269 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.696405 18843 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:15.696813 18843 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:15.697158 18843 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:15.698856 18841 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:15.699301 18841 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.700115 18841 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.701508 18841 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.704804 18149 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:15.669803 18854 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:23:15.717331 18829 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:15.717517 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:15.721096 13794 raft_consensus.cc:606] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:15.722406 13794 raft_consensus.cc:614] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:15.720075 18149 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:23:15.686108 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.745280 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:15.753459 13646 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:15.762001 13649 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:15.769316 13792 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:15.706562 18724 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.774394 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:15.768595 18850 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:15.782572 18850 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:15.782848 18724 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:15.783094 18850 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:15.773854 13793 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:15.777902 13725 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:15.796327 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.769543 18849 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.807439 18730 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:15.809408 18849 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:15.814383 18852 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.812834 13643 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:15.810225 18815 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:23:15.815187 18730 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:15.816639 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.787348 18654 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:23:15.821177 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:15.835561 13793 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.822049 13799 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:15.836776 13800 raft_consensus.cc:606] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:15.837160 13800 raft_consensus.cc:614] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:15.832237 18808 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:15.839525 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:15.845522 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 159693us
I20260709 12:23:15.849854 13796 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:15.865592 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:15.866639 13800 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.846500 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.879992 13649 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:15.885208 18730 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.886785 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:15.849076 18859 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.887699 14416 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:15.888120 13720 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:15.889212 18859 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:15.892637 18570 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:15.893126 18570 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.848227 18849 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:15.894088 18570 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.894672 18849 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:15.895670 18570 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:15.895608 18849 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.897822 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:15.898643 13799 raft_consensus.cc:606] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.899703 18852 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:15.901427 18864 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.902535 13722 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:15.903620 18864 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:15.902334 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:15.907984 13800 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:15.872445 13727 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:15.893674 18850 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:15.828112 18645 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:23:15.905571 13799 raft_consensus.cc:606] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:15.919703 18730 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:15.920778 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:15.923290 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:15.926867 18850 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:15.927760 18730 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:23:15.925496 18718 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:15.926448 18863 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:23:15.928947 18819 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:15.933246 13793 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:15.906797 18851 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:15.938746 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:15.963812 13644 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:15.942330 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:15.878315 18843 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:15.945181 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:15.950955 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:15.976449 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:15.952176 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:15.979462 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:15.980711 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:15.942401 18730 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:15.983722 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:15.991031 18871 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.159s	user 0.001s	sys 0.002s
I20260709 12:23:15.992300 18603 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:16.004994 13722 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:23:16.005213 18603 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.005725 18603 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:16.006196 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:15.993683 18859 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:16.007440 18859 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.007861 18859 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:16.008942 18491 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:16.013504 18862 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:16.014508 18850 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:15.994800 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.015010 18850 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.015244 18862 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:16.015574 18850 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:16.015729 18862 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:16.016260 18868 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:16.016729 18868 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.017074 18868 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:16.018148 18862 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.018438 18845 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:16.019941 18845 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:16.020099 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:16.020948 18845 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.019070 18864 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:16.022547 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.041710 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.027118 13793 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:16.042788 13647 raft_consensus.cc:2368] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17.
I20260709 12:23:15.963624 18851 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.046741 18724 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:16.051030 18724 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.051368 18724 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:15.951593 18844 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.033979 18864 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.054708 18864 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.054400 18840 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:15.957496 18841 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:23:16.058943 18851 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:16.034407 18839 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:23:16.065622 18862 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:16.078354 18752 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.119778 13767 leader_election.cc:400] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17.
I20260709 12:23:16.120332 13767 leader_election.cc:403] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:16.083849 18869 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:16.127992 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43709us
I20260709 12:23:16.131388 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.137307 13647 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:23:16.137766 13647 pending_rounds.cc:85] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 5
I20260709 12:23:16.137988 13647 pending_rounds.cc:107] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:23:16.138208 13647 raft_consensus.cc:2889] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:16.132680 18614 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:23:16.157774 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 116162us
I20260709 12:23:16.161118 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.163879 18841 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:16.167347 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:16.168330 14416 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:16.169020 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.153821 18810 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:16.158376 18716 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.132360 18752 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.169128 18843 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:16.192915 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144253us
I20260709 12:23:16.164450 18841 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.204572 13644 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:16.194877 18716 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.232870 18841 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:16.234093 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:16.234952 14420 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:16.237437 18603 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.240190 18716 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:16.254731 13643 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:23:16.194989 18810 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.260519 18810 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:16.263682 18752 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:16.265908 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:16.267019 14424 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:16.222579 18868 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.268306 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.269083 18752 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:16.269462 18752 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.269783 18752 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:16.271545 18874 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:23:16.171125 18859 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.274626 18859 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.275530 18859 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:16.276816 18859 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:16.277194 18859 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.277565 18859 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:16.277953 18850 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.281502 18844 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.283025 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:16.283411 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:16.240427 18838 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:16.241395 13725 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:16.241778 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:16.286720 13726 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:16.287324 14420 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:16.245182 18717 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:16.269938 18868 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.289155 13725 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:16.251830 18730 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.283056 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.299607 13727 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:16.297677 13718 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:16.294520 13649 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:16.327045 13724 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:23:16.334358 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.337066 13722 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:23:16.325153 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:16.338508 13721 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:16.313390 14416 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:16.341857 13645 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:16.277572 18614 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17.
I20260709 12:23:16.345191 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.286023 18838 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:16.359944 13727 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:16.292605 18717 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:16.365198 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.290869 18850 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:16.378876 13719 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:16.379036 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.381347 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:16.383097 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 214150us
W20260709 12:23:16.387154 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.384091 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.392107 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:16.394822 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.380525 18844 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:16.418007 18839 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 12:23:16.423396 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51683.3us
W20260709 12:23:16.449362 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70340.1us
I20260709 12:23:16.434860 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.388178 18845 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:16.362241 18570 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:23:16.406056 18808 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:16.377956 18557 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:23:16.442422 18603 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.442988 18849 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:16.505306 18881 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:23:16.505472 18849 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.436901 18724 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.437151 18844 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.397380 18730 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:16.456459 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:16.424515 13793 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:16.391654 18614 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:23:16.391049 18745 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:23:16.420239 18868 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:16.391625 18842 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:23:16.458976 18843 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.524034 13644 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:16.462592 18817 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:16.523056 18603 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:16.508538 18845 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:16.533195 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:16.533602 18614 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:16.534583 18849 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:16.535331 18879 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 12:23:16.536044 18885 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 12:23:16.542009 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 158273us
I20260709 12:23:16.543696 13726 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:16.511646 18889 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:23:16.522848 18844 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:16.524248 18815 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:23:16.462862 18864 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.551688 14416 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:23:16.550419 18603 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.507839 18808 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:16.569274 14416 pending_rounds.cc:85] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 10
I20260709 12:23:16.573243 14416 pending_rounds.cc:107] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:16.573556 14416 raft_consensus.cc:2889] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:16.552671 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 117961us
I20260709 12:23:16.551625 18614 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:16.579188 18724 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.537510 18717 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.579211 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:16.582247 14420 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:16.562536 18817 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:16.546097 18843 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.591478 13693 leader_election.cc:400] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 9. Current term is 10.
I20260709 12:23:16.592278 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.575160 18603 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.613135 18849 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.613984 13723 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:23:16.615517 18752 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.616664 18845 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.599889 18716 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:23:16.604039 18848 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:16.651119 18848 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.651589 18848 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:16.652673 18752 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:16.653404 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62030.9us
I20260709 12:23:16.564520 18898 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 12:23:16.623287 18851 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 12:23:16.654796 18851 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.657011 18851 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.621829 18810 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:23:16.658463 13718 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:16.658771 18810 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:16.659117 18810 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:16.659924 18810 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.626950 18817 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.671792 13794 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:16.573642 18808 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:16.637183 13643 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:16.640173 13647 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:16.649586 18902 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.655129 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:16.629240 18864 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.663471 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:16.675406 13799 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:16.664137 13793 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:16.667979 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:16.678341 14416 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:16.678974 18902 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.641821 18849 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.681972 18849 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.682416 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:16.671836 18810 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:16.693291 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:16.625639 18844 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.697230 14416 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:16.702137 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.683048 18851 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:16.682181 18902 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.691689 18898 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:16.723840 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:16.730019 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:23:16.728247 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:16.731642 13725 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:16.732527 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:16.635218 18614 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.739563 13723 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:16.718717 18864 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:16.737270 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:16.741417 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66151.5us
I20260709 12:23:16.695784 18745 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:23:16.743559 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.720482 18716 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:16.745332 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:16.703500 18849 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:23:16.746982 14416 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:16.703315 18906 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:23:16.732740 18854 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:16.757787 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.733186 18851 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:16.737672 18844 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:16.729612 18898 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:16.721908 18859 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.761194 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.769299 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:16.769908 13725 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:16.745967 18862 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:16.757618 18149 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.769969 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.808802 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.810374 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.812597 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:16.813321 14416 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:16.761477 18869 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:16.761556 18724 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:16.818524 18149 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:16.821365 18913 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:23:16.815680 18884 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:16.815531 18880 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:16.802409 18843 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.824416 18603 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.815851 18886 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:16.826472 18883 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:16.804011 18864 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:16.828346 18859 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.801627 18845 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.766711 18854 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:16.822044 18851 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:16.817186 18862 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:16.844551 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.846814 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.848281 13647 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:16.815557 18868 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:16.849560 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.846585 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.852551 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.853163 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.853752 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.820205 18863 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.761214 18716 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.864185 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:16.868431 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.868710 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:16.869048 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113289us
W20260709 12:23:16.925755 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61779.8us
I20260709 12:23:16.871048 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.929272 13725 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:16.929342 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:16.876118 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.877612 13727 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:16.845777 18849 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:16.876367 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.888101 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.933080 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.933373 13647 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:16.883344 18862 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:16.934381 18843 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:16.952677 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:16.993284 13725 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:16.930112 18863 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.000491 18863 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.877035 18848 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.002132 13792 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:16.992342 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:16.880535 18845 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:16.935043 18844 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.026705 13647 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:17.028216 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:17.029633 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:17.031760 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.034718 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:17.038007 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:17.003029 18491 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.040691 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:17.041767 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.043637 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.045831 13721 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:17.043943 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:17.046725 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:17.048478 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:17.056144 18730 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.002s
I20260709 12:23:17.062762 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:17.064296 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:16.938511 18849 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.065726 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.066846 13647 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:17.119242 18603 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.129096 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.126204 13797 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:17.067075 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:17.130332 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.130475 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:17.130635 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.130978 13797 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:17.131937 18917 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:17.132412 18917 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.132843 18770 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:17.061689 18491 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:17.069648 18911 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:23:17.027688 18854 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.070837 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:17.071856 18914 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:23:17.039057 18841 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:17.144248 18841 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:17.144745 18841 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:17.148973 18849 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.150863 18848 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.133903 18770 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.151680 18848 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.153286 18848 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.154776 18770 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:17.088085 18845 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.156149 18845 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.157133 18848 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:16.881994 18838 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.106160 18716 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.159021 18716 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.159614 18716 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.160172 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.161433 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:17.161717 18716 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.161994 13725 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:17.162768 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.132763 18917 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.163724 13720 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:17.164752 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.103248 18149 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.132972 18673 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.166955 18673 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.169801 18673 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:17.169865 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:17.035032 18844 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.163023 13722 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:17.178505 13722 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:17.156760 18845 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.132548 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.137794 18491 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:17.185447 13797 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:17.214232 13794 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 7.
I20260709 12:23:17.179972 18810 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.191655 13722 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.226120 13722 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.217087 18716 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.228060 18917 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.229562 18843 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:17.240326 18917 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.240927 18843 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.244984 18869 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:17.215305 18850 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:23:17.247053 18850 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:17.247414 18850 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:17.250181 13722 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:17.252120 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:17.253086 13797 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.248274 18716 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.231786 18810 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.311797 18717 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.318959 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2681) took 1009 ms. Trace:
I20260709 12:23:17.319442 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2680) took 1019 ms. Trace:
I20260709 12:23:17.320706 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:17.324493 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.319885 13726 rpcz_store.cc:276] 0709 12:23:16.299998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.301888 (+  1890us) service_pool.cc:224] Handling call
0709 12:23:16.306462 (+  4574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.565925 (+259463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.319152 (+753227us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400157cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:17.319365 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535533440}
I20260709 12:23:17.325333 13722 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:17.319454 13718 rpcz_store.cc:276] 0709 12:23:16.309696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.310552 (+   856us) service_pool.cc:224] Handling call
0709 12:23:16.310857 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:16.658868 (+348011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:17.318687 (+659819us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54003bf454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:17.318893 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798910208}
I20260709 12:23:17.323902 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.331436 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.333781 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:17.334772 13722 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:17.338333 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:17.350970 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 95370.9us
W20260709 12:23:17.352289 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.571s (SENT) [suppressed 14 similar messages]
I20260709 12:23:17.352936 13723 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:17.330920 18716 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.306573 18850 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:17.354777 18808 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.665s	user 0.014s	sys 0.000s
I20260709 12:23:17.367939 13643 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:17.377342 18924 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.377595 18844 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.378122 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:17.378216 18844 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.379164 18844 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.378027 18924 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.381901 18924 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.383324 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:17.384524 18849 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:17.384922 18849 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.384995 18924 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:17.385422 18849 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:17.353940 18925 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.385492 18924 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.377220 13797 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:17.388598 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.388854 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:17.389467 13797 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:17.390434 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.390898 13797 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:17.391503 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.392333 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.392771 13797 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:17.393373 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.393818 13797 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:17.395897 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:17.407259 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.386598 18925 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.409256 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.396993 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:17.385730 18862 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.415652 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:17.382119 18850 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.382123 18844 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.420166 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.419782 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:17.394737 18924 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.422977 13797 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
W20260709 12:23:17.425432 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73180.7us
I20260709 12:23:17.436461 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:17.417987 18845 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.387178 18808 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.435374 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:17.439565 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50830.3us
I20260709 12:23:17.422319 18843 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:17.431100 18844 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:17.450407 18928 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:17.450976 18928 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.451496 18928 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:17.455423 18923 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:17.456069 18923 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.456490 18923 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.458406 18916 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:17.438215 18716 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.459228 18916 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:17.459415 18716 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.460497 18916 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:17.462935 18922 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:23:17.466756 18925 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.442943 18843 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.468119 18862 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.468261 18843 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:17.447016 18845 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:17.470064 18845 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.458487 18810 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.473253 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:17.473392 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.474207 13793 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 7: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:17.473978 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.474989 13794 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:17.476943 18927 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:17.442824 18844 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.482952 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:17.483677 13793 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
W20260709 12:23:17.484418 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70823.7us
I20260709 12:23:17.470513 18845 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:17.470468 18716 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.478030 18906 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:17.496636 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:17.501943 13721 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:23:17.496799 18518 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:17.482824 18844 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:17.497949 18808 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:17.514153 18810 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:17.484769 18929 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:17.501696 18716 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:17.522775 18929 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.523002 18898 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.524099 18898 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:17.529045 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.536049 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:17.537182 13797 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:17.531667 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:17.555473 14420 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:17.557466 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:17.558390 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:17.558671 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:17.559253 13646 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:17.516289 18518 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:17.555912 18884 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:17.560410 18518 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:17.572211 18867 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.999s	user 0.010s	sys 0.014s
I20260709 12:23:17.534515 18906 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:17.574512 18930 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:23:17.575757 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:17.597152 18908 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.665s	user 0.004s	sys 0.000s
I20260709 12:23:17.598681 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 755) took 1022 ms. Trace:
I20260709 12:23:17.599165 13800 rpcz_store.cc:276] 0709 12:23:16.576362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.576513 (+   151us) service_pool.cc:224] Handling call
0709 12:23:16.576861 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:16.606644 (+ 29783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:16.606681 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:16.612802 (+  6121us) log.cc:844] Serialized 26 byte log entry
0709 12:23:16.721247 (+108445us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:16.722291 (+  1044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:16.722378 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:17.596367 (+873989us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:17.598334 (+  1967us) raft_consensus.cc:1685] finished
0709 12:23:17.598358 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:17.598604 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21953,"raft.queue_time_us":35886,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":8264320,"thread_start_us":107894,"threads_started":1,"wal-append.queue_time_us":237422}
I20260709 12:23:17.591483 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:17.594075 18936 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:23:17.606365 17735 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:17.606952 17735 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:17.613121 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53500.8us
I20260709 12:23:17.613457 13793 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:23:17.548354 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:17.623124 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 91584.8us
I20260709 12:23:17.624141 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:17.625196 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71074.4us
I20260709 12:23:17.629901 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:17.630906 14416 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:17.632611 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:17.634132 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:17.637895 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:17.608002 17735 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.641637 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:17.642014 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:17.637892 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163297us
I20260709 12:23:17.644131 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:17.645037 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:17.649181 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:17.650002 14424 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:17.645802 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 97574.5us
W20260709 12:23:17.653141 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.659154 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:17.630075 18884 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.647101 18869 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:17.646679 18716 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:23:17.636915 18868 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:17.670328 13797 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:17.671427 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:17.678232 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:17.641467 18848 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:17.683282 14416 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:23:17.666574 18614 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:17.693523 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:17.654510 18927 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:17.700237 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:17.677525 18770 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:23:17.678637 18883 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 12:23:17.722333 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67020.7us
I20260709 12:23:17.726137 18938 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:17.726617 18770 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:17.727240 18770 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:17.724207 18929 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.715147 18942 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.729444 18883 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:17.729151 18942 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.729864 18883 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:17.730950 18883 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.728645 18929 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:17.732270 18883 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:17.714673 18935 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:17.738607 18901 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 12:23:17.740270 18921 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:17.740931 18921 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:17.741876 18921 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.722702 18943 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:17.726712 18938 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:17.681406 18884 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.763350 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:17.697685 18848 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:17.771759 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:17.772950 18745 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.773526 18745 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.780098 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:17.759554 18938 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:17.763381 18935 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:17.728633 18770 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.783090 18645 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:17.826798 18645 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.827900 18848 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.747519 18925 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:23:17.745002 18849 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.780462 18851 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:17.849145 18841 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.007s
I20260709 12:23:17.786744 18906 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.788535 18942 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.856654 18770 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:17.886922 18924 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.889417 18924 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.891227 18928 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.892117 18884 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.895066 18845 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.896148 18935 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.907860 18849 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.908574 18938 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.910531 14420 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:17.866994 18841 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.899435 18844 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.900861 18900 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:17.847831 18927 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.921923 18923 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.917840 18841 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.897907 18845 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:17.958863 18900 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:17.978652 18923 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:17.976272 18843 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.506s	user 0.025s	sys 0.000s
I20260709 12:23:17.991724 18843 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:17.993964 18843 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.995700 18844 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:17.991158 18938 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.055099 18900 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:18.168054 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.000676 18928 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:18.321184 13727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.013s	sys 0.023s
W20260709 12:23:18.323562 13727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.014s	sys 0.023s
I20260709 12:23:18.351229 18745 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.360368 18724 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:18.361117 18724 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:18.366585 18869 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.367291 18869 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.367877 18724 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.369925 18724 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.383495 13796 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
W20260709 12:23:18.442176 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.034s	sys 0.056s
W20260709 12:23:18.442672 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.034s	sys 0.058s
W20260709 12:23:18.448140 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.020s	sys 0.058s
W20260709 12:23:18.449117 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.022s	sys 0.058s
W20260709 12:23:18.449481 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 761687us
I20260709 12:23:18.461118 18938 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.461861 18938 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.464287 18938 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:18.465428 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2146) took 1619 ms (client timeout 1701 ms). Trace:
I20260709 12:23:18.465786 18869 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.468850 14424 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:23:18.465988 13796 rpcz_store.cc:269] 0709 12:23:16.846036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.846221 (+   185us) service_pool.cc:224] Handling call
0709 12:23:18.465173 (+1618952us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:18.465326 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3339351936}
I20260709 12:23:18.472970 18645 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.481752 13646 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:18.484418 13643 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:23:18.488978 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.489408 18869 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.554612 18869 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.555554 17735 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:18.557278 18869 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.529126 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:18.540603 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:18.562075 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 795074us
I20260709 12:23:18.541949 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.562940 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:18.544512 18838 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:18.548413 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.549240 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:18.550395 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:18.552886 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:18.565052 14419 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:18.565290 14423 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19.
I20260709 12:23:18.491032 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.501502 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3058) took 1283 ms. Trace:
I20260709 12:23:18.570816 13645 rpcz_store.cc:276] 0709 12:23:17.217759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.217878 (+   119us) service_pool.cc:224] Handling call
0709 12:23:17.232811 (+ 14933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.301997 (+ 69186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:17.302038 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:17.302088 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:17.302131 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.501119 (+1198988us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b54002b39d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:18.501188 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.501386 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158551936}
I20260709 12:23:18.573678 14419 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
W20260709 12:23:18.574569 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.582221 18852 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.582805 18863 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.582913 18852 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.583600 18863 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.585418 18852 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:18.585877 18863 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.598070 18927 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.598706 18869 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.598917 18927 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.599277 18869 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.601464 18869 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.606190 18928 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.606932 18928 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:18.609460 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.609704 18925 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.610405 18925 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.611326 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.612012 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:18.612790 18925 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.614773 18819 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.615212 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.615305 18819 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.615952 13797 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:18.619304 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.620079 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.620719 13800 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:18.588577 18149 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:18.625715 18149 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:18.626098 18149 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:18.627003 18149 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.628276 18149 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:18.591349 18770 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:18.591805 17735 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:18.595376 13794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.003s	sys 0.003s
W20260709 12:23:18.629722 13794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.004s	sys 0.003s
I20260709 12:23:18.629621 18770 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.630604 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3086) took 1114 ms. Trace:
I20260709 12:23:18.630805 17735 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.631989 18770 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:18.631073 13794 rpcz_store.cc:276] 0709 12:23:17.516132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.516290 (+   158us) service_pool.cc:224] Handling call
0709 12:23:17.516668 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.516928 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:17.658425 (+141497us) log.cc:844] Serialized 14 byte log entry
0709 12:23:18.630172 (+971747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:18.630232 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:18.630330 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.630410 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.630542 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19109,"prepare.queue_time_us":229366,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2003,"raft.queue_time_us":31878,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6345,"thread_start_us":1087787,"threads_started":2}
I20260709 12:23:18.632644 17735 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:18.633446 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.634604 14424 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:18.635232 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.636054 13794 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:18.630601 13796 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:18.638971 18716 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.637971 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:18.644286 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.647500 13797 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:18.640653 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:18.651861 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.601464 18927 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.639560 18716 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.609532 18928 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.655997 18928 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:18.656497 18928 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:18.657390 18928 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.661616 18716 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.668099 18724 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.668843 18724 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.641806 18925 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.670449 18869 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.671443 18883 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.671959 18883 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.675515 18947 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:18.679721 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:18.670845 18925 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.671254 18951 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:18.683620 13800 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:18.688201 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.688764 13800 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:18.693118 13796 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:18.695637 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.693490 13794 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:18.699189 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.700582 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.702692 13794 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:18.694375 18898 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:18.671300 18869 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:18.708976 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68984.3us
I20260709 12:23:18.710072 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:18.691236 18897 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 12:23:18.713008 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71353.6us
I20260709 12:23:18.707657 18898 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:18.712893 18928 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:18.707410 18883 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.715545 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:18.712602 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:18.718116 18968 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:18.718621 18968 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.718968 18968 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:18.719056 13615 leader_election.cc:400] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19.
I20260709 12:23:18.720328 18724 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.720410 18970 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:18.721460 18716 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:18.722018 18716 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:18.722997 18716 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.725449 18951 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.726032 18951 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.716997 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:18.730080 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:18.732179 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:18.732774 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:18.730080 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:18.734117 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:18.738361 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:18.750463 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.752208 18976 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:23:18.752362 18928 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:18.752612 18972 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:18.753445 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.753448 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:18.753544 18898 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.754357 18869 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.754837 14419 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:23:18.755435 18967 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:18.760015 14421 raft_consensus.cc:2368] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20.
I20260709 12:23:18.739941 18724 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:18.754772 18951 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.758279 17735 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.757555 18972 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:18.793335 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:18.759100 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.759696 18898 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:18.763829 18973 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:18.767958 13798 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.008s	sys 0.004s
I20260709 12:23:18.775628 18979 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.777540 13645 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:18.777426 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:18.780555 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:18.796967 13798 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.009s	sys 0.005s
I20260709 12:23:18.739825 18970 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:18.788633 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:18.787050 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:18.790589 17735 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.798934 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:18.736428 18883 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:18.796589 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:18.797302 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:18.782207 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:18.798074 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.799875 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.803179 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.799220 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:18.713753 18844 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:23:18.798492 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 771) took 1323 ms. Trace:
I20260709 12:23:18.811573 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:18.812513 13800 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:18.824201 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.825250 13796 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:18.827987 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.829617 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.831454 13796 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:18.832952 18883 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:18.838056 17735 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.838320 18883 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.832984 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:18.827124 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:18.804450 13798 rpcz_store.cc:276] 0709 12:23:17.475120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.476678 (+  1558us) service_pool.cc:224] Handling call
0709 12:23:17.476976 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:17.477201 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:23:17.648878 (+171677us) log.cc:844] Serialized 14 byte log entry
0709 12:23:18.568348 (+919470us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b18003c3b80. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:18.798118 (+229770us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:18.798176 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:18.798232 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.798321 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:18.798439 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24587,"prepare.queue_time_us":221113,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":566,"raft.queue_time_us":30578,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5396,"spinlock_wait_cycles":61497728,"thread_start_us":1277783,"threads_started":2,"wal-append.queue_time_us":15366,"wal-append.run_cpu_time_us":735,"wal-append.run_wall_time_us":1003187}
I20260709 12:23:18.844401 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.844812 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.845712 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:18.811338 18724 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:18.853826 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.863969 13798 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:18.820838 18869 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:18.866936 13798 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:18.861824 18970 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:18.877614 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.800962 18973 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:18.880524 18973 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.891045 13800 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:18.894665 18724 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:18.811038 18979 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.902382 18724 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.905036 18979 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:18.905681 18979 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:18.911177 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44185.6us
I20260709 12:23:18.906026 18979 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:18.912868 13764 leader_election.cc:400] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20.
I20260709 12:23:18.850073 18716 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:18.882061 18973 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:18.914309 13764 leader_election.cc:403] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:23:18.917657 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 121158us
W20260709 12:23:18.922792 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70546.2us
I20260709 12:23:18.865415 18977 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:23:18.914548 18925 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:18.916924 18979 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:18.922575 18958 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:18.931784 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.930831 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:18.928646 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:18.913968 18770 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:23:18.875975 18928 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:18.940810 18957 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.653s	user 0.002s	sys 0.002s
I20260709 12:23:18.940517 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:18.927354 18938 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:23:18.942356 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:18.921789 18716 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:18.937695 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 110710us
I20260709 12:23:18.959692 18967 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:18.960788 18967 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.961498 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.962251 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:18.936400 18869 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:18.917604 18972 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:18.970002 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44879.4us
I20260709 12:23:19.012742 18958 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20.
I20260709 12:23:19.003213 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:19.025437 13721 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.523s	user 0.010s	sys 0.010s
I20260709 12:23:19.026521 13721 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:18.942849 18977 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:19.027940 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.022063 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:19.029217 13795 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.033030 13800 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:23:19.033390 13800 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 7
I20260709 12:23:19.033591 13800 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:23:19.033808 13800 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:19.033984 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:19.040149 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1048) took 1531 ms (client timeout 1590 ms). Trace:
W20260709 12:23:19.045084 13721 rpcz_store.cc:269] 0709 12:23:17.496094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.496307 (+   213us) service_pool.cc:224] Handling call
0709 12:23:19.027057 (+1530750us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:19.025178 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:19.020542 18901 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:19.018659 18851 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:18.970367 18970 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:19.055691 18970 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.056011 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:19.056128 18970 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.056213 18989 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:19.056681 13793 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:19.046586 18770 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.041589 18993 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:19.039422 18987 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:19.058322 18770 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.985327 18985 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.039179 18863 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:23:19.060143 18985 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.060671 18770 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:18.966322 18978 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.054729 18901 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:19.062577 18978 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.063272 18901 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.064716 18901 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.020637 18976 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:19.065964 18976 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:18.982206 18645 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.055909 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.076359 13648 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:18.968112 18869 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:19.078660 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53564.9us
I20260709 12:23:18.975870 18844 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.101526 18844 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.110571 18990 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:19.111352 18990 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:19.031567 18977 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.111886 18977 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:19.112401 18990 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:19.115098 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.117879 13793 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:19.145972 13795 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:19.121315 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:18.989615 18968 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.115262 18992 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:19.156467 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79172.9us
I20260709 12:23:19.156522 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:19.145901 18928 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:19.165508 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.167606 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.166488 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.086233 18645 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.173897 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:19.179982 18967 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.158466 18989 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:19.162213 18985 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.172099 18925 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:19.180625 18967 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.188294 18982 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:19.215379 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.201817 18716 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.202164 18991 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:19.171245 18992 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.173908 18928 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.166471 18869 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.222817 18925 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:19.296388 18724 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.304831 18851 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:19.305011 18994 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:23:19.305172 18883 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.305500 18851 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.305797 18883 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.307926 18883 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.304992 18724 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.309387 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.310798 18724 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.311384 18724 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:19.315147 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.317840 13793 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:19.320199 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.303700 18990 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:19.325729 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1051) took 1664 ms. Trace:
I20260709 12:23:19.330366 13795 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:19.326153 13727 rpcz_store.cc:276] 0709 12:23:17.661407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.661537 (+   130us) service_pool.cc:224] Handling call
0709 12:23:17.661900 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:18.324064 (+662164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:18.324104 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:18.324743 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:23:18.354866 (+ 30123us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:18.365870 (+ 11004us) log.cc:844] Serialized 14 byte log entry
0709 12:23:18.366439 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:18.368318 (+  1879us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:19.209633 (+841315us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180046a8c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:19.312900 (+103267us) raft_consensus.cc:1685] finished
0709 12:23:19.325499 (+ 12599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:19.325661 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6727,"prepare.queue_time_us":13520,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":18095,"raft.queue_time_us":31550,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":3959,"spinlock_wait_cycles":6252416,"thread_start_us":691549,"threads_started":3,"wal-append.queue_time_us":30621}
W20260709 12:23:19.332207 19002 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:23:19.267135 18982 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.323091 18869 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.312466 18851 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.307962 18968 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.275827 18991 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.325394 18724 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260709 12:23:19.358767 18998 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 12:23:19.372838 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60745.9us
W20260709 12:23:19.374030 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:19.359376 18928 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.360504 18645 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.351606 18925 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.378607 18982 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.381371 18844 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.362504 13648 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:23:19.337430 18990 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.384445 18990 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:19.370173 18991 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.338464 18883 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 12:23:19.366881 18995 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:23:19.366940 19001 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:23:19.341727 18989 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:19.405213 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.415652 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:19.382309 18841 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:19.434244 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.437943 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.440862 13650 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:19.441212 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:19.438935 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.447381 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.448235 13643 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 7.
I20260709 12:23:19.425303 18770 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.450237 13646 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:19.427836 18841 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:19.451582 13647 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:19.451184 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:19.443670 18883 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:19.457409 18770 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.472231 18770 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:19.458424 18851 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:19.474712 18851 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.475154 18851 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:19.485890 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.484242 18997 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:23:19.490980 13797 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:19.491117 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.449676 18989 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.499375 13647 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:19.459227 18967 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.510908 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.485280 18883 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.512569 18997 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.518951 18997 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.523357 18997 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.483732 18992 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.530385 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.537223 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.530779 18883 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.531644 18841 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.547143 18841 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:19.551092 18979 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.616s	user 0.014s	sys 0.000s
I20260709 12:23:19.551607 18979 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.552345 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.557505 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.558202 13650 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:19.560530 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.561292 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 6.
I20260709 12:23:19.541579 18992 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:19.584142 18977 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.590888 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:19.592442 13652 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:19.595777 18970 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.539s	user 0.008s	sys 0.006s
I20260709 12:23:19.596390 18970 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.605068 18970 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.608237 18977 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:19.614197 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57217.1us
I20260709 12:23:19.617551 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:19.618559 13643 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:19.621230 13652 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:19.617532 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:19.622619 18897 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:19.623068 18897 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.623414 18897 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:19.625291 13652 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:23:19.614965 18992 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:19.631489 18978 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.634845 18897 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.642038 13652 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:23:19.648631 18970 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.639254 18897 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:19.661108 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:19.656503 13792 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:19.653633 18997 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.654474 18992 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.657858 13801 raft_consensus.cc:2368] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 8. Current term is 9.
I20260709 12:23:19.654523 18970 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.658460 18977 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.666632 18979 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.665714 18977 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.674566 19023 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:19.675220 19023 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.665189 18992 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.675604 19023 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:19.677342 18992 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.677917 18992 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.680361 18970 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:19.698069 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.739686 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71123.9us
I20260709 12:23:19.740957 19025 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.741642 19025 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.743147 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.743695 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.745039 19025 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.743294 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.746433 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:19.748440 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:19.756816 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.758067 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:19.663859 18997 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.759779 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.759980 13799 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:19.682471 18977 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.765880 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.770726 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3088) took 1159 ms. Trace:
I20260709 12:23:19.760540 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.771232 13799 rpcz_store.cc:276] 0709 12:23:18.611193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.757806 (+1146613us) service_pool.cc:224] Handling call
0709 12:23:19.770655 (+ 12849us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:19.748668 18970 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.775506 18977 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:19.777343 18977 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:19.779306 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115012us
I20260709 12:23:19.738461 18992 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.780050 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.780377 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:19.781005 18977 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:19.781919 18992 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.782451 19026 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:19.783166 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.787281 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.788193 18979 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.791303 18979 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.793048 18977 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:19.793622 18977 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.786319 18970 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.795822 18977 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.796244 18970 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.797079 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:19.797847 18997 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:19.798435 18997 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:19.761384 19025 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:19.788893 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:19.799479 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.799808 19024 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:19.800724 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.803215 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:19.803958 13649 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:19.794075 18979 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:19.805857 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:19.803086 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:19.806422 19024 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:19.843513 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.879750 13801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2147) took 3031 ms (client timeout 1675 ms). Trace:
W20260709 12:23:19.818156 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.881023 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3072) took 2339 ms (client timeout 2120 ms). Trace:
I20260709 12:23:19.876209 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2111) took 1098 ms. Trace:
W20260709 12:23:19.823568 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.115s	user 0.007s	sys 0.016s
I20260709 12:23:19.817724 13643 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:19.859025 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:19.925920 13646 rpcz_store.cc:276] 0709 12:23:18.777876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.778003 (+   127us) service_pool.cc:224] Handling call
0709 12:23:18.778335 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:19.453398 (+675063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:19.875711 (+422313us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:19.876037 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544328576}
W20260709 12:23:19.925101 13651 rpcz_store.cc:269] 0709 12:23:17.542036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.542299 (+   263us) service_pool.cc:224] Handling call
0709 12:23:19.880789 (+2338490us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54001b6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:19.880917 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4644970240}
W20260709 12:23:19.923051 13801 rpcz_store.cc:269] 0709 12:23:16.848061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:16.848341 (+   280us) service_pool.cc:224] Handling call
0709 12:23:19.879472 (+3031131us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:19.879626 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6281445760}
W20260709 12:23:19.929337 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:19.929772 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.221s	user 0.009s	sys 0.023s
W20260709 12:23:19.946794 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 2.25318e+06us
I20260709 12:23:19.949429 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:19.857627 18996 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.706s	user 0.000s	sys 0.003s
W20260709 12:23:19.955441 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 152430us
I20260709 12:23:19.885671 18977 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:19.958065 18977 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.818166 18842 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:19.826449 18992 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:19.818478 19025 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:19.886010 18997 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.889706 18976 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.894392 18970 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.889290 18724 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:19.925324 19024 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:19.989125 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:19.984254 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 178510us
I20260709 12:23:19.959506 18897 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:19.842376 19023 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.970197 18979 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:19.990483 18897 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:19.963375 18842 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:19.992991 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:19.997908 13646 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.004643 19023 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.006305 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:20.006724 19030 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:20.007254 19030 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.009495 19030 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.010535 19032 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.011116 19032 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.013329 19032 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.010098 19030 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:19.989208 18976 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.001533 18842 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:20.019021 18976 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.019652 18976 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:20.020203 18976 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.970867 18992 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.991051 18970 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:19.984577 18997 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.023777 19025 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:19.994165 18724 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.024997 19030 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.025506 19030 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:20.001823 18897 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:20.016531 19027 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.029462 19027 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.046118 19025 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.046762 18951 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.046847 19025 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.047559 18951 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.049597 18951 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:19.567903 18770 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.462s
I20260709 12:23:19.999321 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:20.114351 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 125310us
I20260709 12:23:20.121243 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:20.123490 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 126977us
W20260709 12:23:20.115001 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 182491us
I20260709 12:23:20.115355 13646 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:20.136152 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:20.135900 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:20.143926 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.145402 13646 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:20.147915 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:20.119457 19004 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:20.136489 18851 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.661s	user 0.016s	sys 0.007s
I20260709 12:23:20.133810 19031 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.166033 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.140678 18972 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.173431 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.161371 19006 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:20.155373 19026 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:20.155470 18836 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.183688 19035 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:20.169970 18978 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.209087 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:20.193778 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:20.195636 19019 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.563s	user 0.000s	sys 0.005s
I20260709 12:23:20.209757 18972 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:20.203718 18836 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.226210 18836 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.218626 18979 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:20.171116 18851 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.208946 19026 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:20.176338 19031 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.238945 19024 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.206027 18842 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.249398 18897 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.256798 19026 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.258292 18897 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.251874 18977 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.253810 18724 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.256072 18842 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.267119 18851 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.265285 19030 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.276038 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.278705 13643 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:23:20.262544 18883 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.718s	user 0.016s	sys 0.000s
I20260709 12:23:20.288167 18883 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.292435 18883 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:20.292541 14424 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 804) took 1753 ms (client timeout 2031 ms). Trace:
I20260709 12:23:20.292711 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.293529 13643 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:23:20.293246 14424 rpcz_store.cc:269] 0709 12:23:18.539464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:18.540277 (+   813us) service_pool.cc:224] Handling call
0709 12:23:20.292315 (+1752038us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:20.292442 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165452032}
I20260709 12:23:20.305279 14416 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:20.316148 13723 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:20.320960 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.319290 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.372613 13724 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:20.339371 13718 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:23:20.345423 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.326093 13722 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:23:20.348505 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.322041 13719 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:20.374958 13726 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:20.373636 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:20.378134 13725 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:20.331220 19031 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.382539 14422 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 12:23:20.361008 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.335479 17735 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.386222 17735 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.191767 13646 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:20.330224 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:20.393139 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63071.5us
I20260709 12:23:20.395123 19021 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.401554 18976 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.405304 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.406418 18900 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.407102 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.429167 13721 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:20.429651 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.431321 13727 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:20.371862 18842 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.439384 18842 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.392045 19030 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.445384 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:20.423224 18841 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.875s	user 0.016s	sys 0.002s
I20260709 12:23:20.447630 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:20.445406 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:20.414680 13643 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:23:20.415647 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.451014 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.426297 13725 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:20.426697 17735 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.346890 18883 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.286391 19024 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.371537 18851 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.410678 18927 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:20.432971 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.434655 18900 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.428349 19021 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.483485 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:20.512653 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.437343 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:20.438724 19038 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:23:20.451925 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.452932 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.454579 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.519129 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.520017 13644 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.456121 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.520475 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.521029 13797 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:20.521622 13799 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:20.521804 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:20.460465 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.522461 13725 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:20.522851 13793 raft_consensus.cc:2368] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:20.523980 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.525422 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.461431 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.448010 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:20.526746 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78957.9us
I20260709 12:23:20.527101 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.527861 18898 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:20.528177 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:20.528311 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:20.529310 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:20.451696 13727 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:20.465031 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.530566 18770 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:20.471850 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.510756 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.511369 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:20.518620 13800 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:20.463603 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.514914 18819 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.528386 18898 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.529949 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:20.531232 18770 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:20.446815 18979 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:23:20.463627 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.532160 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.536863 13693 leader_election.cc:400] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:20.539644 13725 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:20.590162 13693 leader_election.cc:403] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:20.540729 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.482785 19006 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.451916 19030 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:20.613293 13643 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:20.584090 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54217.4us
I20260709 12:23:20.589915 13723 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:20.611877 18752 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.599787 17735 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.616521 18752 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.617295 17735 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.618897 18752 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:20.606232 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:20.621701 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84977.5us
I20260709 12:23:20.627007 13723 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:20.464509 19031 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.446501 19035 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:20.609650 13644 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:20.587723 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.532135 19026 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.634773 19039 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:20.636381 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.637754 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:20.717362 13796 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:20.638723 19021 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.640093 13646 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:20.642830 13793 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:20.483358 18842 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.644446 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.723704 13651 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:20.634912 17735 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.647301 13723 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:20.645494 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:20.650092 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:20.643467 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:20.480199 18897 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.651136 18752 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.731405 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:20.654443 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08003188a0 after lost signal to thread 12472
W20260709 12:23:20.733206 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800317ea0 after lost signal to thread 13584
W20260709 12:23:20.734973 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800317de0 after lost signal to thread 13676
W20260709 12:23:20.735316 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800317e00 after lost signal to thread 13677
I20260709 12:23:20.512193 18851 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.737960 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:20.738768 13796 raft_consensus.cc:606] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:20.739130 13796 raft_consensus.cc:614] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:20.740053 18851 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.725754 19026 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.740700 18752 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.743328 18752 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.620177 18841 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:20.746691 18841 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:20.747779 18841 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.733949 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:20.655767 18770 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.663354 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.666710 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.754098 13798 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:23:20.754217 13643 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:23:20.665403 13801 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:20.757103 13643 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:20.757483 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:20.667542 14420 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.004s	sys 0.007s
I20260709 12:23:20.668499 13647 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:20.675776 19025 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.583842 18898 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.680613 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.762156 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45dc2f34593649778ec4b80d29638572"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:20.762872 13801 raft_consensus.cc:606] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:20.681824 18900 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.763223 13801 raft_consensus.cc:614] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:20.684670 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.764348 13645 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:20.714906 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.700742 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:20.725229 13799 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:20.726444 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81009.4us
I20260709 12:23:20.484184 18883 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.730531 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.769438 13725 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:20.690227 19031 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.770627 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:20.771598 19031 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.772709 19027 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:20.773444 13721 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:23:20.731405 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:20.776197 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.737633 13651 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
W20260709 12:23:20.738158 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08001bfd40 after lost signal to thread 13751
W20260709 12:23:20.779819 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800107980 after lost signal to thread 13752
W20260709 12:23:20.781652 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800297dc0 after lost signal to thread 13825
I20260709 12:23:20.730732 18979 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:20.829691 18979 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:20.830230 18979 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:20.830601 18970 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:20.833376 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:20.758294 14420 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.186s	user 0.005s	sys 0.007s
W20260709 12:23:20.834514 14390 connection.cc:256] RPC call timeout handler was delayed by -1.050171792999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
I20260709 12:23:20.673758 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:20.699012 19028 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.835343 13719 raft_consensus.cc:2376] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10. Re-sending same reply.
I20260709 12:23:20.835405 19028 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.836611 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.837260 13719 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:20.837808 19028 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.783434 17735 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.732885 18897 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.784911 18752 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.839342 18752 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.840551 18897 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.841504 18752 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.786931 19035 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:20.842067 18518 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:20.844436 18819 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:20.788551 19026 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.845153 18819 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:20.789760 19044 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.263s	user 0.003s	sys 0.002s
I20260709 12:23:20.789233 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:20.847028 13801 raft_consensus.cc:606] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:20.792018 18951 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:20.848784 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:20.849900 19042 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:23:20.852612 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.855020 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.793304 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:20.805156 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3072) took 3736 ms (client timeout 1608 ms). Trace:
I20260709 12:23:20.806182 13800 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:23:20.858425 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:20.811844 18898 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.813181 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.789705 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:20.858969 13792 rpcz_store.cc:269] 0709 12:23:17.068705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:17.070542 (+  1837us) service_pool.cc:224] Handling call
0709 12:23:20.804864 (+3734322us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:20.805051 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5737700224}
I20260709 12:23:20.815770 13799 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:20.814082 13725 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:23:20.861565 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68359.5us
I20260709 12:23:20.862548 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.825840 18978 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.821870 19039 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:20.838541 17735 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:20.782114 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5ca0 after lost signal to thread 13826
I20260709 12:23:20.855352 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:20.886732 14419 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:20.856019 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:20.887696 13822 debug-util.cc:398] Leaking SignalData structure 0x7b080038d440 after lost signal to thread 14449
I20260709 12:23:20.887456 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:20.855968 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:20.888007 13822 debug-util.cc:398] Leaking SignalData structure 0x7b080033f0e0 after lost signal to thread 14450
I20260709 12:23:20.888270 13801 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:20.857139 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79171.9us
I20260709 12:23:20.838140 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.811547 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.894819 13724 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:20.822819 18883 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.863813 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.896788 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:20.897610 13722 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:20.897998 18997 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.865013 19006 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:20.899549 19006 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:20.899997 19006 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:20.864884 18958 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:20.896715 18992 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.792475 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.906585 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:20.870757 13725 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:20.878407 13648 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:20.908205 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.908895 13725 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:23:20.879871 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 90220.4us
I20260709 12:23:20.913218 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.913951 13647 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:23:20.868525 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:20.914721 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.915617 13651 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:20.916746 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:20.919878 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:20.888504 13721 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:20.921578 18958 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:20.922068 18958 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.922449 18958 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:20.880842 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:20.925573 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.921622 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.927323 13721 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:20.928525 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.929239 13721 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:20.931370 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:20.932291 18752 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:20.932433 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:20.932718 18752 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.933089 18752 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:20.934285 18841 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:20.934698 18841 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:20.935096 18841 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:20.942142 18752 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:20.944583 18752 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:20.930014 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.967206 13721 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:20.968264 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.969167 13721 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:20.884394 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:20.972716 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:20.973528 13646 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:20.985203 13646 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:20.996547 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:20.992772 13722 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:20.993935 13723 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:21.007215 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.012492 13722 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:21.018723 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.024528 17735 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:20.993777 19028 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:21.026485 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:21.024284 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.033490 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.035003 13722 raft_consensus.cc:2368] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:23:21.035840 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.036188 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.001391 18897 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:21.049149 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:20.885691 18928 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.143s
I20260709 12:23:21.058053 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:21.059547 13722 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:20.993854 18978 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.064918 18978 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:21.066107 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 355593us
I20260709 12:23:21.067181 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.069288 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.070463 13722 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:21.073837 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.074302 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.003937 18927 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:23:21.045447 18841 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.053728 18900 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:21.088202 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:21.095754 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 256153us
I20260709 12:23:21.098609 13722 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:21.058704 18897 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:21.099558 13793 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:21.099931 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.107221 13793 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:21.109493 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:21.111522 13720 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:21.112509 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 244330us
I20260709 12:23:21.115612 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.116074 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.117169 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.118067 13794 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:23:21.118655 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.079902 18928 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:23:21.121457 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:21.105896 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.123441 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:21.124562 13722 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:21.126919 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.095110 18900 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.133064 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.134135 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.134320 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.134649 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.134864 13647 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:21.135471 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.135715 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.131109 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.131266 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.139050 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.139050 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.139803 13722 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:21.098445 19034 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.140766 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:21.147580 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 152605us
I20260709 12:23:21.139803 13652 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:21.142679 17735 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.151407 17735 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.141237 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.144162 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.153139 13649 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:21.144096 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.145095 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:21.149487 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:21.161777 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:21.163945 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.164649 13724 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:21.167622 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.132491 19026 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:21.172802 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 288447us
I20260709 12:23:21.169802 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.140831 18900 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:21.172480 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.169818 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:21.176782 13648 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:21.178028 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.180830 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.181769 13724 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 18 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:23:21.182353 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.175014 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.187681 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.192288 18819 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.166769 18978 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:21.193734 18978 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:21.194561 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.195324 13720 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:21.199373 13724 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:21.200467 13652 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:23:21.194931 19035 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.207489 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:21.209429 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.233783 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 819) took 1650 ms. Trace:
W20260709 12:23:21.232432 14391 connection.cc:256] RPC call timeout handler was delayed by -1.8705504630001997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2469s for a response
I20260709 12:23:21.186628 18898 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:21.239588 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.240322 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:23:21.160068 19034 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:21.236120 14391 connection.cc:256] RPC call timeout handler was delayed by -1.6883475120002913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
I20260709 12:23:21.261966 13724 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:21.200659 18977 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.200125 19039 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.266772 13648 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:21.266355 13793 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:21.271808 13724 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:21.272716 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:21.274925 14419 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:23:21.275835 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.276489 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.285036 13652 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:23:21.184893 18951 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
W20260709 12:23:21.279510 14391 connection.cc:256] RPC call timeout handler was delayed by -1.5983475120001458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 12:23:21.291827 14391 connection.cc:256] RPC call timeout handler was delayed by -1.5299475120000352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260709 12:23:21.292328 14391 connection.cc:256] RPC call timeout handler was delayed by -1.2372578170002271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2686s for a response
W20260709 12:23:21.292651 14391 connection.cc:256] RPC call timeout handler was delayed by -1.0261382070002583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2687s for a response
I20260709 12:23:21.289067 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:21.234372 14420 rpcz_store.cc:276] 0709 12:23:19.563741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:19.564024 (+   283us) service_pool.cc:224] Handling call
0709 12:23:19.564433 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:19.564740 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:23:20.834607 (+1269867us) log.cc:844] Serialized 14 byte log entry
0709 12:23:21.130632 (+296025us) log.cc:844] Serialized 14 byte log entry
0709 12:23:21.213092 (+ 82460us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:21.213174 (+    82us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:23:21.213266 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:21.213365 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:21.213543 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11208,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":98,"raft.queue_time_us":124873,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":158,"thread_start_us":1627141,"threads_started":3,"wal-append.queue_time_us":366126}
I20260709 12:23:21.286293 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:21.231732 18819 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:21.296747 13721 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:21.297315 13722 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:21.297489 13723 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:21.186091 19026 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.303189 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:21.267251 18928 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:21.206199 18958 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:21.322332 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 140113us
I20260709 12:23:21.204957 19035 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.331277 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:21.323602 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 153854us
I20260709 12:23:21.332723 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:21.334362 19055 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:23:21.334874 18992 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:21.341369 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.344106 13720 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:21.231812 19006 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.316327 18970 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:21.346134 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.203020 18897 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.356336 13801 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:21.200945 18978 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.321440 18928 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.358999 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:21.359637 13793 raft_consensus.cc:606] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.360405 13726 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:23:21.286554 18977 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.358567 13800 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:21.298282 18898 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:23:21.293107 19039 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.378415 19039 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.358891 18841 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:21.350767 18518 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.006s
I20260709 12:23:21.367822 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:21.363916 18928 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.320046 18819 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.382248 19057 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:21.387554 13721 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:21.388404 19057 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:21.389129 13721 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:23:21.389482 13721 pending_rounds.cc:85] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:23:21.389724 13721 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:23:21.389864 19057 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:23:21.378512 19062 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:21.395704 19062 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:21.345656 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 162707us
I20260709 12:23:21.389982 13721 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:21.375149 13726 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:21.369140 13720 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:23:21.384223 13795 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
W20260709 12:23:21.390583 18979 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.004s
I20260709 12:23:21.396183 19062 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.379341 19069 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:21.404486 13727 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:23:21.393558 19043 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:23:21.400033 18901 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.413508 18901 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.414676 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:21.367110 18992 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:21.422125 17735 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:21.410838 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:21.359992 19061 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:21.431258 13720 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:21.382635 18841 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:21.435667 18997 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.384029 18958 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:21.513628 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 220354us
I20260709 12:23:21.414774 19026 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:21.430106 18992 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:21.519199 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73492.9us
W20260709 12:23:21.487327 19050 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.339s	user 0.002s	sys 0.000s
I20260709 12:23:21.403534 19056 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:21.354663 19059 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.178s
I20260709 12:23:21.404004 18518 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:23:21.461158 18970 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.422967 18979 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.400113 19060 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.125s
W20260709 12:23:21.517962 19051 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:23:21.443125 18841 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:21.561904 19063 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:21.582634 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 249744us
I20260709 12:23:21.566473 18992 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.508994 18997 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.506896 19064 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.093s
W20260709 12:23:21.583741 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226686us
I20260709 12:23:21.590029 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:21.582984 19063 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:21.574529 19056 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:21.584976 19054 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.345s	user 0.001s	sys 0.001s
I20260709 12:23:21.594964 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:21.587576 19052 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 12:23:21.597707 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:21.576385 18970 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:21.602200 13645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.003s	sys 0.019s
W20260709 12:23:21.602592 13645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.003s	sys 0.019s
I20260709 12:23:21.605929 18901 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:21.566445 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47046.7us
I20260709 12:23:21.612967 19023 raft_consensus.cc:993] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:21.614835 19035 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.617727 18997 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.591872 18883 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:21.626667 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 151349us
I20260709 12:23:21.623898 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:21.594121 19034 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.594492 19063 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:21.600047 18819 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:21.601691 19027 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:21.633813 18951 raft_consensus.cc:1081] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:21.658622 19077 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:21.665977 19023 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:21.665714 18977 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.680402 19023 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:21.680977 19023 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.680694 19056 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.687107 18979 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.669113 19006 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:21.626041 18841 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.694818 18928 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.695576 18841 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:21.693130 19077 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.698200 19077 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:21.699672 18978 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:21.700757 18928 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:21.739162 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 100841us
W20260709 12:23:21.582377 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 241086us
I20260709 12:23:21.750053 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:21.751456 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:21.754954 13548 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 15, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 15 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:21.759323 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135405us
W20260709 12:23:21.762776 19047 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.180s	user 0.003s	sys 0.001s
W20260709 12:23:21.765784 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 172995us
I20260709 12:23:21.768224 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:21.775128 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:21.785115 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:21.789569 18970 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.789614 18898 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:21.790863 19073 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:21.791378 18898 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.791962 18898 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:21.794551 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.007s	sys 0.006s
W20260709 12:23:21.795284 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.007s	sys 0.006s
I20260709 12:23:21.804162 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:21.806020 19064 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:21.806493 19006 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:21.807128 19006 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:21.807634 19006 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:21.803906 18979 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:23:21.816766 19072 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:21.818022 18819 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:21.818370 19084 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:21.819197 18819 consensus_queue.cc:1243] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:21.821309 19084 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:21.821784 19084 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:23:21.841676 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 243897us
I20260709 12:23:21.843770 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:21.846432 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.846894 19086 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:21.846522 18970 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:21.851219 18970 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.851658 13548 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:21.852864 19072 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:21.853612 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:21.857836 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68574.8us
I20260709 12:23:21.862329 19056 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:21.862993 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102105us
I20260709 12:23:21.863765 19086 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:21.864425 13617 leader_election.cc:400] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
W20260709 12:23:21.876971 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:21.878528 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:21.879545 13720 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:21.879793 13726 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:21.881081 19069 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:21.881635 19069 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:21.883318 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:21.883090 19069 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.856834 19084 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.885473 19084 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:21.886287 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.887383 13645 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:23:21.891021 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:21.891577 18928 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:21.893229 18977 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:21.896967 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.897050 18977 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:21.900008 18977 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.913282 18819 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:21.913893 18819 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:21.914225 19087 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:23:21.919103 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.920053 13648 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:21.921931 18970 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:21.923650 19086 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:21.924945 19086 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:21.925359 19086 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:21.926033 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:21.927601 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:21.928553 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:21.930507 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 290109us
I20260709 12:23:21.930845 19087 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:21.931455 19087 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:21.931972 19087 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:21.933537 13644 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:23:21.934695 18900 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.735s	user 0.013s	sys 0.004s
I20260709 12:23:21.935235 18900 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.935811 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:21.937160 14421 raft_consensus.cc:2368] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10.
I20260709 12:23:21.937083 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:21.938633 18883 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:21.940747 18883 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:21.941138 18883 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.941529 18883 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:21.942399 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:21.942749 19027 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:21.943259 19027 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:21.943620 18852 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:21.944463 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:21.944936 18852 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:21.946730 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:21.943634 19027 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:21.947889 18970 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:21.949112 18970 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:21.949486 18970 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:21.949805 19069 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:21.942098 13615 leader_election.cc:400] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10.
I20260709 12:23:21.950598 13615 leader_election.cc:403] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:21.929937 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:21.969467 19089 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:21.973179 19088 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:21.973891 19088 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:21.974390 19088 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:21.978963 13548 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 11 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:21.980849 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47146.7us
I20260709 12:23:21.980233 18900 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:21.983163 13720 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:21.983702 13795 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:21.985122 18852 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:21.986243 13801 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:21.988432 13794 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: [14.14-14.14]
I20260709 12:23:21.989013 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:21.992334 13793 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:21.993907 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:21.999020 19089 consensus_queue.cc:1243] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 15.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:23:21.999742 19089 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:22.000095 19089 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:23:22.001739 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60068.1us
I20260709 12:23:22.002632 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.004766 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.006474 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.007081 18979 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.006574 19072 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:22.009464 19072 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:22.010697 19072 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.014073 14421 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:22.014767 14421 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:22.016201 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:22.018535 19090 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 12:23:21.999037 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:22.043370 14421 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.044694 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.045147 14421 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.020289 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.046584 13644 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
W20260709 12:23:22.020406 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137280us
I20260709 12:23:21.983729 13548 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 13, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 13 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.021060 19086 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.027329 19021 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:22.026872 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97171.3us
I20260709 12:23:22.030306 18972 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:22.030913 19064 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:22.031704 18970 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.039106 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:22.019022 18819 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.048657 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.001415 19089 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.054641 13645 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:22.066061 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:21.973893 19062 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.008s
I20260709 12:23:22.079967 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.090898 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.099678 19006 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.101338 13646 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:22.103964 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:22.104290 18883 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.056296 19021 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:22.109334 13720 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:22.097543 18979 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10.
I20260709 12:23:22.106400 19062 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.118659 19061 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:22.119266 18928 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:22.119359 19023 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.122567 19078 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:22.124039 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:22.124282 19074 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 12:23:22.127135 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88209.7us
I20260709 12:23:22.128162 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.129777 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.130801 18970 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:22.132594 18992 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.011s
I20260709 12:23:22.133095 18992 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.130909 19085 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.135355 18898 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.136665 18819 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.137629 18898 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:22.138891 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90566.5us
I20260709 12:23:22.139019 19006 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.139842 19006 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:22.143106 18883 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:22.144263 19058 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:22.145592 19058 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:22.145911 19058 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:22.146584 14419 raft_consensus.cc:2368] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12.
I20260709 12:23:22.147178 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.149075 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.152485 19023 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:22.154909 18997 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:22.155673 18997 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.154955 13548 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.156056 18997 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:22.146210 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.159548 13548 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 11, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 11 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.161955 13548 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.117945 19062 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:22.166163 13799 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:22.167815 14421 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:22.167821 13648 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:22.173784 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45343.8us
I20260709 12:23:22.164669 13548 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 11, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 11 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.169489 19021 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.181789 18992 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.101894 19077 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.187269 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.190578 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.109464 18900 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:23:22.137537 19027 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.185854 19056 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.214725 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.201984 18898 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:22.194195 18977 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:22.217859 13643 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
W20260709 12:23:22.224397 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.229444 13652 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:22.215520 18992 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.232884 19076 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.234477 19039 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:22.235144 19039 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.237686 19039 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.238411 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.239880 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.240768 13647 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.241119 18977 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.243472 18977 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.244005 18841 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:23:22.245865 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:22.250435 19067 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:23:22.259429 13646 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:23:22.259236 13548 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:23:22.249925 19086 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.233632 19076 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:22.272632 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 143010us
I20260709 12:23:22.213279 18972 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:22.219242 18898 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:22.220636 19035 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:22.280102 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104610us
I20260709 12:23:22.290403 13548 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.292393 13548 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:23:22.295782 13544 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 231) took 24833 ms (client timeout 15000 ms). Trace:
W20260709 12:23:22.298408 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.298473 13647 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:22.283437 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:22.216980 19084 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:23:22.293733 13548 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 10, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 10 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.315117 13615 leader_election.cc:400] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12.
I20260709 12:23:22.315476 13615 leader_election.cc:403] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:22.317340 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:22.349628 13644 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:22.284807 18898 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:23:22.356145 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:22.361147 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 90512.1us
W20260709 12:23:22.357592 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78265.6us
I20260709 12:23:22.312901 18979 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:22.358517 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:22.314401 18841 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:22.377403 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 171735us
I20260709 12:23:22.378991 13617 leader_election.cc:400] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:23:22.379359 13617 leader_election.cc:403] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:22.379968 13548 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 12, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 12 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.382325 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:22.386827 18883 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:22.387969 18883 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.389395 19077 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:22.390281 19077 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:22.390852 19077 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:22.391261 18883 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.392035 19077 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.392724 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:22.393558 13720 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:22.386875 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69324.3us
I20260709 12:23:22.225101 19058 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:22.411862 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.350668 19088 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:22.435469 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 220484us
I20260709 12:23:22.426707 13548 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.359057 19035 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:22.441967 19094 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
W20260709 12:23:22.442454 18978 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.742s	user 0.007s	sys 0.012s
I20260709 12:23:22.391300 19076 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:22.423842 19087 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:22.449626 19063 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.011s
I20260709 12:23:22.399050 19039 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:22.395324 18992 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.451606 19027 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.397300 19056 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:22.438118 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72878.1us
I20260709 12:23:22.384442 18977 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:23:22.487386 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.487924 13649 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:23:22.426360 18979 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:22.483737 13720 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
W20260709 12:23:22.435333 13690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.525s (SENT) [suppressed 30 similar messages]
W20260709 12:23:22.492020 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56722.3us
W20260709 12:23:22.474951 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43488.3us
I20260709 12:23:22.456056 13548 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.496583 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.471724 18972 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:23:22.483691 19039 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.499238 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.496843 13548 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.505262 13652 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:22.505808 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.503315 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.484166 19088 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:22.510794 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.513614 13643 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:22.492502 19085 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:22.528354 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.532440 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.508417 19039 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:22.535267 14421 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:22.537736 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 113717us
I20260709 12:23:22.490940 19063 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:22.542353 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.545939 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:22.543520 13548 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 2 to 10, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.557677 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:22.563321 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.566210 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.566478 19072 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:22.566673 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.567445 14422 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:22.572803 19006 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:22.578790 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.579586 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:22.557650 13548 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 9, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 9 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.584211 13792 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:22.296341 13544 rpcz_store.cc:269] 0709 12:22:57.461877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:57.462153 (+   276us) service_pool.cc:224] Handling call
0709 12:22:58.347669 (+885516us) spinlock_profiling.cc:243] Waited 98 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:23:22.295464 (+23947795us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:16.066455 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:16.174851 (+108396us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:16.184675 (+  9824us) write_op.cc:435] Acquired schema lock
0709 12:23:16.188721 (+  4046us) tablet.cc:662] Decoding operations
0709 12:23:16.225127 (+ 36406us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:16.225281 (+   154us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:16.225328 (+    47us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:23:16.437960 (+212632us) tablet.cc:701] Row locks acquired
0709 12:23:16.443532 (+  5572us) write_op.cc:260] PREPARE: finished
0709 12:23:16.443722 (+   190us) write_op.cc:270] Start()
0709 12:23:16.443885 (+   163us) write_op.cc:276] Timestamp: P: 1783599796443649 usec, L: 0
0709 12:23:16.443920 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:23:16.444712 (+   792us) log.cc:844] Serialized 4736 byte log entry
0709 12:23:17.030811 (+586099us) op_driver.cc:464] REPLICATION: finished
0709 12:23:17.367405 (+336594us) write_op.cc:301] APPLY: starting
0709 12:23:17.377865 (+ 10460us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:17.577780 (+199915us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:17.577906 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:17.835071 (+257165us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:17.874584 (+ 39513us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:23:17.874668 (+    84us) write_op.cc:312] APPLY: finished
0709 12:23:17.875357 (+   689us) log.cc:844] Serialized 113 byte log entry
0709 12:23:17.875675 (+   318us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:17.875945 (+   270us) write_op.cc:454] Released schema lock
0709 12:23:17.876052 (+   107us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:17.876692 (+   640us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:17.876737 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1340769,"reported_tablets":50,"spinlock_wait_cycles":225366272,"child_traces":[["op",{"apply.queue_time_us":320984,"apply.run_cpu_time_us":24033,"apply.run_wall_time_us":514878,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54383,"num_ops":12,"prepare.queue_time_us":176229,"prepare.run_cpu_time_us":10963,"prepare.run_wall_time_us":587635,"raft.queue_time_us":229445,"raft.run_cpu_time_us":12593,"raft.run_wall_time_us":246516,"replication_time_us":586703,"thread_start_us":463813,"threads_started":4,"wal-append.queue_time_us":250588,"wal-append.run_cpu_time_us":17466,"wal-append.run_wall_time_us":2168826}]]}
I20260709 12:23:22.542825 19068 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:23:22.587522 13793 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:23:22.589442 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:22.591326 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.595377 19089 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.602180 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.587569 18979 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.604948 18852 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:22.605870 18852 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:22.617141 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.616739 13548 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.620510 18852 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.624595 19087 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:22.626368 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.651487 13548 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 15, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 15 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.654846 19108 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:22.655810 19108 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.658882 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:22.659225 18897 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:22.659504 18992 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:22.659891 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152276us
I20260709 12:23:22.660902 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.661598 19035 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:22.662379 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.666440 19109 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.667618 19109 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:22.668450 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77885.4us
W20260709 12:23:22.672751 18997 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.001s
I20260709 12:23:22.673305 18997 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.674202 18752 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:22.674799 18752 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:22.675158 19106 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:23:22.675311 18752 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:22.675949 18997 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:22.679191 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54072.7us
I20260709 12:23:22.679896 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:22.677534 19068 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:22.689692 13548 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.690889 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:22.659004 19028 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:22.667657 19063 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:22.700773 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.700332 19087 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:22.702899 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:22.702425 13548 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 11, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 11 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.705411 14419 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:22.714663 13617 leader_election.cc:400] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:22.715101 13617 leader_election.cc:403] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 12:23:22.718360 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.720243 19006 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:22.720815 19006 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:22.707751 19021 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.000s
I20260709 12:23:22.722358 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:22.727131 19006 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:22.733107 19113 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.107s	user 0.002s	sys 0.000s
I20260709 12:23:22.734586 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.735474 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.736335 18978 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:22.746677 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:22.747990 19064 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:22.748967 19064 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:22.750583 19071 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:22.750943 18897 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.751195 19071 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:22.751480 18897 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:22.751678 19071 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:22.753607 14424 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:22.689885 18972 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:23:22.720332 18951 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 12:23:22.727327 18997 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:22.757009 18997 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:22.758165 18997 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.759404 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:22.761091 19078 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:22.761583 19078 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:22.762567 19078 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.763782 18972 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:22.764216 18972 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:22.764284 19058 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:22.759526 19006 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:22.767194 18897 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.757161 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:22.769969 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47737.5us
I20260709 12:23:22.775307 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.776369 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.776793 13548 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 10, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 10 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.778565 13548 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 15, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 15 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.809016 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.780611 13548 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.810056 13647 raft_consensus.cc:2368] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:23:22.809154 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:22.823479 13720 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:22.823568 13548 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 10, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:23:22.826346 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50071.1us
I20260709 12:23:22.834393 13643 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:22.837065 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:22.844467 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.788589 19074 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:22.848634 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:22.777232 19108 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:22.765527 18972 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.850605 13646 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:22.852231 19087 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:22.860611 19087 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.861119 19039 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:22.846040 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:22.840445 18951 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:22.864913 19074 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:22.866426 19074 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.770180 19021 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:22.870536 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 113953us
W20260709 12:23:22.873649 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.870774 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:22.843623 18852 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.854223 13548 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 18, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 18 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:22.885903 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:22.889082 13548 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 13, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 13 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:22.897806 13548 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 12 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:22.901486 13793 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: [14.14-14.14]
W20260709 12:23:22.903788 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.905827 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:22.908741 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:22.916946 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:22.917697 13726 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:22.928812 13548 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:22.936131 13720 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:22.933838 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.939708 13767 leader_election.cc:400] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:23:22.940397 13767 leader_election.cc:403] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:22.876654 18898 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.923967 18972 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:22.949860 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:22.958432 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.918359 19063 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:23:22.900305 18852 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:22.949400 19076 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.960178 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:22.950217 19070 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:23:22.961355 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 274468us
I20260709 12:23:22.951025 18951 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:22.960960 13548 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:23:23.011087 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51005.7us
I20260709 12:23:23.013669 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:23.016765 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:23.020766 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.020987 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.022563 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.023821 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:23.024766 19117 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:23:23.025017 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.025646 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.026806 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.026929 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.027464 14417 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:23.028141 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.030213 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.030714 19123 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:23.031934 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:23.033461 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61595.5us
I20260709 12:23:23.034974 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.035562 13800 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:23.035887 18852 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:23.033177 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.036288 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.009999 19070 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:23.037245 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.038887 19070 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.040526 19070 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:23.021044 18951 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.030321 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.022032 13548 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:23:23.023995 19120 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:23:23.048102 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:23.053522 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.057462 14419 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
W20260709 12:23:23.066494 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49797.5us
W20260709 12:23:23.062388 19094 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:23:23.069795 14418 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:23.025645 14425 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:23.080888 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.085651 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.086818 13548 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:23.009915 19069 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:23.092113 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.061535 14422 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:23.090533 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.069357 18979 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:23.102495 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:22.980118 19115 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:23.099857 14418 raft_consensus.cc:2376] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10. Re-sending same reply.
I20260709 12:23:23.104157 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:23.101336 13792 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 12:23:23.106060 13792 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:23.090860 19067 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:23.131083 19067 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:23.131981 19067 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:23.133046 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.133853 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:23.085458 18928 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:23.108667 19126 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:23:23.135284 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.136063 13718 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:23.137420 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.138206 18752 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.140612 18752 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:23.111749 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.112035 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.108696 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.142488 14419 raft_consensus.cc:2376] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12. Re-sending same reply.
I20260709 12:23:23.121479 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.099681 18970 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:23.122414 19098 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:23:23.123534 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.125047 14417 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:23.126713 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.127460 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:23.127957 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.129213 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.138229 13721 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:23.014427 19076 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.151055 13723 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:23.153605 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:23.154189 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173585us
I20260709 12:23:23.154841 14417 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:23.155897 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.109103 19069 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:23.176124 14416 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:23:23.178692 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.175370 13727 raft_consensus.cc:2376] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9. Re-sending same reply.
I20260709 12:23:23.185330 13720 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:23.187567 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:23.177090 19056 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.003s
I20260709 12:23:23.188190 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.173755 19039 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.197257 14419 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:23.198063 14417 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:23.207163 13721 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:23.211968 13723 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:23.197980 19056 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12.
I20260709 12:23:23.178617 18970 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.206490 19028 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:23.217300 19028 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.237334 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.247110 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.250741 14421 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:23.250777 19069 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.223599 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.238245 14423 raft_consensus.cc:2376] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10. Re-sending same reply.
I20260709 12:23:23.261313 13724 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:23.263206 13722 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:23.266321 13800 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:23:23.267844 18852 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:23.268579 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:23.270076 13800 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:23.270539 19084 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:23.275664 18972 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:23.277240 19114 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:23.276136 19056 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:23.314431 14424 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:23.320148 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:23.325173 19115 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:23.326571 14425 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:23:23.326468 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.327759 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.326992 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.329396 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.330169 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.330938 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.331732 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.332226 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.332370 13793 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:23.332794 13801 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:23.334906 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:23.335397 19006 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.004s
I20260709 12:23:23.335918 13796 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 9: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:23.337114 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.338352 19078 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:23.268550 18852 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.343631 18897 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:23.345046 18852 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:23.361676 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.363363 19084 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:23.369585 13722 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:23.374931 18970 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.378752 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.378980 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:23.380086 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.380353 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.380443 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:23.380944 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.381822 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:23.382495 13796 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:23.380115 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.383708 13793 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
W20260709 12:23:23.355893 19122 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.169s	user 0.000s	sys 0.004s
W20260709 12:23:23.404454 19071 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.652s	user 0.015s	sys 0.004s
I20260709 12:23:23.458994 19071 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.461362 19076 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:23.461786 19071 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:23.396241 19114 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.466696 19114 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.468230 19067 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.498304 13795 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:23.508314 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 195221us
I20260709 12:23:23.508778 19114 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.509727 19067 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:23.510612 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:23.512315 13798 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:23.512981 19074 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:23.514038 19114 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:23.514528 19074 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:23.514638 19114 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.515309 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3133) took 1553 ms. Trace:
I20260709 12:23:23.515728 13792 rpcz_store.cc:276] 0709 12:23:21.961926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:21.998473 (+ 36547us) service_pool.cc:224] Handling call
0709 12:23:23.515104 (+1516631us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:23.515214 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246023552}
I20260709 12:23:23.517145 19114 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.518471 19084 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:23.517683 19114 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.519125 19114 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.521332 19114 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.523346 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.524597 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.525730 19058 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:23.526142 19058 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:23.526598 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.527539 13652 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:23.527040 19058 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.529121 19058 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:23.469810 18972 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.529819 18972 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.531416 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.531977 18972 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.513432 13796 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:23.531435 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.533751 13652 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:23.534763 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.535666 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.536572 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.537551 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.537732 13795 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:23.538182 19130 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.315s	user 0.003s	sys 0.001s
I20260709 12:23:23.538442 13644 raft_consensus.cc:2368] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14.
I20260709 12:23:23.539790 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.540380 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:23.542292 13644 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:23.544085 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.544831 13644 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:23.546636 13644 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:23.548292 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.548313 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.552187 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:23.557821 13795 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:23.563381 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.569170 19084 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:23.570650 13792 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:23.571425 13643 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:23.573602 18972 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.574103 18972 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.576542 18972 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.578850 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:23.582417 19076 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:23.583921 19076 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:23.584442 19076 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:23.585421 19076 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.586624 13650 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:23:23.591595 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.592412 14423 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:23.603171 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:23.606566 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.608458 18897 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:23.608955 18897 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:23.614305 13646 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:23.616339 13652 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:23.590093 18951 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:23.630755 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.613780 18897 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:23.615172 19069 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.647502 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.655994 18972 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.656690 18972 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.609894 19084 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:23:23.652817 19085 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 12:23:23.674417 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.676914 18972 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.682554 19084 consensus_queue.cc:1243] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:23.683151 19074 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:23.683619 19074 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:23.684881 19074 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.652765 19069 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:23.691593 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99113.6us
I20260709 12:23:23.693056 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:23.695720 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45060.4us
I20260709 12:23:23.691816 19074 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:23.705111 19126 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:23.703069 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:23.705698 19126 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:23.706189 19126 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:23.707913 14391 leader_election.cc:400] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14.
I20260709 12:23:23.698123 18972 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.720520 18972 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.712306 19072 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:23.725443 18898 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:23.714663 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:23.729363 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.749953 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:23.736913 19128 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.579s	user 0.000s	sys 0.009s
I20260709 12:23:23.755066 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.757066 13792 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:23.758783 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:23.730499 19069 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.761399 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.737362 19021 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:23.762486 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69520.3us
I20260709 12:23:23.766106 13797 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:23.769438 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:23.775913 19116 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.933s	user 0.003s	sys 0.013s
W20260709 12:23:23.779922 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.782332 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:23.783687 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2164) took 1658 ms. Trace:
I20260709 12:23:23.784961 18928 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:23.786221 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.790311 18970 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.790889 18970 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.791920 19021 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:23.792068 19143 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:23.792397 19021 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:23.792567 19143 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:23.792858 19021 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:23.793277 18970 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:23.794699 13690 leader_election.cc:336] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.555s (SENT)
I20260709 12:23:23.795220 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:23.796793 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.796918 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.797573 13725 raft_consensus.cc:2368] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12.
I20260709 12:23:23.797950 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.798727 13724 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:23.797578 13718 raft_consensus.cc:2368] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12.
I20260709 12:23:23.799746 18972 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.800870 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.788318 13797 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:23.798797 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.803099 13722 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:23.803798 19086 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:23.805965 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.806394 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.808547 18979 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.809175 18979 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.809893 19006 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.810534 19006 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.810895 19109 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.811437 19109 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.812775 18992 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.813289 18992 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.793844 18970 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:23.827826 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 120070us
I20260709 12:23:23.849721 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.852092 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:23.854777 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:23.861163 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.862555 13646 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:23.865929 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71289.4us
W20260709 12:23:23.867633 13690 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.558s (SENT)
W20260709 12:23:23.868602 13690 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.573s (SENT)
I20260709 12:23:23.872104 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.873140 13648 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:23.873045 13649 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:23.875229 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:23.877781 13797 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:23.784133 13651 rpcz_store.cc:276] 0709 12:23:22.123666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:22.165066 (+ 41400us) service_pool.cc:224] Handling call
0709 12:23:22.178671 (+ 13605us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:22.190641 (+ 11970us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:23:22.436517 (+245876us) log.cc:844] Serialized 14 byte log entry
0709 12:23:22.688168 (+251651us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:22.706207 (+ 18039us) log.cc:844] Serialized 26 byte log entry
0709 12:23:22.706426 (+   219us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:22.706463 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:22.706532 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:22.721774 (+ 15242us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b18002e69c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:23.775184 (+1053410us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:23.782056 (+  6872us) raft_consensus.cc:1685] finished
0709 12:23:23.782081 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:23.782250 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35795,"prepare.queue_time_us":53447,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":19931,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":25592,"spinlock_wait_cycles":14900352,"thread_start_us":397078,"threads_started":2,"wal-append.queue_time_us":352205}
I20260709 12:23:23.883473 13649 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:23:23.886293 13690 leader_election.cc:336] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.621s (SENT)
I20260709 12:23:23.824473 19109 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.888065 18970 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:23.890885 19150 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:23:23.891615 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62214.5us
I20260709 12:23:23.892158 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.885447 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.893685 13724 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:23.904124 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:23.871958 18979 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:23.908313 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.793793 19143 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:23.897198 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:23.910106 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:23.892982 19148 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:23:23.914332 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.916314 19039 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.918578 18970 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:23.920719 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:23.922202 19126 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.884619 19071 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:23.934345 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:23.937625 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 169239us
I20260709 12:23:23.939502 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.943504 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:23.951195 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:23.951836 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.953846 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:23.960222 13644 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:23:23.961454 13725 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:23.961563 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.932083 19084 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:23.962276 13796 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:23.960829 13644 pending_rounds.cc:85] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 13
I20260709 12:23:23.952404 19039 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:23.936582 19143 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:23.968593 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 82365.1us
I20260709 12:23:23.974005 13647 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:23:23.965473 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:23.979624 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:23.980257 13724 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:23.952093 19076 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:23.982131 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:23.986841 18978 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:23.987257 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.987964 19058 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:23:23.961719 18928 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:23:23.975250 19157 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:23.975328 18970 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:23.986937 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:23.995481 19006 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.964702 19158 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:23.997473 18992 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:23.983266 19115 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:24.002574 13792 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:23.969370 13644 pending_rounds.cc:107] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:23:24.008859 13797 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:24.028954 13647 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:23:24.036559 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97883.6us
I20260709 12:23:24.045451 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.036640 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:24.047348 13796 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.050064 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:23.968148 13727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:23:24.055394 13725 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:24.051916 19155 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.180s	user 0.007s	sys 0.000s
I20260709 12:23:24.056602 13644 raft_consensus.cc:2889] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:24.058139 13645 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:24.054352 19153 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.188s	user 0.000s	sys 0.007s
I20260709 12:23:24.088680 13801 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:24.090935 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:24.085978 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.091089 13646 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:23:24.054373 19151 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.008s
I20260709 12:23:24.084632 13727 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:23.994087 19021 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.095762 13727 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:24.098029 18898 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.098826 19039 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:24.098654 18898 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.101125 18898 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:24.102249 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44227.6us
I20260709 12:23:24.103471 18928 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:24.103797 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:24.103874 18928 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:24.103502 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:24.105319 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:24.044926 18897 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.106616 19071 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:24.107115 19071 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:24.107595 19071 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:24.108659 18897 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:24.109465 18897 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:24.111406 13724 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:24.111644 18770 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:24.112447 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.113617 18770 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:24.115531 19058 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:24.115950 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.102360 19123 raft_consensus.cc:2764] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:23:24.116704 13726 raft_consensus.cc:2368] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:23:24.058148 19115 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:24.119755 19115 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:24.119807 19021 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:24.120548 19108 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:24.121618 13799 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.058369 18972 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:24.148269 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:24.149632 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 113163us
I20260709 12:23:24.154448 13799 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:24.155093 14425 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:23:24.155437 14425 pending_rounds.cc:85] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 11
I20260709 12:23:24.155651 14425 pending_rounds.cc:107] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:23:24.155891 14425 raft_consensus.cc:2889] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:24.156558 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51344.3us
I20260709 12:23:24.162945 13795 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:24.153990 19069 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:24.165306 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.166762 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:24.167512 19162 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.182s	user 0.000s	sys 0.003s
I20260709 12:23:24.150382 18819 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:24.171347 19067 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:24.173833 19028 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:24.174798 13724 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.179168 13727 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.182401 13718 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:24.145434 18972 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:24.189725 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:24.200659 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 219552us
I20260709 12:23:24.149184 19006 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:24.092067 19158 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.203583 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.178308 19067 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:24.144852 19058 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:23:24.208390 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.218425 13719 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:24.209355 13727 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:24.217074 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:24.154160 19089 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:23:24.221802 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115678us
I20260709 12:23:24.178423 19085 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:23:24.199744 19021 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:24.184688 18978 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:23:24.179194 19076 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:23:24.204510 19039 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.217913 19067 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:24.240392 19156 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:23:24.256529 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.227284 19126 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:24.219626 19072 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:23:24.226027 18970 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:24.058895 19157 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.263383 19107 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:24.263643 13725 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.264201 19114 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:24.250978 19161 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:24.264343 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.279568 13718 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:24.239339 19021 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:24.272275 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.258431 19159 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:24.285949 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68954us
W20260709 12:23:24.271726 19074 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.570s	user 0.014s	sys 0.000s
W20260709 12:23:24.313705 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.332508 13727 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:24.333698 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.291101 13798 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.240191 19006 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.344121 13690 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:24.345492 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94793.2us
I20260709 12:23:24.346812 13720 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:24.222198 18928 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:24.349275 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 168573us
I20260709 12:23:24.275408 19107 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:24.278981 19114 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:24.240907 18972 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.259951 19039 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.286880 19021 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.354565 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:24.346685 19156 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:23:24.348799 19164 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:23:24.281301 18978 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:23:24.384279 13727 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:24.393391 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:24.393473 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 58458.8us
I20260709 12:23:24.394140 13727 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:24.394785 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:24.394790 18928 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:24.412348 19168 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:23:24.354506 18951 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:24.285487 19159 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.346616 18970 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.440454 18992 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:24.441692 19166 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:23:24.443152 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.461202 19107 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.442974 19114 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.458343 19084 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:24.460203 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.497761 19084 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:24.498293 13726 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:24.378165 19161 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:24.389042 19169 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:23:24.416090 18972 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:24.491647 18978 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:24.491860 13718 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:24.398442 19028 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
W20260709 12:23:24.538609 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 185031us
W20260709 12:23:24.533479 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 138815us
W20260709 12:23:24.544553 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 152366us
I20260709 12:23:24.549579 13727 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:24.555023 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.533248 19114 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.558521 13726 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:24.548012 18978 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.572647 18978 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:24.555737 18992 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:23:24.574285 18992 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:23:24.574659 18992 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:24.575556 18992 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:24.578498 19158 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:24.580617 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:24.569164 19071 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:24.590415 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.524968 19084 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.591703 19084 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:24.594678 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:24.595623 13648 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:24.490687 18951 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.607442 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:24.598510 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:24.618193 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69776.2us
I20260709 12:23:24.585707 18897 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:24.620393 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:24.620692 13719 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:24.620501 13725 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:24.545011 19006 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:24.627321 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 73989.5us
W20260709 12:23:24.633198 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.599871 18992 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:24.670644 13725 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:23:24.643579 18978 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.675819 13719 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:24.671749 13725 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:23:24.678288 13725 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:23:24.678622 13725 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:24.673426 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:24.648999 19170 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.647738 19006 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.648656 19175 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:24.650343 19108 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.690497 19085 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:24.698357 19006 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.699564 18970 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:24.670184 19176 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:23:24.705449 19069 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:24.706286 19149 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:24.686791 19170 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:24.727545 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.679555 19039 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:24.711397 19143 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.732s	user 0.022s	sys 0.007s
I20260709 12:23:24.684592 19157 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:24.698040 19159 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:24.770783 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.733976 18819 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:24.734325 19158 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:24.793399 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.793780 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.794148 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.794661 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:24.803390 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:24.795207 13726 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:24.807508 13727 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:24.810117 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:24.810881 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 162101us
W20260709 12:23:24.781620 19115 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.001s
W20260709 12:23:24.812875 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.759892 19069 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:24.683828 18978 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.819317 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:24.831418 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:24.834061 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:24.840704 13727 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:24.734622 19108 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.780777 19149 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:24.845896 19149 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.846992 19108 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:24.849680 13792 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:24.847337 19149 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:24.849932 13801 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:24.799213 19072 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:24.801851 19071 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:24.850647 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:24.859477 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:24.806581 18819 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:24.855211 13793 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:24.781896 19180 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.859167 13718 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:24.861660 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.852476 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:24.862820 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 150297us
I20260709 12:23:24.856462 19058 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.866073 19076 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:24.867164 19186 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:24.901332 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.872195 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:24.856302 19158 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:24.882418 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.740273 19085 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.909528 13720 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:23:24.920781 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:24.910858 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:24.859025 19072 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:24.931933 13727 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:24.779287 19175 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.948866 13722 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:24.949510 13726 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:24.841949 19115 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:24.953255 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88601.9us
I20260709 12:23:24.898865 18951 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:24.915055 19186 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.904037 19157 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:24.962941 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 138566us
I20260709 12:23:24.963860 13724 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:23:24.994578 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83818.8us
I20260709 12:23:24.995641 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:24.997290 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:24.999202 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 127085us
W20260709 12:23:25.007961 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48443.7us
I20260709 12:23:24.899010 19180 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:24.966715 19159 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:24.966327 19039 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:24.902623 19058 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:25.032722 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.049183 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:24.965440 19143 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:24.974642 19189 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:25.051548 13799 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:25.052136 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:25.060765 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106927us
W20260709 12:23:25.055751 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152547us
W20260709 12:23:25.061909 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66368.2us
I20260709 12:23:25.062824 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:25.063139 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 96751.8us
I20260709 12:23:25.027405 19157 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:24.941709 19072 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:25.077431 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50469.5us
I20260709 12:23:25.063772 19039 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:25.066977 18972 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.002s
I20260709 12:23:25.068557 19158 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:25.070463 19159 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:25.173300 19107 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:25.081349 19172 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:23:25.060985 19189 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:25.081368 19006 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:25.107935 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45198.4us
I20260709 12:23:25.109380 19071 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:25.110208 19194 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:23:25.119144 19072 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:25.120219 19085 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:25.122531 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44136.5us
W20260709 12:23:25.120776 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71697.4us
W20260709 12:23:25.135672 19188 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:23:25.150801 19067 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.890s	user 0.010s	sys 0.014s
I20260709 12:23:25.068094 14418 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:23:25.153537 18992 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.005s
I20260709 12:23:25.067624 19143 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.172609 19207 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:24.974277 19069 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.182683 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:25.183779 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 226173us
I20260709 12:23:25.202689 19039 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:25.241477 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.235122 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.124568 19176 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:25.123771 19021 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.735s	user 0.016s	sys 0.006s
W20260709 12:23:25.241385 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:25.242116 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59576.2us
I20260709 12:23:25.243006 13725 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:25.132741 19058 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:25.250223 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.256992 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.247174 18992 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:25.238970 19074 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.267890 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:25.254747 19114 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.002s
I20260709 12:23:25.261548 18770 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.247248 19180 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.281469 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.282238 14421 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
W20260709 12:23:25.272616 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 203283us
I20260709 12:23:25.274456 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.260185 19058 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.240782 19067 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.277697 19209 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:23:25.279247 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54903.5us
I20260709 12:23:25.307513 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3163) took 1499 ms. Trace:
W20260709 12:23:25.311192 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.263967 19157 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:25.313403 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.313864 18978 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.314110 18970 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:25.314951 18970 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:25.285375 19169 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.275369 19069 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:25.254007 19021 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.322992 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:25.310390 13800 rpcz_store.cc:276] 0709 12:23:23.787547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:23.787731 (+   184us) service_pool.cc:224] Handling call
0709 12:23:23.925042 (+137311us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:23:23.925316 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:23.971372 (+ 46056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:23.971421 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:24.610530 (+639109us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:23:24.626927 (+ 16397us) log.cc:844] Serialized 26 byte log entry
0709 12:23:24.852009 (+225082us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:24.852065 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:24.942954 (+ 90889us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:24.943025 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:25.286482 (+343457us) raft_consensus.cc:1685] finished
0709 12:23:25.286503 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:25.286665 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5957,"raft.queue_time_us":6416,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5114,"spinlock_wait_cycles":107783168,"thread_start_us":181475,"threads_started":1,"wal-append.queue_time_us":286949}
I20260709 12:23:25.308351 18770 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:25.309687 19058 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.330830 19085 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:25.335394 19085 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:25.335772 18951 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:23:25.336345 19085 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.337000 18951 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:25.337402 18951 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:25.332448 19058 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:25.334074 19143 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:25.338766 19175 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.334372 18978 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:25.257108 19176 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.339707 18978 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.285933 18992 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.334594 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:25.342599 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.343456 18978 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.344159 19085 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:25.345355 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.346750 18951 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.329101 13646 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:25.355049 18951 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:25.336468 19169 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.356916 18819 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.358453 13793 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:25.360905 18819 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:25.273171 19074 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.316946 19170 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.339840 19186 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.290200 19189 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.373452 19189 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.377197 13797 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:25.378136 19158 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:25.378788 19158 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.379206 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:25.380589 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.381004 19158 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:25.381995 18992 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:25.384325 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:25.385643 13645 raft_consensus.cc:2410] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 16 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:23:25.384289 19158 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:25.387521 19158 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:25.387903 18970 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:25.388773 19021 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:25.388835 19158 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.389727 13795 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:25.392292 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3177) took 1130 ms. Trace:
I20260709 12:23:25.392830 13795 rpcz_store.cc:276] 0709 12:23:24.261832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:24.262300 (+   468us) service_pool.cc:224] Handling call
0709 12:23:24.262610 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:25.390163 (+1127553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:25.392038 (+  1875us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000d1bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:25.392232 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2591658368}
I20260709 12:23:25.335906 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:25.398706 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:25.400350 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:25.400085 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73800.6us
I20260709 12:23:25.401605 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:25.389576 18970 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:25.402959 19126 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:25.403739 19126 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:25.346138 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:25.411998 19149 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
I20260709 12:23:25.441622 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:25.442879 18898 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.411343 19074 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:25.404218 19126 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:25.413596 19169 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.417879 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.444770 14425 raft_consensus.cc:2410] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 16 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 12:23:25.444653 19126 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:25.418987 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91571.1us
I20260709 12:23:25.445756 13649 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:25.447440 19089 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:25.448992 19074 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:25.449299 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:25.449806 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:25.450169 13724 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:25.422226 19176 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:25.459095 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44667.2us
I20260709 12:23:25.460246 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.432698 19189 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:25.311939 19115 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:25.476807 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:25.465126 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:25.477711 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.477963 13792 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:25.452572 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:25.452848 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 106804us
I20260709 12:23:25.455940 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:25.453732 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:25.464434 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:25.484103 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:25.465025 14421 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:25.467139 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:25.485697 13719 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:25.458995 18970 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.470436 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.487923 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.489044 14421 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:25.486884 14425 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:25.502450 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.508298 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.509032 14425 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:25.503330 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:25.511749 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.512385 14421 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:25.515383 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:25.464272 19187 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:25.534775 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:25.519341 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54296.9us
I20260709 12:23:25.533496 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:25.536430 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:25.544142 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:25.544490 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44361.3us
I20260709 12:23:25.545969 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.509944 19149 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:25.513479 18951 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:25.564708 19197 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:25.566804 18978 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:25.567289 19107 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:25.464617 19169 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:23:25.531095 19187 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.539073 19170 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:25.552062 19115 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:25.513865 19028 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:23:25.551540 19174 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:25.500514 19189 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.555758 19123 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.568868 19149 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.603618 19149 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:25.605191 19214 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:25.588969 19180 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:25.533952 19021 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:23:25.516851 18970 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.571767 18978 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.610423 18978 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:25.611549 19189 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:25.593120 19174 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.613335 19174 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.544194 19071 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:23:25.600553 19067 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:25.590049 19115 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.615000 19115 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:25.617344 18970 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:25.587606 19170 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.618163 19170 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:25.620532 19039 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.622848 19039 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:25.573624 19107 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:25.623433 19107 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:25.589488 19187 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:25.630286 19180 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:25.631371 19180 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.575266 18951 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:25.592846 18928 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:23:25.635133 18928 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:25.635609 18928 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:25.635171 18951 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.636471 18928 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.637676 18928 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.587998 19197 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:25.607191 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:25.639174 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:25.638976 19197 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.641619 19114 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:25.642139 19175 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:25.608815 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255f10165e844f2cb50d654996d639cf"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:25.642227 19114 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:25.646894 19114 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:25.642712 19175 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:25.651747 19175 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.643536 13792 raft_consensus.cc:606] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:25.653678 13792 raft_consensus.cc:614] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:25.643239 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:25.646092 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 332765us
I20260709 12:23:25.678248 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.679977 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.681767 19216 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:25.681807 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:25.682072 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:25.683092 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 132217us
W20260709 12:23:25.695236 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 149330us
I20260709 12:23:25.725601 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:25.597479 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 216938us
I20260709 12:23:25.638157 19115 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:25.722113 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.729900 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:25.732043 13726 raft_consensus.cc:2376] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9. Re-sending same reply.
I20260709 12:23:25.730590 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:25.733188 13724 raft_consensus.cc:2368] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
W20260709 12:23:25.730800 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50945.7us
I20260709 12:23:25.727965 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:25.734858 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:25.734000 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 372123us
I20260709 12:23:25.738199 19067 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:25.739197 19067 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.750622 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:25.751722 13719 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:25.727380 13690 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:25.754030 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.756196 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:25.757689 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:25.757112 13724 raft_consensus.cc:2376] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14. Re-sending same reply.
I20260709 12:23:25.758478 14425 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:25.762319 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.779742 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:25.780485 13720 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:25.777834 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:25.734742 19222 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:25.737174 19217 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:25.794443 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 241420us
I20260709 12:23:25.796759 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:25.799396 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.755445 19067 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:25.806496 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 79202.5us
I20260709 12:23:25.765658 18897 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:25.780819 19157 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:25.778822 19216 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:25.815986 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:25.854024 13798 raft_consensus.cc:606] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:25.854774 13798 raft_consensus.cc:614] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:25.818743 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.780562 18979 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.837610 19158 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:25.821215 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.839684 19067 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:25.821841 19109 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:25.848151 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:25.783664 19189 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:25.889859 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 127630us
I20260709 12:23:25.858377 13720 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:25.803975 19056 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.851016 19069 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.800078 19217 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:25.881524 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62857.3us
W20260709 12:23:25.882966 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86362.6us
W20260709 12:23:25.892690 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44624.5us
I20260709 12:23:25.867693 19176 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.923049 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.924324 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:25.866323 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 88574.1us
I20260709 12:23:25.930315 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:25.931401 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.882202 18970 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:25.935973 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:25.938100 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:25.881003 19067 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:23:25.895609 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 74494us
I20260709 12:23:25.889117 19180 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:25.921716 19197 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:25.920277 19159 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:25.895856 19157 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:25.893937 19222 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:25.893731 18897 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:25.880882 18979 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.924957 19028 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:25.925057 19217 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:25.889676 19174 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:25.998608 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44200.5us
I20260709 12:23:25.970340 19180 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.957700 19067 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:25.890595 18978 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:25.981194 19158 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.982224 19216 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:25.919808 19056 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:25.894649 19189 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:25.964705 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:26.020138 13800 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:25.936892 19069 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:26.022504 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49530.1us
W20260709 12:23:26.026153 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103246us
I20260709 12:23:26.091595 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:25.940150 19176 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:26.077486 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 106985us
W20260709 12:23:26.071507 19202 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:23:26.085669 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:26.089309 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65444.1us
I20260709 12:23:26.092618 14418 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:26.070343 18852 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:26.095506 13766 leader_election.cc:400] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12.
I20260709 12:23:26.107906 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:25.979245 19159 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:26.108315 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 184102us
I20260709 12:23:26.119184 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:26.120574 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.126706 14425 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:23:26.082726 19197 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:26.127394 14425 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 10
I20260709 12:23:25.992262 19107 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.087038 19115 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.133471 14419 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:23:26.134958 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:26.137535 14425 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:23:26.141239 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.143496 14425 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:26.153936 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.154913 13646 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:26.159682 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:25.990221 19187 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.160751 13646 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:26.083475 19170 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.162855 18897 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.173878 19176 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:26.098740 19149 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.184902 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:25.979725 19222 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:26.162892 18928 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.522s	user 0.014s	sys 0.001s
I20260709 12:23:26.158632 19028 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:25.994378 19157 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.203998 13647 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:26.222182 19174 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.225776 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.226303 19222 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:26.231369 19107 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.233675 19028 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:26.234920 19028 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.193123 19241 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.273043 13643 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:26.249822 14418 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
W20260709 12:23:26.276062 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65246.1us
I20260709 12:23:26.282078 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.283106 13719 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:26.289348 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 191869us
I20260709 12:23:26.302815 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:26.309929 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 169606us
I20260709 12:23:26.322117 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:26.322992 13800 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:26.325651 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:26.333460 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:26.310410 19218 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.347s	user 0.000s	sys 0.002s
I20260709 12:23:26.310552 19246 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.327374 19184 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:26.310756 19115 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:26.357599 13719 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:26.360651 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:26.370730 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:26.334301 19186 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:26.281397 19159 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.333173 19180 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.336090 18978 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.394085 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54361b2742ab4613b55749cbb699113b"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:26.404275 13797 raft_consensus.cc:606] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:26.395028 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 239496us
I20260709 12:23:26.402100 13800 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:26.407255 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.408454 13649 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:26.397647 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.346962 19069 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.416571 13798 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:23:26.308506 19197 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.389369 19107 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:26.374197 19189 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:26.422094 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88880.1us
I20260709 12:23:26.427928 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:26.442444 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 334580us
I20260709 12:23:26.444060 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:26.446655 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.375314 19184 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.420135 19056 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.452049 19265 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:26.453390 19114 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.806s	user 0.008s	sys 0.006s
I20260709 12:23:26.462606 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.427866 19069 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.472666 19069 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.475178 19069 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.364888 19246 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.476516 19184 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.477329 19246 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.478153 13793 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:26.478565 19264 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:26.467626 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.481319 19069 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.484632 13648 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:26.485430 13645 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:23:26.485913 13645 pending_rounds.cc:85] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 5
I20260709 12:23:26.486192 13645 pending_rounds.cc:107] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:23:26.486811 13645 raft_consensus.cc:2889] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:26.448737 19107 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:26.504685 19187 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.469703 19074 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.550539 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.551445 13645 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:26.560819 19175 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.909s	user 0.010s	sys 0.005s
I20260709 12:23:26.486922 19114 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:26.581089 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58084.7us
I20260709 12:23:26.489495 19265 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:26.588959 18979 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.586848 18898 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.591029 18898 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.592507 19227 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:26.593319 18898 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:26.548578 19107 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.615954 19174 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:26.616925 19174 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.621608 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:26.636353 19149 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:26.641517 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:26.644169 19174 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.649638 19235 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:26.650388 19235 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.653483 19123 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:26.656819 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.658416 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:26.659632 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47264.1us
W20260709 12:23:26.653330 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53636.9us
I20260709 12:23:26.663846 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:26.654052 19235 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.654471 19123 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:26.520924 19170 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.659124 13647 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:26.681445 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.682222 13797 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:26.684991 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.697893 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.665479 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:26.702441 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.702436 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.728215 13793 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:26.728320 13800 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:26.729435 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:26.730777 19260 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:23:26.734573 18970 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:26.735213 18970 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.735590 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:26.737131 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.737500 18970 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.737813 13798 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:26.701833 19245 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:23:26.739238 19157 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:26.740417 19245 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:26.742069 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.583882 19175 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.745244 19175 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:26.745707 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 306543us
W20260709 12:23:26.747514 19067 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.598s	user 0.013s	sys 0.000s
I20260709 12:23:26.748037 19067 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.745393 13724 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:23:26.753152 13646 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:26.748230 19241 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:26.754043 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.754844 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:26.756902 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.757109 13652 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 12: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:26.758144 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50640us
I20260709 12:23:26.758617 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.737134 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:26.759410 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:26.761894 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.761859 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.763872 13720 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:26.765046 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.765477 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.765679 13720 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:26.766872 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.569382 19074 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.736194 19069 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:26.759667 13726 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:26.767498 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.778546 19265 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:26.799201 14425 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:26.770347 19175 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:23:26.800472 19074 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.801789 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.801822 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.802737 13727 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:23:26.767028 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.804023 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.768322 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.769090 13722 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:26.768807 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:26.799257 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 209733us
I20260709 12:23:26.806633 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:26.808509 13652 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:26.811414 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.812177 13722 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:23:26.812563 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:26.813360 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 82917us
I20260709 12:23:26.814968 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:26.817090 13718 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:26.817052 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:26.821347 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:26.821352 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62069.4us
I20260709 12:23:26.822640 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.771039 19067 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.825218 14425 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:26.831125 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.823225 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:26.795027 19123 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.839573 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:26.757684 18979 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:26.840596 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:26.844318 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.849077 13719 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:23:26.837498 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:26.768565 19241 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:26.846946 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.862974 13652 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:26.781200 19114 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:26.862129 19176 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:26.865337 19176 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.865706 19176 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.870716 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.872591 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.870673 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.873699 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.874475 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.874586 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.875780 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.876196 19235 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:26.876426 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:26.876648 13719 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:26.876724 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.877519 19234 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:26.878369 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:26.878762 19262 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.133s	user 0.000s	sys 0.005s
I20260709 12:23:26.852919 19074 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:26.880111 19268 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:23:26.882988 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.832835 19175 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:26.883881 13650 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:26.884168 19175 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.884606 19175 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:26.879693 19074 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:26.886523 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78856.6us
I20260709 12:23:26.886745 19074 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:26.870754 19241 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:26.890662 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.890435 19241 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.891052 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.875723 19170 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:26.894426 19170 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.839895 19216 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:26.897090 19216 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:26.877040 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.899261 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.899326 19216 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:26.879648 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.876433 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.900899 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:26.901463 13793 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:26.902117 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.903127 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.903430 13722 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:26.903769 13800 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:26.904398 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:26.903745 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 325736us
I20260709 12:23:26.878859 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.906399 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.906745 13719 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:26.907459 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.878005 19234 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.905014 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:26.917160 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.919450 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.853157 19258 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:26.921475 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.922536 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.876631 19235 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:26.873759 19217 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.817s	user 0.010s	sys 0.004s
I20260709 12:23:26.923229 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:26.924760 13798 raft_consensus.cc:606] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.925122 13798 raft_consensus.cc:614] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:26.927326 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:26.927989 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:26.931234 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.928336 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:26.933938 13726 raft_consensus.cc:2368] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:23:26.942541 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:26.958235 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.958458 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.908659 18979 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:26.924150 19234 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.960580 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.961279 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.959744 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.964592 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:27.017282 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:26.962886 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.940229 19107 raft_consensus.cc:2764] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:26.927040 19235 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.020776 13644 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:26.923434 19258 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:26.972573 19264 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:26.974051 13718 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.033198 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.977752 13726 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:27.034588 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.978180 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:27.035007 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:26.979780 19149 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:26.982647 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:26.986466 19170 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:26.999897 19257 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.263s	user 0.000s	sys 0.004s
I20260709 12:23:27.001546 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:27.008133 13647 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:23:27.039637 19107 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.047070 19107 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.049019 19190 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:27.049568 19107 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.049690 19190 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:27.050211 19190 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:27.052263 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:27.053148 13799 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:27.033252 19258 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:26.926568 19069 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:27.063812 19069 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.064457 19069 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:27.009043 13649 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:26.969244 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:27.065843 13547 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 8 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 11 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:27.073276 19149 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:27.078949 18979 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.086686 19264 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.087500 19264 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.091961 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:27.093076 13727 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 8.
I20260709 12:23:27.097422 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:27.018375 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2806) took 1175 ms. Trace:
I20260709 12:23:27.102187 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:27.019289 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53186.4us
W20260709 12:23:27.021984 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 207104us
I20260709 12:23:27.016315 13794 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:26.944869 19174 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:27.029078 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52682.9us
I20260709 12:23:26.886065 19176 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.031891 13650 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:26.977842 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:26.956552 19254 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:27.008488 13652 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:27.113293 13649 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:27.113189 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:27.121799 13647 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:27.131718 19176 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.132856 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:27.133600 13792 raft_consensus.cc:606] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:27.135413 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 118276us
I20260709 12:23:27.136411 13617 leader_election.cc:400] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:23:27.139515 13617 leader_election.cc:400] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:23:27.139809 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:27.141873 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:27.138499 13793 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:27.154336 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:27.154498 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:27.113739 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.102156 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:27.099653 13725 rpcz_store.cc:276] 0709 12:23:25.788304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.821461 (+ 33157us) service_pool.cc:224] Handling call
0709 12:23:25.821888 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:25.997405 (+175517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.997441 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:26.021019 (+ 23578us) log.cc:844] Serialized 26 byte log entry
0709 12:23:26.155485 (+134466us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:26.191054 (+ 35569us) log.cc:844] Serialized 14 byte log entry
0709 12:23:26.579018 (+387964us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b180024cb80. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:23:26.856515 (+277497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.963058 (+106543us) spinlock_profiling.cc:243] Waited 66.9 ms on lock 0x7b5400027bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:26.963124 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.963193 (+    69us) raft_consensus.cc:1685] finished
0709 12:23:26.963211 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:26.963369 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6360,"prepare.queue_time_us":35972,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":512638,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":33081,"spinlock_wait_cycles":161215616,"thread_start_us":481708,"threads_started":3,"wal-append.queue_time_us":244997}
I20260709 12:23:27.163682 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:27.164445 13727 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:23:26.947466 19217 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:27.171676 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69598.8us
I20260709 12:23:27.172734 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.182060 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.165908 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:27.027670 19186 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:23:27.199851 14423 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.151635 19170 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:27.221303 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62199.2us
I20260709 12:23:27.209218 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.210191 13727 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:27.228188 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:27.199882 18992 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:27.236812 13801 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:27.199884 18852 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:23:27.195482 19107 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.201203 19126 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.192472 19114 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:27.246614 19114 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.247150 19114 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:27.247764 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50974.2us
I20260709 12:23:27.251055 19126 raft_consensus.cc:993] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:27.251705 19126 raft_consensus.cc:1081] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:27.251809 19217 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:27.246889 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67386.6us
I20260709 12:23:27.259972 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:27.237124 19258 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.170936 19254 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:23:27.264240 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55101.9us
I20260709 12:23:27.225498 19271 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:23:27.267057 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:27.265793 19217 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:27.277941 14420 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:27.273358 19223 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.279199 19271 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:27.279323 19223 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.279810 19271 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:27.280789 19271 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.281746 19223 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:27.225956 19264 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.286412 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:27.286645 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:27.272601 19257 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:23:27.261164 19067 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:23:27.266232 19254 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.262764 19157 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:27.300618 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:27.301496 13649 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:27.282627 18970 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:23:27.286617 19272 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:27.326437 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2804) took 1538 ms. Trace:
I20260709 12:23:27.330600 14424 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:27.333451 13650 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.295251 19067 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:27.337126 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:27.338560 14418 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:23:27.339998 14416 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:27.297808 19254 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:27.326892 13723 rpcz_store.cc:276] 0709 12:23:25.787811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:25.789305 (+  1494us) service_pool.cc:224] Handling call
0709 12:23:25.789664 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:25.790038 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:25.790069 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:25.790598 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:23:26.229968 (+439370us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b18001042c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:23:26.461674 (+231706us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:26.499325 (+ 37651us) log.cc:844] Serialized 14 byte log entry
0709 12:23:26.738561 (+239236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:26.738640 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:26.999020 (+260380us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:27.326165 (+327145us) raft_consensus.cc:1685] finished
0709 12:23:27.326202 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:27.326371 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238681,"prepare.queue_time_us":33094,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":249658,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":2465,"spinlock_wait_cycles":105188992,"thread_start_us":231498,"threads_started":1,"wal-append.queue_time_us":919577}
I20260709 12:23:27.291167 19216 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:23:27.343966 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.350185 14416 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:27.323171 19126 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:27.316771 19249 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:27.296787 19223 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:23:27.316735 19217 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:23:27.316576 18992 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:23:27.342669 19067 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:23:27.302271 19157 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.320721 19241 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:23:27.350234 19227 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.344043 19272 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:27.353606 19126 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:27.346205 19021 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:23:27.353693 18951 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:23:27.369714 19249 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:27.421403 19175 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.004s
I20260709 12:23:27.435432 19175 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.438925 13646 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:27.406190 19227 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.440155 19190 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.441922 19227 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:27.438023 19175 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:27.442440 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:27.447852 19217 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:27.449204 19241 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:27.450562 19241 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:27.451820 19241 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.453542 19241 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.401934 19074 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.456787 19074 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:27.457872 19278 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:23:27.459749 19272 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.419570 19126 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.472923 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:27.420542 18970 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:27.493777 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.495595 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:27.450034 19217 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:27.499135 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.479239 19190 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:27.503707 19157 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.504556 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.505590 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:27.506044 19249 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.503814 19217 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.508280 19217 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.513260 14418 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:27.480445 18992 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:27.515077 18992 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:27.498883 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 307835us
I20260709 12:23:27.519578 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:27.516095 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:27.522167 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:27.523615 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.525110 14418 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:27.527269 14416 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:27.492163 19074 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:27.490752 18970 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:27.493441 18951 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:27.536273 19249 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:27.538126 19174 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:27.571166 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 398484us
I20260709 12:23:27.572206 14417 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:27.512766 19235 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.638921 19114 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.641819 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.664270 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:27.673221 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:27.556620 19074 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:27.566195 19159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.013s	sys 0.007s
I20260709 12:23:27.555960 18951 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.680289 13652 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:27.691341 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.670593 19186 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:27.671208 19174 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:27.577116 19249 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.696735 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:27.699043 19186 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:27.700318 19234 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.000s
I20260709 12:23:27.567418 19254 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.706804 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:27.708863 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.710568 13643 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17.
I20260709 12:23:27.714178 14418 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:27.691772 19159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.014s	sys 0.007s
I20260709 12:23:27.702119 18951 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:27.726330 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.731467 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:27.733361 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51662.2us
I20260709 12:23:27.735918 14421 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 10: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:23:27.715442 19283 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:23:27.745647 13767 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17.
I20260709 12:23:27.746189 13767 leader_election.cc:403] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:23:27.740707 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:27.710992 19287 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:27.768507 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 430948us
I20260709 12:23:27.714664 19234 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.770191 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.782874 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:27.793151 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:27.785295 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:27.692126 18970 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:27.783784 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:27.801527 13721 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 8.
I20260709 12:23:27.733629 19175 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:27.794831 13718 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:23:27.705510 19174 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:27.798919 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53440.6us
I20260709 12:23:27.807235 13724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54361b2742ab4613b55749cbb699113b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:23:27.808068 13724 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:27.808475 13724 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:27.812392 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:27.734908 19071 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:23:27.840060 13649 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:27.853977 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50249.6us
I20260709 12:23:27.793313 18979 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.792953 18992 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.868491 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:27.820595 19263 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:27.790026 19287 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.862493 18970 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:27.878304 18992 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.819676 19175 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:27.896026 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.895358 18992 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.880909 18979 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.879036 19157 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:27.887019 19256 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:27.930399 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 160327us
I20260709 12:23:27.931838 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:27.923285 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 216595us
I20260709 12:23:27.935426 18992 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.938392 19186 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.940017 19254 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.942433 18852 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:27.947281 14420 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:27.914522 19114 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.920528 19157 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:27.954385 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:27.958338 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:27.955245 13643 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:27.958714 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:27.949504 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47553.6us
I20260709 12:23:27.926430 19107 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.914000 19289 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:27.961051 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.959017 14420 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:27.929946 18979 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:27.955621 19277 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.552s	user 0.009s	sys 0.011s
I20260709 12:23:27.950657 19249 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.980199 13650 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:27.943117 18852 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.979918 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:27.984319 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 191239us
I20260709 12:23:27.990365 18852 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:27.992529 19299 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:27.993228 19299 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:27.993590 19307 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:27.994102 18951 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:27.997710 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:27.998541 13797 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:27.997912 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.001741 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.002235 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.002941 13795 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 9.
I20260709 12:23:27.970636 19291 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.006839 13795 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:28.008435 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.981106 19234 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:27.998755 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:27.943974 19256 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:28.046689 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48017.2us
I20260709 12:23:28.007319 13795 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:27.949043 19126 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.019286 18979 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:27.992573 19295 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:23:28.024267 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.026260 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:28.058068 13801 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:27.995134 19107 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.051127 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.051831 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.059224 13726 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:28.048839 13795 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.060298 13795 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:28.055475 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.061080 13722 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.056556 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.059291 13718 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:23:27.999625 19289 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:28.064896 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.072988 14420 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:28.010749 19249 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:27.994850 18951 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.079222 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:27.995117 19021 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:27.973762 19305 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:23:28.080868 13797 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:28.085498 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.047969 13724 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:28.081986 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69834.4us
W20260709 12:23:28.062963 19067 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.007s
I20260709 12:23:28.045630 19258 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.088063 13727 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:28.088063 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:28.096585 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 93055.9us
I20260709 12:23:28.098773 13617 leader_election.cc:400] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9.
I20260709 12:23:28.104461 13801 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:28.058185 19174 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.141656 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.063476 19234 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:28.144942 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.125376 13722 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:28.146775 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.075563 19254 raft_consensus.cc:2764] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:23:28.147830 13793 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:28.151117 13724 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.154392 19234 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.162453 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.086999 19271 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:28.162667 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.141857 19114 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.169961 13794 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:28.177453 19286 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:28.178045 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.180269 19186 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:23:28.169656 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81685.4us
I20260709 12:23:28.181553 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.172217 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1085) took 1569 ms. Trace:
I20260709 12:23:28.183514 14423 rpcz_store.cc:276] 0709 12:23:26.594622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.628377 (+ 33755us) service_pool.cc:224] Handling call
0709 12:23:26.628714 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.221127 (+592413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.163516 (+942389us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:28.163696 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312803712}
I20260709 12:23:28.186794 19311 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.187633 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.190349 19315 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:28.190554 19311 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.190970 19315 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.193168 19315 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:28.193229 19311 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.195919 13799 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:23:28.023104 19157 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.141745 19289 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:28.173717 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 193938us
W20260709 12:23:28.206477 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 153125us
I20260709 12:23:28.207405 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.208786 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:28.209095 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:28.097491 18979 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:28.172926 13793 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:28.165093 19234 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:28.242863 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99462.9us
I20260709 12:23:28.245007 13793 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:28.141283 19067 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.250449 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.097236 19021 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.260793 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:28.261343 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:28.251024 19069 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.186s	user 0.012s	sys 0.008s
I20260709 12:23:28.247802 19315 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:28.250389 19272 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:28.249068 19271 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:28.257138 19300 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:28.265102 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:28.242511 19175 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:28.520828 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 255807us
I20260709 12:23:28.269624 19069 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:28.524077 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 263730us
I20260709 12:23:28.514881 19067 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:28.450733 19310 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:23:28.253547 19197 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 12:23:28.454777 19263 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 12:23:28.477949 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:28.342542 19316 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 12:23:28.446971 19217 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.938s	user 0.011s	sys 0.004s
W20260709 12:23:28.448992 19241 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.995s	user 0.014s	sys 0.002s
I20260709 12:23:28.355080 18951 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:28.407727 13726 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:28.342478 19186 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:28.491586 19282 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.551507 19282 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.513374 19259 raft_consensus.cc:993] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:23:28.491958 19299 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.341351 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:28.237977 18992 consensus_queue.cc:1175] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 10 status { last_received { term: 10 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 10 index: 10 } }
I20260709 12:23:28.407178 13718 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:23:28.310662 19285 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:23:28.341472 19315 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:28.557466 19315 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.343210 19321 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:28.561853 19321 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.539664 19263 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17.
I20260709 12:23:28.535854 18852 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:23:28.562515 19321 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:28.260180 19287 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.554028 19282 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.342026 19304 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.566943 19304 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:28.417421 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 341672us
I20260709 12:23:28.358507 19300 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.572906 19300 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:28.340091 13795 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:23:28.573552 13795 pending_rounds.cc:85] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 10
I20260709 12:23:28.573819 13795 pending_rounds.cc:107] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:23:28.574054 13795 raft_consensus.cc:2889] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:28.548625 19186 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:28.575253 19304 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:28.452890 19298 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.448s	user 0.000s	sys 0.001s
I20260709 12:23:28.452409 19021 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:28.340425 19286 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 12:23:28.410563 19307 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:28.270967 19303 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.451s
W20260709 12:23:28.453945 19320 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:23:28.421075 19126 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:28.500437 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 922) took 1173 ms. Trace:
I20260709 12:23:28.403422 19271 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:28.441924 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.616s (SENT) [suppressed 25 similar messages]
I20260709 12:23:28.587976 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.409983 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:28.269866 18979 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:23:28.420215 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 2954) took 1640 ms. Trace:
I20260709 12:23:28.325443 19291 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.634784 19291 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.615682 19258 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.636420 19258 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.637598 19291 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.308350 19272 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:28.636466 19235 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.641580 19258 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.644749 19157 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.649600 18951 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:28.521811 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:28.528556 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2207) took 1225 ms. Trace:
I20260709 12:23:28.491833 19234 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.529407 19245 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:28.541199 14424 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:23:28.665146 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.661703 13649 rpcz_store.cc:276] 0709 12:23:27.303552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.306124 (+  2572us) service_pool.cc:224] Handling call
0709 12:23:27.306452 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.854386 (+547934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.528157 (+673771us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:28.528484 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226569088}
W20260709 12:23:28.542362 19323 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:23:28.542750 19313 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:28.543767 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.524300 19318 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:28.549248 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 367773us
I20260709 12:23:28.673838 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.620846 19327 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:28.575718 18970 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.002s
I20260709 12:23:28.581772 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:28.594882 19324 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:28.597069 19271 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.592316 19306 raft_consensus.cc:1081] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:23:28.690220 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.690653 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.693463 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.694114 19271 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:28.690820 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.685096 19324 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:28.696313 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.698418 19216 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.698501 19291 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:28.698956 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.699163 19216 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.699275 19291 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.701371 19216 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.697374 19324 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.592679 19307 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:28.618588 13720 rpcz_store.cc:276] 0709 12:23:26.766514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:26.801512 (+ 34998us) service_pool.cc:224] Handling call
0709 12:23:28.406890 (+1605378us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:28.407086 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976399104}
I20260709 12:23:28.667591 19315 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:28.575474 19186 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.629144 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 875) took 1073 ms. Trace:
I20260709 12:23:28.717278 14416 rpcz_store.cc:276] 0709 12:23:27.555562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.555725 (+   163us) service_pool.cc:224] Handling call
0709 12:23:27.556080 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.709541 (+153461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.709574 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:27.709640 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:23:27.709674 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.628757 (+919083us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:28.628828 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.629042 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352365952}
I20260709 12:23:28.630177 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 926) took 1296 ms. Trace:
I20260709 12:23:28.651993 19074 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.307533 19301 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:28.724929 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.725615 19186 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:28.655248 14419 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:23:28.658218 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:28.663247 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 141538us
I20260709 12:23:28.727588 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.667085 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.729136 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:28.729369 13794 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:28.619477 19289 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:28.670463 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.350798 19256 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:28.678494 19249 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:28.680498 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 238616us
W20260709 12:23:28.667874 18898 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:23:28.664534 19245 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:28.687361 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.686067 18992 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.774057 19245 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.774514 18992 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.775480 19245 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:28.777458 18992 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:28.689602 13801 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:28.753624 19313 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:28.778452 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.779044 19313 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:28.703389 19327 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.780018 19313 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.690155 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.780031 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.691493 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1098) took 1405 ms. Trace:
I20260709 12:23:28.784389 13644 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:28.586876 14425 rpcz_store.cc:276] 0709 12:23:27.326722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.336878 (+ 10156us) service_pool.cc:224] Handling call
0709 12:23:28.500024 (+1163146us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:28.500229 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252781312}
I20260709 12:23:28.784314 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.707042 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:28.550727 19217 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:28.584983 19282 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:23:28.711135 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:28.792893 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81847.6us
I20260709 12:23:28.793377 13794 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:28.793659 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.713294 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:28.714972 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.719478 13724 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:23:28.719009 14417 rpcz_store.cc:276] 0709 12:23:27.333967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.334094 (+   127us) service_pool.cc:224] Handling call
0709 12:23:27.346962 (+ 12868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.681541 (+334579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.629858 (+948317us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54007125d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:28.630088 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517303168}
W20260709 12:23:28.794806 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81595us
I20260709 12:23:28.794824 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 10.
I20260709 12:23:28.796250 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.796916 13720 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:28.714143 19271 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:23:28.738953 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.666654 18951 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.749663 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.751559 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.713810 19291 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:28.759542 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65367.7us
I20260709 12:23:28.760211 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:28.760563 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.806128 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.763837 13646 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:23:28.765890 13722 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:28.766429 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.770593 19069 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:28.771700 13650 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:23:28.772375 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.772840 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:28.773031 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:28.784284 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:28.790344 14419 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:23:28.784605 14421 rpcz_store.cc:276] 0709 12:23:27.285931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.286112 (+   181us) service_pool.cc:224] Handling call
0709 12:23:28.691270 (+1405158us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54004b6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:28.691416 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027952512}
I20260709 12:23:28.722653 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:28.798130 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:28.801874 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.804091 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.758713 19318 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:28.817935 13797 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 8: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:28.806613 13649 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:28.817481 13793 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:28.726243 19234 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:28.807942 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56531.6us
I20260709 12:23:28.762140 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.822649 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.822961 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.823920 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.810040 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.810853 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:28.815850 13645 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:28.825588 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:28.763091 19301 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:23:28.844862 19301 consensus_queue.cc:1243] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:23:28.845223 19294 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.910s	user 0.000s	sys 0.004s
I20260709 12:23:28.845479 19301 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:28.845897 19301 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:28.846652 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2208) took 1542 ms. Trace:
I20260709 12:23:28.847096 13647 rpcz_store.cc:276] 0709 12:23:27.304447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.304592 (+   145us) service_pool.cc:224] Handling call
0709 12:23:27.304988 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:27.305490 (+   502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.305522 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:27.320618 (+ 15096us) log.cc:844] Serialized 26 byte log entry
0709 12:23:27.724224 (+403606us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:27.724300 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:27.724417 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:28.801733 (+1077316us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:28.846331 (+ 44598us) raft_consensus.cc:1685] finished
0709 12:23:28.846355 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.846584 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11550,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":2814,"spinlock_wait_cycles":18974592,"thread_start_us":369403,"threads_started":1,"wal-append.queue_time_us":576874}
I20260709 12:23:28.849680 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.712760 19272 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.858893 19254 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:23:28.860769 13795 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:28.862030 18951 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:28.865746 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.805634 19217 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.867445 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.804908 19170 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:28.869328 19069 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:28.869621 19217 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.869962 19069 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:28.871683 13726 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:23:28.868680 13727 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:28.831946 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:28.875705 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43921.1us
I20260709 12:23:28.898423 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.818306 19322 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:28.811079 18992 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.901296 19271 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:28.911625 19271 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:28.912321 19271 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:28.908815 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:28.919569 13726 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:28.920852 13793 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:28.958755 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:28.817466 19074 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:28.967162 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3274) took 1262 ms. Trace:
I20260709 12:23:28.967659 13652 rpcz_store.cc:276] 0709 12:23:27.704948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.716035 (+ 11087us) service_pool.cc:224] Handling call
0709 12:23:27.718225 (+  2190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:27.915165 (+196940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:27.915315 (+   150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:27.915434 (+   119us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:27.915476 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:28.966891 (+1051415us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:28.966940 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:28.967082 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452272512}
I20260709 12:23:28.978076 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:28.979043 13797 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:28.979887 13792 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:28.980700 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.980983 13727 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:23:28.982774 13793 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:28.984290 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.922962 19318 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:28.900779 19321 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:28.987527 19318 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.988956 19321 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:28.989379 13719 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:28.902046 19217 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:23:28.870893 19069 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:28.995172 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:29.002632 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 180559us
I20260709 12:23:28.888765 19235 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:29.008536 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.011957 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:28.846807 19301 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.005792 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:29.010579 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.901242 19170 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:29.021533 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.022063 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.013808 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:29.023512 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 229913us
I20260709 12:23:29.025183 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:28.974428 19311 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:29.028023 13645 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:29.028541 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.030184 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:29.033772 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:29.035003 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.037932 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.039217 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.040483 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.040747 13792 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:29.042371 13793 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:29.043058 13793 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:29.044651 13727 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:23:28.859953 19307 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.044453 13793 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.056479 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.063244 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.064010 13797 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:29.064283 13801 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:29.070329 13793 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:29.075340 19334 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:23:29.078320 19305 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:29.079061 19305 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:29.099416 19074 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:29.027089 19313 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:29.102651 19313 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.103268 19313 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:29.103504 19217 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.104245 19217 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.107177 19217 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:29.119406 19234 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.119155 19170 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.120213 19234 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.002074 19322 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:29.125684 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:28.955596 18992 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.125232 19234 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.129060 18992 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.129962 13721 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:29.137458 13725 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:23:29.155339 13725 pending_rounds.cc:85] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 9
I20260709 12:23:29.155717 13725 pending_rounds.cc:107] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:23:29.156047 13725 raft_consensus.cc:2889] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:29.158437 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:29.169435 14420 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:23:29.159349 13727 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:29.049492 19321 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:29.028910 19256 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 12:23:29.136569 19301 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.061283 19311 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:29.219576 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:29.219787 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.150812 19217 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.158586 18979 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:29.244122 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 175413us
I20260709 12:23:29.173072 19157 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.259954 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:29.266355 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:29.312898 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46623.5us
I20260709 12:23:29.313895 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:29.270795 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:29.125206 19322 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.249298 19342 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:29.297767 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:29.320511 13652 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:29.318833 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:29.229235 19339 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.223703 19299 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.341779 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 925) took 2011 ms. Trace:
I20260709 12:23:29.261682 19327 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.345229 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:29.361055 13647 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:29.239553 19321 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:29.358515 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:29.291558 19249 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.353041 14424 rpcz_store.cc:276] 0709 12:23:27.333514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.334678 (+  1164us) service_pool.cc:224] Handling call
0709 12:23:27.335028 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:28.668854 (+1333826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.341397 (+672543us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540028ffd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:29.341693 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2772102912}
I20260709 12:23:29.280597 18992 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.220033 19300 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 12:23:29.331913 19107 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:29.308548 19159 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.332037 19315 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:23:29.390745 19227 raft_consensus.cc:2764] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:23:29.393020 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 367929us
I20260709 12:23:29.375730 19335 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.310631 19217 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.405892 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:29.242862 19074 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:23:29.423197 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104969us
I20260709 12:23:29.427479 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:29.407375 19337 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:23:29.326692 18979 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.331913 19157 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.352144 19342 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.397461 19107 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.438724 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:29.416944 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:29.401998 19235 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.445349 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:29.377247 19299 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.420078 19159 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.450834 19256 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:29.452952 19321 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.436007 19074 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.370165 19339 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.393855 19249 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.443310 19336 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:23:29.486770 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:29.486635 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:29.456674 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:29.433384 19315 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:29.489880 19315 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.454109 19157 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:29.494206 19157 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.495296 19157 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.497469 19157 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.472417 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:29.455128 19107 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.398893 18992 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.480903 13793 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:29.484949 13724 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:29.488559 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:29.501355 18992 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.457140 19326 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:29.457217 19286 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:29.522246 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:29.516842 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60260.8us
I20260709 12:23:29.488088 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:29.520296 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:29.432287 19335 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.465716 19074 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:29.532042 13719 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:29.463132 19327 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:29.465557 19234 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:29.545805 19234 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:29.546597 19234 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.547106 19234 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:29.535774 19286 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:23:29.552353 19286 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:29.552855 19286 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.553335 19286 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:29.556011 19159 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:29.557086 19159 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.560523 19249 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:29.566152 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.566952 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.567481 13793 raft_consensus.cc:2368] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:29.566579 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.569021 13798 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:29.570720 19342 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:29.571014 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:29.571565 19342 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:29.572638 19342 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:29.557761 19159 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.456804 19235 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.567884 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:29.579236 19184 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:29.438324 19311 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.523690 19217 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.529772 18979 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.520270 19300 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:23:29.579077 19259 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:23:29.654203 19299 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.646111 19159 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:29.595324 19327 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:29.677150 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:29.685855 13721 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.531s	user 0.015s	sys 0.000s
I20260709 12:23:29.686519 13721 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:29.672168 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:29.697832 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:28.777856 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:29.713646 13548 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 254) took 18576 ms (client timeout 15000 ms). Trace:
I20260709 12:23:29.717099 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:29.720165 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:29.712419 13547 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 9 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.691901 19256 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:29.738835 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:29.741199 13723 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:29.745782 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:29.692031 19159 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.714931 19282 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.764201 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:29.765302 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:29.773607 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:29.737924 19307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.011s	sys 0.019s
I20260709 12:23:29.728209 19326 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:29.781248 19326 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:29.781750 19326 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:29.714555 19287 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.786775 19287 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.789958 19287 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:29.790953 19307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.014s	sys 0.019s
I20260709 12:23:29.728107 19300 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:29.715169 13548 rpcz_store.cc:269] 0709 12:23:11.131635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:11.131866 (+   231us) service_pool.cc:224] Handling call
0709 12:23:29.708401 (+18576535us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:23.623139 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:23.623826 (+   687us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:23.623878 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:23:23.623897 (+    19us) tablet.cc:662] Decoding operations
0709 12:23:23.679216 (+ 55319us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:23.679367 (+   151us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:23.679410 (+    43us) tablet.cc:685] Acquiring locks for 33 operations
0709 12:23:23.734593 (+ 55183us) tablet.cc:701] Row locks acquired
0709 12:23:23.734651 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:23:23.734880 (+   229us) write_op.cc:270] Start()
0709 12:23:23.735064 (+   184us) write_op.cc:276] Timestamp: P: 1783599803734805 usec, L: 0
0709 12:23:23.735112 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:23:23.735986 (+   874us) log.cc:844] Serialized 12847 byte log entry
0709 12:23:24.164576 (+428590us) op_driver.cc:464] REPLICATION: finished
0709 12:23:24.414591 (+250015us) write_op.cc:301] APPLY: starting
0709 12:23:24.414740 (+   149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:25.337074 (+922334us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b180024cee0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:25.352524 (+ 15450us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:25.352575 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:26.214394 (+861819us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:26.214999 (+   605us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=66,key_file_lookups=66,delta_file_lookups=0,mrs_lookups=0
0709 12:23:26.215062 (+    63us) write_op.cc:312] APPLY: finished
0709 12:23:26.216029 (+   967us) log.cc:844] Serialized 284 byte log entry
0709 12:23:26.378484 (+162455us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:26.425232 (+ 46748us) write_op.cc:454] Released schema lock
0709 12:23:26.446600 (+ 21368us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:26.474113 (+ 27513us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:26.474156 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":997458,"reported_tablets":43,"spinlock_wait_cycles":231808,"child_traces":[["op",{"apply.queue_time_us":224721,"apply.run_cpu_time_us":82623,"apply.run_wall_time_us":2059523,"cfile_cache_hit":134,"cfile_cache_hit_bytes":142331,"num_ops":33,"prepare.queue_time_us":162358,"prepare.run_cpu_time_us":32441,"prepare.run_wall_time_us":204105,"raft.queue_time_us":167226,"raft.run_cpu_time_us":5547,"raft.run_wall_time_us":113425,"replication_time_us":429268,"spinlock_wait_cycles":82905984,"thread_start_us":507831,"threads_started":5,"wal-append.queue_time_us":451381,"wal-append.run_cpu_time_us":47715,"wal-append.run_wall_time_us":2201380}]]}
W20260709 12:23:29.806505 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 442007us
W20260709 12:23:29.806783 19245 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.031s	user 0.014s	sys 0.002s
I20260709 12:23:29.855676 19324 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:29.739271 19302 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:23:29.738588 19256 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.847622 19241 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.797482 19159 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:29.881954 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 393470us
I20260709 12:23:29.886526 14418 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:23:29.887672 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 361765us
I20260709 12:23:29.818914 19282 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:29.904124 19354 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.918895 19364 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:29.885094 19326 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.904531 19307 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:29.968073 19301 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.722s	user 0.014s	sys 0.001s
I20260709 12:23:29.883168 19241 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:29.905014 18979 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.905977 13726 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:29.889650 19235 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.077616 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:29.901247 19271 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.988s	user 0.017s	sys 0.005s
W20260709 12:23:29.955502 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67996.3us
I20260709 12:23:29.989283 13547 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 5 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 8 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:29.869555 19339 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:29.916082 19352 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:23:30.018982 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 113540us
I20260709 12:23:29.912592 14424 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:30.000173 19356 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.823496 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:30.045385 19287 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:30.021085 19353 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:30.046627 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:30.047398 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 250367us
W20260709 12:23:30.051312 13718 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.002s	sys 0.020s
I20260709 12:23:30.059149 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:30.121673 13718 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.002s	sys 0.020s
I20260709 12:23:30.121897 13719 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:30.122541 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 2987) took 1434 ms. Trace:
I20260709 12:23:30.060407 19184 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.998440 19186 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:23:30.024875 19327 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:29.827262 19300 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.104099 13723 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:30.110153 19301 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:29.885044 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:30.077236 19350 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:23:30.107785 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:30.119138 13724 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:23:30.124163 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77635.7us
I20260709 12:23:30.126528 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:30.109452 19322 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.123133 13718 rpcz_store.cc:276] 0709 12:23:28.688079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.688235 (+   156us) service_pool.cc:224] Handling call
0709 12:23:28.688565 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:28.709990 (+ 21425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:28.710025 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:28.711904 (+  1879us) log.cc:844] Serialized 26 byte log entry
0709 12:23:28.913743 (+201839us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:29.401834 (+488091us) log.cc:844] Serialized 14 byte log entry
0709 12:23:30.122177 (+720343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.122291 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.122340 (+    49us) raft_consensus.cc:1685] finished
0709 12:23:30.122353 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.122480 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20960,"prepare.queue_time_us":528951,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":121440,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":4708,"thread_start_us":1353517,"threads_started":3,"wal-append.queue_time_us":182512}
I20260709 12:23:30.143324 13799 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:30.045303 19321 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 12:23:30.111413 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 288111us
I20260709 12:23:30.147969 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:30.120007 19217 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:30.125625 19339 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:30.197096 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 89398.3us
I20260709 12:23:30.155921 19256 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:30.145865 19184 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.276165 19217 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.276404 19184 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:30.276701 19217 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:30.277091 19184 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:30.277849 19184 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:30.074838 19307 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.279459 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:30.139055 19069 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:30.094650 18979 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.113210 19271 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.286463 13547 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:30.285778 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 159497us
W20260709 12:23:30.199417 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 314450us
I20260709 12:23:30.293468 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:30.296036 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:30.296893 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:30.150671 19302 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.300595 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2225) took 1021 ms. Trace:
I20260709 12:23:30.123359 19287 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.269712 19301 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.049883 19354 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.120821 19356 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.137014 19353 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.070197 19364 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.287048 19069 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:30.389315 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 276559us
I20260709 12:23:30.318552 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:30.339298 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:30.404129 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:30.385849 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 260610us
W20260709 12:23:30.335752 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60430.4us
W20260709 12:23:30.380160 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226680us
W20260709 12:23:30.524748 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:30.596401 19378 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:30.730232 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 391052us
W20260709 12:23:30.507449 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 83081.4us
I20260709 12:23:30.403625 13796 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:30.358922 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:30.316134 13650 rpcz_store.cc:276] 0709 12:23:29.278802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.278970 (+   168us) service_pool.cc:224] Handling call
0709 12:23:29.279349 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.279738 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.279767 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.280323 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:23:29.461886 (+181563us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:29.461948 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:29.462041 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.300360 (+838319us) raft_consensus.cc:1685] finished
0709 12:23:30.300384 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.300517 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33993,"raft.queue_time_us":903,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":27078,"thread_start_us":153558,"threads_started":1,"wal-append.queue_time_us":419726}
I20260709 12:23:30.371459 13722 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:30.359653 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:30.655714 14424 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:30.748770 13643 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:30.649819 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:30.758760 13650 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:30.301851 19327 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:30.319322 19107 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.819s	user 0.010s	sys 0.006s
W20260709 12:23:30.421332 19358 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.386s	user 0.002s	sys 0.000s
W20260709 12:23:30.771780 19074 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.177s	user 0.017s	sys 0.000s
W20260709 12:23:30.523134 19360 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.454s	user 0.000s	sys 0.001s
W20260709 12:23:30.510576 19313 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.406s	user 0.015s	sys 0.003s
W20260709 12:23:30.593186 19361 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.482s	user 0.000s	sys 0.002s
W20260709 12:23:30.640779 19370 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.369s	user 0.002s	sys 0.000s
I20260709 12:23:30.315054 19159 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:30.308378 19291 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:30.635396 19383 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:30.824564 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68571.9us
I20260709 12:23:30.554385 19241 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:30.760293 19367 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.445s	user 0.002s	sys 0.000s
I20260709 12:23:30.288929 19339 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.728796 19069 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:23:30.829282 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.033s	sys 0.049s
W20260709 12:23:30.864593 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.033s	sys 0.049s
W20260709 12:23:30.865051 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 560996us
I20260709 12:23:30.766645 19374 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:23:30.761205 19355 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:23:30.302631 19184 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.749785 19378 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:30.836440 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:30.755632 19271 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.839037 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:30.761729 19300 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:30.840189 13650 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:23:30.318117 19322 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.538378 19326 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:30.826398 19327 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:30.586912 19254 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:30.870873 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:30.877480 19398 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:30.882444 19385 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:30.934108 19385 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:30.884414 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 111686us
I20260709 12:23:30.361809 19302 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.796622 19311 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 12:23:30.937776 19384 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:23:30.938043 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:30.938557 19388 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:23:30.940305 19302 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:30.863584 19372 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.467s	user 0.000s	sys 0.002s
I20260709 12:23:30.940795 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:30.857082 19313 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.942070 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 890) took 1064 ms. Trace:
W20260709 12:23:30.943287 19324 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.087s	user 0.014s	sys 0.001s
I20260709 12:23:30.942463 14423 rpcz_store.cc:276] 0709 12:23:29.877914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.878094 (+   180us) service_pool.cc:224] Handling call
0709 12:23:29.878484 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.878867 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.878896 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.888708 (+  9812us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.029048 (+140340us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:30.029089 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.029175 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:30.941829 (+912654us) raft_consensus.cc:1685] finished
0709 12:23:30.941847 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:30.941996 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2290,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":1694,"thread_start_us":89202,"threads_started":1,"wal-append.queue_time_us":388365}
I20260709 12:23:30.943552 19313 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.943866 19245 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.864111 19291 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.944645 19394 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:30.945497 19394 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:30.824095 19379 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:23:30.946584 19291 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.899729 19378 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:30.947222 19287 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.927596 19241 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.948259 19241 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.949193 19301 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.899143 13718 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:30.910461 19353 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:30.914646 19390 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.950973 19258 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:30.899957 19387 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:30.951407 19390 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:30.917414 19371 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.382s	user 0.000s	sys 0.001s
I20260709 12:23:30.922744 13725 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:30.929042 19159 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:30.952076 19387 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:30.957479 19286 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.404s	user 0.010s	sys 0.005s
I20260709 12:23:30.838320 19107 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:30.958657 19159 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.960032 19107 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:30.960822 19217 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.683s	user 0.008s	sys 0.005s
I20260709 12:23:30.954753 19390 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.961550 13647 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:30.868747 19383 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.931284 19184 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:30.932165 19375 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:23:30.901316 19259 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:30.944381 19245 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:30.916726 19356 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:30.963847 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:30.843796 19074 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:30.980911 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 338992us
I20260709 12:23:30.927839 19327 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.003129 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.019111 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.019987 13723 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:31.026489 19387 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.027369 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.028733 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.031314 19354 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.032343 19241 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.033849 19339 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:30.997406 19382 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:31.035205 19382 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.036388 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3302) took 1198 ms. Trace:
I20260709 12:23:31.038491 19159 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.038826 19074 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:30.965782 19254 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.040817 19254 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:31.042081 19234 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.494s	user 0.018s	sys 0.000s
I20260709 12:23:31.042584 19234 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.043952 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3301) took 1207 ms. Trace:
I20260709 12:23:31.044802 19234 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.017668 19377 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:30.951634 19258 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.046286 19377 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.048281 19258 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.048642 19377 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.043855 19107 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:31.051343 19107 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:31.051932 19107 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:31.051440 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2840) took 1586 ms. Trace:
I20260709 12:23:31.052928 13650 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:31.056224 13650 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:31.030056 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.036777 13648 rpcz_store.cc:276] 0709 12:23:29.837771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.861318 (+ 23547us) service_pool.cc:224] Handling call
0709 12:23:29.876954 (+ 15636us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:29.881523 (+  4569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:29.881567 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:29.888094 (+  6527us) log.cc:844] Serialized 26 byte log entry
0709 12:23:30.046759 (+158665us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:30.046811 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:30.047718 (+   907us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:31.036132 (+988414us) raft_consensus.cc:1685] finished
0709 12:23:31.036158 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.036316 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4080,"raft.queue_time_us":4633,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":5564,"thread_start_us":131432,"threads_started":1,"wal-append.queue_time_us":505149}
I20260709 12:23:31.014611 19400 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:31.048292 19364 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.060156 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:31.044402 13645 rpcz_store.cc:276] 0709 12:23:29.836927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.838668 (+  1741us) service_pool.cc:224] Handling call
0709 12:23:29.839010 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.284422 (+445412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.284456 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.284509 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:30.284559 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.043703 (+759144us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:31.043753 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.043886 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023784320}
I20260709 12:23:31.055948 13722 rpcz_store.cc:276] 0709 12:23:29.464686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.464902 (+   216us) service_pool.cc:224] Handling call
0709 12:23:29.465312 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.742218 (+1276906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.051178 (+308960us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54000f5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:31.051372 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083096448}
W20260709 12:23:31.020673 13798 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.006s	sys 0.005s
I20260709 12:23:30.958045 19286 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.061621 19234 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.072116 19234 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.066421 19356 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:31.075568 19356 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:31.076152 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3000) took 2315 ms (client timeout 1600 ms). Trace:
W20260709 12:23:31.076629 13721 rpcz_store.cc:269] 0709 12:23:28.760840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.783946 (+ 23106us) service_pool.cc:224] Handling call
0709 12:23:31.075868 (+2291922us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54001a5754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:31.076011 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":722073472}
I20260709 12:23:31.065239 19170 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.077622 19170 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.079696 19383 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.080396 19324 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:30.965703 19217 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.086776 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:31.051271 19245 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.090492 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.079710 13720 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:23:31.063663 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 209767us
I20260709 12:23:31.095115 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.037249 19382 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.102123 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:31.063578 13798 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.006s	sys 0.006s
I20260709 12:23:31.094403 19364 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:31.097385 13651 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:31.112099 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.110474 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.112303 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:31.114398 13648 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:31.114091 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.115971 14424 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:31.111006 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108007us
I20260709 12:23:31.117146 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.117640 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.118309 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.118317 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.119557 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.119694 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.120167 13793 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:31.120368 14423 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:31.121025 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.121299 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.075030 19234 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.122565 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.131045 14420 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:31.123683 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.123955 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.123742 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.123967 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.126135 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.133462 14416 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:31.133702 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.121240 19382 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.121408 19339 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.096647 19287 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.135224 13800 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:31.133816 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:31.135234 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.137282 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:31.147063 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.150235 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.133426 14425 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:31.138677 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.140367 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:31.152787 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:31.141690 14422 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:31.142103 13725 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:23:31.151466 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:31.154981 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.156757 19286 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.158130 19383 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:31.158684 19395 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.159441 19383 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:31.145653 19234 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:31.160737 19234 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.162369 19287 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.163100 19339 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.159489 19395 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.165059 19287 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.165532 19339 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:31.162396 19170 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.142858 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.167927 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:31.169838 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.142313 19406 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:31.170903 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.171429 19379 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:31.171232 19406 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.172209 19379 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:31.173168 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.102572 19326 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.174665 19402 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:31.143272 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.176270 19374 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:31.176644 19374 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:31.122612 19364 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.177762 19374 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.143631 19321 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:31.111461 19245 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.179273 19364 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.179584 19245 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.180529 19312 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:31.180689 19234 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.181002 19312 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:31.166079 19395 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.182641 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.109959 19385 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.187081 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.187355 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.145844 19382 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.174645 19326 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.189292 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2852) took 1655 ms. Trace:
I20260709 12:23:31.189484 19382 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.189857 19326 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:31.189706 13720 rpcz_store.cc:276] 0709 12:23:29.534462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.561508 (+ 27046us) service_pool.cc:224] Handling call
0709 12:23:31.079447 (+1517939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:31.096350 (+ 16903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.189100 (+ 92750us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54001a5750. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:31.189229 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3489780480}
I20260709 12:23:31.190778 19356 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.142195 19217 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.188009 13645 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
W20260709 12:23:31.191989 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47077.6us
W20260709 12:23:31.192584 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1131) took 1518 ms (client timeout 1730 ms). Trace:
I20260709 12:23:31.192689 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.197270 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:31.193311 13793 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:31.193660 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.201066 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.177152 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.195034 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.206660 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:31.205899 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.208230 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:31.192508 19217 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:31.219659 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.221423 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:31.210686 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.211294 13648 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:31.197120 13726 rpcz_store.cc:269] 0709 12:23:29.674582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.697470 (+ 22888us) service_pool.cc:224] Handling call
0709 12:23:31.192410 (+1494940us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:31.192520 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475974400}
I20260709 12:23:31.217525 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.218170 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.218760 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.220875 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.212136 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:31.223295 13645 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:31.221918 13649 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:31.222345 13800 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:31.173991 19406 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.227082 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.228482 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.229776 13649 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:31.237895 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.238674 13726 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:31.239065 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.203826 13650 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:31.230208 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93616.7us
I20260709 12:23:31.230584 19324 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:31.228741 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.245985 13652 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 16. Ops: []
I20260709 12:23:31.231873 14425 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:23:31.250385 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.251602 14425 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:31.253000 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.232402 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3309) took 1228 ms. Trace:
I20260709 12:23:31.232416 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:31.224119 14420 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.138064 19400 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:31.235756 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.241598 13693 leader_election.cc:400] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:31.267115 13693 leader_election.cc:403] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:31.268297 13693 leader_election.cc:400] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:31.268522 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.243837 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.094784 19107 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.269394 14425 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:31.173506 19379 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.243647 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.257239 13645 rpcz_store.cc:276] 0709 12:23:30.004154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.231501 (+1227347us) service_pool.cc:224] Handling call
0709 12:23:31.231855 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.232072 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.232097 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:31.232141 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:31.232178 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.232233 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.232342 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:31.193866 19258 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.234575 19241 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:31.181891 19245 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.272790 19390 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:31.302397 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.317023 13648 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:31.268761 13693 leader_election.cc:403] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:31.239969 19387 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:31.299700 19217 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:31.233534 19395 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:31.194154 19074 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:23:31.311568 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.304102 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:31.232535 19382 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.231642 19321 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:31.231276 19159 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:31.320592 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 88369.7us
I20260709 12:23:31.312800 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:31.392056 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79379.1us
I20260709 12:23:31.301833 19184 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:31.394134 19184 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.394567 19184 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:31.390633 19321 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.395643 19321 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:31.336249 19380 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:23:31.320327 19258 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.306748 19287 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:31.403733 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:31.342469 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.277204 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:31.349696 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.020s	sys 0.065s
I20260709 12:23:31.304149 19406 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:31.312947 19327 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.304585 19378 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.271529 19069 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:31.375803 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 132228us
I20260709 12:23:31.241338 19383 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:23:31.333488 19245 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:31.322418 19356 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:31.354794 19363 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:31.218158 19326 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:23:31.245716 19234 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:31.181877 19312 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:31.379791 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 111786us
I20260709 12:23:31.408758 19355 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:31.393545 19159 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:31.399984 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96005.3us
I20260709 12:23:31.388352 19395 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:31.405794 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.020s	sys 0.065s
W20260709 12:23:31.435992 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 786246us
I20260709 12:23:31.384418 19217 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.417410 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.417806 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.393187 19107 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.399296 19318 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:31.401388 19286 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.400771 19364 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.424237 14419 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:31.409024 19259 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:31.406751 19406 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:31.396896 19382 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.428629 14417 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:31.430621 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.430999 19362 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.382184 14420 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:31.433123 18852 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.475612 19400 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:31.440127 19395 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.424614 19356 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:31.434334 19159 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.467358 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.442241 19318 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:31.442772 19217 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.443811 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.448179 19355 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:31.468907 13767 connection.cc:256] RPC call timeout handler was delayed by -1.5036919829999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
I20260709 12:23:31.441485 19312 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.447523 19363 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:31.431320 19234 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.442611 19300 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.489164 19307 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.491734 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3291) took 1770 ms. Trace:
I20260709 12:23:31.471462 19406 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:31.494350 19404 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:23:31.496557 19327 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:31.464841 19259 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.503512 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:31.515193 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71515.1us
I20260709 12:23:31.473829 19326 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:31.540915 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.541690 13796 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:31.541951 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.528306 13652 rpcz_store.cc:276] 0709 12:23:29.721579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.725543 (+  3964us) service_pool.cc:224] Handling call
0709 12:23:29.726340 (+   797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.270530 (+1544190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.491396 (+220866us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002224d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:31.491654 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3485347584}
I20260709 12:23:31.449435 19286 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.541553 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.480989 18852 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.464582 19364 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.518038 19355 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.519472 19363 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:31.587405 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45960.3us
I20260709 12:23:31.479229 19362 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.518049 19403 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:31.519085 19387 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:31.629571 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.585317 19383 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:31.511467 19254 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.512720 19318 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:31.552945 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:31.533433 19408 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:23:31.546809 19259 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.543533 19300 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:31.646624 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.586752 19326 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:31.622888 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.656296 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3259) took 1499 ms. Trace:
I20260709 12:23:31.620967 19258 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.623111 19400 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:31.631170 19372 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.094s	user 0.000s	sys 0.002s
W20260709 12:23:31.670576 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 117850us
I20260709 12:23:31.626765 19327 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:31.634964 19363 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:31.631964 19355 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.634567 19387 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.686663 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:31.693878 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.695127 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:31.694977 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 190372us
I20260709 12:23:31.696444 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.664180 19074 raft_consensus.cc:2764] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:23:31.688434 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.676420 13798 rpcz_store.cc:276] 0709 12:23:30.156438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.156619 (+   181us) service_pool.cc:224] Handling call
0709 12:23:30.286539 (+129920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.286794 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:31.113181 (+826387us) log.cc:844] Serialized 14 byte log entry
0709 12:23:31.655743 (+542562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:31.655837 (+    94us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:31.655921 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.656024 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.656207 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74749,"prepare.queue_time_us":146133,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":84135,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":7545,"thread_start_us":1258711,"threads_started":3,"wal-append.queue_time_us":295384}
I20260709 12:23:31.673333 19157 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:31.614234 19356 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:31.702013 19415 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.276s	user 0.002s	sys 0.000s
I20260709 12:23:31.714833 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.687402 19327 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.699617 19387 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:31.689953 19069 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.669476 19254 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.721809 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:31.722856 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.698894 19378 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.736639 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.738193 13726 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:23:31.738328 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86107.7us
I20260709 12:23:31.739321 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:31.686658 19421 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.740152 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.700317 19324 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:31.720208 13652 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:23:31.701478 19383 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:31.711369 19409 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:23:31.713126 19400 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:31.725386 19327 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:31.735677 13724 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:31.700135 19394 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:31.739382 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:31.741140 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.761909 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.722317 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:31.741936 13726 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:31.743114 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.743575 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.744951 13798 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:31.745368 19383 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.730468 19356 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:31.746220 19414 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:31.768329 13800 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:23:31.746704 19382 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:31.726063 19326 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.747519 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:31.772912 13652 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:31.748446 19286 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.749087 19234 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:31.721349 19157 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:23:31.761828 19421 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.733728 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.785305 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.786199 19362 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.787138 19413 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:31.787025 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.763401 13727 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:31.787971 19413 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.789418 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:31.770388 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.790621 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:31.791230 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:31.791524 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:31.767529 13726 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:31.772600 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.773631 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:31.748307 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.794903 13795 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:31.775408 14425 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:23:31.776278 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54919.1us
I20260709 12:23:31.789721 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.796248 13798 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 8.
I20260709 12:23:31.818639 13649 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:31.819875 19429 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:31.820189 19364 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.820474 19429 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.820967 19429 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:31.782897 19417 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 12:23:31.825232 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 114369us
I20260709 12:23:31.825439 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:31.827566 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.827983 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.828347 13690 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:31.824337 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.764186 19258 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:31.829676 13722 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:31.768815 19383 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.838990 13652 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:31.842731 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.851121 13798 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:31.842514 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.852406 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.852339 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.857442 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.861102 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.782979 19254 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.866720 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.867726 13652 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:31.834009 19157 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:31.868768 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:31.869390 13722 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:31.866670 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.884174 14424 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:23:31.910233 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.911448 13652 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:23:31.912245 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:31.843032 19258 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.913851 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.914355 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:31.843708 19184 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.924185 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:31.924913 13649 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:31.842746 19400 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 12:23:31.935700 13722 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
W20260709 12:23:31.930588 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:31.929289 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:31.937423 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.940495 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:31.912953 19307 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:23:31.947937 19438 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:23:31.951547 19282 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.928839 19426 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:31.965602 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:31.966507 19245 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:31.973610 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2859) took 2224 ms. Trace:
I20260709 12:23:32.029996 19400 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:32.029089 13719 rpcz_store.cc:276] 0709 12:23:29.749411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.122873 (+373462us) service_pool.cc:224] Handling call
0709 12:23:30.123194 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.770792 (+647598us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:31.012444 (+241652us) log.cc:844] Serialized 14 byte log entry
0709 12:23:31.218210 (+205766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:31.218283 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:31.218331 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:31.973287 (+754956us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54000285d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:31.973355 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:31.973531 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47644,"prepare.queue_time_us":300965,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":14278,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":12381,"spinlock_wait_cycles":1488725888,"thread_start_us":373647,"threads_started":2,"wal-append.queue_time_us":258806}
I20260709 12:23:31.980130 19258 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.984894 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:31.929436 19300 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:31.998221 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.007247 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:32.008249 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.006633 19227 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:32.021184 19355 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.038086 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.038966 13719 raft_consensus.cc:2376] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10. Re-sending same reply.
I20260709 12:23:32.026815 19184 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.026302 19387 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.028074 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.045756 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.030484 19400 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:31.915550 19157 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:32.032418 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.035521 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:32.035643 13722 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:32.023032 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:32.014089 19107 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.002s
W20260709 12:23:32.034246 19395 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.544s	user 0.014s	sys 0.000s
I20260709 12:23:32.071488 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.071914 19395 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:31.945325 19159 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.074615 19395 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.076588 19374 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.076812 19159 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.049512 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:32.080166 19434 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:32.081291 19327 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.083133 19421 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.084976 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.089373 19355 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.089437 13800 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:23:32.053460 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:32.049953 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.070917 19394 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:32.042117 19443 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.073637 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:32.050748 19321 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.651s	user 0.015s	sys 0.000s
I20260709 12:23:31.991549 19307 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.050664 19400 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.089952 19383 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.033381 19426 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.098913 19335 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.099105 19394 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.120302 19394 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:32.048269 19227 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:32.112210 19406 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.565s	user 0.014s	sys 0.000s
W20260709 12:23:32.122305 19217 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.003s
I20260709 12:23:32.113116 19321 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.108616 19429 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.141825 19217 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.148449 19217 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.149821 19322 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.149711 19259 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.137387 19406 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.153795 19259 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.151741 19321 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.151345 19305 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.155884 19406 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.157110 19387 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.167469 19426 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.168653 19327 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.161082 19368 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:32.169224 19368 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:32.134032 19363 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.172670 19429 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.173061 19383 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.173367 19363 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.171041 19368 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.174147 19307 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.167553 19413 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.175110 19368 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:32.183007 19434 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:32.187813 19420 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:32.188367 19420 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:32.167713 19400 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.189648 19420 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.195936 19400 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.185227 19394 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:31.988417 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.292403 19227 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.294003 19307 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:32.295390 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 306986us
I20260709 12:23:32.297160 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 2867) took 1461 ms. Trace:
I20260709 12:23:32.297638 19307 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:32.299039 19394 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.237478 13645 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:32.239792 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.054440 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.297706 13723 rpcz_store.cc:276] 0709 12:23:30.835964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.836146 (+   182us) service_pool.cc:224] Handling call
0709 12:23:32.296639 (+1460493us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54002f6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:32.296859 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288409472}
I20260709 12:23:32.241909 13724 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:32.047441 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.009367 13649 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:32.312882 13650 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.313721 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54768 (request call id 3307) took 2315 ms. Trace:
I20260709 12:23:32.051631 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:32.315541 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 263803us
I20260709 12:23:32.249169 13652 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19.
W20260709 12:23:32.250922 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 171412us
I20260709 12:23:32.274027 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3304) took 2321 ms. Trace:
I20260709 12:23:32.309664 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.275897 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3303) took 2375 ms. Trace:
I20260709 12:23:32.277772 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3299) took 2441 ms. Trace:
I20260709 12:23:32.279868 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 903) took 1151 ms. Trace:
I20260709 12:23:32.154567 19318 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.138154 19157 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.324043 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.325066 13727 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:32.319501 13648 rpcz_store.cc:276] 0709 12:23:31.128477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.317982 (+189505us) service_pool.cc:224] Handling call
0709 12:23:31.318465 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.547933 (+229468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.547984 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:31.548048 (+    64us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:23:31.548103 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.279269 (+731166us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.279339 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.279624 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527063808}
I20260709 12:23:32.328277 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2219) took 3565 ms. Trace:
I20260709 12:23:32.329106 13643 rpcz_store.cc:276] 0709 12:23:28.762960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:28.955689 (+192729us) service_pool.cc:224] Handling call
0709 12:23:28.999303 (+ 43614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.749250 (+1749947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.327240 (+1577990us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54001b6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.328058 (+   818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4022898432}
W20260709 12:23:32.196125 19432 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.283s	user 0.003s	sys 0.000s
I20260709 12:23:32.318922 13647 rpcz_store.cc:276] 0709 12:23:29.835971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:29.837962 (+  1991us) service_pool.cc:224] Handling call
0709 12:23:29.854583 (+ 16621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:30.962541 (+1107958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.277021 (+1314480us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400096d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.277548 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2545155456}
I20260709 12:23:32.289176 19425 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.331686 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:32.331602 19425 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.318765 13646 rpcz_store.cc:276] 0709 12:23:29.900110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.074042 (+173932us) service_pool.cc:224] Handling call
0709 12:23:30.074562 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.357827 (+283265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.357862 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.357910 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:30.357954 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.275172 (+1917218us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54001e6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.275273 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.275659 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650653952}
I20260709 12:23:32.263592 14425 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:32.314159 13650 rpcz_store.cc:276] 0709 12:23:30.003466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.830647 (+827181us) service_pool.cc:224] Handling call
0709 12:23:32.283251 (+1452604us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54002224d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393beba3d 00007f5398bbbcd0 00007f5392140254 00007f53921401e2 00007f5392140111 00007f539214008c 00007f539213f342 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:32.313590 (+ 30339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346765696}
I20260709 12:23:32.309276 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.301949 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.323154 19318 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.332089 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.318423 13644 rpcz_store.cc:276] 0709 12:23:29.952612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.095187 (+142575us) service_pool.cc:224] Handling call
0709 12:23:30.124134 (+ 28947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.825732 (+701598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.825766 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.825887 (+   121us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:30.825946 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.273321 (+1447375us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.273418 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.273752 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1613007744}
I20260709 12:23:32.313529 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.320676 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2871) took 1198 ms. Trace:
I20260709 12:23:32.336581 13721 rpcz_store.cc:276] 0709 12:23:31.119179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.119408 (+   229us) service_pool.cc:224] Handling call
0709 12:23:31.119714 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.195593 (+ 75879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.195624 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:31.195664 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:31.195695 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.316950 (+1121255us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b540021fcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.317064 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.317393 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174003584}
W20260709 12:23:32.321678 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.297976 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.340353 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.340909 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.342582 19307 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:32.342818 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45190.8us
I20260709 12:23:32.340390 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:32.345263 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.346133 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.330015 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.339007 19425 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.348958 13723 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:32.362416 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 843) took 1962 ms. Trace:
I20260709 12:23:32.363123 13792 rpcz_store.cc:276] 0709 12:23:30.390335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.390472 (+   137us) service_pool.cc:224] Handling call
0709 12:23:30.427915 (+ 37443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:30.932157 (+504242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:30.932189 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:30.932249 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:23:30.932284 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.351709 (+1419425us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54002cb0d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.351817 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.352178 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158470400}
I20260709 12:23:32.378152 13726 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:32.379431 13649 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:32.379359 19404 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:32.379962 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.381428 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:23:32.382355 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:32.383821 14419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 936) took 2092 ms (client timeout 2034 ms). Trace:
W20260709 12:23:32.384435 14419 rpcz_store.cc:269] 0709 12:23:30.291615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.291804 (+   189us) service_pool.cc:224] Handling call
0709 12:23:32.383197 (+2091393us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54004c3e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:32.383487 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1551345920}
I20260709 12:23:32.384845 19400 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:32.386209 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.402683 13727 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:32.405557 19404 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:32.406349 19404 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:32.410328 13650 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:32.411337 19394 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:32.411854 19394 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.412267 19394 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:32.378774 19318 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:23:32.415047 19318 consensus_queue.cc:1243] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:32.416967 19318 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.418464 19425 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:32.419389 19425 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.421237 13722 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:32.422704 19307 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:32.424062 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.424912 19307 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:32.426288 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.426518 19404 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.428201 19404 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:23:32.429495 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:32.431033 13799 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:32.433971 13796 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:23:32.434726 13721 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:32.436015 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74941.8us
I20260709 12:23:32.436196 18979 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:32.439348 19453 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:23:32.440109 19379 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:32.441891 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.443660 19374 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:32.406682 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.444231 19374 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:32.444522 19241 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:32.445029 19241 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:32.447083 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.446954 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.436277 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.452845 13650 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:32.458786 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.458864 13801 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:32.459805 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.460479 19326 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.460860 14423 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:32.462786 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.462916 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.465790 14417 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:23:32.466776 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44685.3us
I20260709 12:23:32.445699 19374 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.467095 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:32.468346 19174 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.469244 19174 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.471810 18852 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.472277 19422 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:32.472903 19422 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:32.473327 19422 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:32.467149 19425 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.474505 14420 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:32.475452 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:32.477067 19307 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:32.478431 19174 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:32.417807 19318 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.469269 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:32.478585 19307 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:32.478899 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.478886 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:32.477119 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:32.467218 19186 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.460752 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:32.468394 19374 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:32.483964 19307 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:32.495748 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:32.496501 13648 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:32.499686 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.479153 19241 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.506784 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.508213 14419 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:32.509353 19227 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:32.509639 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.509961 19227 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:32.510439 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:32.510599 19227 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:32.511327 19400 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:32.512172 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.513636 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:32.514081 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3269) took 1335 ms. Trace:
I20260709 12:23:32.514585 13793 rpcz_store.cc:276] 0709 12:23:31.178531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.180597 (+  2066us) service_pool.cc:224] Handling call
0709 12:23:31.180931 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.198131 (+ 17200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.513591 (+1315460us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.513868 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27589888}
I20260709 12:23:32.502207 19425 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:32.517135 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2877) took 1372 ms. Trace:
I20260709 12:23:32.520838 19318 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.504493 19263 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:32.517722 13725 rpcz_store.cc:276] 0709 12:23:31.145150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.157120 (+ 11970us) service_pool.cc:224] Handling call
0709 12:23:31.157604 (+   484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:31.214190 (+ 56586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.214240 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:31.214330 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:31.214382 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.516505 (+1302123us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54000280d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.516609 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.516894 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129505024}
I20260709 12:23:32.525123 19311 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:32.536300 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.541895 19311 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:32.538661 19241 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.534668 19400 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:32.549444 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88945us
W20260709 12:23:32.555625 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78741.4us
I20260709 12:23:32.560416 13649 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:32.563252 13647 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:32.578775 19318 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:32.579789 19318 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.576718 13646 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:32.585775 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108171us
W20260709 12:23:32.584118 19450 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.136s	user 0.001s	sys 0.002s
I20260709 12:23:32.621191 19420 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:32.615895 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 856) took 1121 ms. Trace:
I20260709 12:23:32.623283 13794 rpcz_store.cc:276] 0709 12:23:31.419174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.420367 (+  1193us) service_pool.cc:224] Handling call
0709 12:23:31.420793 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:31.421379 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.421424 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:31.424305 (+  2881us) log.cc:844] Serialized 26 byte log entry
0709 12:23:32.345070 (+920765us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b1800332aa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:23:32.501901 (+156831us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:32.527691 (+ 25790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.527796 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:32.527987 (+   191us) raft_consensus.cc:1685] finished
0709 12:23:32.528016 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.540735 (+ 12719us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1719,"raft.queue_time_us":13824,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":7557,"spinlock_wait_cycles":8692224,"thread_start_us":108349,"threads_started":1,"wal-append.queue_time_us":1086336}
W20260709 12:23:32.624697 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.584949 19400 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.587625 19311 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.659560 19383 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:32.661242 19383 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.654062 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:32.666143 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52584us
W20260709 12:23:32.670177 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.683324 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:32.684705 14423 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
W20260709 12:23:32.686667 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54525.5us
I20260709 12:23:32.687397 19354 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:32.689038 19446 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:32.690656 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:32.691455 19311 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:23:32.702945 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 131821us
W20260709 12:23:32.715895 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98116.3us
I20260709 12:23:32.701069 19400 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.705072 19394 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.731470 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:32.735190 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.729468 19403 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:23:32.740461 19354 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:32.738962 19318 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.725541 19307 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.728168 19400 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:32.749240 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:32.739336 19241 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:32.757270 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 84488.8us
I20260709 12:23:32.754992 19354 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.765434 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.728581 19227 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.766636 19419 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:23:32.768836 19458 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:32.770097 19458 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:32.771064 19458 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:32.773054 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 219515us
I20260709 12:23:32.773880 19241 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.791568 13721 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:23:32.793006 13725 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:23:32.766103 19354 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.796905 13648 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:32.803318 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:32.804674 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.763321 19400 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.785473 19383 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.794415 19419 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:32.810559 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95381.1us
I20260709 12:23:32.816533 19241 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.818750 13725 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 12:23:32.822455 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.822753 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.824594 13724 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:32.829864 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:32.836185 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.840585 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2239) took 1118 ms. Trace:
I20260709 12:23:32.841316 13645 rpcz_store.cc:276] 0709 12:23:31.722497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.722623 (+   126us) service_pool.cc:224] Handling call
0709 12:23:32.837495 (+1114872us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b54001b6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:32.840478 (+  2983us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158317312}
I20260709 12:23:32.840701 19400 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.832182 19403 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:32.845805 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49683.2us
W20260709 12:23:32.848481 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66216.5us
W20260709 12:23:32.840863 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99869.2us
I20260709 12:23:32.851121 19458 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.851399 19227 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.853600 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.854152 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:32.854818 13725 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:32.855000 19305 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:32.856523 19394 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:32.857125 19457 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:23:32.858300 19272 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:32.862221 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.872251 19434 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:32.872964 19434 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:32.874341 19368 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.610s	user 0.004s	sys 0.017s
I20260709 12:23:32.877862 13649 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:32.899423 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:32.880055 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 895) took 1764 ms. Trace:
I20260709 12:23:32.879869 19422 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.910555 14421 rpcz_store.cc:276] 0709 12:23:31.115070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.138510 (+ 23440us) service_pool.cc:224] Handling call
0709 12:23:31.138857 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:31.212904 (+ 74047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.212956 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:31.213671 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 12:23:32.376342 (+1162671us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b18003b20c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:23:32.528136 (+151794us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:32.531332 (+  3196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.879538 (+348206us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b54005965d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.879599 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:32.879654 (+    55us) raft_consensus.cc:1685] finished
0709 12:23:32.879667 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.879848 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61641,"raft.queue_time_us":43338,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":41755,"spinlock_wait_cycles":162970112,"thread_start_us":107075,"threads_started":1,"wal-append.queue_time_us":1312247}
I20260709 12:23:32.882650 13793 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:32.913291 13795 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:23:32.913525 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.915771 13793 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:32.913293 19307 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:32.917299 13793 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:32.917614 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:32.884603 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.918382 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.918869 13722 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:32.919059 19307 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:32.919313 13725 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:32.885073 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.919749 19307 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:32.920266 13721 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:32.886375 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.921387 13727 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:32.920894 19307 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.921561 19422 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:32.887630 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.922917 13726 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:32.923151 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:32.891444 13800 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:23:32.924965 19446 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:32.918877 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62917.7us
I20260709 12:23:32.891937 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.928467 13719 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:32.894562 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.930001 13801 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:32.894728 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:32.895069 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:32.930986 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.931525 13796 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:32.932312 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.896580 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.932986 13801 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:32.897073 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:32.935694 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:32.900393 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:32.900820 19318 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:32.934190 13801 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:32.941387 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:32.944024 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:32.878787 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3008) took 1143 ms. Trace:
I20260709 12:23:32.930946 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:32.945942 13723 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 10. Current term is 11. Ops: []
W20260709 12:23:32.904789 19459 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:32.908535 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:32.881117 19263 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:32.886704 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:32.946820 19335 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:32.948379 19335 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:32.949604 19335 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.945442 13720 rpcz_store.cc:276] 0709 12:23:31.735730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.735881 (+   151us) service_pool.cc:224] Handling call
0709 12:23:31.764079 (+ 28198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:31.808236 (+ 44157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:31.808274 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:31.808866 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:23:32.241748 (+432882us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:32.241924 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:32.878167 (+636243us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b54001a5254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:32.878214 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:32.878304 (+    90us) raft_consensus.cc:1685] finished
0709 12:23:32.878318 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:32.878594 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1155,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":7471,"spinlock_wait_cycles":100497280,"thread_start_us":431807,"threads_started":1,"wal-append.queue_time_us":326310}
I20260709 12:23:32.955919 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.961278 13723 raft_consensus.cc:2368] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:23:32.962597 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.963665 13719 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:32.966501 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:32.967140 19394 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:32.967659 13723 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:32.967963 19394 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:32.969044 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60619.8us
I20260709 12:23:32.969985 19368 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:32.970587 19368 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:32.971740 19368 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:32.975908 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:32.874202 19434 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:32.966600 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:32.977330 13720 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:32.977438 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46658us
W20260709 12:23:32.999560 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55126.8us
I20260709 12:23:33.001291 13796 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:33.020614 13792 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:32.984973 19420 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:33.018353 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:33.046355 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:33.047720 13792 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.056936 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.058120 13721 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:33.059229 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:33.059958 13721 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:33.065591 19434 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:33.035030 19394 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:33.068665 19326 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.002s
I20260709 12:23:33.070505 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.071273 19382 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.095671 19382 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.071444 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:33.097920 13797 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:23:33.102756 13797 pending_rounds.cc:85] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 12
I20260709 12:23:33.103108 13797 pending_rounds.cc:107] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:23:33.103426 13797 raft_consensus.cc:2889] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:33.074432 19318 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:33.075377 19174 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:33.076419 19069 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:33.080411 19385 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:33.081012 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.082453 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.083139 19404 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:33.083652 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:33.084681 13725 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:33.084755 19354 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:33.085194 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:33.085968 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85255us
I20260709 12:23:33.098709 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:33.070539 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:33.099800 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.099958 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.101047 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:33.101228 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.112825 13723 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.113140 13726 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.113304 13720 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.116529 13645 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:33.093832 19394 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:33.122182 13795 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:33.048018 19420 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.108984 19404 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:33.133209 13726 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:23:33.133772 19404 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.134665 19354 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.135620 19404 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:33.137111 13725 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:33.137822 19354 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.139894 13647 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:33.140482 13719 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.141202 19157 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:33.142092 19157 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:33.143863 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.600s (SENT)
I20260709 12:23:33.143975 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:33.144663 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:33.145164 13652 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:23:33.146471 19460 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:33.163079 19403 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:33.163734 19403 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:33.164829 19403 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.170989 19429 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:33.173717 13767 leader_election.cc:400] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19.
I20260709 12:23:33.174139 13767 leader_election.cc:403] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:33.175457 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52057.7us
I20260709 12:23:33.123559 13649 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:33.130754 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.181466 13645 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:33.184094 19305 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19.
I20260709 12:23:33.185563 19305 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:33.185575 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:33.121791 19227 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:33.125982 19394 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:33.193984 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.211658 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 170684us
I20260709 12:23:33.201067 19227 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:33.255053 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 129232us
W20260709 12:23:33.257840 19452 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:23:33.259366 19326 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:33.259635 19420 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.260675 19382 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.260876 13693 leader_election.cc:400] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:23:33.261266 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 845) took 2863 ms. Trace:
I20260709 12:23:33.261746 13797 rpcz_store.cc:276] 0709 12:23:30.392098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:30.392256 (+   158us) service_pool.cc:224] Handling call
0709 12:23:30.392580 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:33.103975 (+2711395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.246734 (+142759us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b54000e65d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:33.255321 (+  8587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6220285312}
I20260709 12:23:33.263744 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:33.261284 13693 leader_election.cc:403] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:33.279587 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.274276 18852 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:33.274668 19227 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:33.286737 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.286914 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.281140 19335 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:23:33.289590 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 185257us
I20260709 12:23:33.289076 19460 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:23:33.280519 19368 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.303864 19256 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.304610 19256 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.295938 19174 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:23:33.306571 19354 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:33.309115 19461 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.309942 19461 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.313412 19461 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.315275 19256 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:33.317384 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 2875) took 2158 ms. Trace:
I20260709 12:23:33.317889 13718 rpcz_store.cc:276] 0709 12:23:31.126138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:31.152389 (+ 26251us) service_pool.cc:224] Handling call
0709 12:23:31.152727 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:31.153183 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:23:31.231062 (+ 77879us) log.cc:844] Serialized 14 byte log entry
0709 12:23:32.841897 (+1610835us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b1800389c20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:23:32.842201 (+   304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:32.843029 (+   828us) log.cc:844] Serialized 26 byte log entry
0709 12:23:33.118475 (+275446us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:33.118513 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:33.122646 (+  4133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:33.270366 (+147720us) raft_consensus.cc:1685] finished
0709 12:23:33.270527 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:33.284381 (+ 13854us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":78909,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":28223,"raft.queue_time_us":1196,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":289,"spinlock_wait_cycles":28084608,"thread_start_us":353800,"threads_started":2,"wal-append.queue_time_us":590244,"wal-append.run_cpu_time_us":3467,"wal-append.run_wall_time_us":1335879}
I20260709 12:23:33.320580 19368 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.323498 19368 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:33.329439 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.332419 13721 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:33.338613 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158759us
W20260709 12:23:33.339023 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 183829us
W20260709 12:23:33.336375 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 231605us
W20260709 12:23:33.338296 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47052.5us
I20260709 12:23:33.356593 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.347936 19241 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.386559 19368 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:33.388742 19461 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:33.393353 19461 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:33.394474 19461 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.396565 19461 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:33.405503 19403 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:33.411711 19307 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:33.412870 19307 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:33.414426 19419 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:33.415642 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 230172us
I20260709 12:23:33.414911 19419 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:33.419368 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:33.424963 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59369.8us
W20260709 12:23:33.443514 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80197.4us
W20260709 12:23:33.444398 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 265278us
I20260709 12:23:33.410038 19394 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.445688 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.449793 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:33.452337 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.454329 19472 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:23:33.456419 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:33.461439 18852 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:33.464128 13652 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:33.464385 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.465065 13644 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.466154 19478 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:23:33.377198 19241 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.462067 18852 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:33.568471 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80645.2us
W20260709 12:23:33.572613 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98534.8us
I20260709 12:23:33.438160 19429 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:23:33.575927 19241 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.553845 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:33.583519 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:33.444375 19474 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:33.586582 13727 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:33.587783 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.588613 13647 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:33.571077 19469 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:33.606592 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.607488 14425 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:23:33.611788 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:33.621141 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:33.613186 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.666s (SENT) [suppressed 7 similar messages]
I20260709 12:23:33.634081 13649 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:33.635924 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163961us
I20260709 12:23:33.592128 19474 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:33.642349 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:33.606624 19394 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:33.615324 19422 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.639206 19475 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:33.669674 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.675047 13647 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:33.634593 19382 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.670688 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:33.638840 19326 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:33.641237 19446 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:33.687963 19446 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:33.689252 19446 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.690863 19305 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:33.707356 19227 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.644866 19474 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:33.710000 19227 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:33.712154 13798 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:23:33.720868 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 244701us
I20260709 12:23:33.724212 19382 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:33.727121 19480 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:23:33.727300 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:33.727685 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72060.6us
I20260709 12:23:33.728737 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:33.614710 19419 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.674666 19475 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:33.661801 19422 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:33.733212 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 137786us
I20260709 12:23:33.727221 19382 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.728754 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:33.768226 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.768759 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:33.773937 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111104us
I20260709 12:23:33.778553 19385 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:33.779376 19385 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:33.779888 19385 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:33.780927 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.781877 14425 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:33.784403 19422 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.785805 19461 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.791464 19307 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:33.791927 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63537.1us
I20260709 12:23:33.792105 19307 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:33.792742 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:33.794516 19335 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:33.794592 19461 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.796558 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:33.801949 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:33.798334 19482 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.790102 19385 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.799057 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:33.802891 19311 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:33.795255 19335 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:33.793076 19307 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.797307 19461 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:33.765678 19475 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:33.761607 19446 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:33.810171 13644 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
W20260709 12:23:33.819851 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45983.1us
W20260709 12:23:33.836752 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108406us
I20260709 12:23:33.847400 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:33.825361 19460 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.521s	user 0.000s	sys 0.011s
I20260709 12:23:33.827701 19442 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:33.873200 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:33.820217 19335 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:33.827659 19272 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:33.887939 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.895831 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:33.877239 13722 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.834538 19461 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.876461 19460 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:23:33.886691 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:33.881822 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:33.881796 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:33.888434 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.880340 19474 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.892197 19485 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:33.889369 13726 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:23:33.823037 19385 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:33.893498 19434 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.929692 19434 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.930377 19461 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.845129 19422 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.932289 19422 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.820274 19311 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:33.935140 19311 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.936367 19474 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:33.937417 14425 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:33.882742 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:33.876413 14421 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:33.875779 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:33.938700 13718 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:33.939282 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:33.940706 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:33.942550 19460 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:33.942734 19368 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:33.943496 19368 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:33.944609 19368 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.878949 13727 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:23:33.819082 19482 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.889086 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.947325 13721 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:33.842171 19241 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:33.949512 14417 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:33.945648 19475 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.950402 19241 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:33.954383 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:33.956073 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:33.903349 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 101606us
W20260709 12:23:33.903990 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 266399us
I20260709 12:23:33.876160 19404 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:33.955143 13719 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:33.923013 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.864639 19227 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:23:33.926002 18852 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.926590 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.922839 19442 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:23:33.925976 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:33.927654 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63851.2us
I20260709 12:23:33.927897 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.928457 19493 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:33.879937 19489 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.171s
I20260709 12:23:33.895684 19272 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:33.946772 13727 pending_rounds.cc:85] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 7
I20260709 12:23:33.958635 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:33.960809 13722 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:33.969300 19382 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:23:33.970252 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45777.7us
I20260709 12:23:33.990206 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.991088 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:33.970863 19442 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:33.971596 18852 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:33.971544 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.995899 13725 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:33.972159 13797 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:33.973868 19157 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:33.975659 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:33.978433 19488 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:33.979617 13796 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:23:33.980819 13794 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:33.984982 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.989765 13727 pending_rounds.cc:107] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:23:34.001029 13727 raft_consensus.cc:2889] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:33.981397 19335 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:33.995724 19422 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.005657 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.007345 13719 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:33.994803 19493 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:34.009007 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:33.963992 19434 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.017655 13720 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:34.021060 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:33.994995 19272 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.019820 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:34.032336 13725 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:23:33.995004 19382 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:34.036610 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78044.5us
I20260709 12:23:34.037643 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.039099 19241 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.045672 19486 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:34.046358 19486 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:34.047024 19382 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.048561 19382 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:34.054482 19491 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:34.054988 14418 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:34.000718 19488 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.069128 19404 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:34.084837 19241 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.098937 19475 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.098868 19241 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.064131 19429 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 12:23:34.086843 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49352.4us
I20260709 12:23:34.112906 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.087400 19482 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.033877 19307 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:34.088689 14416 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:34.088397 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.118659 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:34.060132 19489 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:34.009778 19493 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.094794 14417 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:34.065061 19472 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:23:34.060885 19488 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.097497 19335 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.114938 14390 leader_election.cc:400] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:23:34.119565 14418 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:34.122874 13724 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:34.152868 14390 leader_election.cc:403] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:23:34.112040 19241 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.118992 19305 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.118708 19307 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:34.153852 13718 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:34.160002 19482 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:34.166538 19307 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:34.174526 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86282.4us
I20260709 12:23:34.175606 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:34.176937 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62653.5us
I20260709 12:23:34.177711 19335 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12.
I20260709 12:23:34.118556 19069 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:23:34.179020 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.179226 19069 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:34.191164 19069 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:34.192377 19069 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.194221 19069 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:34.161208 19484 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.382s	user 0.000s	sys 0.003s
I20260709 12:23:34.154805 19434 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.197656 19174 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:34.198064 19434 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.198565 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.207507 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:34.187994 19307 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.131319 19486 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.185810 19272 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.122655 19489 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:34.202065 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:34.182768 19482 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:34.208410 19174 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:34.142338 19475 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:34.182130 19404 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.238221 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:34.241648 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 232728us
I20260709 12:23:34.227233 19335 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:34.243405 13722 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:23:34.243475 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:34.227092 19489 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:34.230792 19475 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:34.245785 19475 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.248061 19475 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:34.246773 19434 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:34.251672 19272 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:34.253113 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:34.253439 19305 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:23:34.267350 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65350us
I20260709 12:23:34.242353 19374 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:34.297605 19485 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.302546 19374 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:34.303941 19374 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.315515 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.317518 14420 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:34.266804 19335 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:34.320951 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1138) took 1041 ms. Trace:
I20260709 12:23:34.282842 19382 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.321478 14421 rpcz_store.cc:276] 0709 12:23:33.277365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.281547 (+  4182us) service_pool.cc:224] Handling call
0709 12:23:34.318986 (+1037439us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:34.319120 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353387392}
I20260709 12:23:34.326347 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3298) took 1315 ms. Trace:
I20260709 12:23:34.327039 13798 rpcz_store.cc:276] 0709 12:23:33.022171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.030847 (+  8676us) service_pool.cc:224] Handling call
0709 12:23:34.326023 (+1295176us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54003d54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:34.326170 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383602432}
I20260709 12:23:34.335357 19311 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:34.288468 19069 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:23:34.352399 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.321031 18898 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:23:34.359433 18898 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:34.359983 18898 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:34.330220 19488 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.364389 19488 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.374912 19335 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.376545 19489 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.380451 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.380690 19307 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.381379 13721 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:34.381819 19368 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:34.382375 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:34.382682 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.383145 13721 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:34.383380 13724 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 13: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:34.385084 19485 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:34.385334 13721 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:34.386626 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.334461 19382 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.388362 13719 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:34.389252 19382 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:34.387104 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:34.385654 19485 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:34.391475 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:34.394593 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.361295 18898 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:34.406001 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.408027 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:34.402961 19460 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:34.420213 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.421784 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.422488 13727 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:34.389797 19482 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:23:34.407601 19157 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.407163 19311 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:34.433605 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:34.389976 19368 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.420068 19472 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:34.401561 19241 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:23:34.444602 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:34.421239 19460 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.407428 19307 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:23:34.451052 19489 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.455271 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.456159 13727 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:34.456171 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.450915 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.452061 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:34.457376 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.457612 13721 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.458029 13647 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:34.460327 13643 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:34.447392 19472 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:34.452147 19460 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:34.471141 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.437239 19157 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.473745 13793 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:23:34.474943 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.475003 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.481113 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.461959 19307 consensus_queue.cc:1243] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:34.464175 18898 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:34.487344 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.491014 13800 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.489321 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.465744 19256 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.497603 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.498646 13645 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:34.500941 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.501714 13648 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:34.502697 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.477855 19472 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.507419 13801 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:34.510156 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.514770 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.517508 13644 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:34.513239 19368 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.478807 19475 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.520021 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.517372 13799 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:34.520040 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.537007 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.537711 13801 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:34.537699 13721 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:34.528265 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.542758 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:34.550627 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 110298us
I20260709 12:23:34.551801 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:34.552824 19157 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.537974 13794 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:34.555202 19419 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:34.515180 19382 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:34.555511 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.572216 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.555789 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.558981 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:34.532153 19485 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.579039 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.579276 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.555992 19368 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:34.574349 19382 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:34.518323 19256 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.581410 19157 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.591131 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:34.596361 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:34.584640 19475 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:34.600358 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.572283 19422 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:23:34.610349 19485 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:34.614537 19335 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:34.592727 19382 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:34.630807 19256 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:34.593958 19307 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.640924 19335 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.603766 19477 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:34.642606 19477 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.642987 19477 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:34.643970 19335 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.635450 19503 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:34.645668 19503 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:34.646888 19478 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:34.647663 19478 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:34.655236 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 65684.1us
I20260709 12:23:34.656352 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:34.651311 19422 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:34.659449 13800 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:23:34.660207 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81298.1us
I20260709 12:23:34.661762 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.662925 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.622828 19488 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:34.674110 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.674866 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.675418 13648 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:34.675832 13652 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:34.676412 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:34.676666 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.678644 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:34.680087 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.681016 13645 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:34.686951 14419 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:34.650105 19460 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.669085 19157 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:34.692191 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.692863 19157 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:34.693151 13652 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:34.693331 19157 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.700721 13793 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:34.702240 19486 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:34.707684 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46006.6us
I20260709 12:23:34.709223 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:34.709854 19461 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.716544 19475 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:34.717209 19475 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:34.659181 19514 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:34.718770 19514 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.670368 19488 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.721910 19488 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:34.703773 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.725337 19489 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:34.725795 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:34.726722 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:34.727414 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:34.729235 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:34.730937 19478 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:34.731860 19506 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.393s	user 0.003s	sys 0.000s
I20260709 12:23:34.732882 19374 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:34.733444 19374 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.735483 19420 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:34.735807 19335 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:34.654917 19485 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.736756 19420 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:34.738570 19485 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.738839 19404 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:34.739306 19335 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:34.739710 19335 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:34.740087 19478 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:34.740523 19478 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:34.741459 19478 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.627417 19434 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:34.714576 19241 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:23:34.715372 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:34.711337 19307 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:34.664749 19422 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:34.744199 19503 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.738667 19374 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.764040 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.744975 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.745422 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:34.731963 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69163.2us
I20260709 12:23:34.746387 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.770785 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.772495 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:34.712538 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:34.773943 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61518.2us
I20260709 12:23:34.747581 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:34.711115 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.752132 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.748057 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.756606 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.757153 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.775764 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:34.781124 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67033us
I20260709 12:23:34.783938 19241 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:34.784101 19461 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:34.784665 19241 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.784981 19461 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.788268 19241 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.768142 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:34.771440 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:23:34.790936 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.739305 19404 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:34.753129 19460 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:34.801641 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.802980 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:34.759205 19434 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.804174 19491 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.804203 19305 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:34.805801 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:34.844976 14424 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:34.805707 19422 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.846692 13648 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:34.847153 19422 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.810461 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:34.794628 19256 raft_consensus.cc:2764] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:34.812872 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.810994 13647 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:34.849709 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.761477 19368 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:34.798121 19479 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:23:34.814486 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89968.5us
I20260709 12:23:34.797099 19311 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.851572 19263 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:34.809010 19307 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:34.877276 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.814057 19503 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:34.879173 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68738.2us
I20260709 12:23:34.799698 19474 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:34.879873 13797 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:34.861562 19478 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:34.862928 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87368.2us
I20260709 12:23:34.872313 19491 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:34.740540 19335 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:34.888635 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.790005 19461 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:34.888665 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.873061 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.923782 13648 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:34.889298 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93400.7us
W20260709 12:23:34.905532 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:34.796718 19493 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:34.897424 19263 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:34.942365 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137771us
I20260709 12:23:34.916838 19374 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:34.861183 19472 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.946808 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:34.912082 19479 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:34.951892 13647 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:23:34.952199 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 150640us
I20260709 12:23:34.954376 14424 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:34.953543 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:34.927745 19493 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:34.958155 19335 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:34.958487 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.962807 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:34.963172 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.963829 13801 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:34.964160 13650 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:34.965612 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:34.967777 19477 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.969445 19479 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:34.972069 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:34.972846 13648 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:34.945482 19263 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:34.979616 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:34.948704 19374 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:34.991077 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:34.979542 19516 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:23:35.040064 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:35.040796 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 78024.9us
I20260709 12:23:34.976400 19493 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:34.965665 19434 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:35.038002 19374 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:34.954600 19382 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.062865 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:35.068706 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:35.069681 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:35.070607 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123951us
I20260709 12:23:34.948098 19404 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:35.087857 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54601.9us
I20260709 12:23:35.090237 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.091300 13801 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:35.061336 19307 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:35.103255 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.104110 13801 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:35.103349 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.105274 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:23:35.085887 19491 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:35.088845 19477 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:35.101475 19518 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:23:35.106709 19404 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:35.124253 19517 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 12:23:35.125349 19526 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:23:35.128837 19472 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.167037 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:35.134923 19419 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.010s
I20260709 12:23:35.180358 13646 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:23:35.174754 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 253328us
I20260709 12:23:35.135957 19532 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.197482 19157 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.005s
I20260709 12:23:35.198040 19157 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.201601 19475 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:35.205008 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 259941us
W20260709 12:23:35.205533 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 118865us
I20260709 12:23:35.183547 19354 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.209275 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 167582us
I20260709 12:23:35.211192 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:35.213232 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.186918 19525 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:35.213940 13647 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:35.214223 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:35.217636 13801 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:35.215494 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.176623 19529 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:23:35.216863 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:35.221930 13648 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:35.222431 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:35.224145 19538 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:23:35.255292 13797 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:35.255558 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:35.242070 13725 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:35.257681 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.264868 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 238) took 40903 ms (client timeout 15000 ms). Trace:
W20260709 12:23:35.265623 13547 rpcz_store.cc:269] 0709 12:22:54.361353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:54.361666 (+   313us) service_pool.cc:224] Handling call
0709 12:23:35.264397 (+40902731us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:30.837954 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:30.945461 (+107507us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:30.945505 (+    44us) write_op.cc:435] Acquired schema lock
0709 12:23:30.945528 (+    23us) tablet.cc:662] Decoding operations
0709 12:23:31.022168 (+ 76640us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:31.054076 (+ 31908us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:31.054122 (+    46us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:23:31.054610 (+   488us) tablet.cc:701] Row locks acquired
0709 12:23:31.054632 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:23:31.054836 (+   204us) write_op.cc:270] Start()
0709 12:23:31.054973 (+   137us) write_op.cc:276] Timestamp: P: 1783599811054782 usec, L: 0
0709 12:23:31.054999 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:23:31.116883 (+ 61884us) log.cc:844] Serialized 1674 byte log entry
0709 12:23:32.081512 (+964629us) op_driver.cc:464] REPLICATION: finished
0709 12:23:32.184271 (+102759us) write_op.cc:301] APPLY: starting
0709 12:23:32.184396 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:32.187839 (+  3443us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:32.187973 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:32.195385 (+  7412us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:32.195769 (+   384us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 12:23:32.196052 (+   283us) write_op.cc:312] APPLY: finished
0709 12:23:32.196435 (+   383us) log.cc:844] Serialized 49 byte log entry
0709 12:23:32.196740 (+   305us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:32.196902 (+   162us) write_op.cc:454] Released schema lock
0709 12:23:32.196969 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:32.197877 (+   908us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:32.197919 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2553829,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":98117,"apply.run_cpu_time_us":9157,"apply.run_wall_time_us":16681,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20879,"num_ops":4,"prepare.queue_time_us":244733,"prepare.run_cpu_time_us":15542,"prepare.run_wall_time_us":422471,"raft.queue_time_us":643132,"raft.run_cpu_time_us":42006,"raft.run_wall_time_us":92268,"replication_time_us":1026260,"thread_start_us":579981,"threads_started":4,"wal-append.queue_time_us":189488,"wal-append.run_cpu_time_us":21730,"wal-append.run_wall_time_us":1938688}]]}
I20260709 12:23:35.259181 19472 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:35.281503 19514 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:35.293447 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78048.5us
I20260709 12:23:35.296881 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:35.241460 19256 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:35.203364 19532 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.302359 19541 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:23:35.236086 19157 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.230212 19382 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:35.303542 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:35.307204 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 87081.4us
I20260709 12:23:35.307310 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:35.306456 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.316566 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:35.278055 19461 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:35.321774 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:35.322144 13651 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:35.322579 19475 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.213724 19354 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.325176 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:35.325352 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.279846 19335 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:35.330354 13644 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:23:35.334748 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:35.335850 14421 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:23:35.340018 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 110951us
I20260709 12:23:35.341742 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.336483 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.347390 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:35.303117 19422 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.302706 19256 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:35.351290 13647 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:35.281821 19472 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:23:35.359112 13722 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:35.367951 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:35.370558 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.372071 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:35.225737 19525 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.375226 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.376405 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:35.376680 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:35.377290 13647 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:35.378691 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:35.378937 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.382213 13649 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:35.382248 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:35.346206 19461 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:35.388654 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:35.389837 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53449.1us
I20260709 12:23:35.330628 19477 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:23:35.413015 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:35.349211 19536 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:23:35.348798 19335 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.438894 13725 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:23:35.433905 19486 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.729s	user 0.013s	sys 0.000s
I20260709 12:23:35.362986 19422 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.400158 19527 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:35.419598 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.411305 19531 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:23:35.458737 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82133.1us
I20260709 12:23:35.417279 19475 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:35.437753 19419 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:35.455238 13645 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:35.417496 19538 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.459196 19272 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:35.463438 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:35.398880 19474 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:23:35.475214 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:35.468189 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48682.3us
W20260709 12:23:35.470947 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99917.1us
I20260709 12:23:35.464289 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.486833 19554 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:35.487551 19554 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.490180 19554 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:35.455760 19531 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:35.418654 19374 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.490844 19554 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:35.495880 19434 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.496460 19561 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:35.499420 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.469103 19479 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:35.470779 19420 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:35.471055 19335 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:35.517421 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:35.494067 19460 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:35.536271 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.491741 19475 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:35.539428 13792 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:35.540395 13797 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:35.496757 19539 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:35.500372 19514 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:35.472180 19538 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.554348 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.557164 19354 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.561739 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:35.630316 19488 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.908s	user 0.010s	sys 0.005s
I20260709 12:23:35.538070 19461 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.540132 19460 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:35.545848 19420 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:35.521567 19272 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:23:35.564502 19535 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:35.480978 19256 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.516525 19434 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.629297 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:35.629801 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 216885us
I20260709 12:23:35.609112 19485 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.638734 13643 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
W20260709 12:23:35.570508 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 65020.5us
I20260709 12:23:35.576762 13800 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
W20260709 12:23:35.543082 18898 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.004s	sys 0.043s
I20260709 12:23:35.536055 19477 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.604513 19493 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:35.663497 19546 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:23:35.666206 19420 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:35.682924 18898 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.005s	sys 0.044s
I20260709 12:23:35.603434 19531 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.681800 19256 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:35.667881 19460 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:35.607442 19539 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.686240 19525 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.654850 19488 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.690245 19477 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.714314 19558 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:23:35.715375 19558 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:35.654632 19475 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.715721 19558 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:35.716526 19558 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.717717 19558 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:35.674788 19535 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:35.719092 19535 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.720866 19535 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:35.687415 19422 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:35.723091 19493 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:35.729507 19404 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.000s
I20260709 12:23:35.730510 19566 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 12:23:35.653914 19263 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.004s
W20260709 12:23:35.699847 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 219350us
I20260709 12:23:35.689005 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:35.712931 13795 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:35.751143 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2295) took 1087 ms. Trace:
I20260709 12:23:35.751571 13795 rpcz_store.cc:276] 0709 12:23:34.664140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.718425 (+ 54285us) service_pool.cc:224] Handling call
0709 12:23:34.718876 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.750026 (+1031150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:35.750818 (+   792us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:35.751060 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285496704}
I20260709 12:23:35.713809 19539 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:35.734458 13798 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:35.753850 19525 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.754508 19525 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.755228 19565 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:35.756412 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67473.4us
I20260709 12:23:35.757002 19525 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.760388 19565 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:35.762399 19263 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.770233 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2261) took 1097 ms. Trace:
I20260709 12:23:35.685916 19485 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.744928 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:35.780172 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:35.780905 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:35.776783 13646 rpcz_store.cc:276] 0709 12:23:34.673117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.673227 (+   110us) service_pool.cc:224] Handling call
0709 12:23:35.764721 (+1091494us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:35.770156 (+  5435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148445056}
W20260709 12:23:35.783735 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:35.747316 19311 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.860s	user 0.010s	sys 0.010s
I20260709 12:23:35.794196 14420 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:35.795696 14421 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:35.796048 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:35.800194 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:35.807485 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:35.800843 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:35.822871 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58341.5us
I20260709 12:23:35.784615 19541 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:35.828187 14424 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:35.848091 13799 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
W20260709 12:23:35.859910 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79114.9us
I20260709 12:23:35.861130 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.806835 19311 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:35.830957 19566 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:35.831138 19272 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:35.882877 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.885632 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:35.883520 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:35.887145 13721 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:23:35.882727 19479 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:35.890622 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:35.891584 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:35.896100 13726 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:35.896459 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:35.894213 13644 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:23:35.895256 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:35.898053 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:35.901160 13651 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:35.876014 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:35.899412 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:35.905133 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:35.906060 13719 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:35.915464 19461 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:35.915668 19568 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:35.916256 19461 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:35.918715 19461 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.884953 19272 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:35.920511 19272 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:35.924455 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.924932 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:35.927525 19490 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:35.927840 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:35.928555 13800 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:35.929729 19522 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.930423 19522 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.933171 19522 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:35.882889 19566 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:35.959466 19566 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:35.966176 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90330us
I20260709 12:23:35.967310 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:35.943250 19567 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:23:35.949704 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:35.970422 13792 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:35.949748 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:35.972083 13800 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:23:35.972352 19307 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:35.973013 19307 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.975788 19307 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:35.952340 19374 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:35.890189 19354 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:35.954011 19477 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:35.896684 19536 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:23:35.955355 19434 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:35.885183 19553 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.265s
I20260709 12:23:35.942534 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:35.905412 19479 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:35.983004 19479 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:35.995460 13651 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:23:35.996038 13651 pending_rounds.cc:85] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 7
I20260709 12:23:35.996322 13651 pending_rounds.cc:107] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:23:35.996618 13651 raft_consensus.cc:2889] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:35.995190 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.998667 19538 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.000799 19485 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.003297 13721 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:36.003583 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.004985 13793 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:23:36.006935 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.009274 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:35.938529 19311 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:36.010347 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.055166 19311 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:36.055400 13719 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:36.011439 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.012871 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.015707 13800 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:36.056182 19311 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.016826 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.010221 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:36.057960 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:36.021214 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.001565 19272 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:36.058671 19575 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:23:36.058843 13643 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:36.059181 19503 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:36.020484 13797 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:36.059613 19503 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.030401 19553 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.030416 19461 raft_consensus.cc:2764] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:36.060590 19272 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.032220 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:36.040316 19570 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.203s	user 0.004s	sys 0.000s
I20260709 12:23:35.989822 19531 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:36.016214 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:36.044023 19475 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:23:36.063027 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46930.2us
I20260709 12:23:36.048884 13798 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:23:36.063773 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:36.050779 19419 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.065021 19419 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.041401 19519 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.066514 19539 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:36.067085 19539 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:36.066895 19519 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.068033 19539 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.068512 19419 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.069185 19519 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.052541 19488 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.060022 19503 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:36.058106 13651 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.070441 19538 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:36.070058 19488 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:36.072082 19488 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:36.073014 19488 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.073689 19550 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:36.074172 19550 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:36.075156 19550 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.075578 19519 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.076396 19550 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.076180 19519 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.079523 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.079921 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.080631 13800 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:36.080929 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.094970 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:36.105212 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:36.101850 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 286719us
I20260709 12:23:36.110718 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.110919 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.111588 13797 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:36.112915 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:36.116361 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.116415 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:36.117106 13793 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:36.117434 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:36.121235 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.121552 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.124603 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.125986 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.126391 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.126703 13719 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:36.128875 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:36.102311 19461 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:36.152441 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:36.102080 19553 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.149570 13643 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:36.128234 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:36.142704 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.161397 13646 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:23:36.157972 13651 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:36.163371 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:36.110504 19565 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:36.109899 18898 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.167024 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:36.167907 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.173483 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.174625 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:36.147742 19556 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:36.125669 19272 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:36.108436 19307 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:23:36.140297 19335 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:36.175514 19527 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:36.176107 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:36.140372 19434 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:36.132596 19157 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:36.133324 19514 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.133193 19485 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.132191 19472 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:23:36.176174 19556 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:36.183777 14422 raft_consensus.cc:2410] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:23:36.178404 18898 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.183162 19553 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:36.125809 19477 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:23:36.178493 19272 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.183583 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:36.165478 19475 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:36.164201 19382 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:23:36.208066 13795 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:36.166915 19565 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.190352 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:36.150454 19374 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:23:36.176340 19368 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.186048 19569 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.183861 19551 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:36.184405 19335 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:36.120186 19460 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.162807 19503 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.208278 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.191433 19157 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:36.185729 19537 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.093s
I20260709 12:23:36.252925 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:36.219285 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57356.8us
I20260709 12:23:36.254592 13648 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:36.253518 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:36.255563 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 134401us
I20260709 12:23:36.256327 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:36.256386 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:36.190151 19434 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:36.221447 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93295.3us
I20260709 12:23:36.206920 19272 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.261057 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.257351 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:36.207803 19475 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:36.249871 19558 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.002s
I20260709 12:23:36.268886 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 876) took 1312 ms. Trace:
W20260709 12:23:36.266460 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76212.7us
I20260709 12:23:36.220630 19335 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:36.299059 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:36.300019 14420 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:36.280683 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:36.301383 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44120.8us
I20260709 12:23:36.303102 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.298625 13645 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:36.298090 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:36.200558 19461 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.312778 14422 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:36.311897 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:36.214306 19551 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.260075 19569 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:36.212387 19565 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.321259 14423 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:23:36.346031 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48010.2us
I20260709 12:23:36.218387 19479 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:36.283905 19519 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:36.295476 19527 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:36.358444 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 177011us
I20260709 12:23:36.265795 19485 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.297593 13799 rpcz_store.cc:276] 0709 12:23:34.956844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:34.957360 (+   516us) service_pool.cc:224] Handling call
0709 12:23:36.268586 (+1311226us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b540012a054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:36.268786 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756969216}
I20260709 12:23:36.338748 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:36.348978 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88003.3us
I20260709 12:23:36.304591 19475 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.321040 19558 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:36.324262 19579 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:23:36.383350 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44713us
I20260709 12:23:36.384505 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:36.311832 19514 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:36.220715 19368 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.341033 19551 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.344429 19374 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:36.400615 19374 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.400851 19553 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:36.401068 19374 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:36.350551 19569 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.402437 19569 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:36.402613 13647 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 12:23:36.407516 19577 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:23:36.408509 19584 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:36.409473 19584 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.344689 19532 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.296124 19556 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.411015 19556 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:36.359205 19582 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:36.394003 19404 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.415563 19503 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:36.415354 19404 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.416074 19460 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:36.393288 19583 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.419104 19583 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.422667 19404 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:36.408739 19576 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:23:36.397171 19157 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.328305 19523 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:36.341670 19566 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.352357 19479 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:36.328027 19574 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:36.359397 19527 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:36.426034 19368 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.388367 19558 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.427364 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.428023 19263 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.431685 19522 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:36.433105 19434 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.433303 19565 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.429616 19374 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.438868 19514 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.518605 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:36.518462 13542 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:36.525229 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:36.491245 19558 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:36.528080 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.530823 14418 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:23:36.400027 19581 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.341547 19461 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.441886 19420 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:36.442838 18898 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:36.391449 19485 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.525689 19553 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:36.548884 19485 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.448722 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.549777 13645 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:36.521338 13542 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:36.451505 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2280) took 1077 ms. Trace:
I20260709 12:23:36.551005 13644 rpcz_store.cc:276] 0709 12:23:35.373985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:35.377294 (+  3309us) service_pool.cc:224] Handling call
0709 12:23:35.377594 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:35.897703 (+520109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:36.451196 (+553493us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b5400198054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:36.451401 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1187239808}
I20260709 12:23:36.410013 19584 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:36.459329 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74962.3us
I20260709 12:23:36.465530 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.472934 13721 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:23:36.472952 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.477231 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.483353 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.462013 19460 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:36.490705 13719 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:36.462359 19354 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.500730 14421 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
W20260709 12:23:36.567762 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 94949.8us
I20260709 12:23:36.504923 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:36.515026 14423 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:36.569615 14423 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:36.515422 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.571977 13718 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:36.475000 19527 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:36.504535 19522 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.474227 19582 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.504853 19537 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:36.517354 19374 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.530853 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.584337 13724 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:23:36.542033 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.544517 19581 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.546620 19461 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.559185 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:36.559589 13651 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:36.475294 19523 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:36.560925 13725 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:36.574276 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.570823 14423 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.551017 19491 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:36.607259 14423 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:36.611776 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52765us
I20260709 12:23:36.614028 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.574993 19485 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.614233 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.621781 13720 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:36.622519 13725 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:36.575325 19256 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:23:36.614833 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:36.617911 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.557757 19584 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.615868 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.580509 19460 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:36.624387 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.675462 13726 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:23:36.627691 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 244991us
I20260709 12:23:36.682560 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:36.690904 13725 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:36.609247 19491 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:36.542686 19536 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:23:36.576745 19553 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:23:36.707381 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.633534 19382 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:36.710505 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:36.737051 13724 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:36.723486 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99221.1us
W20260709 12:23:36.675072 19576 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:23:36.619900 19583 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.746605 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:36.752281 13726 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:36.689929 13722 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:36.620949 19461 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:36.588865 19582 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.739351 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.763288 13799 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:36.764094 19591 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.734282 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:36.765062 14420 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:36.741137 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:36.741634 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.742184 13725 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:36.730563 14423 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:23:36.735056 14417 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:36.580144 19354 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.747201 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:36.727699 19593 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.782390 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.783481 13722 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:36.633338 19485 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.785189 13724 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:23:36.749581 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.554194 13542 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 9 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 11 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:36.759744 19461 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:36.731616 19584 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:36.792891 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.740062 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:36.765532 19586 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:23:36.793772 13798 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:36.766114 13795 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:36.771992 14423 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 9
I20260709 12:23:36.796296 14423 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:23:36.796682 14423 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:36.766008 14425 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:36.774473 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:36.691964 19335 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.786230 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.743065 19566 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.792248 19461 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:36.797904 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.605647 19537 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.796006 13649 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:36.730377 19475 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.766880 19263 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.810920 13727 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:23:36.813026 13798 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:23:36.814320 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:36.818606 13797 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:36.822235 19602 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.824210 19590 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.825059 19590 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.823071 19602 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.835914 13797 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:36.835775 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.837456 13727 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:23:36.837644 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:36.793329 19584 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.858000 19584 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:23:36.842723 19550 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.766s	user 0.014s	sys 0.002s
I20260709 12:23:36.845453 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:36.846565 19569 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.880438 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:36.883325 19556 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:36.883912 19556 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.884366 19556 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:36.846827 13542 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 12 to 16, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 16 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:36.793684 19523 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.852033 19581 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.852509 19420 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:23:36.794384 19272 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.857414 19582 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:36.859603 19594 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:23:36.859731 19537 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.791638 19593 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.899917 14416 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:36.879688 19553 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:36.861138 14424 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:36.866061 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.867468 13722 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:23:36.867606 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:36.817493 19069 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:23:36.734706 19460 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:36.901707 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:36.903993 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68346.3us
I20260709 12:23:36.909008 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.906117 13720 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:36.913548 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:36.810050 19368 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 12:23:36.911569 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:36.920450 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:36.922657 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:36.925707 19557 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:36.926446 19557 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:36.927209 19272 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.925280 13720 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:36.931545 14425 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:23:36.834342 19263 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:36.937650 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 71741us
I20260709 12:23:36.939098 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:36.940335 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:36.920501 19596 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:23:36.958498 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.936249 19583 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:36.963248 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.956637 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.957484 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:36.964146 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 214884us
I20260709 12:23:36.965598 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:36.966876 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:36.916276 19514 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:36.930248 19581 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:36.970114 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:36.971622 13723 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:36.981925 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:36.982939 13651 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:36.988766 19540 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:36.978403 19603 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:36.990767 19590 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:36.991114 19602 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.992530 19569 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:36.993134 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.963819 19583 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:36.994745 19583 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:36.995019 19335 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.935524 19582 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:23:36.946377 19531 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:37.002135 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.008731 13720 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:36.889698 13542 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 8 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:36.993844 19382 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:36.966468 19532 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:37.016957 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.024389 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.025352 13648 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:36.984292 19593 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.010913 19536 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.034793 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:23:37.012357 18898 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:37.012470 19540 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:37.011224 19531 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:37.046047 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.012633 19354 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:36.985193 19558 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.049788 13725 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.020337 19335 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:37.050349 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 110081us
I20260709 12:23:37.035790 19557 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:37.027305 19532 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:37.050850 19531 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.067451 19532 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.047998 18898 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.072942 13724 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:37.046351 19604 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.012305 19603 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.076951 19354 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:37.056345 19335 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.147890 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.149025 13645 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:37.051146 19540 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:37.097579 19551 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.000s
I20260709 12:23:37.074193 19503 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:23:37.107514 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:37.168015 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.169112 13718 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:37.108675 19472 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:37.111651 19558 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.115852 13726 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:37.118445 19591 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:37.113638 19491 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.119837 13725 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:37.136147 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.143214 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.156445 19585 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.193053 13644 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:37.195019 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.195983 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:37.155840 13799 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:37.162364 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.164084 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.165200 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.166484 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.168082 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.208307 13792 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:37.107802 19583 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.075695 18898 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:37.049980 19527 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.172333 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.074977 19475 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.175163 13720 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:37.177510 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.176048 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.150583 19540 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.193084 19382 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.201090 19585 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.210501 13793 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:37.231171 14419 raft_consensus.cc:2368] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:37.184340 19591 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:37.212373 13725 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:37.216336 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.218245 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:37.170915 19551 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.222177 13798 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:23:37.226998 13794 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:23:37.228832 19491 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:37.251995 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.235553 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:37.229377 19503 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:37.254532 19503 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.257216 19503 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.263512 19551 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.264966 19591 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.266289 19591 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:37.267424 19354 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:37.267860 19354 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:37.268777 19354 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:37.228152 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 262630us
I20260709 12:23:37.221262 19531 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.234997 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.273533 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:37.248838 19556 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.292541 19531 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.293818 19533 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:37.227316 19263 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.237862 19382 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.313443 19382 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.313598 13797 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
W20260709 12:23:37.314376 19605 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:23:37.319357 19603 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:37.321491 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:37.322326 14423 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:37.322417 19556 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.323510 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 931) took 1071 ms. Trace:
I20260709 12:23:37.323648 19536 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:37.323925 14421 rpcz_store.cc:276] 0709 12:23:36.251839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.252445 (+   606us) service_pool.cc:224] Handling call
0709 12:23:36.265058 (+ 12613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:36.579766 (+314708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.322831 (+743065us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:37.323438 (+   607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541339008}
I20260709 12:23:37.326634 19615 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:37.327132 19615 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.327459 19615 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.328187 13651 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:37.306079 19461 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.237347 13542 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 8 to 12, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 12 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:37.270247 19354 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:37.307763 19583 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.309453 19585 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.314597 13800 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:37.345448 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.302939 19533 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.261313 19604 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:37.306170 19335 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.350991 19531 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:23:37.356160 19531 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:23:37.357689 19522 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.359589 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.360396 13727 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.358219 19533 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.341308 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:23:37.348210 13723 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:37.309473 19527 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.379912 13727 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:37.383172 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.389760 13719 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:23:37.384187 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.389882 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.384948 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.386003 19584 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.390631 13725 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:37.352807 19604 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.392419 19565 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.392520 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.393909 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.395332 14425 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:23:37.390631 13727 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:37.394833 13721 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:37.390928 13726 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.398367 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:37.395627 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:37.343943 19486 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:23:37.328352 19479 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:37.409284 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.431852 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:37.406985 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:37.433598 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78727.7us
I20260709 12:23:37.442699 13726 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:37.456318 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:37.404630 19311 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:37.462335 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:37.469009 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.439107 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.472278 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.430717 19491 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.484779 13644 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:23:37.488204 13723 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:37.458884 19537 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:37.435200 19420 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:37.526000 19420 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:37.528807 19524 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:23:37.529337 19524 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:37.529052 19540 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.470695 19527 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:37.410132 19533 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:23:37.489331 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:37.497931 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:37.480450 19486 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.475317 19382 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:37.484293 19593 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:37.437987 19585 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:23:37.481421 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:37.535058 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:37.474824 19539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.175s	user 0.000s	sys 0.020s
I20260709 12:23:37.490118 19374 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:37.481068 19335 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:23:37.539245 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50054.9us
I20260709 12:23:37.475103 19311 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.566416 19540 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:37.568970 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87645.4us
I20260709 12:23:37.588546 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.589712 13652 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:37.558377 19539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.314s	user 0.001s	sys 0.022s
I20260709 12:23:37.534291 19475 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.598701 19540 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:37.537061 19527 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.557971 19587 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:37.558653 19551 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:37.540860 19382 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:37.538976 19625 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:37.493155 19491 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.592450 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:37.593583 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58670.8us
W20260709 12:23:37.595863 19578 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:23:37.410804 13542 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:37.538615 19537 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:37.592077 19335 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:37.542616 19532 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:37.623863 14425 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:37.640990 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.599774 19474 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:37.637378 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 115925us
I20260709 12:23:37.560413 19556 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:37.595541 19584 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.595228 19461 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.607758 19527 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.623072 19382 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:37.651743 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92408.7us
I20260709 12:23:37.584986 19486 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.598097 19475 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.628746 19488 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:37.539569 19524 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.661463 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:37.607977 19587 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:37.707445 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.670832 19335 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.672401 19491 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.673444 13643 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:37.672479 19311 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.674340 19486 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.674439 19488 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.677138 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.678553 19535 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:37.637705 19625 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.637537 19485 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.643865 19532 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.681028 19479 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:37.661594 19533 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.630597 19551 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.659106 19474 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.684312 19615 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.684796 19472 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:37.686110 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.640574 19540 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:37.744447 19540 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.746673 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.747543 13648 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:37.748509 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.749264 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.749619 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.749899 13650 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:37.750538 19631 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.751933 19475 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:37.723222 19535 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:37.751744 19631 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.706590 19630 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:37.753610 19535 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.712757 13652 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:37.754938 19535 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:37.754698 19630 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.755676 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.756912 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.644250 19539 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:37.725383 19524 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:37.722083 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.724557 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.724538 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.765652 13797 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:37.765805 13644 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:37.766784 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.767295 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.767676 13644 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:37.659754 19593 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:37.688593 13542 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:37.734506 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.768523 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.769644 13797 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.709954 19069 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:37.737522 19533 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.750748 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.732718 19636 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:37.711772 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.757474 13652 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:37.713047 13792 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
W20260709 12:23:37.746698 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58808.1us
I20260709 12:23:37.773240 14419 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:37.774950 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.696277 13645 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:37.738149 19472 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:37.763720 13799 raft_consensus.cc:2376] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18. Re-sending same reply.
I20260709 12:23:37.764442 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:37.730366 19532 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.803061 13645 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:37.804000 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.805125 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.805644 13793 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:37.726531 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49510.6us
I20260709 12:23:37.806478 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3402) took 1387 ms. Trace:
I20260709 12:23:37.767901 13651 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:37.665035 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:37.809965 13800 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:37.811432 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.813227 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.652776 19604 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.737835 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.816251 19609 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:37.816740 19609 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:37.817618 19609 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.819077 19609 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:37.823148 19531 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:37.823664 19374 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:37.824198 19374 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.823820 19531 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.776252 19539 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:37.827344 19539 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.829051 19604 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:37.829253 14422 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:37.757319 19630 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:37.719309 18898 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.807091 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.833279 13650 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:37.806871 13649 rpcz_store.cc:276] 0709 12:23:36.419043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.427032 (+  7989us) service_pool.cc:224] Handling call
0709 12:23:37.806313 (+1379281us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:37.806420 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734968704}
I20260709 12:23:37.834712 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.842543 13798 raft_consensus.cc:2368] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18.
I20260709 12:23:37.842996 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.843137 13796 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:37.843678 13799 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.844270 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:37.845062 14422 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:23:37.721520 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 151098us
I20260709 12:23:37.793726 13652 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:23:37.797194 19631 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:37.798172 19627 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:37.800184 19488 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:37.847016 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.848565 13797 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:23:37.855232 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48244.4us
I20260709 12:23:37.856200 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.860276 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:37.862839 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.865918 13648 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:37.879433 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:37.883850 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 665661us
I20260709 12:23:37.890081 13796 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:37.892421 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.900487 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.901503 13651 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:37.902064 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49381.6us
I20260709 12:23:37.902558 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.902909 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:37.903419 13643 raft_consensus.cc:2368] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:37.904606 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:37.905325 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:37.905826 19488 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:37.906332 19488 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:37.906726 19488 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:37.884016 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:37.908377 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:37.909386 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3340) took 1097 ms. Trace:
I20260709 12:23:37.909488 19522 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:37.910053 19522 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:37.910459 19522 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:37.910879 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 63961.1us
I20260709 12:23:37.831158 19636 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:37.911091 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:37.920534 13649 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:37.909905 13795 rpcz_store.cc:276] 0709 12:23:36.811984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.828941 (+ 16957us) service_pool.cc:224] Handling call
0709 12:23:36.833584 (+  4643us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:36.834486 (+   902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:36.834520 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:37.376916 (+542396us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:23:37.377509 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:23:37.479248 (+101739us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:37.479288 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:37.481160 (+  1872us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:37.909036 (+427876us) raft_consensus.cc:1685] finished
0709 12:23:37.909061 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:37.909305 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29232,"raft.queue_time_us":949,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":26464,"spinlock_wait_cycles":8983424,"thread_start_us":101238,"threads_started":1,"wal-append.queue_time_us":276170}
I20260709 12:23:37.758134 19537 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:37.869355 19628 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:37.960533 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.925468 13799 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
W20260709 12:23:37.929208 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46626.7us
I20260709 12:23:37.914453 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:37.932708 14391 leader_election.cc:400] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18.
I20260709 12:23:37.963821 14391 leader_election.cc:403] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:23:37.943818 19625 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:37.944887 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.946017 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:37.946275 19603 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:37.946993 19531 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:37.852489 19597 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.952929 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:37.923592 19558 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:37.965516 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.773438 13542 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:37.957957 13793 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:37.863386 19630 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 12:23:37.955164 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:37.876036 19634 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:37.972031 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:37.973016 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 199191us
I20260709 12:23:37.988597 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.989401 13726 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:37.964465 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:37.935907 19583 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:37.993392 13793 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:23:37.990403 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44476.1us
I20260709 12:23:37.988131 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:37.996553 13724 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:37.974964 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:37.989096 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:37.947955 19637 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:38.004855 19637 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.005043 19531 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:38.005328 19637 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:37.874698 19382 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:38.000509 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62813.6us
I20260709 12:23:38.001549 13725 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:23:37.777122 19587 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.007885 19382 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.009001 13798 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:37.969966 19553 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:38.011993 13792 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:38.014240 13724 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:38.008483 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:38.011583 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:38.024823 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:38.030349 19069 consensus_queue.cc:1175] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 20 status { last_received { term: 19 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:38.046893 13693 leader_election.cc:400] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:38.047425 13693 leader_election.cc:403] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:23:38.047533 13690 leader_election.cc:400] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:38.050107 13690 leader_election.cc:403] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:38.049388 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:37.991725 19634 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.054082 19630 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:38.056851 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:38.057523 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:37.992561 19606 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:37.972990 19631 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:23:38.000890 19604 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:38.067353 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:38.072741 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98473.2us
W20260709 12:23:38.084779 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.084986 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76570.7us
I20260709 12:23:38.015538 19583 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:38.085780 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:38.020395 19531 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:38.051790 19598 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:38.050684 19368 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:37.981706 19603 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.103585 19557 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.050907 19157 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11.
I20260709 12:23:37.960067 19593 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.101269 19630 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:38.048870 19553 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.068288 19461 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:38.082330 19335 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.061002 19634 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.133416 19488 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:37.986881 19597 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.025175 19636 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.138195 19420 raft_consensus.cc:993] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:38.119740 19368 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.129454 19551 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.069767 19606 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.176017 14422 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:38.129550 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:38.176994 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47541.2us
I20260709 12:23:38.052070 19475 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:23:38.048440 19460 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
W20260709 12:23:38.147462 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 215152us
I20260709 12:23:38.179255 19311 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:38.180028 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.181140 19643 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:38.181450 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:38.182902 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.063459 19558 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
W20260709 12:23:38.152671 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 164603us
I20260709 12:23:38.158543 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.187135 13795 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:38.185904 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
W20260709 12:23:38.164237 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 116898us
W20260709 12:23:38.139003 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53345us
I20260709 12:23:38.141000 19474 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.175350 19368 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.197906 19598 raft_consensus.cc:1081] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:38.139585 19263 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:38.203150 19263 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:38.204699 19263 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.207134 13723 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:23:38.208724 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.209545 13724 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:38.165045 18898 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.211602 19531 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.212472 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.173328 19615 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.215497 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.215730 19553 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.216375 13721 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:38.217105 19587 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:38.217334 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.151119 19632 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.226008 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:38.226593 19632 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.227005 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.228710 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.229907 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.220537 13801 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:38.222635 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:38.165383 13542 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 7 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 8 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:38.185493 19419 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:38.225476 19522 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.230142 19585 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:38.218724 19485 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:38.230360 19420 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:38.244834 19374 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.230367 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:38.252899 13798 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:38.252063 13542 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 9 to 10, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 10 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:38.141983 19591 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:38.252115 19420 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.254829 19420 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:38.231840 19583 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.156673 19491 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:38.230132 19631 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:38.238955 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:38.182824 19652 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:38.226058 19472 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.258138 19419 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.262404 19472 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.264428 19631 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.264832 19631 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:38.263428 19598 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:38.240588 19311 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:38.266482 19598 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:38.266794 19311 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:38.262135 19419 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.267356 19311 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:38.268054 19522 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.268527 19311 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.269810 19311 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:38.269949 19374 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.142627 19557 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.241920 19603 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.242434 19503 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:38.250036 19533 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.220703 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:38.176115 19157 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:38.275072 13798 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:38.252279 19585 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:38.219427 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.278512 13719 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:38.278697 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:38.279225 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:38.245896 14425 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:38.233804 19488 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.228844 19632 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:38.258631 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68024.6us
I20260709 12:23:38.250808 13797 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:23:38.239312 19486 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.286510 19486 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.272975 19557 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.266870 19598 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:38.276501 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:38.213479 19532 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:38.221648 19527 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.006s
I20260709 12:23:38.277693 19652 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:23:38.275590 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55045.3us
I20260709 12:23:38.247418 19597 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.313228 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.300251 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:38.314394 13726 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:38.307240 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:38.277887 18898 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:38.249753 19606 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.315910 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:38.317967 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:38.318485 19557 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:38.321027 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:38.321623 19551 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:38.325579 19652 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.313239 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:38.343618 13726 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:38.282407 19585 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:38.356812 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.315400 14422 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
W20260709 12:23:38.329391 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 271925us
I20260709 12:23:38.358196 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:38.360361 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:38.361349 13652 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:38.362874 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:38.364517 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:38.365334 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:38.334482 13801 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:38.318109 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:38.346551 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:38.350855 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.351526 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:38.302994 13724 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:23:38.368772 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:38.369134 13796 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:38.369390 13724 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:38.370048 13724 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:38.355955 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:38.356714 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:38.357537 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:38.305043 19157 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:38.387343 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:38.390605 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:38.403903 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82935us
I20260709 12:23:38.405078 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:38.406885 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:38.408245 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:38.273167 19368 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.406116 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:38.413380 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:38.414208 13796 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:38.417258 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:38.422883 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:38.423573 13648 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:38.431828 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:38.432938 14424 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:38.429965 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:38.389971 19604 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:38.445572 13726 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:38.395303 19584 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.388420 19069 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:23:38.410826 19557 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:38.497666 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.498333 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:38.499055 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.356935 19652 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:38.480711 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:38.431551 19633 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:38.388255 19522 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:38.490101 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:38.506131 13799 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:38.483528 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:38.492029 13690 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:38.357039 19597 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 12:23:38.396121 18898 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:38.456727 19644 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:38.387730 19157 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.342523 19632 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 12:23:38.433588 19603 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:38.502661 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:38.514923 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:38.335274 19527 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.509281 14418 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:23:38.435374 19645 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:38.446168 19558 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:38.454494 19551 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:38.455029 19606 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:38.454905 19565 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:38.521589 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:38.542543 19488 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:38.508209 13690 leader_election.cc:403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:23:38.543787 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51941.6us
I20260709 12:23:38.523046 13646 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:23:38.473958 19461 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:38.545351 19461 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.548170 19461 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.452816 13724 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.505525 19633 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:38.507772 19532 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.549700 13718 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:23:38.538728 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:38.562249 13722 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:38.554102 13726 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 8.
I20260709 12:23:38.504693 19069 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:38.555931 13645 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:38.537890 13648 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:38.561069 13794 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:38.569830 13724 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.571249 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:38.571938 13724 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:38.573421 13724 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:23:38.511332 19651 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:38.588621 13724 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:38.492642 19584 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.603067 13650 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:38.532105 19551 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:38.567500 19637 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 12:23:38.626194 19637 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.549414 19645 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:38.560364 19633 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:38.662192 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 159662us
I20260709 12:23:38.662907 14423 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:38.588936 19603 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:23:38.552923 19558 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:38.713721 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:38.638607 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:38.620241 19609 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.791s	user 0.014s	sys 0.000s
I20260709 12:23:38.567830 19461 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:38.620393 13796 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:38.666571 13693 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:38.667344 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:38.719745 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53324.9us
I20260709 12:23:38.564433 19488 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:38.720674 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:38.732080 13652 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:23:38.734001 13652 pending_rounds.cc:85] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 10
I20260709 12:23:38.734325 13652 pending_rounds.cc:107] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:23:38.734632 13652 raft_consensus.cc:2889] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:38.718164 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:38.676193 19655 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:23:38.721215 13724 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:38.660372 19637 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.654976 19479 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:23:38.716933 19553 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.719836 19523 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:38.625520 19551 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:38.631071 19639 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:23:38.788580 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.741349 19461 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:38.745721 19644 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:38.791865 19527 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:38.792153 19661 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:23:38.620265 19651 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.670811 19557 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.825436 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.675587 18898 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.669315 19604 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.847059 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:38.785825 19553 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.848505 13724 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:38.850231 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.790818 19523 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.747123 19069 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.819084 19368 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:38.865794 19666 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 12:23:38.867019 19664 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:23:38.870954 19678 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:38.872884 19651 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:38.877943 19419 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 12:23:38.828172 19382 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:38.881906 19420 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.627s	user 0.005s	sys 0.003s
I20260709 12:23:38.843766 19657 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:38.823872 19558 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:38.890808 19660 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:23:38.894232 19659 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 12:23:38.801862 19645 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.845773 19157 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:38.850828 19263 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:38.857203 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:38.921463 19461 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:38.930361 19420 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:38.906029 19472 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.639s	user 0.008s	sys 0.000s
I20260709 12:23:38.828377 19488 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:38.938995 19523 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:38.940184 19644 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:38.941205 19587 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.005s
I20260709 12:23:39.002614 19263 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:38.943375 19585 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
I20260709 12:23:38.962863 19584 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.957716 13722 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:38.839792 19673 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 12:23:38.904191 19597 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:38.947319 19634 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.003s
I20260709 12:23:38.973058 19491 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:38.953734 19419 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:38.968307 19631 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.703s	user 0.007s	sys 0.006s
I20260709 12:23:38.978786 19420 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:38.978891 13795 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:38.979570 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3351) took 1247 ms. Trace:
I20260709 12:23:39.020529 13794 rpcz_store.cc:276] 0709 12:23:37.732401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.737542 (+  5141us) service_pool.cc:224] Handling call
0709 12:23:38.979348 (+1241806us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54001608d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:38.979503 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713521536}
I20260709 12:23:38.981853 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:38.965482 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108365us
I20260709 12:23:38.966403 19382 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:38.952135 19652 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:39.018602 19335 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.855s	user 0.012s	sys 0.000s
I20260709 12:23:39.030620 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.031359 13645 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:39.033061 13649 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18.
I20260709 12:23:39.035779 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.036438 13643 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:39.036667 19419 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.022946 19634 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.042883 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:39.043579 19634 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:39.045688 13720 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:39.049199 14416 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:39.022349 19491 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:39.046626 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:39.056923 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:39.028888 19606 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:39.067050 13720 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:39.056257 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:39.017328 19585 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.075992 13643 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:23:39.034613 19335 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18.
I20260709 12:23:39.013736 19263 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.096175 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.097563 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.099472 13644 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:39.110406 19683 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:23:39.129860 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:39.130543 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.130321 14421 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:39.073329 19606 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:39.052906 19597 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.025372 19631 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.066720 19419 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:39.087170 19523 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:23:39.098248 19667 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:23:39.086517 19587 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:23:39.088444 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:39.140415 13643 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:39.168767 13643 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:39.170064 13643 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:39.164245 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:39.167543 19491 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:39.182509 19684 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:23:39.183257 19382 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:39.157876 19523 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:39.223393 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:39.204607 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.224625 19523 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.226629 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:39.208371 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:39.227454 19434 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:39.229241 13799 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:23:39.209017 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:39.230372 13801 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:39.144531 19597 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.151687 19419 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.180928 19488 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:39.234133 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:39.177139 19553 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.240240 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:39.240998 14424 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:39.241838 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3420) took 1645 ms. Trace:
I20260709 12:23:39.242324 13646 rpcz_store.cc:276] 0709 12:23:37.596684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:37.597193 (+   509us) service_pool.cc:224] Handling call
0709 12:23:37.605898 (+  8705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.544765 (+938867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.241541 (+696776us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54001e7b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:39.241767 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108416384}
I20260709 12:23:39.246932 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2314) took 1008 ms. Trace:
I20260709 12:23:39.247429 13795 rpcz_store.cc:276] 0709 12:23:38.238126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.244207 (+  6081us) service_pool.cc:224] Handling call
0709 12:23:38.247611 (+  3404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:39.022765 (+775154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.246668 (+223903us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400102cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:39.246861 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681267456}
I20260709 12:23:39.247965 19597 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.250463 13800 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:39.251962 13720 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:39.253319 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:39.256071 13721 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:39.257082 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:39.257865 13726 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:39.258777 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:39.186018 19420 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:23:39.280405 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:39.259423 13652 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:23:39.177722 19606 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.267658 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:39.305650 19606 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:39.249488 19488 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.306305 19488 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:39.260350 19536 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:39.310681 19536 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:39.272213 19522 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.273687 19419 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:39.276608 19680 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.262s	user 0.003s	sys 0.000s
I20260709 12:23:39.293784 19598 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.286123 19591 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:39.295266 19491 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:39.295950 19633 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.000s
W20260709 12:23:39.298141 19551 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
W20260709 12:23:39.300516 19687 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 12:23:39.303288 19486 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:39.303619 19603 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.002s
W20260709 12:23:39.313616 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59637.6us
I20260709 12:23:39.262377 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:39.325946 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:39.333675 13792 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:39.342622 13792 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:39.345494 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:39.286790 19652 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.319785 19522 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:39.348732 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:39.315529 19653 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.349371 13652 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:39.372251 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:39.372999 13646 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:39.373281 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.374529 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.374778 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2315) took 1136 ms. Trace:
I20260709 12:23:39.375123 13652 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.347031 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.375175 13793 rpcz_store.cc:276] 0709 12:23:38.238469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.238672 (+   203us) service_pool.cc:224] Handling call
0709 12:23:39.374556 (+1135884us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001004d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:39.374718 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445491968}
W20260709 12:23:39.351624 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.368311 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:39.312693 19536 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.319480 19475 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:39.379251 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53453.4us
I20260709 12:23:39.362671 13792 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.380285 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:39.411031 13792 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:39.412365 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:39.415899 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 153633us
I20260709 12:23:39.321768 19639 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
W20260709 12:23:39.420593 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 194051us
W20260709 12:23:39.421190 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75811.1us
I20260709 12:23:39.418507 13647 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
I20260709 12:23:39.425503 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:39.427721 13725 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:23:39.429556 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:39.430207 13718 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:39.373769 19603 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13.
I20260709 12:23:39.434760 13767 leader_election.cc:400] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18.
I20260709 12:23:39.410794 19558 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:39.439042 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:39.392990 19585 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:39.339438 19591 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.393100 19565 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:39.421135 19684 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:23:39.443164 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2306) took 1004 ms. Trace:
I20260709 12:23:39.468608 13648 rpcz_store.cc:276] 0709 12:23:38.439209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.568596 (+129387us) service_pool.cc:224] Handling call
0709 12:23:38.569067 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.577420 (+  8353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:38.577471 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:38.580055 (+  2584us) log.cc:844] Serialized 26 byte log entry
0709 12:23:38.733838 (+153783us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:38.733885 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.733977 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.275115 (+541138us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:39.434184 (+159069us) raft_consensus.cc:1685] finished
0709 12:23:39.434215 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.443043 (+  8828us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7513,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"spinlock_wait_cycles":7170944,"thread_start_us":153259,"threads_started":1,"wal-append.queue_time_us":364309}
I20260709 12:23:39.406550 19584 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.391443 19486 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:39.447240 13647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:23:39.488307 13647 raft_consensus.cc:606] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:39.488744 13647 raft_consensus.cc:614] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:39.425740 19491 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:23:39.346859 19551 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.499725 13718 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:39.424784 19382 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:39.424640 19553 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:39.424324 19604 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:39.473948 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:39.341506 19633 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.448103 19558 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:39.483676 19532 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:39.405287 19653 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.412948 19475 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.479568 13652 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:39.497293 13792 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:39.454205 19479 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:23:39.533481 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2308) took 1080 ms. Trace:
I20260709 12:23:39.498905 19491 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:39.537516 13651 raft_consensus.cc:2368] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:23:39.536597 13650 rpcz_store.cc:276] 0709 12:23:38.452825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.669448 (+216623us) service_pool.cc:224] Handling call
0709 12:23:38.718939 (+ 49491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:38.719364 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:38.719395 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:38.736554 (+ 17159us) log.cc:844] Serialized 26 byte log entry
0709 12:23:38.957647 (+221093us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:38.969647 (+ 12000us) log.cc:844] Serialized 14 byte log entry
0709 12:23:38.969923 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:38.970003 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:39.191236 (+221233us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003764e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:39.527415 (+336179us) raft_consensus.cc:1685] finished
0709 12:23:39.533264 (+  5849us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:39.533417 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22537,"prepare.queue_time_us":13604,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":651,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":10089,"spinlock_wait_cycles":26364672,"thread_start_us":170767,"threads_started":1,"wal-append.queue_time_us":265695}
I20260709 12:23:39.516580 19565 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:39.543511 13648 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:39.526093 19558 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:39.513114 13724 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:39.520356 19635 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:39.546439 13720 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
W20260709 12:23:39.537446 19524 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:23:39.525839 19671 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.529400 19645 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:39.533706 19553 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:39.529186 19604 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:39.617285 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:39.541620 19491 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:23:39.492974 19584 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.499687 19486 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:39.637400 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.629s (SENT) [suppressed 10 similar messages]
I20260709 12:23:39.659070 19584 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.663728 19591 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.667191 19604 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.578352 19697 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.583506 19609 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:39.647785 19645 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:39.636807 19671 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.576431 19565 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:39.705470 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 282713us
I20260709 12:23:39.586501 19635 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:39.709347 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 261316us
I20260709 12:23:39.688442 19609 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:39.691154 19697 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.736584 19475 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:39.786115 19695 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:23:39.789044 19635 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.712898 19553 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.795735 19486 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.706195 19488 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:39.798094 19488 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.800443 19697 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.804733 19671 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:39.808037 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 426144us
I20260709 12:23:39.809253 19565 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.809451 19606 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:39.810282 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 488070us
I20260709 12:23:39.810061 19606 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.812372 19606 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.812175 19645 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.815282 19603 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.815829 19603 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.781165 19591 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:39.817013 19591 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.817835 19603 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.819115 19591 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.820408 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:39.821076 14425 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:39.821388 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:39.822192 14424 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:39.822643 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:39.825421 19693 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:23:39.833067 13724 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:39.834601 13793 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:23:39.836843 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:39.837606 13792 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:39.839190 13724 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:23:39.844444 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:39.843245 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 393835us
I20260709 12:23:39.852043 13727 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:39.871338 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:39.880678 13727 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:39.891762 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:39.894932 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3103) took 1397 ms. Trace:
I20260709 12:23:39.895593 13719 rpcz_store.cc:276] 0709 12:23:38.497891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.498036 (+   145us) service_pool.cc:224] Handling call
0709 12:23:39.894606 (+1396570us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b54003fd4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:39.894839 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1991750528}
I20260709 12:23:39.912182 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:39.918388 19598 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:39.942045 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50435.2us
I20260709 12:23:39.880781 19486 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.843245 19551 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:39.870664 19639 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.956414 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:39.846655 19591 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:39.959924 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73131.5us
I20260709 12:23:39.966427 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:39.980759 13652 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:23:39.997685 14425 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:39.940272 19704 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:39.940411 19698 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:39.955927 19382 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:40.015189 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44409us
I20260709 12:23:40.047331 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:40.006242 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:40.054853 13648 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:23:40.054806 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:39.954392 19633 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:39.954340 19653 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:39.959851 19503 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:39.940575 19643 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
W20260709 12:23:40.063936 13765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.003s	sys 0.014s
I20260709 12:23:40.052524 13652 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:40.015017 19587 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.046610 19701 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:23:40.089174 13765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.003s	sys 0.014s
I20260709 12:23:39.886735 19609 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:39.959463 19486 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.015872 19639 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.052415 19584 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 12:23:40.079294 19419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.002s	sys 0.016s
I20260709 12:23:40.026808 19591 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.088825 19631 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:40.140810 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:40.064092 19491 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.145601 13801 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:40.143107 13793 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
W20260709 12:23:40.231949 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.229887 13650 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:40.242801 13643 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:23:40.125653 19503 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:40.078889 19335 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:23:40.131313 19374 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:40.256460 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:40.275755 13799 raft_consensus.cc:606] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:40.240669 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:40.188709 19653 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.188133 19703 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:40.101950 19587 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.057636 19704 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.258848 13796 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:40.243775 19708 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:40.129879 19715 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.232720 19537 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:23:40.280143 19631 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:40.282629 19434 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:40.305109 19553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.041s	sys 0.051s
W20260709 12:23:40.331120 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.326391 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 443473us
I20260709 12:23:40.244132 19382 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.338433 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:40.314932 19645 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.293259 19374 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.300215 19703 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.287997 19653 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:40.330992 19553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.041s	sys 0.053s
I20260709 12:23:40.366600 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:40.367359 13799 raft_consensus.cc:606] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:40.367741 13799 raft_consensus.cc:614] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:40.330350 19434 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:40.372756 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:40.369853 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.376379 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:40.313247 19708 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.380345 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:40.324788 19631 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.381873 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:40.355448 19460 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.376786 19673 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:40.377317 19710 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.323s	user 0.001s	sys 0.001s
I20260709 12:23:40.319583 19715 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.387835 19698 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.398005 19522 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.990s	user 0.015s	sys 0.002s
I20260709 12:23:40.357309 19645 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.396728 19721 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:40.435117 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:40.438185 13725 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.437108 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:40.434070 13726 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:23:40.439271 13799 raft_consensus.cc:606] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:40.439783 13799 raft_consensus.cc:614] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:40.436954 13721 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:40.434813 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:40.431337 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:40.445581 13723 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:40.442116 13718 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:40.455711 13719 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 9: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:40.451951 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:40.453158 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:40.454195 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:40.457583 13801 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:23:40.458519 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:40.459388 13718 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.426548 19730 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:40.422048 19703 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:40.509081 13718 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:40.437144 19486 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.421444 19491 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.398916 19460 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.473393 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:40.595286 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 170162us
I20260709 12:23:40.596067 13722 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:40.580405 13726 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:40.581435 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:40.598884 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144281us
W20260709 12:23:40.445019 19705 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:23:40.429574 19479 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:40.424476 19698 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.578874 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:40.623674 14421 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:40.583067 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:23:40.608796 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2305) took 2153 ms (client timeout 1501 ms). Trace:
W20260709 12:23:40.581213 19710 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:23:40.399556 19673 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:40.624972 13651 rpcz_store.cc:269] 0709 12:23:38.438235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:38.485053 (+ 46818us) service_pool.cc:224] Handling call
0709 12:23:40.590918 (+2105865us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:40.591104 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2342717696}
I20260709 12:23:40.430238 19522 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.627170 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:40.579317 19553 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.637804 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.005s	sys 0.033s
W20260709 12:23:40.637859 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.011s	sys 0.026s
I20260709 12:23:40.616475 19587 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:40.664300 19704 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.656565 19533 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:23:40.649252 13652 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:23:40.677276 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.011s	sys 0.026s
I20260709 12:23:40.677356 19553 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:40.639721 19709 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
W20260709 12:23:40.679392 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.005s	sys 0.033s
W20260709 12:23:40.673070 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.591763 19591 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:40.679867 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 623328us
W20260709 12:23:40.677809 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 690878us
I20260709 12:23:40.679505 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3462) took 1329 ms. Trace:
I20260709 12:23:40.680642 19553 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.681671 19706 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:40.680887 13646 rpcz_store.cc:276] 0709 12:23:39.350111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.404959 (+ 54848us) service_pool.cc:224] Handling call
0709 12:23:39.416902 (+ 11943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:39.417248 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:39.417278 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.437324 (+ 20046us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.836163 (+398839us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:39.836211 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.836287 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:40.679231 (+842944us) raft_consensus.cc:1685] finished
0709 12:23:40.679257 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:40.679421 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":60905,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":9502,"thread_start_us":361356,"threads_started":1,"wal-append.queue_time_us":674901}
I20260709 12:23:40.684823 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:40.686299 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:40.650972 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:40.685449 19553 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.616719 19639 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:40.688719 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:40.689384 19587 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:40.689863 19587 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:40.691151 14416 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:23:40.691293 19587 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.654872 19732 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.693646 19732 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.637265 19734 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.696139 19734 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.690193 19553 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.698889 19587 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:40.702729 19734 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:40.638829 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.007s	sys 0.026s
W20260709 12:23:40.703972 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.007s	sys 0.026s
W20260709 12:23:40.704694 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 748336us
I20260709 12:23:40.706018 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:40.707718 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:40.708406 19734 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:40.727543 14425 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:40.745707 13651 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:40.449653 19434 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.750806 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:40.758894 13646 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:23:40.752758 13719 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:40.762440 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:40.744297 19704 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:40.772038 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:40.773020 13645 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
W20260709 12:23:40.776340 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 175958us
I20260709 12:23:40.755173 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:40.777494 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:40.786727 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:40.787118 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:40.749954 19263 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.785048 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3445) took 1727 ms. Trace:
I20260709 12:23:40.791067 13644 rpcz_store.cc:276] 0709 12:23:39.057389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.118492 (+ 61103us) service_pool.cc:224] Handling call
0709 12:23:39.124639 (+  6147us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:39.125062 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:23:39.423323 (+298261us) log.cc:844] Serialized 14 byte log entry
0709 12:23:39.713357 (+290034us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:39.716523 (+  3166us) log.cc:844] Serialized 26 byte log entry
0709 12:23:39.716900 (+   377us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:39.716958 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:39.717072 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:40.784831 (+1067759us) raft_consensus.cc:1685] finished
0709 12:23:40.784853 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:40.784982 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":66601,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":1085,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":15834,"thread_start_us":505991,"threads_started":2,"wal-append.queue_time_us":483609}
I20260709 12:23:40.769556 19639 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.789645 19479 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:40.799012 19419 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:40.799885 19263 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.800522 19598 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.881s	user 0.013s	sys 0.000s
I20260709 12:23:40.800935 19719 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.145s
I20260709 12:23:40.799043 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.789655 19704 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:40.809165 19728 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
W20260709 12:23:40.811144 19729 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:23:40.899123 19522 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:40.900170 19597 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:40.900507 19069 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.823977 19479 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:40.823513 19604 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.770167 19734 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:40.770200 19631 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:40.828272 19726 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 12:23:40.828482 19715 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:40.828739 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:40.911846 13719 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:23:40.830384 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:40.912586 13721 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:40.912791 13725 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:40.830868 19486 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.833516 19475 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:40.772075 19712 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:40.860839 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:40.935701 13726 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:23:40.806795 19735 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.392s	user 0.000s	sys 0.001s
I20260709 12:23:40.864483 13646 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:23:40.798347 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:40.793700 19653 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:40.830397 19733 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:40.812135 19374 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:40.890777 19419 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:40.895344 19737 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
I20260709 12:23:40.886984 19472 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:23:40.896073 19759 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:23:40.897112 19722 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:40.904699 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 147023us
I20260709 12:23:40.925021 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:40.824508 19523 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:23:40.962302 19733 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:40.952409 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 154169us
I20260709 12:23:40.945139 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:40.971824 13723 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:23:40.923342 19479 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:40.891719 19708 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:40.835249 19639 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.900779 19597 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.926326 19732 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:40.964740 19734 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.962036 19645 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.964998 19419 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.965126 19731 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:23:40.866006 19263 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.943552 19712 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:40.979897 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:40.949395 19460 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:40.955523 19475 consensus_queue.cc:1243] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:23:40.959919 19704 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:23:40.957320 19653 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.989217 19750 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:40.997406 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3466) took 1034 ms. Trace:
I20260709 12:23:40.925877 19604 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:40.901038 19069 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.003295 19708 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:41.041738 13649 rpcz_store.cc:276] 0709 12:23:39.962879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.963060 (+   181us) service_pool.cc:224] Handling call
0709 12:23:39.963517 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:40.358434 (+394917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:40.358484 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:40.367131 (+  8647us) log.cc:844] Serialized 26 byte log entry
0709 12:23:40.646871 (+279740us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:40.646910 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.646975 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:40.997082 (+350107us) raft_consensus.cc:1685] finished
0709 12:23:40.997110 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:40.997315 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2658,"raft.queue_time_us":139740,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":10824,"thread_start_us":673685,"threads_started":2,"wal-append.queue_time_us":627205}
I20260709 12:23:40.935509 19486 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:40.971738 19749 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:41.026077 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46323.7us
I20260709 12:23:41.021596 19673 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:40.797339 19730 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:41.005031 19719 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:41.038326 19742 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:40.983632 19722 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.055693 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:41.066174 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:41.082871 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 223609us
I20260709 12:23:41.118800 13792 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:41.085114 13800 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:41.035759 19704 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:41.115345 13795 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:41.121964 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:41.123739 13799 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:23:41.124115 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:41.125104 19704 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:41.061523 19713 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:41.133551 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.068445 19719 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:41.139696 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86721.8us
I20260709 12:23:41.140656 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:41.129613 19591 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.047067 19653 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.103288 19749 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:41.142504 19719 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:41.152904 19653 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.158711 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:41.163439 13795 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
W20260709 12:23:41.148932 19558 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.560s	user 0.015s	sys 0.003s
I20260709 12:23:41.174206 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:41.179858 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:41.181088 14424 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:41.114114 19742 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:23:41.147878 19698 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.177307 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:41.065119 19750 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.199258 13792 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:23:41.214460 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:41.193845 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:41.186700 19767 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:23:41.224345 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83801.3us
I20260709 12:23:41.212533 13764 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:41.135941 19712 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.195190 19558 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.147563 19591 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.210958 19553 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.219245 19639 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:41.221807 19776 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:41.210762 19748 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:41.234565 19731 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:41.272308 13723 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:41.143115 19713 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.277997 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:41.280637 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.281193 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:41.281344 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:41.281525 13649 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:41.278594 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:41.279219 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:41.284605 13646 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:41.287104 19773 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:23:41.290427 19713 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:41.290969 19752 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:23:41.293468 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.294212 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:41.292209 19263 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.242626 19597 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.236902 19770 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:41.264554 19712 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:41.295521 19776 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:41.305217 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:41.308650 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.244038 19634 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:23:41.276539 19731 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.309319 13646 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:41.309830 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.313503 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:41.339658 13650 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:41.338177 13649 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:41.308346 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:41.325028 19558 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.318468 19731 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:41.348363 19069 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.319482 19604 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.336030 19486 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.307833 19770 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.369040 19770 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:41.293166 19749 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.307265 19479 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.377290 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:41.377740 19776 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:41.387275 13693 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.733s	user 0.016s	sys 0.008s
I20260709 12:23:41.388728 19704 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:41.392226 19704 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.395758 19733 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.282703 19748 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.192168 19730 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.426049 13723 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:41.375397 19479 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.431712 19719 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:41.432627 19746 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.659s	user 0.004s	sys 0.001s
I20260709 12:23:41.434762 19719 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:41.402287 19773 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:23:41.436939 19652 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.446828 19652 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.439886 19604 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:41.449522 19604 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.452040 19604 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:41.441716 19606 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:41.469126 19606 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.494751 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:41.502643 19652 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:41.524271 13693 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.870s	user 0.017s	sys 0.008s
W20260709 12:23:41.527267 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.89045e+06us
I20260709 12:23:41.531387 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.536432 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.537037 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.539089 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.540962 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.541832 13649 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:41.546419 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:41.548089 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 330137us
I20260709 12:23:41.557178 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:41.565500 19748 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:41.564188 19069 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:41.574003 13792 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:23:41.556905 19770 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:41.564375 19652 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:23:41.600828 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.597976 13644 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:41.600385 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:41.602092 13652 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 12:23:41.602082 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:41.604517 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:41.597496 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.590082 19069 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:41.613097 13649 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:23:41.616008 13795 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:23:41.573781 19748 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:23:41.676397 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.677693 13649 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:23:41.746085 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:41.842351 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 440740us
I20260709 12:23:41.846413 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:41.847425 13652 raft_consensus.cc:2368] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:23:41.611997 19069 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:41.615160 19652 consensus_queue.cc:1243] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:23:41.845559 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7433093449999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 12:23:41.860499 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.881618 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:41.840555 19606 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:41.951882 19784 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
W20260709 12:23:41.954820 19786 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
W20260709 12:23:41.956961 19781 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.360s	user 0.000s	sys 0.000s
I20260709 12:23:41.851874 19768 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:41.939218 19779 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.415s	user 0.002s	sys 0.000s
I20260709 12:23:42.047389 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:42.066032 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 461358us
I20260709 12:23:41.945616 19783 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:41.983213 19606 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:42.065819 19749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.013s	sys 0.040s
I20260709 12:23:42.112217 19734 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:42.124859 19749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.013s	sys 0.040s
I20260709 12:23:42.154081 19770 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:42.047094 19768 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.102218 19606 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.099973 19783 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.149394 19734 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.188766 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:41.900354 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:42.223577 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 917) took 1165 ms. Trace:
I20260709 12:23:42.223438 13651 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:42.231279 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:42.226961 13800 rpcz_store.cc:276] 0709 12:23:41.058080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.058878 (+   798us) service_pool.cc:224] Handling call
0709 12:23:41.059365 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:41.122029 (+ 62664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.223208 (+1101179us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b540012a054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:42.223484 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58611072}
I20260709 12:23:42.246165 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:42.246851 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:42.248276 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:42.252193 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:42.264026 13646 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:42.272205 19801 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:42.280026 19801 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:42.280683 19801 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:42.284052 19801 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:42.285650 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3007) took 1855 ms. Trace:
I20260709 12:23:42.286172 13724 rpcz_store.cc:276] 0709 12:23:40.430598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.434208 (+  3610us) service_pool.cc:224] Handling call
0709 12:23:40.434489 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:40.724792 (+290303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:40.724842 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:40.724902 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:40.724950 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.285306 (+1560356us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b5400155ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:42.285370 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.285544 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666957568}
W20260709 12:23:42.289426 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3381) took 2149 ms (client timeout 1755 ms). Trace:
W20260709 12:23:42.290684 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:42.279322 19789 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 12:23:42.283228 19749 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:42.290025 13796 rpcz_store.cc:269] 0709 12:23:40.140316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.140455 (+   139us) service_pool.cc:224] Handling call
0709 12:23:42.289067 (+2148612us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:42.289270 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48025984}
I20260709 12:23:42.297191 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3030) took 1155 ms. Trace:
I20260709 12:23:42.301962 19587 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:23:42.302587 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:42.332818 13649 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:42.302295 19801 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:42.304358 19715 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:42.342314 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.014s	sys 0.042s
W20260709 12:23:42.342866 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.014s	sys 0.042s
I20260709 12:23:42.329928 13721 rpcz_store.cc:276] 0709 12:23:41.141339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.296237 (+154898us) service_pool.cc:224] Handling call
0709 12:23:41.308557 (+ 12320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:41.437562 (+129005us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.437589 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:41.438633 (+  1044us) log.cc:844] Serialized 52 byte log entry
0709 12:23:41.853034 (+414401us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:23:41.939120 (+ 86086us) log.cc:844] Serialized 14 byte log entry
0709 12:23:42.224191 (+285071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:42.224358 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:42.296851 (+ 72493us) raft_consensus.cc:1685] finished
0709 12:23:42.296883 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:42.297101 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128609,"prepare.queue_time_us":60651,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":274894,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":13715,"spinlock_wait_cycles":99072,"thread_start_us":671580,"threads_started":2,"wal-append.queue_time_us":674873}
I20260709 12:23:42.312500 19697 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:42.313100 19557 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:42.307093 19591 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:42.333395 19791 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.269s	user 0.000s	sys 0.002s
I20260709 12:23:42.350535 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:42.350231 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:42.353569 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.354338 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:42.334470 19587 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:42.355882 13722 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:42.356992 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:42.339161 19763 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:42.340932 19715 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:42.341913 19760 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:42.362731 19797 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
W20260709 12:23:42.357347 19796 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:23:42.454015 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:42.333911 19749 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:42.356855 19802 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:42.606946 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:42.358125 19606 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:42.611040 13616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.004s	sys 0.030s
W20260709 12:23:42.611541 13616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.004s	sys 0.030s
I20260709 12:23:42.350229 19697 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.745746 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:42.350463 19557 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.746696 13721 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:42.604434 19535 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:42.748303 19793 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.506s	user 0.001s	sys 0.000s
W20260709 12:23:42.836472 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.449096 19763 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.604754 19760 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.960570 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:42.961876 13724 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 12:23:42.857133 19794 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.588s	user 0.000s	sys 0.002s
I20260709 12:23:42.825522 19768 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:42.967944 13724 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:42.825865 19486 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:42.850745 19734 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:42.972030 13724 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:42.985064 13724 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.986891 13724 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:42.739475 19802 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.975307 19486 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:42.989024 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:42.827504 19535 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.002611 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1221) took 1725 ms. Trace:
I20260709 12:23:42.755843 19587 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.005000 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.006677 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.003296 13718 rpcz_store.cc:276] 0709 12:23:41.277023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.297657 (+ 20634us) service_pool.cc:224] Handling call
0709 12:23:41.298031 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:41.437516 (+139485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:41.437549 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:41.440676 (+  3127us) log.cc:844] Serialized 26 byte log entry
0709 12:23:41.686696 (+246020us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:41.686753 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:41.686843 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.002230 (+1315387us) raft_consensus.cc:1685] finished
0709 12:23:43.002291 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.002507 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138081,"raft.queue_time_us":138320,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":19251,"thread_start_us":226238,"threads_started":1,"wal-append.queue_time_us":706559}
I20260709 12:23:42.969256 19807 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:42.806080 19715 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:42.999123 19805 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:43.007795 19731 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.659s	user 0.017s	sys 0.001s
I20260709 12:23:43.010651 19801 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.021857 19802 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.013080 19808 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.024219 19749 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.026016 19535 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.028182 19806 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:43.030202 19763 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:43.030906 14425 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 958) took 2655 ms (client timeout 2205 ms). Trace:
I20260709 12:23:43.031805 19557 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:43.034435 19739 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 2.210s	user 0.007s	sys 0.021s
I20260709 12:23:43.023043 19697 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.024569 19486 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.040870 19715 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.016645 19807 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.058414 19760 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.040239 19731 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.045013 19808 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.033648 19805 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.068233 19805 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.050168 19806 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.080005 19806 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.068302 19731 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.070787 19808 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.080907 19807 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.074247 19810 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:43.085003 19810 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:43.085005 19790 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.139s	user 0.013s	sys 0.033s
I20260709 12:23:43.088444 19810 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:43.124289 13616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.012s	sys 0.014s
W20260709 12:23:43.125444 13616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.012s	sys 0.014s
W20260709 12:23:43.129316 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.131019 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.131682 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.146306 13720 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:23:43.148020 13721 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:23:43.150599 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.152191 13719 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:43.155405 13722 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:43.155227 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:43.159287 19711 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 3.085s	user 0.006s	sys 0.032s
W20260709 12:23:43.159655 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.007s	sys 0.007s
W20260709 12:23:43.162130 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.007s
W20260709 12:23:43.046474 14425 rpcz_store.cc:269] 0709 12:23:40.375895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.376089 (+   194us) service_pool.cc:224] Handling call
0709 12:23:43.030588 (+2654499us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:43.030777 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805171456}
I20260709 12:23:43.163180 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:43.164382 13718 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:43.167006 13792 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:43.170130 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2314) took 3898 ms. Trace:
I20260709 12:23:43.170701 19808 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.170837 13643 rpcz_store.cc:276] 0709 12:23:39.271697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:39.281752 (+ 10055us) service_pool.cc:224] Handling call
0709 12:23:39.298446 (+ 16694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:40.247745 (+949299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:43.169208 (+2921463us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400115154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:43.169783 (+   575us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170902912}
I20260709 12:23:43.171932 19808 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.173870 13720 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:23:43.176574 13720 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 9
I20260709 12:23:43.180444 13720 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:23:43.198866 13720 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:43.198935 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.204324 19762 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:43.205675 19762 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.209458 19762 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.204424 19643 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:43.211602 19643 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.215683 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.219439 19606 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:43.221446 19606 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:43.225962 13721 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 11.
I20260709 12:23:43.204329 19807 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:43.229820 19810 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:43.232456 19810 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:43.235661 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.233845 19810 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:43.237440 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.227479 19807 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.243666 19807 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.236708 13720 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:43.250849 19815 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:43.252121 19815 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.210021 19808 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.253120 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 924) took 1657 ms. Trace:
I20260709 12:23:43.253736 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:43.256253 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.222023 19606 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:43.253619 13792 rpcz_store.cc:276] 0709 12:23:41.595687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.595841 (+   154us) service_pool.cc:224] Handling call
0709 12:23:43.252771 (+1656930us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:43.252975 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3575839616}
W20260709 12:23:43.195976 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57424.4us
I20260709 12:23:43.237114 19810 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:43.269764 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.271116 13720 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:43.270848 19606 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:43.272557 19798 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.935s	user 0.000s	sys 0.011s
I20260709 12:23:43.272950 19606 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:43.273715 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:43.275808 13800 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:23:43.277747 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.278358 19750 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.278896 13721 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:43.280190 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.286955 19536 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:43.288101 19815 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.272569 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.290197 13719 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:23:43.292232 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.290125 19536 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:43.293215 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:43.295240 19536 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.297116 19536 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.263703 19643 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.302588 13721 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:43.232971 19609 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:43.308856 13718 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:43.315367 19472 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.317430 19810 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.317551 19472 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.319334 19810 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:43.319460 13615 connection.cc:256] RPC call timeout handler was delayed by -1.2340489069997602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
I20260709 12:23:43.320238 19776 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:43.321048 19776 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.321437 19472 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.322875 19419 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:43.323060 19634 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:43.323697 19419 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.323976 19634 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.326749 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.328387 19419 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:43.329456 13615 connection.cc:256] RPC call timeout handler was delayed by -1.044133576999684s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19940000000000002s for a response
I20260709 12:23:43.331135 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:43.332171 14424 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:43.331243 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:43.341732 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:43.338937 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.359320 13720 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:43.357048 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:43.365351 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:43.364765 19776 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:43.371304 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1217) took 2250 ms (client timeout 1567 ms). Trace:
W20260709 12:23:43.371845 13723 rpcz_store.cc:269] 0709 12:23:41.120538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:41.123906 (+  3368us) service_pool.cc:224] Handling call
0709 12:23:43.370782 (+2246876us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:43.371033 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339402112}
W20260709 12:23:43.374575 19069 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.486s	user 0.013s	sys 0.011s
I20260709 12:23:43.383596 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:43.380663 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70253.5us
I20260709 12:23:43.364691 13719 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:23:43.389142 13643 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:23:43.392234 19374 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.406421 19722 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.406761 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:43.377763 19754 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 2.413s	user 0.006s	sys 0.022s
I20260709 12:23:43.378901 19783 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.403401 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:43.375633 19069 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.416465 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:43.417035 13649 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.083s	user 0.021s	sys 0.005s
I20260709 12:23:43.417657 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:43.418497 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 962) took 1158 ms. Trace:
I20260709 12:23:43.418903 13649 rpcz_store.cc:276] 0709 12:23:42.260367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:42.297980 (+ 37613us) service_pool.cc:224] Handling call
0709 12:23:43.418397 (+1120417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:43.419379 19704 raft_consensus.cc:2764] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:23:43.390178 19634 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.412748 19420 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.426664 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.427080 19704 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:43.437443 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.441001 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.442499 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.429955 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:43.444001 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3005) took 3013 ms. Trace:
I20260709 12:23:43.444396 13797 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:43.447016 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.447436 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.448469 19420 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.451025 19420 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.451918 13792 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:43.452291 19704 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.453327 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.455363 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.456323 13798 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:43.444499 13727 rpcz_store.cc:276] 0709 12:23:40.429935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:40.432366 (+  2431us) service_pool.cc:224] Handling call
0709 12:23:40.432719 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:40.433146 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:40.433178 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:40.442851 (+  9673us) log.cc:844] Serialized 26 byte log entry
0709 12:23:40.873752 (+430901us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:40.873799 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:40.873879 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:43.376541 (+2502662us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:43.443358 (+ 66817us) raft_consensus.cc:1685] finished
0709 12:23:43.443381 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:43.443729 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2224,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10812,"spinlock_wait_cycles":12587776,"thread_start_us":257042,"threads_started":1,"wal-append.queue_time_us":351290}
I20260709 12:23:43.459949 19704 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:43.462535 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98603.3us
I20260709 12:23:43.462728 19643 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:43.463383 19643 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:43.463861 19643 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:43.453226 19374 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.466766 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.466890 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.467821 19374 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.464030 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.470736 14425 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:43.464543 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.475669 13727 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:43.451803 13801 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:43.472312 13797 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:43.482930 14416 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:43.483070 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.429970 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:43.490633 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:43.505363 14424 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:43.506467 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:43.506834 19803 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:23:43.507321 13649 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:43.507774 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.508940 19587 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:23:43.513823 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.489815 19634 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:43.534523 19634 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.537223 13797 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:43.545089 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.544837 19374 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:43.546103 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:43.545847 19374 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.548472 19374 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.549103 19770 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:23:43.544936 13767 connection.cc:256] RPC call timeout handler was delayed by -1.710777207999854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
I20260709 12:23:43.578871 19069 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.537634 19634 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.548333 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:43.591500 19634 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:43.592082 19634 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:43.592562 19634 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:43.597504 14416 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:43.598826 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 86988.7us
I20260709 12:23:43.597527 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.600124 14423 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
W20260709 12:23:43.617511 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 69284.7us
I20260709 12:23:43.638866 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.640357 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.654995 13797 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:23:43.658906 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.668681 19374 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.670023 13719 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:23:43.669648 19374 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.673092 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.675329 19374 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.678489 14424 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:43.668812 19774 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:43.680269 19774 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.690526 13794 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:43.688213 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:43.694845 19374 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:43.707713 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.715150 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:43.715205 13766 connection.cc:256] RPC call timeout handler was delayed by -1.445258456999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
W20260709 12:23:43.716163 13766 connection.cc:256] RPC call timeout handler was delayed by -1.293158456999663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
W20260709 12:23:43.717806 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.718116 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.718433 19488 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.718778 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.719053 19488 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.719053 14416 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:43.720078 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.720907 13797 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:43.721143 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.721575 19488 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.721937 14421 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:43.722096 13794 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:23:43.707903 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46232us
I20260709 12:23:43.727324 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.728080 14422 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:43.728130 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:43.722602 13724 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.017s
I20260709 12:23:43.698130 19374 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:43.728808 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:43.734344 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:43.735047 19821 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:23:43.735424 13796 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:43.735311 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.737926 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:43.738761 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45399us
I20260709 12:23:43.739295 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.739508 19652 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:43.739646 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:43.740132 19652 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:43.741266 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.738209 14424 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:23:43.742126 19708 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:43.742367 13793 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:23:43.743641 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.737964 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.745371 13794 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:43.745636 13793 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:43.746771 19774 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:43.747254 13719 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:23:43.734046 19523 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:43.747661 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.747838 19523 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.749413 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.743609 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.751554 13792 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:23:43.753011 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.738533 19374 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.746630 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.754721 13724 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:23:43.741329 19652 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.763927 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:43.764784 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 335019us
I20260709 12:23:43.756990 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:43.768896 13724 pending_rounds.cc:85] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:23:43.770586 13724 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:23:43.771000 13724 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:43.775066 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.779078 13719 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:43.783591 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:23:43.785074 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:43.785642 13690 connection.cc:256] RPC call timeout handler was delayed by -1.7640648139999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27690000000000003s for a response
W20260709 12:23:43.786101 13690 connection.cc:256] RPC call timeout handler was delayed by -1.729409078000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
I20260709 12:23:43.783793 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.765761 19606 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:43.791766 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.795344 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:43.799381 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:43.800304 19523 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:43.802737 19744 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:43.803203 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61199.2us
I20260709 12:23:43.804723 19759 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:43.805133 19759 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:43.805508 19759 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:43.825978 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:43.830878 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:43.792759 19774 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:43.826273 19536 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.005s
W20260709 12:23:43.845817 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.844949 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:43.850975 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54110.9us
I20260709 12:23:43.854791 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.856050 14424 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:23:43.856161 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:43.850221 14422 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:43.857096 13649 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:43.862655 14421 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:43.867416 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.850162 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.873658 13798 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:43.881167 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 388608us
I20260709 12:23:43.883632 13797 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:23:43.887706 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.895105 13796 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:43.894843 19759 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.897502 19759 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:43.900023 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.900804 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.901760 13722 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:43.901872 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.902653 18898 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:43.903009 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:43.903316 18898 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:43.903632 13649 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:43.904443 18898 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.906564 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:43.907264 13693 leader_election.cc:400] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:23:43.906822 19460 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:43.907613 13798 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:23:43.909474 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 145772us
I20260709 12:23:43.910753 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:43.911677 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:43.912423 13792 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:23:43.912855 13792 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 10
I20260709 12:23:43.913110 13792 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:23:43.891515 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.913341 13792 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:43.906858 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:43.900913 13726 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:43.891000 14421 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:43.845069 19606 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:43.923835 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.924901 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.925647 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:43.919152 19815 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:43.923840 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.925704 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.930112 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:43.930527 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.937773 13794 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:43.941586 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:43.945847 13800 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.669s	user 0.013s	sys 0.007s
I20260709 12:23:43.947253 13800 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:43.876362 19774 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.959326 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:43.882941 19643 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:43.971841 13796 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:43.941030 19815 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:44.042987 14425 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:44.043316 13798 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:43.956861 19587 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:43.984778 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:44.004892 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:43.960667 19722 raft_consensus.cc:2764] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:23:43.954406 13727 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:44.035533 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.041432 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.041675 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:43.985082 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:44.044620 19815 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:44.051503 13649 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:44.051741 14421 raft_consensus.cc:2368] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
W20260709 12:23:44.057557 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.060307 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:44.045656 19587 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:44.063670 19715 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.902s	user 0.011s	sys 0.012s
I20260709 12:23:44.064087 19587 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:44.064409 19826 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:23:44.068854 19774 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:44.060204 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:44.072604 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.080572 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:44.078481 13650 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
W20260709 12:23:44.079358 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44764.6us
I20260709 12:23:44.070093 19460 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.066159 19420 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:23:44.087862 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 286844us
I20260709 12:23:44.084086 19460 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.088629 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.083055 19374 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:44.108952 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.113569 14424 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:23:44.115947 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80564.2us
I20260709 12:23:44.116228 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:44.123117 19720 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:44.123911 19720 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.132970 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.133150 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.133788 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.134542 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.135186 19536 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:44.135686 19536 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.136404 19536 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:44.137354 13796 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:23:44.137660 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.138525 13794 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:44.114952 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:44.143291 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47449.9us
I20260709 12:23:44.143076 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.144418 13797 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:44.144302 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.149030 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.100080 19488 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:44.129921 19587 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.169261 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.170701 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:44.171352 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.109935 19744 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:44.185295 19744 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.185959 19744 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:44.103889 19715 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:23:44.107537 19420 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:44.173204 19643 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:44.173720 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.192147 13794 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:44.171703 19069 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.177927 13650 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:44.180156 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.184191 19460 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.196465 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.172938 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.198489 19715 consensus_queue.cc:1243] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:23:44.200557 19587 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.175561 19606 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:44.201761 19834 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.255s	user 0.000s	sys 0.004s
I20260709 12:23:44.106138 19800 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
W20260709 12:23:44.223503 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 102761us
I20260709 12:23:44.226788 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:44.217615 19715 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:44.216496 19460 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:44.238857 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:44.231923 19634 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.008s
I20260709 12:23:44.247805 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.214277 19069 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.249161 13643 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:44.250459 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:44.219614 19606 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:23:44.235389 19715 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:44.237617 19720 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.268929 19720 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:44.276371 19069 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.246447 19634 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.278486 14423 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:23:44.280735 19634 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.248374 19643 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:44.282706 19643 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:44.284006 19643 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.287067 19420 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.313333 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:44.314584 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73924.2us
I20260709 12:23:44.316236 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:44.317099 13795 raft_consensus.cc:606] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.317490 13795 raft_consensus.cc:614] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:44.324344 13643 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:23:44.329507 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:44.331465 13799 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:23:44.332777 13724 raft_consensus.cc:2376] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20. Re-sending same reply.
I20260709 12:23:44.334756 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:44.337875 13725 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:44.331707 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.331008 18898 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:23:44.341209 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:44.340503 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.342197 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:44.342660 18898 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:44.343109 18898 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:44.327273 19634 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.346421 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
W20260709 12:23:44.350975 19609 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.037s	user 0.017s	sys 0.004s
I20260709 12:23:44.370610 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.371997 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:44.372682 13643 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:44.373780 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 448233us
I20260709 12:23:44.376619 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.378574 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:44.386085 19803 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.386989 19803 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.389963 19803 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.397068 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:44.397377 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:44.397857 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:44.373543 19335 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:23:44.410092 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.408774 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 245244us
W20260709 12:23:44.411310 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.411974 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.416472 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.421480 19803 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:44.401124 19745 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.430899 19745 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.431829 13693 leader_election.cc:400] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:23:44.374071 19815 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.432327 13693 leader_election.cc:403] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:23:44.434486 13797 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:44.435122 19744 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.437304 19744 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.440042 19536 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:44.441219 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.442458 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:44.444320 19488 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:44.434059 19815 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:44.414348 19715 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.455879 19715 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:44.440332 19803 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:23:44.448395 19335 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:44.459295 19335 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.470955 19803 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:44.471624 19803 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:44.472121 19803 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:44.479948 18898 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.482051 18898 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:44.502717 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.354408 19634 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.503835 19536 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:44.511682 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108803us
I20260709 12:23:44.511684 19644 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:44.513589 19644 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.514046 19644 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:44.519656 19745 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.526567 19460 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.527274 19460 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.530828 19460 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.534176 13725 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 12.
I20260709 12:23:44.535718 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:44.538398 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 424647us
W20260709 12:23:44.546653 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 429010us
W20260709 12:23:44.559211 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 460328us
I20260709 12:23:44.527269 19634 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.585390 13690 leader_election.cc:400] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:23:44.586820 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:44.595295 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57789.7us
W20260709 12:23:44.607537 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 414129us
I20260709 12:23:44.611805 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:44.612265 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:44.617178 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81605.8us
I20260709 12:23:44.631613 14424 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 12:23:44.632128 14424 pending_rounds.cc:85] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 15
I20260709 12:23:44.624046 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:44.632378 14424 pending_rounds.cc:107] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 12:23:44.632702 14424 raft_consensus.cc:2889] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:44.601917 19652 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:44.615195 19536 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:23:44.641995 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 265429us
I20260709 12:23:44.653735 19609 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:44.650995 19745 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:44.655434 19745 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:44.655469 19609 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.655953 19745 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:44.656083 19609 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.658447 19609 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:44.658661 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 321774us
I20260709 12:23:44.659511 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.660914 19335 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:44.661537 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:44.643788 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:44.664891 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:44.635970 19652 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:44.666580 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:44.645332 19536 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:44.673368 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:44.680653 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5379574360003971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:23:44.681180 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4839574360003098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 12:23:44.681545 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4263574360002167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 12:23:44.681983 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3318574360005186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 12:23:44.682471 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3219574360005026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 12:23:44.682919 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3021574360004706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:23:44.683219 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1482574360002218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
I20260709 12:23:44.681489 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:44.683517 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1293574360001912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
W20260709 12:23:44.683996 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0015574360004393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260709 12:23:44.687808 19844 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:44.688561 19844 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.690160 19652 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.636277 19843 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:23:44.693861 13719 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:44.669829 19643 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:44.672747 19759 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:44.695353 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.697470 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.699906 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.679117 19536 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:44.696784 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.755s (SENT)
I20260709 12:23:44.706537 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:44.707208 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59436.2us
I20260709 12:23:44.700416 19849 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:44.712561 19849 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.712945 19849 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:44.706015 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.714963 13719 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9.
I20260709 12:23:44.716231 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.716872 13719 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:44.717840 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.719942 19844 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.720397 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:44.720645 13724 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:44.721457 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.721601 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:44.722210 13724 raft_consensus.cc:2376] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 9. Re-sending same reply.
I20260709 12:23:44.723081 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.723536 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.724731 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.725240 13719 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
W20260709 12:23:44.723104 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.569s (SENT) [suppressed 30 similar messages]
W20260709 12:23:44.726711 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93402.1us
I20260709 12:23:44.727258 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:44.727766 19840 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:23:44.728092 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:44.729990 19420 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:44.730739 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.726701 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.731789 13720 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:44.732013 13718 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:23:44.732970 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.733201 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.734432 19842 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:44.707566 13721 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
W20260709 12:23:44.736892 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50548.6us
I20260709 12:23:44.737624 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:44.734871 19759 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:44.700729 19643 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:44.740607 13718 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:44.742347 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.743031 13725 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:44.740412 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:44.728103 13719 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 12:23:44.751890 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49904.7us
I20260709 12:23:44.766644 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:44.773475 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65171.5us
I20260709 12:23:44.774359 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.747341 19643 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:44.790437 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67381.3us
I20260709 12:23:44.777825 19845 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:44.758904 19609 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:44.783183 19759 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.788223 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:44.741621 19842 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:44.777463 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.772553 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:44.791747 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87561.4us
I20260709 12:23:44.792467 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:44.793491 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.801339 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:44.801620 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:44.805066 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:44.801146 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:44.806778 14421 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:44.802222 18898 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:23:44.807231 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:44.807633 13693 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.629s (SENT)
I20260709 12:23:44.808569 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:44.791375 19845 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:44.818291 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68810.4us
I20260709 12:23:44.796468 19606 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:44.790066 19803 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.798537 19708 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:44.834405 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:44.834832 19708 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:44.835340 19708 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:44.836668 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.837651 13646 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:44.820566 19843 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:44.814667 19800 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:44.839255 19800 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:44.839687 19800 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:44.797070 19609 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:44.842526 19609 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:44.826002 13651 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:44.822896 13643 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:44.801002 19374 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:44.825594 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.825914 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:44.826843 19839 raft_consensus.cc:2764] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:23:44.827700 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:44.874570 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 155264us
I20260709 12:23:44.828205 13800 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:23:44.820674 19761 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:44.829042 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76114.3us
I20260709 12:23:44.820292 19069 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:44.828917 13544 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 13, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 13 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.798739 19634 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:44.817341 19845 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:44.845635 19842 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:44.775210 19841 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 12:23:44.895535 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92247.5us
I20260709 12:23:44.910641 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:44.905853 14422 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:23:44.905946 14416 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:23:44.907785 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44623.7us
I20260709 12:23:44.908147 19862 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:44.914291 19420 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:44.895988 19634 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:44.915187 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.895061 19864 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:44.919037 13544 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 18, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 18 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:44.845302 19606 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.921277 13724 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:44.922400 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:44.927942 19803 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.943226 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.952302 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:44.953955 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78294.9us
I20260709 12:23:44.954895 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.949579 19745 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.957705 19644 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.958562 19745 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:44.933763 19634 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:44.952800 19523 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:44.962029 13643 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:44.962296 19606 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:44.962635 19644 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:44.962092 14418 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:23:44.912127 19536 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:44.965408 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97154.2us
I20260709 12:23:44.966857 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.967042 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.930490 19069 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:23:44.974820 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.977628 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.981787 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:44.986997 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:44.966161 19715 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:44.999650 19715 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.003026 13649 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:45.004804 13643 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:44.968415 19850 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:23:45.012446 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56825.8us
I20260709 12:23:45.028640 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.002110 19715 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:44.979348 19523 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:44.993634 19708 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:44.997963 19849 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.070485 19866 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:45.074549 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.079200 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:45.082072 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 158958us
I20260709 12:23:45.084852 13650 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:23:45.084929 14423 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:23:45.085685 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148184us
I20260709 12:23:45.091756 19745 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:45.092234 19745 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:45.092878 19849 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.093199 19745 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.093878 19523 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.095237 19643 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.098666 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.099435 13648 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:23:45.101588 19536 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:45.102576 14422 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:23:45.087288 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:45.082234 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82281.7us
I20260709 12:23:45.102571 19759 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:45.075160 19715 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:23:45.131156 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:45.133114 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45907.5us
I20260709 12:23:45.133783 19745 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.134574 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.136821 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.137885 19606 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:23:45.138664 19839 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:45.139613 19839 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:45.140548 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.141837 19839 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:45.142470 19843 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:45.143085 19846 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:45.143654 19491 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:45.144475 19871 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.124936 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:45.158744 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.163615 19843 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:45.163792 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.164198 19843 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:45.166585 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:45.168552 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:45.168192 19871 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.169804 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.124912 19759 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:45.170595 13644 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:45.154011 19606 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:45.171206 19871 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.171595 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.172752 13645 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:45.172847 13644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:23:45.173597 13644 raft_consensus.cc:606] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.174006 13644 raft_consensus.cc:614] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:45.174692 13647 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:45.175305 19643 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.177335 14425 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
W20260709 12:23:45.184207 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60123.3us
I20260709 12:23:45.191972 13798 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:23:45.087769 19868 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 12:23:45.139714 19587 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:23:45.141975 19845 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.204833 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:45.190635 19643 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:45.210808 13800 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:45.215411 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:45.218551 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.215306 13652 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:45.222416 13798 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:45.210752 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:45.226161 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.228440 13651 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:23:45.229447 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72010.8us
I20260709 12:23:45.227461 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:45.212006 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:45.189374 19842 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:23:45.204918 19868 consensus_queue.cc:1243] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:23:45.211033 19873 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:23:45.233537 14417 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:23:45.234436 19852 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.443s	user 0.004s	sys 0.002s
I20260709 12:23:45.234606 14420 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:45.237320 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:45.258982 13643 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:23:45.224823 19643 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.240222 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:45.242586 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:45.210649 19335 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:23:45.262660 14419 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:45.233460 19708 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:45.273478 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:45.232052 19374 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:45.271311 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.230899 19839 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:45.240561 19864 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:45.171242 19759 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.165545 19843 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.274381 13726 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:45.274230 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:45.271346 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.284183 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:45.301654 13727 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:45.302697 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:45.261620 19846 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:45.267443 19803 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:45.298578 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:45.304559 13723 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:23:45.289779 19841 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:45.281637 19839 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.263948 19715 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:23:45.257514 19720 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:23:45.315585 13650 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:45.274907 19875 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:23:45.319924 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:45.332782 13799 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:45.323292 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52110.1us
I20260709 12:23:45.334762 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.336151 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.337393 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.328994 13643 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:23:45.306370 19842 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.306445 19846 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.307633 19873 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:45.282956 19536 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:23:45.313374 19862 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:45.311429 19803 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:45.321282 19839 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.308005 19800 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.363741 19841 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:45.364598 19841 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:45.366933 19866 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:45.367276 19536 consensus_queue.cc:1243] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:45.367717 19866 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:45.378793 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.349459 19868 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:45.348155 19864 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:45.386562 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:45.389700 13799 raft_consensus.cc:606] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.390061 13799 raft_consensus.cc:614] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:45.350455 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.351157 19846 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:45.415036 14424 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:45.369232 19803 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:45.435261 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163978us
I20260709 12:23:45.385407 19745 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:45.385156 19868 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:45.339550 19609 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.436092 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:45.351512 19842 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.392397 19845 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:45.393085 19864 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:45.430670 19643 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:45.455902 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105645us
I20260709 12:23:45.476961 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.479094 13648 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:45.446846 19883 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:45.369129 19866 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.453080 19865 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:45.505143 13647 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:45.481493 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.456840 19864 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:45.450775 19868 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:45.509884 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:45.525213 13801 raft_consensus.cc:606] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.454249 19720 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:23:45.514014 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:45.526363 19891 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.094s	user 0.002s	sys 0.000s
I20260709 12:23:45.526950 19609 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:45.531191 19873 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:23:45.503932 19845 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:45.532612 19800 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:45.533538 19634 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.004s
I20260709 12:23:45.534430 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:45.498181 19883 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:45.503782 19865 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:45.559635 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:45.569617 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88211.1us
I20260709 12:23:45.570395 13651 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:45.579088 13799 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:45.510596 19715 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:23:45.561053 19335 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:45.589352 13648 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:23:45.587888 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.562198 19845 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:45.594390 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 672612us
I20260709 12:23:45.563776 19536 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:45.598714 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.611214 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.583798 19884 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:23:45.580075 19883 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:45.615619 19866 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.620131 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:45.623624 19881 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:45.629590 19720 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:45.594676 19335 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.630501 19335 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:45.633811 13650 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:45.579645 19634 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.611734 19845 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:45.594321 19491 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:23:45.597949 19523 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
W20260709 12:23:45.672694 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81857.7us
I20260709 12:23:45.632980 19374 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:45.680415 14416 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:45.598387 19609 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:23:45.673115 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:45.687809 19523 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:23:45.688282 19523 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:45.689163 19523 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.674785 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:45.704802 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 143193us
I20260709 12:23:45.679960 19374 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.706921 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:45.708376 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 120579us
I20260709 12:23:45.710268 19873 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:45.711531 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.713606 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:45.699040 19842 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:45.719041 13544 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:45.707083 19523 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:45.713979 19374 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:45.705586 19800 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:45.728155 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:45.740096 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:45.741261 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:45.743976 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 132845us
I20260709 12:23:45.747611 19842 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:45.741971 19866 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:45.748370 19842 consensus_queue.cc:1243] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:45.750198 19866 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:45.750622 19866 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.738974 19800 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:45.751905 19800 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:23:45.757570 19860 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.853s	user 0.004s	sys 0.000s
W20260709 12:23:45.760146 19897 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:45.767019 19634 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.768338 19866 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.770602 19866 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:45.773960 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.719506 19609 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:45.775120 19609 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:45.775868 19609 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:45.777933 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59702.2us
I20260709 12:23:45.783859 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:45.795673 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54471.2us
I20260709 12:23:45.823159 13725 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:23:45.796330 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 97207.8us
I20260709 12:23:45.795683 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.836261 14416 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:23:45.800720 13726 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:45.836881 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:45.815440 19905 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:23:45.792420 19491 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:23:45.819494 19715 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:45.852593 19715 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:45.796064 19803 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.794850 19907 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:45.799297 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:45.843333 19911 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:23:45.844640 19874 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:23:45.853122 19715 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:45.830485 19839 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:45.847407 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45693.6us
I20260709 12:23:45.855183 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.859687 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:45.868300 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:45.862764 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.877425 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.884840 13793 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:23:45.846649 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.879562 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:45.881731 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.888047 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:45.888589 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.899160 19883 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.902767 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.872046 19720 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:45.905119 14416 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:45.902956 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.873508 19907 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:45.911199 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51641.3us
I20260709 12:23:45.911036 18898 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:45.913007 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.911693 19745 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:23:45.919243 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:45.920359 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:45.919760 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.921450 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.931671 19609 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.935055 19883 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.918238 19745 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:45.936877 19846 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.842365 19864 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.915387 19720 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.939168 13647 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:45.939788 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.940064 19864 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.940650 13793 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:45.941643 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.941845 19803 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:45.942421 13792 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15.
I20260709 12:23:45.943516 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.943871 19374 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.913137 19907 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:45.944716 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.945637 13792 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:45.946656 19800 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.947244 19335 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:45.948696 19587 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:45.949363 19800 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:45.879143 19874 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:45.822219 19868 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.936910 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:45.951720 14418 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:45.938138 19745 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:45.952664 13793 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:45.975171 13719 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:45.953193 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.952466 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:45.976296 13795 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:45.916888 19841 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.552s	user 0.007s	sys 0.007s
I20260709 12:23:45.978852 13725 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:23:45.979007 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.978711 18898 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:45.976573 13792 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:45.973342 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.983397 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.984813 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:45.985508 13727 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:45.989140 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:45.991568 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54776us
I20260709 12:23:45.940668 19720 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:45.908946 19875 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:23:45.996805 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.997632 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:45.995579 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:46.001644 14416 raft_consensus.cc:2368] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:45.950114 19800 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:23:46.002151 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.003654 19609 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.004920 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.011754 13792 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:46.008229 13799 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:45.979768 19864 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:45.979208 19374 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:45.950884 19420 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:23:45.997890 19708 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:23:45.902143 19845 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.029290 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:46.052796 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 140745us
I20260709 12:23:46.054873 13793 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:46.056340 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:45.979308 19606 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:23:46.063205 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:45.944656 19890 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.133s
W20260709 12:23:46.057956 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 58469.4us
I20260709 12:23:46.059015 19924 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:23:46.066952 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 138035us
I20260709 12:23:46.069571 13690 leader_election.cc:400] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:46.070698 13798 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:23:46.071961 19862 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:46.072533 19862 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.073299 19846 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:46.074066 19864 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:46.074561 19864 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:46.075016 19864 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:46.079118 19918 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:23:46.086223 19606 consensus_queue.cc:1243] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:23:45.986160 19841 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.070000 13690 leader_election.cc:403] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:23:46.090793 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.095441 13652 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:46.103703 13794 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:23:46.104244 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.104982 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.105154 13797 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:46.108284 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:46.109308 14421 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:46.110386 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:46.110667 14418 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:46.112311 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.115461 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.116717 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.120069 19523 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:46.120666 19523 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:46.121179 19523 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:46.126844 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.108532 19609 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:46.132368 19609 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:46.132762 19609 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.135239 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:46.138679 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:46.149082 19845 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.160800 13793 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:46.162961 13650 raft_consensus.cc:1217] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[11.8-14.10]   Dedup: 11.8->[12.9-14.10]
I20260709 12:23:46.167695 13650 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [11.8-14.10]
W20260709 12:23:46.168221 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57939.7us
I20260709 12:23:46.158810 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:46.166098 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53842us
I20260709 12:23:46.169091 13651 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.169839 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:46.170387 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:46.171927 13792 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:46.171597 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:46.172377 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.170894 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:46.170986 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 103381us
I20260709 12:23:46.185959 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.189735 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.191916 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.164086 19845 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.087901 19907 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.211278 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.215166 19491 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:46.215404 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.220659 19523 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.221649 19925 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:46.222348 19536 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:46.223965 19907 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.199926 19903 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:46.186451 19634 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:46.199143 19883 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:23:46.226425 19868 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.227886 19841 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.190603 19909 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.253242 19634 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:46.253561 19909 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.238649 13643 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:23:46.199378 19902 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:46.241552 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 110461us
I20260709 12:23:46.250680 19536 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:46.214540 19839 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:46.265163 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49837.7us
I20260709 12:23:46.265930 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:46.245775 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.252100 19862 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.267359 18898 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:46.268541 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.269683 19921 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:46.209944 19890 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 12:23:46.279105 19919 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:46.272078 13795 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:46.264874 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:46.282723 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.248924 19846 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:46.285655 19634 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:46.288581 19916 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.455s	user 0.002s	sys 0.000s
I20260709 12:23:46.271669 19902 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.263783 19720 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:46.266875 19536 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:46.276573 19715 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.297125 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.300578 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.309414 13799 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:46.310622 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.292408 19841 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:23:46.294732 19924 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:46.324755 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.297439 19883 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:46.323891 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:46.293524 19720 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:46.342046 18898 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:46.330541 19924 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.324388 19862 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.334360 19720 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.333184 19909 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.348846 19722 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:23:46.347857 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:46.351517 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.364431 13651 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:46.368229 19934 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.368861 19934 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.370134 19865 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.371392 19722 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:23:46.371603 19865 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:46.374395 19609 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:46.375062 19913 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.522s	user 0.000s	sys 0.002s
I20260709 12:23:46.371838 19722 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:46.378121 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.378777 19722 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.380060 19722 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.380435 19335 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:46.383436 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.372573 19875 raft_consensus.cc:993] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:46.386431 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.387140 13796 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:46.388609 19921 raft_consensus.cc:1081] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:23:46.339140 19931 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:46.361205 19864 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.349970 19924 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:46.393290 19864 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.393939 19609 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.396245 19715 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.397125 19523 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:46.397502 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.399128 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:46.404466 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56732.7us
I20260709 12:23:46.407941 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:46.435364 13720 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:23:46.435364 13796 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:46.438688 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:46.439654 13724 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:46.449276 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:46.453693 13720 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:46.460737 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:46.462014 13719 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 15
W20260709 12:23:46.470716 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 119271us
I20260709 12:23:46.471966 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:46.477147 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 177627us
I20260709 12:23:46.478188 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.440434 19934 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:46.480660 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:46.450088 19931 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:46.484403 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.487394 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.488401 13651 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:46.494683 13645 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:23:46.495630 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.496366 13651 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:46.470149 19335 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:46.467159 19875 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:46.478221 19420 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:46.504171 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.481199 19909 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:46.366986 19536 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.515412 19536 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:46.483398 19715 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:46.507313 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.506141 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.491231 13721 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:23:46.509987 19868 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.484498 19902 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.523803 13651 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:46.524091 13721 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.528668 13721 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.531069 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.532930 19868 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.535900 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.537698 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:46.539602 14421 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:46.541814 19909 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:46.542838 19909 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:23:46.477181 19491 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:23:46.548906 19925 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:46.561424 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.570430 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.517067 19335 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:46.577466 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.584465 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:46.616466 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85497.9us
I20260709 12:23:46.573453 19871 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:23:46.595202 19335 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.581895 19634 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:23:46.602779 19925 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:46.602653 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:46.558319 19491 consensus_queue.cc:1243] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:46.611217 13644 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:23:46.611326 19869 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:46.611455 19902 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:23:46.614817 13724 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:46.616207 13719 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:46.587883 19720 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.618374 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.516335 19875 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.622391 13648 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:46.626179 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:46.621579 13721 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.628911 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:46.629709 13724 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:46.619890 19069 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:46.632723 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:46.630715 19643 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:46.642398 19932 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:46.639736 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.638088 13644 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:23:46.647130 19875 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:46.647329 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:46.654762 13721 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.655783 13648 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:46.655814 13724 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:46.661015 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:46.663007 13648 pending_rounds.cc:85] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 12
I20260709 12:23:46.663406 13648 pending_rounds.cc:107] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:23:46.663661 13648 raft_consensus.cc:2889] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:46.667637 19634 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:23:46.665337 19643 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:23:46.671037 19931 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:23:46.650930 19890 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:46.673106 19890 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:46.673547 19890 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:46.676357 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.680131 19924 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.682488 19849 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:46.683183 19843 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:46.654295 19925 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:46.687433 19925 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:46.687913 19925 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:46.688971 18898 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:46.689373 18898 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:46.690222 18898 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.692046 19759 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:46.672143 19874 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.692471 13797 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:46.692924 19874 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:23:46.699306 18898 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:46.702521 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 996) took 1799 ms. Trace:
I20260709 12:23:46.665052 19708 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:46.703184 14424 rpcz_store.cc:276] 0709 12:23:44.903588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.903719 (+   131us) service_pool.cc:224] Handling call
0709 12:23:44.904181 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.427471 (+523290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.702189 (+1274718us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b540028ffd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:46.702426 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174249856}
I20260709 12:23:46.665525 19909 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:46.748557 13644 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:23:46.727725 19909 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:46.699433 19924 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:46.761857 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:46.781471 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:46.841718 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 185326us
I20260709 12:23:46.835032 19909 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:46.838622 19933 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:23:46.862939 19720 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.866442 13796 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:23:46.867251 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:46.868098 13798 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:46.852416 19924 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:23:46.875773 19924 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:46.876396 19924 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.880894 19951 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:23:46.883452 19374 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:46.886150 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43548.2us
I20260709 12:23:46.886373 19715 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:23:46.886024 19374 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.889871 19890 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:46.897219 19881 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:46.897686 19881 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:46.898245 19374 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:46.901911 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:46.899714 19890 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.916507 13721 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:46.920686 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:46.924321 19846 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:46.925359 19846 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:46.934888 19420 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:46.935654 19420 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:46.941087 19924 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.941790 19870 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:46.947301 19870 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:46.948386 19870 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.949477 19870 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.950300 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:46.951550 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3521) took 1421 ms. Trace:
I20260709 12:23:46.952104 13652 rpcz_store.cc:276] 0709 12:23:45.530116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.531021 (+   905us) service_pool.cc:224] Handling call
0709 12:23:45.531388 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.096018 (+564630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.951152 (+855134us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b54001482d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:46.951426 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291294976}
W20260709 12:23:46.952834 19910 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.146s	user 0.000s	sys 0.006s
I20260709 12:23:46.954301 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 964) took 1396 ms. Trace:
I20260709 12:23:46.954521 19606 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:46.954998 19606 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:46.954711 13801 rpcz_store.cc:276] 0709 12:23:45.557758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.558713 (+   955us) service_pool.cc:224] Handling call
0709 12:23:45.587261 (+ 28548us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.599172 (+ 11911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.599212 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:45.635751 (+ 36539us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.710281 (+ 74530us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:23:45.710336 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.710431 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.908883 (+1198452us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:46.953944 (+ 45061us) raft_consensus.cc:1685] finished
0709 12:23:46.953966 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:46.954202 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5580,"raft.queue_time_us":5884,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":18385,"spinlock_wait_cycles":63525760,"thread_start_us":18724,"threads_started":1,"wal-append.queue_time_us":70090}
I20260709 12:23:46.906126 19902 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:46.957695 19902 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.958923 19923 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:46.959344 19923 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:46.959976 19902 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:46.960317 19923 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.962291 13727 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:23:46.962772 13727 pending_rounds.cc:85] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:23:46.963061 13727 pending_rounds.cc:107] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:23:46.963351 13727 raft_consensus.cc:2889] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:23:46.971450 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:46.961995 19923 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:46.943466 19924 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:23:46.982141 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94559.4us
I20260709 12:23:46.908397 19955 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:46.983414 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:46.985953 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:46.898645 19881 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:46.995283 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.985679 19955 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:47.005045 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:47.007365 13799 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:46.993988 19864 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:23:47.014227 14418 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:46.985826 19715 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:23:47.020398 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:47.002948 19924 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:47.020915 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.985466 19925 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.021554 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:47.027864 13799 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:47.030566 19955 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.031925 19955 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:47.039436 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:47.040462 13792 raft_consensus.cc:2368] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:47.041050 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:47.043154 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0115897239998048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
I20260709 12:23:47.045276 19925 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.017549 19881 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:47.014492 19865 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:23:47.059166 19932 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:47.059778 19932 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.049063 19715 consensus_queue.cc:1243] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:23:47.073086 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.082286 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:47.083287 13648 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:47.084201 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:47.085414 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:47.086831 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.088152 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.089236 13648 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:47.091533 19909 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.048431 19720 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:47.103663 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.094156 19420 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.106052 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.099969 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:47.067524 19745 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:23:47.100453 19956 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:47.048633 19924 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:47.074597 19925 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:47.098073 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.108307 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.109612 13718 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:23:47.112389 13719 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:23:47.129742 13719 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:23:47.130040 13719 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:47.113737 19909 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.114090 13725 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [14.8-15.9]
I20260709 12:23:47.114751 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:47.116554 19956 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.116705 19932 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.122154 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3518) took 1593 ms. Trace:
I20260709 12:23:47.122991 13722 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:23:47.125723 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.128484 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:47.130420 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.131899 13648 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:47.134505 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.141657 13724 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:47.140808 13722 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:47.144728 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.144512 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.142073 13647 rpcz_store.cc:276] 0709 12:23:45.528719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.556422 (+ 27703us) service_pool.cc:224] Handling call
0709 12:23:45.572025 (+ 15603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:45.939589 (+367564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.121807 (+1182218us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400095454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:47.122051 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843727104}
I20260709 12:23:47.146983 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.159821 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.162317 13718 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:47.149163 13722 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.163816 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.168349 13647 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:47.150791 19374 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:47.170637 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.172111 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.173270 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.173828 13725 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.175689 19956 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.162992 19874 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:23:47.130604 19925 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:47.192579 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.190536 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.191190 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:47.130568 19924 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.207047 13722 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.212694 18898 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:23:47.220265 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.222067 13724 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:47.221524 13647 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:23:47.229158 19960 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:23:47.230191 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.231160 13645 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:47.162348 19934 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:23:47.246665 13718 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:47.249285 13725 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:47.251235 19868 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:47.254918 19957 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:23:47.256798 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:46.701716 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:47.289011 19906 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.458s	user 0.000s	sys 0.005s
I20260709 12:23:47.292088 13693 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15.
I20260709 12:23:47.317479 13693 leader_election.cc:403] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:47.297668 13793 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:47.243147 19374 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.258272 19420 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:47.300923 13647 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:47.299913 19965 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:23:47.305155 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 139468us
I20260709 12:23:47.300429 19934 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:47.309101 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.311560 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.319634 13647 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:47.327816 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:47.328886 13724 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:47.326287 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:47.335412 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.325032 19893 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15.
I20260709 12:23:47.337234 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.345747 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:47.346601 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:23:47.338781 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46904.4us
I20260709 12:23:47.330928 19932 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.352788 13648 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:47.348367 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.343919 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.340011 13719 raft_consensus.cc:2368] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:47.331111 19845 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:23:47.350369 13643 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:47.350461 19374 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.322903 19965 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.350481 13647 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.437357 13643 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:47.440147 13648 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:23:47.437631 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111497us
I20260709 12:23:47.325958 19934 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.441668 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:47.444715 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:47.445529 13648 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:47.339022 19968 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.462726 13798 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:23:47.462457 19968 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.465687 19968 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:47.466463 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:47.471858 19870 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.004s
I20260709 12:23:47.443639 19893 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:47.473836 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.476806 13647 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:47.482012 13643 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:47.483526 13723 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:23:47.485880 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 137635us
I20260709 12:23:47.486804 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.487524 13648 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:47.496731 13648 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:47.496999 13723 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:47.444669 19850 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:47.499596 19850 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:47.500181 19850 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:47.503696 19846 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.578s	user 0.005s	sys 0.010s
I20260709 12:23:47.504210 19846 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.505290 13800 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:47.505949 19925 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.506740 13798 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:23:47.508715 19845 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:47.509662 19845 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:47.510005 19845 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:47.510854 19934 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.511008 19965 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.510916 19845 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.512303 19934 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:47.512513 19965 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:47.512529 19845 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:47.515496 19842 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:47.515978 19842 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.516887 19842 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.486027 19912 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:47.518801 19912 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.519192 19912 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:47.523373 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:47.523840 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3520) took 1994 ms. Trace:
I20260709 12:23:47.524125 19951 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.498934 13643 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:23:47.537540 13725 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
W20260709 12:23:47.525375 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48937.9us
W20260709 12:23:47.540438 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.537593 13722 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:23:47.498931 13648 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:47.480708 19966 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:47.518121 19842 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:47.547719 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1267) took 1756 ms. Trace:
I20260709 12:23:47.549198 13647 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:23:47.544054 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:47.550868 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.548250 13722 rpcz_store.cc:276] 0709 12:23:45.786174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.786349 (+   175us) service_pool.cc:224] Handling call
0709 12:23:45.800341 (+ 13992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:47.122865 (+1322524us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b54001bdb20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:23:47.541876 (+419011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.542147 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33196288}
I20260709 12:23:47.525239 13646 rpcz_store.cc:276] 0709 12:23:45.529636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.534690 (+  5054us) service_pool.cc:224] Handling call
0709 12:23:45.563599 (+ 28909us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:45.607349 (+ 43750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:45.607387 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:45.655457 (+ 48070us) log.cc:844] Serialized 26 byte log entry
0709 12:23:45.703281 (+ 47824us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:45.738230 (+ 34949us) log.cc:844] Serialized 14 byte log entry
0709 12:23:45.738569 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:45.738653 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:46.679895 (+941242us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b18003f82c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:47.288235 (+608340us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:47.523607 (+235372us) raft_consensus.cc:1685] finished
0709 12:23:47.523633 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.523776 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47781,"prepare.queue_time_us":35505,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":44519,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":20476,"spinlock_wait_cycles":13362176,"thread_start_us":26898,"threads_started":1,"wal-append.queue_time_us":164516}
I20260709 12:23:47.537009 19881 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:23:47.556304 19420 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:23:47.571419 19420 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:47.571820 19420 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.572777 19420 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.573980 19420 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:47.576242 19939 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.116s	user 0.010s	sys 0.000s
W20260709 12:23:47.577989 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:47.595053 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.579883 14424 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:23:47.579977 19902 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.580191 19708 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.602208 19876 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:23:47.602855 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.602694 19708 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.603614 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3532) took 1443 ms. Trace:
W20260709 12:23:47.604342 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.580425 19875 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:47.604821 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.605165 19875 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:47.606006 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.580817 19874 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:47.581539 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:47.581709 14419 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:23:47.582121 18898 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:47.582762 19846 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:47.582886 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 991) took 1089 ms. Trace:
W20260709 12:23:47.587539 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.564152 19870 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:23:47.542857 19966 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:47.556247 19917 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:23:47.558068 19955 raft_consensus.cc:2764] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:23:47.591410 19973 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:23:47.585856 19963 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.485s	user 0.001s	sys 0.000s
I20260709 12:23:47.595541 19609 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:23:47.594606 13544 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 14, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 14 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:47.621644 18898 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.601816 19902 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:47.560472 19907 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 12:23:47.580773 14421 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:47.612521 13647 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 12:23:47.611465 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60739us
I20260709 12:23:47.606562 19940 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:47.614982 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52208.7us
I20260709 12:23:47.616024 19720 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:23:47.616286 19929 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.228s	user 0.003s	sys 0.000s
I20260709 12:23:47.617076 19874 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:47.602694 19876 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.611457 19587 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:47.606036 19961 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.392s	user 0.002s	sys 0.000s
I20260709 12:23:47.613379 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:47.618366 19839 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:23:47.614250 19923 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.002s
I20260709 12:23:47.618873 19870 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:47.618671 19912 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.619321 19966 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:47.578975 14418 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:47.610993 13644 rpcz_store.cc:276] 0709 12:23:46.493616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.646442 (+152826us) service_pool.cc:224] Handling call
0709 12:23:46.646848 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.749218 (+102370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.582628 (+833410us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002e54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:47.582820 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232541824}
I20260709 12:23:47.624374 13722 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:23:47.626811 18898 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:47.626988 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.637005 19803 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:47.636696 13544 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.641966 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:47.656361 14419 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:47.644901 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:47.656200 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:47.604003 13650 rpcz_store.cc:276] 0709 12:23:46.159865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.182429 (+ 22564us) service_pool.cc:224] Handling call
0709 12:23:46.182939 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:46.931792 (+748853us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:46.999132 (+ 67340us) log.cc:844] Serialized 14 byte log entry
0709 12:23:47.016122 (+ 16990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:47.016184 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:47.016234 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.603366 (+587132us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b5400095954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:47.603414 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.603549 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76811,"prepare.queue_time_us":71166,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":424,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":1721636224,"thread_start_us":10203,"threads_started":1,"wal-append.queue_time_us":63520}
I20260709 12:23:47.624985 19931 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:47.630817 19940 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:47.690918 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:47.656852 19868 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:47.705192 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:47.659502 19954 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:47.702108 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57283.1us
I20260709 12:23:47.705708 18898 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:47.641430 19874 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.706791 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1273) took 1712 ms. Trace:
I20260709 12:23:47.713299 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:47.718137 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 91255.3us
I20260709 12:23:47.719043 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.721005 13724 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:47.644713 19846 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:23:47.630200 19606 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.665925 19870 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.669273 19803 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:47.726522 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.708114 13544 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:47.735392 13722 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:47.712630 13726 rpcz_store.cc:276] 0709 12:23:45.994304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:45.994466 (+   162us) service_pool.cc:224] Handling call
0709 12:23:45.994797 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.093268 (+ 98471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:46.093303 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:46.152509 (+ 59206us) log.cc:844] Serialized 26 byte log entry
0709 12:23:46.309694 (+157185us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:23:46.309759 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:46.309865 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.615542 (+1305677us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:47.694820 (+ 79278us) raft_consensus.cc:1685] finished
0709 12:23:47.694854 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.706706 (+ 11852us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":63522,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"spinlock_wait_cycles":23326464,"thread_start_us":207780,"threads_started":2,"wal-append.queue_time_us":168327}
I20260709 12:23:47.735891 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:47.743870 13718 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:47.745710 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:47.658767 19968 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 12:23:47.664235 19884 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:23:47.700482 19708 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.666244 19587 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.743213 13544 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 4 to 10, term changed from 14 to 18, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to <none>, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:47.693779 19924 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:47.749845 13721 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:47.755417 13797 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:47.775163 19874 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.767508 19606 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:47.781201 19902 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.782650 19803 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.784528 19803 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:47.694977 19955 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:47.787036 19955 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.705477 19868 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:47.790524 13720 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:23:47.758481 13650 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:47.761270 13799 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:23:47.791221 19868 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.793119 19868 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:47.794715 19870 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.796211 19875 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:47.717875 19922 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:23:47.758996 19883 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.799312 19883 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.753633 19881 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:47.803488 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:47.767679 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:47.805529 13722 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 9.
W20260709 12:23:47.810040 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52490.5us
I20260709 12:23:47.763471 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:47.753507 19932 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:47.727320 19876 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:47.775233 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 239) took 32836 ms (client timeout 15000 ms). Trace:
I20260709 12:23:47.819335 19876 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:47.767752 19903 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:47.767598 13544 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 17, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 17 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.768508 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.768417 19954 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:47.811395 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48052.5us
W20260709 12:23:47.823303 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54898.2us
I20260709 12:23:47.824288 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:47.735694 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:47.801957 19883 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:47.804333 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:47.810400 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:47.828227 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 997) took 1122 ms. Trace:
I20260709 12:23:47.777906 19874 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:47.829123 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93595.9us
I20260709 12:23:47.833003 13544 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 13, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 13 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.836021 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:47.759954 19968 consensus_queue.cc:1243] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:23:47.841804 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:47.841514 13544 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.822098 19951 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:47.843581 13544 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:47.842499 13794 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:47.831305 19863 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:47.827725 19932 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:47.837966 19965 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:47.863080 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.830935 19708 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:47.829713 19606 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:47.831024 13652 rpcz_store.cc:276] 0709 12:23:46.706231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.953292 (+247061us) service_pool.cc:224] Handling call
0709 12:23:46.953717 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.954071 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:46.954099 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:46.981778 (+ 27679us) log.cc:844] Serialized 26 byte log entry
0709 12:23:47.019177 (+ 37399us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:47.038724 (+ 19547us) log.cc:844] Serialized 14 byte log entry
0709 12:23:47.038969 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.039054 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.605385 (+566331us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:47.803884 (+198499us) raft_consensus.cc:1685] finished
0709 12:23:47.828014 (+ 24130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.828163 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5770,"prepare.queue_time_us":19215,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":26937,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":4523,"spinlock_wait_cycles":3642240,"thread_start_us":12087,"threads_started":1,"wal-append.queue_time_us":150657}
I20260709 12:23:47.877071 19966 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.878902 19881 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.879989 19908 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:47.880565 19875 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:47.881126 19374 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:47.819633 13542 rpcz_store.cc:269] 0709 12:23:14.938436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.962197 (+ 23761us) service_pool.cc:224] Handling call
0709 12:23:47.774937 (+32812740us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:39.162602 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:39.167374 (+  4772us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:39.167433 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:23:39.167451 (+    18us) tablet.cc:662] Decoding operations
0709 12:23:39.278458 (+111007us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:39.289788 (+ 11330us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:39.289821 (+    33us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:23:39.311184 (+ 21363us) tablet.cc:701] Row locks acquired
0709 12:23:39.311236 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:23:39.312302 (+  1066us) write_op.cc:270] Start()
0709 12:23:39.312494 (+   192us) write_op.cc:276] Timestamp: P: 1783599819311363 usec, L: 0
0709 12:23:39.312537 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:23:39.417843 (+105306us) log.cc:844] Serialized 4345 byte log entry
0709 12:23:43.156734 (+3738891us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:43.299213 (+142479us) op_driver.cc:464] REPLICATION: finished
0709 12:23:43.458551 (+159338us) write_op.cc:301] APPLY: starting
0709 12:23:43.458704 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:44.026281 (+567577us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:44.048460 (+ 22179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:44.120731 (+ 72271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:44.121102 (+   371us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:23:44.121165 (+    63us) write_op.cc:312] APPLY: finished
0709 12:23:44.121876 (+   711us) log.cc:844] Serialized 105 byte log entry
0709 12:23:44.122171 (+   295us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:44.166589 (+ 44418us) write_op.cc:454] Released schema lock
0709 12:23:44.166672 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:44.167252 (+   580us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:44.167289 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":65658,"reported_tablets":50,"spinlock_wait_cycles":71040,"child_traces":[["op",{"apply.queue_time_us":133552,"apply.run_cpu_time_us":26009,"apply.run_wall_time_us":708726,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50195,"num_ops":11,"prepare.queue_time_us":244944,"prepare.run_cpu_time_us":28638,"prepare.run_wall_time_us":574412,"raft.queue_time_us":136509,"raft.run_cpu_time_us":10317,"raft.run_wall_time_us":62835,"replication_time_us":3986390,"spinlock_wait_cycles":7842176,"thread_start_us":412637,"threads_started":4,"wal-append.queue_time_us":509985,"wal-append.run_cpu_time_us":58186,"wal-append.run_wall_time_us":5170577}]]}
I20260709 12:23:47.860884 13544 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.833171 19967 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:47.906526 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43568.2us
I20260709 12:23:47.832960 19924 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:23:47.862303 19955 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:47.864650 19863 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:47.829587 19903 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.913522 19925 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:47.926129 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.901638 19965 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:47.907800 19923 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:47.958444 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.958312 19965 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:47.960067 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:47.960255 19965 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:47.960992 19932 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:47.961488 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:47.909021 19606 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:47.991498 19606 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:23:47.992478 19974 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:23:47.964600 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3526) took 1811 ms. Trace:
I20260709 12:23:47.927477 19374 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:47.973899 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:47.999305 19966 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.000344 19374 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:48.001257 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:47.996206 13649 rpcz_store.cc:276] 0709 12:23:46.153532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:46.153701 (+   169us) service_pool.cc:224] Handling call
0709 12:23:46.164812 (+ 11111us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:46.184868 (+ 20056us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:23:47.264275 (+1079407us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b18000384a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:23:47.442145 (+177870us) log.cc:844] Serialized 14 byte log entry
0709 12:23:47.861351 (+419206us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:47.862010 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 12:23:47.862206 (+   196us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:47.862281 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:47.862488 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:47.964203 (+101715us) raft_consensus.cc:1685] finished
0709 12:23:47.964321 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:47.964513 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91247,"prepare.queue_time_us":148548,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":2573,"raft.queue_time_us":29152,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":33360,"spinlock_wait_cycles":6865280,"thread_start_us":505011,"threads_started":2,"wal-append.queue_time_us":455310}
I20260709 12:23:47.976173 19909 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:47.962306 19932 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:47.972841 19881 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.003598 19903 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:47.936708 19967 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:47.972540 13544 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 11, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 11 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.918469 19875 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:48.002038 14421 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:47.970046 14416 pending_rounds.cc:85] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 15
I20260709 12:23:48.004902 14416 pending_rounds.cc:107] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:23:48.005229 14416 raft_consensus.cc:2889] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:47.938974 19863 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:23:48.020125 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 279624us
I20260709 12:23:48.000903 19954 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:48.042397 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.043048 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81663.1us
I20260709 12:23:48.042488 13544 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 10, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 10 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.043956 13693 leader_election.cc:400] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:48.045392 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.050872 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:48.052839 18898 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:48.053584 18898 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:48.052759 13544 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:23:48.054034 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 216049us
I20260709 12:23:48.058416 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:48.019747 19925 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:23:48.041791 19850 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
W20260709 12:23:48.061206 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:48.070391 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.066187 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:48.066887 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.067489 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:48.067489 13544 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 8, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 8 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.073643 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:48.077206 13649 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.074018 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:48.079051 13652 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:23:48.079802 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.081929 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:48.082595 13643 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.085939 13651 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:23:48.073760 18898 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.093453 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.053123 19954 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:48.099521 13544 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.100317 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.103287 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:48.106626 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:48.107733 13646 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 12: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:48.088984 19932 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.114409 13651 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:23:48.118124 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:48.117887 19932 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:48.115307 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49205.3us
I20260709 12:23:48.120087 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.102692 19907 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:48.123586 13645 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:48.010973 19875 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.125190 19875 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:48.126067 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:48.127449 19420 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:23:48.092545 19715 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:48.116725 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.130635 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.132444 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.128167 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.133708 19587 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:48.134812 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.136018 19907 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.077472 19850 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.137292 19907 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:48.138012 19940 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.138751 19850 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.139819 19868 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:48.140053 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.140441 19868 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.139839 13647 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:23:48.142390 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:48.144835 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.145216 19839 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:48.145568 13792 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:48.145771 19839 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:48.117983 13544 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 16, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 16 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:48.149691 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.150522 13649 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:48.119045 13650 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:48.161698 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.163101 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.123682 19876 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:48.166927 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.167707 13793 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.170466 13726 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:48.171677 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:48.172175 13719 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:48.172842 19903 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:48.173280 19903 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.173619 19903 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.175652 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:48.176012 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.177441 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:48.179420 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:48.182351 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.077788 19934 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 12:23:48.138059 19907 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.139359 19940 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:48.185117 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.189800 13721 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:48.186381 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:48.186764 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.096014 19865 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:23:48.180382 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.192929 13793 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:48.196930 13800 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:48.202368 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.202725 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.191110 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:48.225852 13643 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:48.226835 13794 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:48.180351 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:48.186103 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:48.235419 13544 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.285449 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.285465 13720 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.202019 19715 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:48.239109 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52453us
I20260709 12:23:48.290982 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.292101 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.189354 19883 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:48.166129 19932 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:48.272073 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.185691 19720 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:48.270457 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.241545 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:48.269394 19868 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:48.313740 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72278.2us
I20260709 12:23:48.229049 19606 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.315757 14421 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:48.315949 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:23:48.270720 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:48.280072 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 204846us
I20260709 12:23:48.280764 13797 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:48.231693 13723 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:23:48.242475 19609 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:48.268355 19374 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:48.298640 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:48.325392 13794 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:48.233515 19876 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:23:48.302471 19420 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:23:48.309880 13652 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:48.233711 19973 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:23:48.317747 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.276109 19902 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.359278 19902 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.361651 19902 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:48.314047 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:48.242938 14418 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:48.362584 19609 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:23:48.362838 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48882.8us
I20260709 12:23:48.363895 13617 leader_election.cc:400] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:48.364240 13617 leader_election.cc:403] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:23:48.364813 19720 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:48.365286 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:48.323891 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:48.366094 19720 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:48.320537 19850 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:48.324070 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:48.307559 19865 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.278210 19984 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:48.334134 13793 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:23:48.379807 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55994.3us
I20260709 12:23:48.379235 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:48.351003 19987 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:23:48.351146 19606 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:48.148535 19968 consensus_queue.cc:1175] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 10 status { last_received { term: 9 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:48.362974 19609 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:48.328145 18898 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:23:48.328231 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57878.1us
I20260709 12:23:48.369429 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:48.370703 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:48.370609 13544 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 16, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 16 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.373394 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:48.368521 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:48.381974 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:48.339632 19980 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:23:48.317241 19715 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:48.325891 19745 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:48.333561 19965 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.276058 19839 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.406656 13652 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:48.407164 14416 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:23:48.405088 19609 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:48.407074 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:48.446010 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:48.447286 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.445822 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:48.415093 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.378837 19967 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.336910 19925 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:23:48.450639 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.451272 18898 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:23:48.390461 19955 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:48.390666 19924 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:48.333917 19932 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:23:48.481006 13544 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 13, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 13 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.400681 13795 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:48.410063 19991 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.390651 19951 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:48.503280 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 176858us
I20260709 12:23:48.409550 19850 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:48.506735 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56180.2us
W20260709 12:23:48.396611 19929 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:23:48.526059 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.509943 13544 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.527714 18898 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.447775 19606 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:48.393838 19884 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:48.409446 19874 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:23:48.449901 19907 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:48.454087 19883 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:48.534638 18898 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:23:48.457374 19868 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:48.515543 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.537075 13544 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:23:48.542491 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163365us
I20260709 12:23:48.540164 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:48.545674 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138839us
W20260709 12:23:48.546922 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93566.4us
I20260709 12:23:48.447099 19965 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.537377 19865 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:48.531286 19932 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:48.455511 19912 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:48.545712 19803 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.761s	user 0.012s	sys 0.003s
I20260709 12:23:48.410180 19871 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.142s
I20260709 12:23:48.555820 19606 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:48.556001 20007 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:48.457893 19745 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 12:23:48.559762 18898 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.559567 19850 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.508594 19924 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.550143 19884 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:48.557322 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:48.573307 19955 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:48.530644 19991 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.584030 18898 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.585911 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:48.589167 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.589679 14416 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:23:48.393294 19984 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.565361 19069 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:48.598243 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.615450 13794 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:23:48.586838 13544 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.596678 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:48.612854 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55598.3us
I20260709 12:23:48.622234 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.624058 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:48.627574 18898 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:48.629966 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:48.632764 13793 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:23:48.632053 13544 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.634590 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.637267 19803 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:23:48.615701 19868 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:48.639498 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:48.541014 19951 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.641786 13723 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:48.561902 19932 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.644555 13544 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 12, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:23:48.647598 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 314239us
I20260709 12:23:48.643293 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:48.614326 19069 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.655081 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:48.582554 19606 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.661026 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:48.640031 19883 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:48.662587 18898 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:48.668120 18898 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.621414 20005 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:48.663591 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 88373.3us
I20260709 12:23:48.640352 19967 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:48.565392 19715 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.614626 19988 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:23:48.653529 19803 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:48.687592 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:48.669588 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.677811 13720 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:48.683519 13544 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 19, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 19 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:23:48.652956 19865 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:48.665716 19069 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.665412 19871 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.686887 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:48.664757 19925 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:48.613831 20004 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:23:48.695824 13544 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 6 to 8, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 8 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.672024 20005 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:48.620971 20006 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 12:23:48.701411 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:48.675343 19967 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:48.709699 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48792.1us
I20260709 12:23:48.684665 20009 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:23:48.704344 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:48.705364 13544 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:23:48.703747 18898 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.701622 19966 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.682675 19881 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:23:48.714792 19883 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.715063 19966 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.719496 20012 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.720134 20012 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.721230 19966 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:48.722752 20000 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:23:48.723158 19865 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.724435 19983 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.725021 19983 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.729722 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.736176 20012 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.736925 19991 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:48.701719 20005 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:48.710619 19967 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:48.751560 14416 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:23:48.759860 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106307us
I20260709 12:23:48.727447 19983 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:48.716008 20013 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:48.702827 19871 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.687187 19955 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.683640 19884 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:48.775350 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 110751us
I20260709 12:23:48.757560 19924 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:48.795266 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:48.766600 19965 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.801301 14424 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:48.789055 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:48.713047 19868 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.758138 19803 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:48.802789 13720 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:48.789891 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:48.802834 13797 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:48.813482 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:48.807557 13723 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:23:48.835933 19990 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
W20260709 12:23:48.837955 19999 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.200s	user 0.001s	sys 0.001s
I20260709 12:23:48.839375 20020 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:48.840022 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3476) took 1092 ms. Trace:
I20260709 12:23:48.840431 20020 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:48.840577 13798 rpcz_store.cc:276] 0709 12:23:47.747577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.751495 (+  3918us) service_pool.cc:224] Handling call
0709 12:23:47.796941 (+ 45446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:47.846005 (+ 49064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:47.846058 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:47.858573 (+ 12515us) log.cc:844] Serialized 26 byte log entry
0709 12:23:48.240540 (+381967us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:48.240607 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.240760 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:48.839772 (+599012us) raft_consensus.cc:1685] finished
0709 12:23:48.839797 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:48.839947 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48415,"raft.queue_time_us":82906,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":316323,"threads_started":1,"wal-append.queue_time_us":557098}
I20260709 12:23:48.844223 20008 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:48.852936 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:48.854036 14416 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:48.866755 19745 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.885075 19924 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:48.851868 20013 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:48.877000 19951 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:48.875981 19984 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:48.873095 19932 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:48.887212 19909 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.876s	user 0.011s	sys 0.004s
I20260709 12:23:48.856416 19925 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:23:48.914431 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 184800us
I20260709 12:23:48.844861 20008 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.891757 19606 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:48.895603 19924 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.950488 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:48.954450 19924 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:48.962450 20011 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.963474 20011 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.967188 20022 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:48.967963 20022 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.969190 19870 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:48.962616 19908 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:48.886469 19745 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:48.986868 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:49.008294 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 296857us
I20260709 12:23:48.946135 19909 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:48.966092 20011 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:48.984586 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:49.041790 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57341.6us
I20260709 12:23:49.043058 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:49.036584 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:48.985546 19924 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:48.988210 19908 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:49.053570 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 286737us
I20260709 12:23:49.060458 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:49.051291 19954 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.936s	user 0.010s	sys 0.005s
I20260709 12:23:49.052314 19982 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:49.044698 19865 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:48.916316 20013 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.052958 20011 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.069720 19924 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:49.099390 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.070477 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:49.104256 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.112063 13650 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:23:49.101274 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.098570 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:49.075314 19609 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.004s
I20260709 12:23:49.110998 13719 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:49.117653 13649 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:23:49.113401 13727 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:49.067453 20022 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:49.115497 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45114us
I20260709 12:23:49.119259 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:49.077991 19954 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:49.123205 13649 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:23:49.008893 19803 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.119817 19865 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.129292 20011 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:49.131618 20031 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:23:49.132253 19868 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.132778 19868 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.123867 19609 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.074232 19759 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:23:49.142143 19745 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.143018 19745 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:49.124221 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:49.143553 19745 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.099490 19909 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.146478 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2364) took 1062 ms. Trace:
I20260709 12:23:49.101259 20009 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:49.146952 13647 rpcz_store.cc:276] 0709 12:23:48.083860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.083978 (+   118us) service_pool.cc:224] Handling call
0709 12:23:48.104351 (+ 20373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:48.142855 (+ 38504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:49.146180 (+1003325us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b54001b6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:49.146416 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80941952}
I20260709 12:23:49.147993 20009 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.149981 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1214) took 1340 ms. Trace:
I20260709 12:23:49.150455 14419 rpcz_store.cc:276] 0709 12:23:47.809316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.818919 (+  9603us) service_pool.cc:224] Handling call
0709 12:23:47.826012 (+  7093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:47.830094 (+  4082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:47.830131 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:47.853422 (+ 23291us) log.cc:844] Serialized 26 byte log entry
0709 12:23:48.065603 (+212181us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:48.065648 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:48.065730 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:48.357001 (+291271us) raft_consensus.cc:1685] finished
0709 12:23:48.357034 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:49.149680 (+792646us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b5400644bd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:49.149916 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1631,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3128,"spinlock_wait_cycles":15242112,"thread_start_us":198836,"threads_started":1,"wal-append.queue_time_us":440050}
I20260709 12:23:49.153049 20023 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:49.146943 19909 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:49.156116 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.157238 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:49.155303 20010 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.157528 18898 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.158006 20010 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.158679 19909 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.153224 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2374) took 1024 ms. Trace:
I20260709 12:23:49.159582 18898 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.168476 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.168574 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.096915 19883 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.107764 19982 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.172384 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.169708 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:49.170404 13800 rpcz_store.cc:276] 0709 12:23:48.128930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.134474 (+  5544us) service_pool.cc:224] Handling call
0709 12:23:49.152906 (+1018432us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:49.153105 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140461440}
I20260709 12:23:49.176448 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:49.158607 20017 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:23:49.177222 13649 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:23:49.211644 13718 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
W20260709 12:23:49.228082 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61488.9us
I20260709 12:23:49.182710 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:49.231832 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 171496us
I20260709 12:23:49.124207 19876 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:49.247128 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.248348 13647 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:49.227576 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.158533 19931 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:49.219630 18898 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:49.256960 13651 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:49.259747 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.253903 13646 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:49.167415 19846 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.265295 19846 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.255056 19909 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:23:49.266844 20011 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.269256 19846 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.271780 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.263029 20008 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.263199 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:49.272993 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:49.157583 20019 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.264468 19745 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.072007 19925 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.278134 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.279575 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.214123 19842 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:23:49.280400 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.263975 13725 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:49.272749 13726 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:49.283871 13800 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:49.284605 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:49.420023 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135511us
I20260709 12:23:49.285470 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:49.295637 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.340615 14424 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:49.247604 19982 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:49.410547 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.411140 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.373805 13651 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:49.426371 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.249419 20035 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.247354 19876 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.412321 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:49.436038 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 150650us
I20260709 12:23:49.436882 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:49.257696 19931 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:49.412182 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 984) took 1770 ms. Trace:
I20260709 12:23:49.428504 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.443663 13651 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:49.421389 14418 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:49.291767 19954 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:49.434235 13797 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:49.442752 13796 rpcz_store.cc:276] 0709 12:23:47.641527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:47.641669 (+   142us) service_pool.cc:224] Handling call
0709 12:23:49.411969 (+1770300us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:49.412114 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093877504}
I20260709 12:23:49.446434 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:49.451623 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:49.450783 19759 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:49.456422 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.458474 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:49.423578 19940 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.186s	user 0.023s	sys 0.000s
I20260709 12:23:49.460529 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.463528 13651 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:49.284296 19803 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.150s
I20260709 12:23:49.212124 19883 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.462489 13726 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:49.467640 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:49.422906 19745 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.435182 19865 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.479508 19745 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.480127 20010 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.412762 19842 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:49.497103 13645 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:49.440948 19876 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:49.496714 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.432488 19884 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.497138 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:49.505348 13796 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
W20260709 12:23:49.494973 19069 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.792s	user 0.016s	sys 0.001s
I20260709 12:23:49.498050 13650 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:49.500517 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.508121 13646 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:49.509187 19069 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.444428 19984 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:49.460219 19587 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:49.512535 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:49.340546 20019 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.481297 19923 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:49.461068 20013 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.518172 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:49.467674 19954 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.468688 20034 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:49.530180 20030 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:49.531996 19883 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.519377 19587 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.530277 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.536629 13793 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:49.539894 19069 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.517180 19984 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.445557 20035 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:49.556560 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:49.522972 19923 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:49.542773 20019 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.538436 20034 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:49.504591 19884 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.568629 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:49.576831 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 137330us
I20260709 12:23:49.577979 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.550450 19984 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:49.572016 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:49.563889 19863 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.517s	user 0.015s	sys 0.004s
I20260709 12:23:49.548185 19587 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:49.552502 19909 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:49.567170 19903 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.393s	user 0.013s	sys 0.008s
I20260709 12:23:49.570847 19923 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.650511 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:49.651316 13726 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:23:49.654897 13644 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.005s
I20260709 12:23:49.655535 13644 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:49.576476 20034 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:49.587071 20016 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:49.661170 20039 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.539567 19931 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.643136 19803 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:49.548291 20030 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.654191 20021 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.657536 19745 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.658347 20024 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:49.569579 19842 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.645972 19909 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.646154 19903 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.642294 19863 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.666138 20016 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:49.667369 19924 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:49.714232 20001 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.179s	user 0.000s	sys 0.010s
W20260709 12:23:49.734596 20036 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.312s	user 0.003s	sys 0.001s
I20260709 12:23:49.738556 19924 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:49.739404 20047 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:23:49.741343 19863 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.747154 19903 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.747848 13652 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:23:49.760429 19985 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:49.767729 19954 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.661682 20015 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:49.734305 19908 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.010s
I20260709 12:23:49.769748 20016 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:49.698469 19803 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.671452 20039 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.789387 19876 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:49.813692 19967 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.046s	user 0.017s	sys 0.001s
I20260709 12:23:49.814275 19967 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.814826 19876 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:49.790177 19985 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:49.720289 20021 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.726747 20024 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.850561 14418 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:23:49.852025 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.853212 14424 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:49.830214 19967 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.809475 19908 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.834674 19924 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.834781 20039 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.811714 20015 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:49.866937 19907 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.306s	user 0.008s	sys 0.010s
I20260709 12:23:49.875379 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:49.880323 13720 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
W20260709 12:23:49.882073 20064 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:23:49.884025 19981 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:49.884486 19981 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:49.885394 19981 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.866218 20044 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:49.886433 19981 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:49.886691 20044 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:49.887468 20055 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:23:49.889190 20044 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:49.873083 20005 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.105s	user 0.011s	sys 0.004s
I20260709 12:23:49.881599 19985 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.891181 13643 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:49.890024 20005 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.866581 19954 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:49.894984 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:49.869376 19803 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.870415 19884 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:49.845355 20034 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:49.879323 19907 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.325s	user 0.008s	sys 0.010s
I20260709 12:23:49.906111 19985 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:49.878413 20035 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.909035 20034 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:49.909564 19870 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.940s	user 0.007s	sys 0.011s
I20260709 12:23:49.931059 13800 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:49.931708 19984 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.933633 18898 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.934161 18898 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.935360 19907 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:49.938225 19907 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.879364 19967 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:23:49.943043 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:49.910768 20030 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.944435 20030 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:49.944765 20024 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.945036 20030 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.946538 19909 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.947031 19931 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:49.947263 20030 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.912519 20015 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:49.909893 20051 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 12:23:49.910719 13795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2379) took 1743 ms (client timeout 1631 ms). Trace:
I20260709 12:23:49.913908 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:49.961418 14424 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:49.914743 19839 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:49.915522 20070 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:49.960587 13795 rpcz_store.cc:269] 0709 12:23:48.166934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.167062 (+   128us) service_pool.cc:224] Handling call
0709 12:23:49.910517 (+1743455us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:49.910635 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272667520}
I20260709 12:23:49.898779 19954 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:49.939074 19908 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:49.940351 18898 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:49.943904 14419 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:49.912810 20005 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:49.982466 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:49.988956 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:23:49.984375 13800 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:49.959971 20053 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.227s	user 0.001s	sys 0.001s
I20260709 12:23:49.983601 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:50.000821 13646 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:49.952620 19923 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:49.934100 19984 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.009194 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:50.010464 13796 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:50.024797 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:50.026854 13795 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:50.045126 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:50.044368 19982 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.002s
I20260709 12:23:50.055819 19923 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.058636 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:50.044122 19908 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.065387 19908 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.069013 19982 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.069852 19883 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.070477 19883 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.049029 19909 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:50.071502 19982 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.072023 19909 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.073514 19759 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:50.077366 13795 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:50.079911 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:50.080667 14424 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:50.084929 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:49.979830 20070 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.090612 19908 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.091893 20070 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.093852 13792 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:50.095371 19909 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:50.096369 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2376) took 1942 ms (client timeout 2172 ms). Trace:
W20260709 12:23:50.096841 13792 rpcz_store.cc:269] 0709 12:23:48.154139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.164407 (+ 10268us) service_pool.cc:224] Handling call
0709 12:23:50.096193 (+1931786us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54000d16d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:50.096308 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4279298304}
I20260709 12:23:50.081789 19759 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.099417 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:50.101675 19759 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.084131 19587 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.086957 20016 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.116703 14419 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:50.117591 14418 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:23:50.118150 19587 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.119040 14421 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:50.119750 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.120973 19931 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:50.121021 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:50.121762 19931 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:50.121949 13645 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:50.122839 19931 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.127317 20016 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.120930 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.130460 13649 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:50.129222 20057 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.401s	user 0.000s	sys 0.004s
I20260709 12:23:50.137104 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.139902 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.140547 13725 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:50.098245 13646 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:50.142071 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.133186 19871 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.145414 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.142793 19839 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:50.146489 19839 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.149005 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.150091 13648 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:50.153779 13648 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:50.155706 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:50.156378 13649 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:50.158425 19839 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.154795 19965 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:50.159492 19965 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.161705 19965 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.155026 19842 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.162194 13645 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:50.162953 19842 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.165750 19842 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.151787 19968 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:50.168135 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.168071 19968 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:50.169057 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.170747 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.173472 18898 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:23:50.175067 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.175314 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.184886 13644 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:50.186112 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.186825 13648 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:50.183144 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.188951 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.193674 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:50.194418 13799 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:50.194520 13648 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:50.194939 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.187469 20003 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:50.176473 19909 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:50.198848 19909 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.199272 19909 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:50.199689 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1294) took 1103 ms. Trace:
I20260709 12:23:50.200327 13718 rpcz_store.cc:276] 0709 12:23:49.096207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.098326 (+  2119us) service_pool.cc:224] Handling call
0709 12:23:50.199344 (+1101018us) spinlock_profiling.cc:243] Waited 97 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:50.199569 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223088000}
I20260709 12:23:50.202739 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.184486 20028 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:23:50.204252 20028 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:50.204583 20028 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:50.205760 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2371) took 1586 ms. Trace:
I20260709 12:23:50.206704 13652 rpcz_store.cc:276] 0709 12:23:48.618955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:48.619192 (+   237us) service_pool.cc:224] Handling call
0709 12:23:48.642928 (+ 23736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:49.787551 (+1144623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.205374 (+417823us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400198054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:50.205670 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2540286208}
I20260709 12:23:50.208655 19587 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:50.209270 19587 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:50.210278 19587 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.211405 19923 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.211957 19923 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.212690 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:50.213852 20003 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:50.214887 20003 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.216225 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:50.212942 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:50.218696 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.183732 19968 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.219725 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:50.220656 14419 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:50.222476 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.223528 13799 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:23:50.205487 20028 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.229979 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:50.225961 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.227005 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.227741 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.230890 14419 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:50.231326 13725 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:50.217872 19923 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.235042 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.235682 13721 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:23:50.237133 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.238880 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:50.239554 13652 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:50.222888 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:50.241665 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.242491 13648 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:50.251885 13725 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:50.253777 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.256400 13727 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:50.260740 14424 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:50.261422 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:50.262758 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:50.263480 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.265777 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:50.269052 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.262778 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.255396 20011 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.280083 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.282680 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.218008 19335 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:50.284426 18898 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.288045 18898 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.289264 13725 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:50.290488 19876 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.290665 19923 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:50.260444 19984 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:50.291656 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:50.292325 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:50.329704 13645 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:50.296219 13726 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:50.296633 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.300716 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:50.302122 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.303504 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.332698 13720 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:50.303907 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.236502 19745 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:23:50.229188 19374 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:50.303907 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.322314 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.350028 13721 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:50.234393 19982 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.229480 20019 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:23:50.324817 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:50.355909 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 94629.6us
I20260709 12:23:50.326131 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.356778 13725 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:50.308555 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.243805 20005 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:50.243676 19908 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.291232 19940 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:23:50.333537 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:50.337893 18898 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.299396 19965 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:50.347465 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.348070 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:50.361629 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53199.3us
W20260709 12:23:50.352494 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 177707us
W20260709 12:23:50.353176 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130359us
I20260709 12:23:50.397384 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.397627 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.230079 20009 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:50.403241 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.404295 13792 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:50.354331 13798 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:50.304565 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.359745 13652 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:50.284372 20012 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.400053 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.395780 13801 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:50.300215 20011 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.323544 19985 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.412985 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.401718 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.360268 19981 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:50.359103 19374 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:50.395881 19903 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.395614 19884 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:50.439682 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.451759 13798 raft_consensus.cc:2368] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:23:50.360868 20066 raft_consensus.cc:2764] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:23:50.449255 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.394806 20030 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:50.355661 19984 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:50.432044 19981 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.411302 20034 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.411413 19839 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:23:50.402912 19965 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.345553 19923 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.471246 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.472925 13798 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:50.474967 13798 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:50.492568 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.787s (SENT) [suppressed 16 similar messages]
I20260709 12:23:50.360355 20005 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.468034 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.497186 13719 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:50.329466 19876 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.448590 19920 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:50.450769 19884 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.360360 19982 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.465843 19720 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:23:50.415344 19908 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.521915 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.591943 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.593024 13722 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:50.596441 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:50.599953 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:50.525458 20005 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:50.477880 20034 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.607051 13727 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:50.505539 13721 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:50.422597 20012 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.422061 20009 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.540305 19923 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.543865 19965 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:50.544934 20085 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:23:50.468115 19981 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:50.552507 20011 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.453065 19903 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.636263 13795 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:50.433787 19985 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.560727 19884 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:50.565114 19984 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:50.517750 20076 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.324s	user 0.001s	sys 0.000s
I20260709 12:23:50.468050 20070 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:50.608327 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 208772us
I20260709 12:23:50.609555 13719 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:23:50.567781 19609 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.615027 19374 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.616180 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.618904 13727 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:50.626560 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.660609 13718 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:23:50.596024 19876 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.660471 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:50.632400 19982 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.634056 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.662518 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:50.664681 13798 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:50.666059 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:50.668228 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.669509 13798 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:23:50.673134 19374 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.680160 20021 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.685598 20028 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:50.688056 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3563) took 1583 ms. Trace:
I20260709 12:23:50.688627 13643 rpcz_store.cc:276] 0709 12:23:49.104464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:49.104640 (+   176us) service_pool.cc:224] Handling call
0709 12:23:49.104900 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:49.894481 (+789581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:50.687699 (+793218us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b54001b6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:50.687979 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807805696}
I20260709 12:23:50.655629 19925 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:50.690747 19925 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.696019 19903 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.665207 20070 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:50.703841 20034 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.704200 19954 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.705081 19954 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:50.715103 19883 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.642918 13722 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:50.631680 19909 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:50.706946 19908 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.732566 19985 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:50.734717 18898 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:50.734776 19920 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:50.736454 19909 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.736317 19920 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:50.737708 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 134896us
W20260709 12:23:50.739033 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:50.739574 20084 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.280s	user 0.003s	sys 0.000s
I20260709 12:23:50.736340 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.744194 13721 raft_consensus.cc:2368] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20.
I20260709 12:23:50.748196 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.748484 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:50.749058 13719 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:50.750931 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.747624 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.681994 20009 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.675383 19720 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:50.694116 19925 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:50.774696 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.776329 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.777683 13618 leader_election.cc:400] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:23:50.777938 13618 leader_election.cc:403] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:50.672235 20011 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:50.763043 19720 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.757062 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.787052 13798 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:50.783612 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:50.785182 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:50.791086 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:50.787055 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:50.793789 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:50.756165 19923 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:50.782326 19908 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:50.804232 18898 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:23:50.785127 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:50.755347 19982 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:50.795352 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:50.814222 13793 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:50.804512 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:50.818657 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:50.795043 19931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.012s	sys 0.004s
I20260709 12:23:50.788009 20011 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.790211 19720 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:50.827940 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 335417us
I20260709 12:23:50.789664 19868 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:50.790926 20013 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:50.801760 19883 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:50.825948 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:50.792755 20093 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:23:50.802724 19923 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:50.860960 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:50.812225 19609 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:23:50.861879 19881 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:50.815482 20034 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:23:50.831410 19931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.012s	sys 0.004s
I20260709 12:23:50.864354 19374 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:50.870649 13726 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:50.837868 19968 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:50.865845 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:50.880223 13799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.007s	sys 0.020s
I20260709 12:23:50.811511 19745 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:23:50.811916 20010 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:23:50.838865 20011 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:50.940209 19374 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:50.841151 19876 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:23:50.987726 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.017s	sys 0.034s
W20260709 12:23:51.018348 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.017s	sys 0.034s
I20260709 12:23:50.965106 19881 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.840972 19940 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:50.860356 19908 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:51.010480 13799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.009s	sys 0.024s
W20260709 12:23:51.010532 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144762us
I20260709 12:23:50.855288 19883 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:50.827039 19982 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.022063 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:51.051263 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:51.056851 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 378171us
I20260709 12:23:51.018795 20023 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:51.035705 19881 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:51.016957 19374 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.034049 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.013653 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:51.065544 14421 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:50.971875 20034 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.116696 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:50.988705 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108161us
I20260709 12:23:51.118814 18898 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:51.027024 19876 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.027684 20012 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.046204 19745 consensus_queue.cc:1243] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:23:51.131467 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 702793us
I20260709 12:23:51.049232 19883 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:51.036657 19925 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.034940 19940 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.133226 18898 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.045250 20009 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.123476 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.117674 19868 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.162519 13792 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:51.114283 20030 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.149890 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.148093 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:51.135241 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 101309us
I20260709 12:23:51.149296 13650 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:51.154273 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.153520 13798 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:23:51.123906 20023 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.168514 13800 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:51.131171 20095 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.193688 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.121778 20070 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.151523 19973 raft_consensus.cc:2764] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:23:51.156862 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.172197 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.206676 18898 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:51.203006 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.210855 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:51.219686 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5799507599999743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 12:23:51.192236 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.141584 19903 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.138873 20034 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:51.159512 20012 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:51.209745 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44195.1us
I20260709 12:23:51.138043 19876 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:51.049045 19908 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.163288 19940 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:51.172156 20023 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:51.043818 19968 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:51.194845 20096 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:23:51.229442 18898 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:51.230418 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:51.122495 19923 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.171227 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.162899 19745 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:51.227749 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.168140 19909 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:51.255872 19909 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.240316 13800 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:23:51.228858 20095 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.258488 20100 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 12:23:51.259258 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.259941 20100 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:51.260269 20095 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.260394 20100 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:51.261364 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.262115 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:51.263062 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:51.241675 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.242518 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.266506 19069 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:51.266603 13722 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:51.267720 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.268332 13722 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:51.197278 19982 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.244010 13725 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:51.172600 20009 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.271112 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.163194 19925 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:51.230403 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73656.6us
I20260709 12:23:51.246834 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.250387 13721 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:23:51.254889 20104 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:51.254786 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.253149 19745 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.285408 19745 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:51.208101 20030 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.286477 20039 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:51.287050 20039 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:51.288235 20030 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.271765 19908 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.288604 20039 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.241158 19973 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:51.271785 13725 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:51.176025 19868 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.260509 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.283051 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.293641 19868 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.283846 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.295583 13721 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:51.284242 19925 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.284427 20104 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.235121 19903 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.258147 19909 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.296399 19923 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:51.295523 19968 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.306573 14421 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:23:51.309527 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 295972us
I20260709 12:23:51.292728 13725 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:51.330727 13797 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:51.338522 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1011) took 1109 ms. Trace:
W20260709 12:23:51.323109 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44505.7us
W20260709 12:23:51.326318 20092 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.464s	user 0.000s	sys 0.001s
W20260709 12:23:51.297618 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 181032us
I20260709 12:23:51.300407 20039 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:51.291673 20011 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.261687 20100 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.340158 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.360628 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:51.343266 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.362653 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.345758 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.363296 13800 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:51.346293 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:51.348650 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.365366 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.353042 13650 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:23:51.357481 13648 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:51.366319 13650 pending_rounds.cc:85] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 8
I20260709 12:23:51.366734 13650 pending_rounds.cc:107] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:23:51.367077 13650 raft_consensus.cc:2889] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:51.340852 19967 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:51.356563 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.344621 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:51.370653 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.368901 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:51.371392 14421 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:51.369407 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.369542 13794 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:51.351933 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.373047 13797 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:51.373562 13649 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:23:51.374302 13648 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:51.376498 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:51.354168 19884 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.007s
I20260709 12:23:51.380591 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.381271 13721 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:51.378428 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.350436 20110 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:23:51.383723 19909 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:51.383919 20110 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:51.384248 19909 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:51.384419 20110 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:51.384757 19909 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.356178 20116 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 12:23:51.387189 20108 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:23:51.394596 20009 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.398402 19884 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.399948 13793 raft_consensus.cc:2368] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:51.400887 19903 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.339074 13799 rpcz_store.cc:276] 0709 12:23:50.228853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.232477 (+  3624us) service_pool.cc:224] Handling call
0709 12:23:50.232849 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.233221 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:23:51.036879 (+803658us) log.cc:844] Serialized 14 byte log entry
0709 12:23:51.338052 (+301173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:51.338119 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:23:51.338174 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.338244 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.338396 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18655,"prepare.queue_time_us":321399,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":2804,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2439,"thread_start_us":1084569,"threads_started":2}
I20260709 12:23:51.402778 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.408782 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.403265 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.404220 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.402760 20117 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:51.400923 19884 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.427980 19069 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:51.425537 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.411132 13799 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:23:51.432782 14421 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:23:51.425251 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.435642 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.379215 20106 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:51.438374 19720 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.003s
I20260709 12:23:51.482642 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.447155 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:51.418905 20005 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.790s	user 0.010s	sys 0.005s
I20260709 12:23:51.412402 20009 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:51.485285 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.448122 19069 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.446899 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.491214 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:51.445338 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.502368 18898 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:23:51.502622 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57416.9us
I20260709 12:23:51.446908 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.440177 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:51.510965 13651 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:51.510106 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
W20260709 12:23:51.515601 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 249591us
I20260709 12:23:51.519608 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.520176 14390 leader_election.cc:400] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20.
I20260709 12:23:51.520654 14390 leader_election.cc:403] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:23:51.484800 19720 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:51.524564 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 87929.4us
I20260709 12:23:51.436940 20104 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.519470 19985 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:23:51.446527 20117 raft_consensus.cc:2764] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:23:51.414042 19745 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.543365 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:51.544894 13652 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:51.449021 20106 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.438951 20118 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:51.556497 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.564781 13792 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:51.565095 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.550434 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:51.557214 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.525840 19998 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 12:23:51.551977 19981 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.903s	user 0.016s	sys 0.001s
I20260709 12:23:51.522483 20093 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:51.516088 20030 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 12:23:51.580755 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 155338us
I20260709 12:23:51.524326 20005 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.586730 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.553812 20009 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:51.569347 20106 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.595688 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.594410 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:51.578212 19965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.009s	sys 0.023s
I20260709 12:23:51.598917 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.603106 13644 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:51.594576 13651 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:51.608632 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.613183 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:51.614302 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.616308 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.579576 19920 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:51.619843 13650 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:51.570058 20126 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:51.522044 20110 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.626194 13724 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:23:51.626926 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.593919 20104 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.579798 20121 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20.
I20260709 12:23:51.631760 19876 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.635151 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1311) took 1114 ms. Trace:
I20260709 12:23:51.635627 13724 rpcz_store.cc:276] 0709 12:23:50.521286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.521565 (+   279us) service_pool.cc:224] Handling call
0709 12:23:51.634900 (+1113335us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:51.635080 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387940608}
I20260709 12:23:51.638164 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:51.639107 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:51.639945 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:51.605373 19965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.010s	sys 0.023s
I20260709 12:23:51.578495 19850 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:23:51.598585 20009 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.581581 19842 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:23:51.582072 19985 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:23:51.672817 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:51.631215 19883 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.641997 13726 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: []
I20260709 12:23:51.646430 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:51.647210 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:51.674667 13797 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.675239 13797 raft_consensus.cc:614] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:51.651846 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.653084 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.612010 19982 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:51.590049 20093 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.581420 19981 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:51.626993 20028 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.901s	user 0.013s	sys 0.001s
I20260709 12:23:51.690058 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:51.661674 19868 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:51.666515 19720 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.662297 20075 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:51.663170 20126 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:51.670411 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.671267 20005 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.660360 13644 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:23:51.681900 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.659202 19839 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.598417 20123 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:23:51.658880 19903 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:51.623351 20118 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:51.691149 13720 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:23:51.665953 20119 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:23:51.692373 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:51.664086 20116 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:51.695106 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:51.662894 20009 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:51.699694 14421 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:51.663004 20128 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:51.665210 20011 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.704291 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:51.715126 19903 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.741045 19903 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:51.741667 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.722402 19839 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.743021 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.743168 19846 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:51.743602 19846 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:51.743703 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:23:51.744831 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.699755 19868 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.744603 19846 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.745491 19868 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:23:51.701987 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93493.5us
I20260709 12:23:51.747013 20117 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:51.747453 20117 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:51.679540 20095 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:51.748018 20095 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.748497 20095 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:51.748256 20117 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.722761 13798 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:51.691920 19973 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:51.750813 20120 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:23:51.652590 20104 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.741679 19876 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:51.756143 20104 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.746078 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.684208 20127 raft_consensus.cc:2764] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:23:51.694670 20110 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:51.703811 20126 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.730700 20116 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.713773 20075 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.727141 20118 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.740058 20128 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:51.746878 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.780161 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:51.757097 19881 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.003s
I20260709 12:23:51.664465 19908 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:23:51.758932 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.787793 13646 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 9.
I20260709 12:23:51.759413 13644 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:51.760296 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.724362 13721 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:51.762336 13798 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:23:51.770587 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:51.795378 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:51.713058 20125 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:51.721217 20113 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 12:23:51.746958 20124 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:23:51.777706 20116 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:51.780881 19846 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:23:51.778360 20115 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 12:23:51.796425 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.766518 19876 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:51.789803 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.837487 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.838809 13644 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:51.744650 19839 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:51.821233 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.783058 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:51.776055 20126 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:51.819620 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:51.847523 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.848572 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:51.849157 13798 raft_consensus.cc:606] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:51.849464 13798 raft_consensus.cc:614] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:51.849543 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:51.841450 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.846910 13646 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:51.798441 20104 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:23:51.799582 19842 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:23:51.855141 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.856705 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.867372 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.867956 13646 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:51.857769 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:51.858644 20039 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:51.803375 20132 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:23:51.859020 19745 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:51.859814 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.877310 13648 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:51.821880 20011 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:51.861882 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:51.881605 13798 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:51.797947 20125 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:51.789314 19881 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.890110 13648 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:51.895509 13646 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:51.900014 13646 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:23:51.861374 20005 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:51.834575 19909 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.902923 13646 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:23:51.873289 20012 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:51.906529 19981 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:51.908315 19609 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:51.803884 20127 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:23:51.908721 19881 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:51.911489 20133 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 12:23:51.912194 20131 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.187s	user 0.001s	sys 0.001s
W20260709 12:23:51.916541 19940 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 12:23:51.853593 19876 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:51.895850 20125 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:51.876961 19839 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:51.918745 13801 raft_consensus.cc:2368] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
W20260709 12:23:51.920881 19923 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.001s
I20260709 12:23:51.922375 20093 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.922752 20100 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:51.925024 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.973974 19609 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:51.838497 20009 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.930647 19745 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:51.979029 19883 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:52.008093 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.008701 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:51.933037 20124 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:23:51.942356 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:52.010713 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 143423us
I20260709 12:23:51.946697 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:51.875285 20039 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:51.953069 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3123) took 1655 ms. Trace:
I20260709 12:23:51.964355 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:51.926009 19973 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:51.901420 13798 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:23:51.968657 20034 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.002s
I20260709 12:23:51.979740 13721 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:51.964198 20132 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:51.990363 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:52.004169 13727 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:23:52.004016 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:52.028496 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:52.028985 13800 raft_consensus.cc:606] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.029557 13800 raft_consensus.cc:614] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:51.926918 20075 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:51.932446 20011 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:52.021862 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:52.034991 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:52.011958 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:51.994515 19839 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.024911 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:52.020195 13723 rpcz_store.cc:276] 0709 12:23:50.297714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:50.297930 (+   216us) service_pool.cc:224] Handling call
0709 12:23:50.298340 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:50.629458 (+331118us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:23:51.118212 (+488754us) log.cc:844] Serialized 14 byte log entry
0709 12:23:51.597817 (+479605us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:51.597883 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:23:51.597940 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.952772 (+354832us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:51.952831 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:51.952980 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10157,"prepare.queue_time_us":641169,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":120778,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":10349,"spinlock_wait_cycles":760749440,"thread_start_us":947569,"threads_started":3,"wal-append.queue_time_us":365087}
I20260709 12:23:52.002526 19868 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.056885 14423 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:23:51.995443 20104 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:23:52.027539 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56838.3us
I20260709 12:23:52.029136 14424 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:52.064126 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:52.064599 14416 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:52.006323 13726 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:52.033458 20034 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.041561 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.009811 19609 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.070542 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:52.070761 19609 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.001358 20138 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:52.079190 13800 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:52.007089 19902 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:52.035071 20116 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.023636 20093 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:23:52.006111 20095 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.082248 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:52.047966 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 101392us
I20260709 12:23:52.048085 13798 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:23:52.049356 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.084465 13722 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:23:52.007257 19745 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.088173 13796 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:23:52.055899 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:52.092464 20141 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:23:52.094751 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53277.2us
I20260709 12:23:52.095607 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.056051 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:51.993175 19940 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:51.900404 20128 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.031047 20136 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:52.087855 20014 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:23:52.059190 14422 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:23:52.061558 19982 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:52.066519 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:52.122632 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56227.5us
I20260709 12:23:52.124722 20009 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.087364 19902 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.110852 19908 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:52.127710 19902 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.128909 13650 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:23:52.129289 19903 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:52.075685 20012 consensus_queue.cc:1175] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 16 status { last_received { term: 10 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:23:52.131274 13651 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:52.057559 19839 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:52.069187 20023 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.004s
I20260709 12:23:52.062590 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:52.096413 19745 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:52.162387 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55393us
I20260709 12:23:52.078907 14424 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:52.081111 13644 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:23:52.085886 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:52.085976 13726 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:23:52.085713 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:52.062551 19883 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:52.103080 13722 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:52.111357 13643 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:52.134289 13647 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:52.076308 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.023882 20039 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.184752 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:52.158226 13792 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
W20260709 12:23:52.244208 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148685us
I20260709 12:23:52.244381 13800 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:52.163879 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:52.247604 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 162023us
I20260709 12:23:52.248490 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:52.249572 13719 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:52.246136 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:52.111248 19967 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.252707 19374 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:52.256515 19374 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:52.134277 19842 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.321507 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:52.322561 13799 raft_consensus.cc:606] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.323372 13799 raft_consensus.cc:614] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:52.269812 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:52.255116 20136 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:52.183672 19973 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.137s
I20260709 12:23:52.320825 20142 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:52.257292 19374 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.262511 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:52.320984 20148 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:23:52.135733 20118 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:52.385645 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 123350us
I20260709 12:23:52.391353 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:52.392830 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:52.394274 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:52.405465 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 321950us
I20260709 12:23:52.406276 19069 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.407480 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:52.408247 13800 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:52.408877 14424 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:23:52.410367 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:52.256630 20095 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.411010 13800 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:52.412869 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:52.414911 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.137817 19982 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.254s
I20260709 12:23:52.377995 20009 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.137638 19968 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.243s
I20260709 12:23:52.256078 20093 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.154s
I20260709 12:23:52.396212 19954 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:23:52.404734 20075 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.406854 19069 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.416096 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:52.425536 19955 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:23:52.426959 20129 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.669s	user 0.020s	sys 0.021s
I20260709 12:23:52.176729 20023 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:52.454866 20140 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.428s	user 0.003s	sys 0.000s
I20260709 12:23:52.457417 20116 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.416704 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:52.417359 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:52.259812 19876 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:52.387372 20138 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.365857 19967 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:52.440125 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45346.5us
I20260709 12:23:52.482285 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3585) took 1125 ms. Trace:
I20260709 12:23:52.405145 20127 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:52.485441 20127 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:52.486018 20127 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:52.451287 19955 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.486645 19955 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:52.487323 20154 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:52.487852 20154 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.488258 20154 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:52.411788 20132 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:52.446098 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:52.489660 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43706.4us
W20260709 12:23:52.490820 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.482841 13645 rpcz_store.cc:276] 0709 12:23:51.356459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.357083 (+   624us) service_pool.cc:224] Handling call
0709 12:23:51.376961 (+ 19878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:51.381129 (+  4168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:51.381162 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:51.407948 (+ 26786us) log.cc:844] Serialized 26 byte log entry
0709 12:23:51.593335 (+185387us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:51.593384 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:51.593460 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:52.425955 (+832495us) spinlock_profiling.cc:243] Waited 7.61 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:52.458788 (+ 32833us) raft_consensus.cc:1685] finished
0709 12:23:52.458818 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:52.482168 (+ 23350us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3719,"raft.queue_time_us":9096,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":169,"spinlock_wait_cycles":17496448,"thread_start_us":102569,"threads_started":1,"wal-append.queue_time_us":252690}
I20260709 12:23:52.489689 19069 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:52.457618 20102 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.334s	user 0.010s	sys 0.009s
I20260709 12:23:52.378779 20136 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.500161 20136 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:52.452491 13719 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:52.139348 20011 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.470407 13797 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:23:52.477236 13800 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:52.478925 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:52.513077 13799 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:52.403990 20013 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:23:52.443871 19954 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:52.520840 20152 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.521452 20012 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.512123 20034 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:52.522024 20012 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.525166 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:52.515878 20125 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.001s
I20260709 12:23:52.517303 20070 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:52.469678 19902 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:23:52.523636 19954 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:52.514091 19931 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 12:23:52.556396 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:52.526152 19903 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.459501 20075 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.553936 13800 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:23:52.511628 20005 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:23:52.468243 20016 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 12:23:52.567189 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:52.532156 20142 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:23:52.512212 20138 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.582536 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:52.544118 20070 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:52.557734 20078 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.593734 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.594120 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.607548 13722 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:52.557734 20132 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:52.596882 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.609925 20070 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:52.615365 20150 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:23:52.616904 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2395) took 1499 ms. Trace:
I20260709 12:23:52.537297 20125 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.589129 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.607553 13727 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:52.617458 13793 rpcz_store.cc:276] 0709 12:23:51.117521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.117696 (+   175us) service_pool.cc:224] Handling call
0709 12:23:52.610520 (+1492824us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54002cb5d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:52.616822 (+  6302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540656896}
I20260709 12:23:52.595485 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.598738 13800 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:23:52.631147 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 467383us
W20260709 12:23:52.634357 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:52.616753 20106 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.992s	user 0.011s	sys 0.003s
I20260709 12:23:52.620771 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:52.572067 20034 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:52.620488 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 204061us
I20260709 12:23:52.625128 19907 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:52.656895 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3596) took 1109 ms. Trace:
I20260709 12:23:52.657325 13647 rpcz_store.cc:276] 0709 12:23:51.547174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.670058 (+122884us) service_pool.cc:224] Handling call
0709 12:23:52.656682 (+986624us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:52.656830 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944207616}
I20260709 12:23:52.660034 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:52.660682 13719 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:52.660822 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:52.645262 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:52.647742 13727 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:52.640552 13800 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:52.649688 13723 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:52.665241 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.667991 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:52.669343 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:52.669711 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:52.672813 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.623397 20078 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.648631 20093 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.664219 20118 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.664379 19907 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:52.680869 13719 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:52.723387 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:52.656030 20023 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:52.674227 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:52.682865 20153 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:23:52.665534 19925 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.654510 20151 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:52.623803 20132 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.666826 20152 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.651674 20106 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.653620 20034 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:52.720484 20149 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:23:52.741196 13723 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:23:52.809134 19839 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.636s	user 0.008s	sys 0.005s
W20260709 12:23:52.810348 19745 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.000s
I20260709 12:23:52.803520 20070 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:52.817397 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 143243us
W20260709 12:23:52.730855 13616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.011s	sys 0.015s
W20260709 12:23:52.714789 20110 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.937s	user 0.013s	sys 0.000s
I20260709 12:23:52.670115 13647 raft_consensus.cc:2368] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:23:52.773233 19883 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:52.802903 20159 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:52.715634 20010 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:52.720930 20021 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.728179 20009 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.864948 13647 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:23:52.859354 13616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.016s	sys 0.018s
I20260709 12:23:52.873979 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1322) took 1286 ms. Trace:
I20260709 12:23:52.860474 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:52.867650 13693 leader_election.cc:400] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:23:52.675060 20156 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.186s
W20260709 12:23:52.863341 20126 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.990s	user 0.014s	sys 0.000s
I20260709 12:23:52.880578 13718 rpcz_store.cc:276] 0709 12:23:51.587869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.588024 (+   155us) service_pool.cc:224] Handling call
0709 12:23:51.604402 (+ 16378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:51.639548 (+ 35146us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:23:51.639950 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 12:23:51.662065 (+ 22115us) log.cc:844] Serialized 14 byte log entry
0709 12:23:51.766827 (+104762us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:51.766887 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:23:51.766952 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.873673 (+1106721us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b54000285d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:52.873745 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:52.873909 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21626,"prepare.queue_time_us":30312,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":2490,"raft.queue_time_us":162954,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":35053,"spinlock_wait_cycles":80098560,"thread_start_us":104421,"threads_started":1}
I20260709 12:23:52.719695 19907 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.887094 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.892280 13693 leader_election.cc:403] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:23:52.718802 20118 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.702834 20012 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.864817 20152 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:52.698621 20093 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.818507 19925 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.875133 19968 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.858578 19745 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.858270 19839 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.859261 20136 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.903069 20176 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:52.866362 20154 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.845227 20151 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.922996 19907 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:52.979722 20070 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.893412 20021 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.981706 19745 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:52.898223 19982 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:23:52.983168 20034 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:52.889285 20127 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.002887 13727 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:52.913576 19876 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.979943 19839 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:52.993667 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:52.898231 20009 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:52.911398 20163 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:52.951717 19609 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:52.897971 19955 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.903839 20126 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:52.923439 20011 consensus_queue.cc:1175] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 9 status { last_received { term: 9 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 9 index: 8 } }
I20260709 12:23:52.974150 20012 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:53.008327 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 140831us
I20260709 12:23:52.984828 20075 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:53.000389 20039 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.742s	user 0.010s	sys 0.003s
I20260709 12:23:52.952267 19968 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.000671 20070 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:52.996464 20166 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:53.030750 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 133096us
I20260709 12:23:53.033294 13719 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:23:52.960558 20152 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:23:52.961308 20176 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.019922 19884 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.046710 13720 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.022651 20164 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.046459 20039 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.075173 20154 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.076766 20039 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.077993 13649 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:23:53.079387 19925 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:53.080088 20167 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:23:53.023531 20034 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:23:53.082037 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3601) took 1403 ms. Trace:
I20260709 12:23:53.082442 13652 rpcz_store.cc:276] 0709 12:23:51.678787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.385164 (+706377us) service_pool.cc:224] Handling call
0709 12:23:52.420232 (+ 35068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.420589 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:23:52.436841 (+ 16252us) log.cc:844] Serialized 14 byte log entry
0709 12:23:52.815970 (+379129us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:52.816534 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.816715 (+   181us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:23:52.818214 (+  1499us) log.cc:844] Serialized 14 byte log entry
0709 12:23:52.931669 (+113455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.931784 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.081779 (+149995us) raft_consensus.cc:1685] finished
0709 12:23:53.081803 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.081964 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20813,"prepare.queue_time_us":17339,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":95,"raft.queue_time_us":260705,"raft.run_cpu_time_us":414,"raft.run_wall_time_us":36113,"spinlock_wait_cycles":22011520,"thread_start_us":424962,"threads_started":2,"wal-append.queue_time_us":426578}
I20260709 12:23:53.084486 19955 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.084807 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:53.086676 20126 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.087023 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2399) took 1126 ms. Trace:
I20260709 12:23:53.087528 13644 rpcz_store.cc:276] 0709 12:23:51.960690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.070214 (+109524us) service_pool.cc:224] Handling call
0709 12:23:53.086687 (+1016473us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.086956 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22613248}
I20260709 12:23:53.088672 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:53.089502 13644 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:53.089540 20169 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:23:53.087011 20178 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.090968 20178 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.092103 20009 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.093385 19909 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.093526 20178 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.094717 20176 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:53.095801 19954 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 12:23:53.057183 20166 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.048604 20128 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:23:53.096277 19954 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:52.995639 19883 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.101495 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.095280 20164 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.096158 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:53.057263 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 106659us
I20260709 12:23:53.098143 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:53.100713 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:53.046169 20012 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:53.099399 13720 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:53.111074 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.108963 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:53.114477 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:53.115159 13652 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:53.110647 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:53.108022 20168 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:53.121697 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:53.122404 13647 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:53.117626 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:53.123870 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.125406 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:53.120004 13644 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.101775 20181 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.157946 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:53.128785 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:53.129168 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107136us
I20260709 12:23:53.159201 13798 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:53.159644 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3529) took 1081 ms. Trace:
I20260709 12:23:53.132835 19903 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:23:53.142059 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3593) took 1696 ms (client timeout 1802 ms). Trace:
I20260709 12:23:53.145759 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3236) took 1278 ms. Trace:
I20260709 12:23:53.163177 13723 rpcz_store.cc:276] 0709 12:23:51.867085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.055656 (+188571us) service_pool.cc:224] Handling call
0709 12:23:53.145540 (+1089884us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54000280d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.145679 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269054464}
I20260709 12:23:53.148958 13645 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:53.154732 20123 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.129973 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.155329 19868 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.155757 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1235) took 1131 ms. Trace:
I20260709 12:23:53.156351 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3518) took 1511 ms. Trace:
I20260709 12:23:53.158594 13647 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.159968 13798 rpcz_store.cc:276] 0709 12:23:52.078217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.083829 (+  5612us) service_pool.cc:224] Handling call
0709 12:23:53.128571 (+1044742us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:23:53.159581 (+ 31010us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4051712}
I20260709 12:23:53.180337 13792 rpcz_store.cc:276] 0709 12:23:51.645063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.647565 (+  2502us) service_pool.cc:224] Handling call
0709 12:23:51.674698 (+ 27133us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:52.247806 (+573108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.156074 (+908268us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54003d54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:53.156284 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111051392}
I20260709 12:23:53.181411 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:53.104305 20106 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.056006 20070 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.167430 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.168661 13644 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:53.190992 13800 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:53.190771 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.191496 13793 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:53.121184 20160 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:23:53.202540 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2400) took 2050 ms (client timeout 2449 ms). Trace:
W20260709 12:23:53.203176 13796 rpcz_store.cc:269] 0709 12:23:51.152136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.154362 (+  2226us) service_pool.cc:224] Handling call
0709 12:23:53.193160 (+2038798us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.202425 (+  9265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001416576}
W20260709 12:23:53.161769 13651 rpcz_store.cc:269] 0709 12:23:51.445647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.630749 (+185102us) service_pool.cc:224] Handling call
0709 12:23:53.141758 (+1511009us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400095454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.141973 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040532224}
I20260709 12:23:53.184710 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:53.185587 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 83403.9us
I20260709 12:23:53.165347 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.112282 19908 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:53.176770 20139 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:53.112713 19982 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.116896 20132 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.191361 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:53.218797 14421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1022) took 1775 ms (client timeout 1598 ms). Trace:
I20260709 12:23:53.161645 19903 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:53.192363 13800 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:53.192538 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:53.182512 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.225755 13647 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:53.227041 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:53.227698 13647 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:23:53.180256 14418 rpcz_store.cc:276] 0709 12:23:52.024387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.024541 (+   154us) service_pool.cc:224] Handling call
0709 12:23:52.024990 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:52.132432 (+107442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:52.132481 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:52.132543 (+    62us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:23:52.132588 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.155439 (+1022851us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:53.155524 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.155689 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246281088}
I20260709 12:23:53.157856 20075 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:53.198290 13801 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:23:53.199306 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.232384 13795 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:23:53.200326 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.232409 13652 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:53.232993 13767 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:53.233610 13799 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 10: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:53.200538 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.127796 20110 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:53.112486 20178 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:53.239540 20178 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:53.240482 20070 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.240576 20178 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.220127 19908 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.242350 20034 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:53.243085 20187 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:23:53.247318 20134 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.216s	user 0.007s	sys 0.001s
I20260709 12:23:53.221149 13793 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:53.221333 13644 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
W20260709 12:23:53.219338 14421 rpcz_store.cc:269] 0709 12:23:51.436062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.438142 (+  2080us) service_pool.cc:224] Handling call
0709 12:23:53.211237 (+1773095us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540010f4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.211384 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454569344}
I20260709 12:23:53.250785 13644 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 12
I20260709 12:23:53.251232 13644 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:23:53.234566 14420 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:23:53.251549 13644 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:53.234114 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:53.189332 20118 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.237855 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:53.253281 13644 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:23:53.253428 13651 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:53.257345 13796 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:23:53.254835 19720 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.222415 19982 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.259522 19720 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.261320 19982 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.262089 13798 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.267005 20127 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.267567 19903 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.214907 20166 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.267011 19954 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.256695 20034 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:53.273945 20034 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.281635 13792 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:53.157166 20168 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.297775 19908 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.232385 20136 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.299417 19984 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:53.299919 19984 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:53.300877 19984 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.302057 20093 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.231442 20075 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:53.298069 20173 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:23:53.325893 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:53.306456 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73566.3us
I20260709 12:23:53.306277 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:53.308923 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:53.328488 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:53.329005 13721 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.333750 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:53.334573 13722 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:23:53.329770 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:53.176509 20181 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.348281 20181 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.320423 20151 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.320920 20021 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.188411 20123 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.321285 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:53.349973 13718 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:53.302232 19984 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:53.351286 20123 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.323135 13651 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:23:53.353958 13651 pending_rounds.cc:85] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 9
I20260709 12:23:53.354420 13651 pending_rounds.cc:107] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:23:53.354733 13651 raft_consensus.cc:2889] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:53.326359 20075 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:53.325304 13719 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:23:53.326174 20136 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:53.361447 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:53.362313 13719 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:53.363368 20125 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:53.365495 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3146) took 1592 ms (client timeout 1735 ms). Trace:
I20260709 12:23:53.332763 20110 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:53.388792 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:53.365983 13720 rpcz_store.cc:269] 0709 12:23:51.772772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.781417 (+  8645us) service_pool.cc:224] Handling call
0709 12:23:53.365291 (+1583874us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400173c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.365427 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2758955904}
I20260709 12:23:53.234671 20012 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.345890 13722 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:53.324193 13724 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:53.357179 13726 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:53.367931 13721 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:53.383884 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:53.312958 13798 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:53.401405 13723 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:23:53.405689 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:53.406850 13721 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 13: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:53.407541 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:53.394438 20110 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:53.393774 20192 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:53.411708 13721 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:53.412925 20034 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:23:53.413774 20184 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:23:53.388221 20136 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.422415 20117 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:53.393973 19839 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:23:53.422791 20136 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.433538 20110 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.439052 20168 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.424846 20164 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.430646 19940 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.445433 19968 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:53.452173 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.452556 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.453002 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.452957 14417 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:53.466945 19871 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.450094 20160 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:23:53.468096 19871 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.467188 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:53.471880 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 239913us
I20260709 12:23:53.477294 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.462699 20187 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:53.478209 14417 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:53.425205 20034 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.479426 20187 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:53.480731 19871 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:53.480933 20195 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:23:53.484825 19884 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:53.485539 19967 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:53.485724 14416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1023) took 1811 ms (client timeout 1601 ms). Trace:
W20260709 12:23:53.486181 14416 rpcz_store.cc:269] 0709 12:23:51.673891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.674051 (+   160us) service_pool.cc:224] Handling call
0709 12:23:53.485527 (+1811476us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54006f72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.485643 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689741568}
I20260709 12:23:53.486581 20078 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.437213 20192 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:53.450153 20075 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.501075 19720 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.607179 19745 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:53.560292 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.572875 13647 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:23:53.608527 19745 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:53.509739 19907 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.005s
I20260709 12:23:53.620607 20075 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:53.602903 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.502832 20138 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.613690 13718 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:53.614468 19745 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.615541 20010 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.488827 20034 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:53.504446 20170 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.179s	user 0.001s	sys 0.002s
I20260709 12:23:53.487533 20187 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.643561 20187 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:53.647773 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3244) took 1049 ms. Trace:
I20260709 12:23:53.515043 20192 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:53.631845 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3587) took 2264 ms (client timeout 1607 ms). Trace:
W20260709 12:23:53.655206 13643 rpcz_store.cc:269] 0709 12:23:51.366905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.369134 (+  2229us) service_pool.cc:224] Handling call
0709 12:23:53.630876 (+2261742us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54001e5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.631454 (+   578us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684946688}
I20260709 12:23:53.628167 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.669842 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:53.670071 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.670468 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:53.673969 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:53.663867 20010 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.678867 20168 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:53.679709 20168 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:53.680387 20016 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:53.681015 14420 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:23:53.681394 20016 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.683681 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:53.648411 13725 rpcz_store.cc:276] 0709 12:23:52.597688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.600103 (+  2415us) service_pool.cc:224] Handling call
0709 12:23:52.600544 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.601005 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:52.601045 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:52.601703 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 12:23:52.870182 (+268479us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:52.870229 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:52.871534 (+  1305us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:53.503533 (+631999us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:53.647062 (+143529us) raft_consensus.cc:1685] finished
0709 12:23:53.647106 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:53.647473 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":960,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":9717,"spinlock_wait_cycles":14304000,"thread_start_us":228580,"threads_started":1,"wal-append.queue_time_us":595983}
I20260709 12:23:53.687254 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.688967 14419 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:53.691120 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.693552 20034 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.694064 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.696112 20010 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.697997 20110 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.699193 20078 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.698956 20110 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.699509 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.700037 20078 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.704304 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:53.706084 20075 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.707192 20075 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.707912 19745 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:53.708545 19745 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.708817 19846 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:53.709106 19745 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:53.709715 19846 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.694741 20034 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.677462 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:53.703926 20110 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.718016 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.719691 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:53.721189 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:53.721537 13794 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 11. Current term is 12. Ops: []
I20260709 12:23:53.723244 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:53.723613 13800 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:53.724247 13792 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:53.681547 20168 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.736613 20016 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:53.738047 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.736415 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:53.741685 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:53.743300 13794 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:53.751082 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:53.753798 20110 raft_consensus.cc:2764] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:23:53.755038 20128 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:53.756263 13798 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:53.756029 20128 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.758464 20138 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.758647 13796 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:53.759486 20138 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.765396 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.764423 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.766975 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.771113 13795 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:23:53.772279 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.773550 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.775416 19940 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:53.776003 19587 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.776357 20110 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.778609 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:53.787627 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2402) took 1763 ms. Trace:
I20260709 12:23:53.787982 19587 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.788272 13645 rpcz_store.cc:276] 0709 12:23:51.961399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.493458 (+532059us) service_pool.cc:224] Handling call
0709 12:23:53.148745 (+655287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.164473 (+ 15728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.724081 (+559608us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400095450. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:53.724385 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445047680}
W20260709 12:23:53.797982 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:53.802788 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.803228 20110 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:53.803939 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:53.804371 19587 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:53.807274 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.808925 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.809845 13801 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:53.812031 19967 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:53.798233 19940 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:53.817027 19903 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:53.799963 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.814459 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:53.817221 13693 leader_election.cc:400] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:53.806023 20110 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.853672 13693 leader_election.cc:403] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:53.853000 20023 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:23:53.815843 20138 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.862356 19940 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:53.865293 20034 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.887209 19846 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:53.870218 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.879750 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3597) took 2327 ms. Trace:
I20260709 12:23:53.880102 20136 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:53.901626 13800 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:53.856287 20078 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.890972 13649 rpcz_store.cc:276] 0709 12:23:51.552694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.963396 (+410702us) service_pool.cc:224] Handling call
0709 12:23:51.963799 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:53.078656 (+1114857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.879074 (+800418us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001b6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:53.879483 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2561776384}
I20260709 12:23:53.901499 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:53.881067 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:53.910403 20128 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:53.914151 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:23:53.908829 13801 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:23:53.918500 20075 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:53.927361 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3519) took 2281 ms. Trace:
I20260709 12:23:53.928069 13797 rpcz_store.cc:276] 0709 12:23:51.645391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:51.686958 (+ 41567us) service_pool.cc:224] Handling call
0709 12:23:51.687610 (+   652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:52.506749 (+819139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.926546 (+1419797us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:53.927085 (+   539us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776953216}
I20260709 12:23:53.936522 13792 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:53.881809 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.917397 20034 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:53.948664 13794 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
W20260709 12:23:53.882352 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 205295us
I20260709 12:23:53.953488 13796 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:23:53.944530 20034 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:53.977363 20011 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:53.978142 20011 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:53.985857 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173593us
I20260709 12:23:53.974906 20014 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:23:53.987653 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.985787 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:53.986343 20034 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:53.999171 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:54.015537 20209 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
I20260709 12:23:53.985986 20136 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.018615 19876 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:23:54.023156 19876 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:54.023360 14419 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:54.024515 19876 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.026304 19876 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:54.027978 20128 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:23:53.944845 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:23:54.033457 13800 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:54.033957 13800 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:23:54.033799 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:54.036325 20013 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:54.036362 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:54.031466 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 251804us
I20260709 12:23:54.037122 20013 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:54.038158 13693 leader_election.cc:400] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:54.038842 20136 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:54.038578 13693 leader_election.cc:403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:54.039516 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:54.040112 20013 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.042716 20136 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:54.042008 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:54.022873 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1330) took 1339 ms. Trace:
I20260709 12:23:54.047564 13793 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:23:54.052335 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.054070 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:54.055684 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:54.056694 13722 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14. Re-sending same reply.
I20260709 12:23:54.058282 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:54.059144 13724 raft_consensus.cc:2376] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19. Re-sending same reply.
I20260709 12:23:54.047251 13727 rpcz_store.cc:276] 0709 12:23:52.662019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.664128 (+  2109us) service_pool.cc:224] Handling call
0709 12:23:52.664415 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:53.003328 (+338913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:53.970628 (+967300us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:54.000928 (+ 30300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777934080}
I20260709 12:23:54.067837 13693 leader_election.cc:400] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:23:54.068279 13693 leader_election.cc:403] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:54.069343 20078 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:23:54.070920 20012 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:23:54.071040 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100776us
I20260709 12:23:54.072104 20012 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.072793 20012 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.078598 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.079735 13725 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:23:54.064180 19745 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.088241 19884 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:23:54.089339 19884 consensus_queue.cc:1243] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:23:54.090476 20182 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:54.091708 20182 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.092504 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.099236 13645 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:54.099927 20136 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:54.100435 20136 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.100857 20136 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:54.106643 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:54.108184 13721 raft_consensus.cc:2376] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14. Re-sending same reply.
W20260709 12:23:54.111227 20171 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:23:54.113320 20117 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.690s	user 0.021s	sys 0.008s
I20260709 12:23:54.114480 13646 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:23:54.115577 19720 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:54.113399 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45838us
I20260709 12:23:54.116225 19720 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.051040 20023 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:23:54.119686 20212 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:23:54.081949 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.121713 13718 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:54.124439 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:54.111375 20152 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:54.135893 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:54.142905 20078 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:54.131513 20128 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 12:23:54.184715 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:54.182624 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90214.4us
I20260709 12:23:54.162993 20138 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:23:54.221004 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.222492 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:54.225992 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.227242 13721 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:54.228786 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.230543 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.231400 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:54.257228 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2401) took 2290 ms. Trace:
I20260709 12:23:54.236554 20011 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:54.200052 20110 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:23:54.272277 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:54.267189 13646 rpcz_store.cc:276] 0709 12:23:51.961167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.063880 (+102713us) service_pool.cc:224] Handling call
0709 12:23:54.251033 (+2187153us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:54.251212 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4713271040}
I20260709 12:23:54.254225 20012 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:54.257885 20016 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:23:54.295466 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.300558 20192 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
W20260709 12:23:54.304010 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.337329 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:54.308956 20012 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:54.319800 19967 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.369989 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.358919 20016 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:23:54.376859 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 244528us
I20260709 12:23:54.373742 13725 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:54.385077 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:54.396210 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 178301us
W20260709 12:23:54.391791 20213 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 12:23:54.404223 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 151481us
I20260709 12:23:54.375993 20016 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:54.372305 20136 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.401949 19903 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.408084 19967 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.422292 19884 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:54.433019 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 242409us
W20260709 12:23:54.461248 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.476140 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:54.426651 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 213013us
I20260709 12:23:54.465454 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:54.471995 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 217855us
I20260709 12:23:54.469851 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.496866 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:54.442919 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.494771 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.474793 19967 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:54.432021 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:54.479193 20224 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:23:54.516903 13648 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:23:54.518002 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 115027us
I20260709 12:23:54.508507 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:54.473454 19745 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:54.457636 20138 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:54.518985 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:54.432634 20136 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:54.524125 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.469244 19720 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:54.461966 20192 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:23:54.432899 19903 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.503043 19884 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.526113 20138 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.539877 19907 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:23:54.543169 13646 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:23:54.482137 20016 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.546092 19903 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:54.547751 20218 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:23:54.550631 20227 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:23:54.551877 20222 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:23:54.566740 20230 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:23:54.552513 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.535741 20231 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:54.434170 20187 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
W20260709 12:23:54.558617 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 134234us
I20260709 12:23:54.561772 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:54.562990 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 129180us
I20260709 12:23:54.566283 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:54.567035 14419 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:23:54.534482 19587 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:23:54.573325 13647 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:54.574728 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.573289 14421 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:54.575418 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.575762 20231 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:54.585886 19846 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:54.572381 20136 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:23:54.588943 19884 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:54.590049 20219 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:23:54.588927 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:54.595124 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 95145.9us
I20260709 12:23:54.565652 20138 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:23:54.607290 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86104.4us
W20260709 12:23:54.627300 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93586.4us
I20260709 12:23:54.598626 19846 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:54.602705 20192 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:54.631088 13644 raft_consensus.cc:1217] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[11.8-16.11]   Dedup: 11.8->[12.9-16.11]
I20260709 12:23:54.598652 20231 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:54.635874 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61327us
I20260709 12:23:54.636488 19967 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:54.638604 19907 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:54.639099 20230 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:54.631776 13644 pending_rounds.cc:85] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 8
I20260709 12:23:54.635248 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:54.641484 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 125376us
I20260709 12:23:54.642874 13647 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:54.640686 13644 pending_rounds.cc:107] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 13.9
W20260709 12:23:54.650287 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74965.2us
I20260709 12:23:54.649221 13644 raft_consensus.cc:2889] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:54.632393 20192 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.658337 13648 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:23:54.660542 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:54.641955 20235 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:54.672148 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.672935 13651 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:54.672657 20235 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.675343 20235 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:54.681748 20240 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:54.682401 20240 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:54.683521 20240 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.684273 19587 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:54.684957 19587 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:23:54.692936 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:54.694501 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.695061 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.696564 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:54.697661 19720 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:54.698364 19720 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:54.698514 20012 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:54.699151 20012 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:54.699604 19720 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.700206 20012 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.693075 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:54.702740 13652 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:54.702636 20221 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:54.702672 20228 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:54.656246 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:54.638155 19967 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 12:23:54.709785 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78609.8us
I20260709 12:23:54.655557 20192 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:54.718061 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:54.652328 20187 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 12:23:54.714893 19903 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:54.718106 20221 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:23:54.743256 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51143.3us
W20260709 12:23:54.745440 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108622us
I20260709 12:23:54.738646 19903 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:54.753355 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83155.1us
I20260709 12:23:54.739837 19876 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:54.755321 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.761739 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:54.738387 20013 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:54.741333 20221 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:54.757508 20034 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.005s
I20260709 12:23:54.754948 19903 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:54.771602 20221 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.762637 19745 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:23:54.778877 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 127871us
I20260709 12:23:54.783349 19745 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:54.784082 20221 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:54.784509 19745 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.787955 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.784154 20164 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:54.789528 20164 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:54.789956 20164 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:54.794338 20136 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:23:54.794895 20136 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:54.773524 20229 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.145s
W20260709 12:23:54.803064 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66709.3us
W20260709 12:23:54.803930 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59442.2us
I20260709 12:23:54.804075 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:54.805128 20241 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:54.806671 20013 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.808614 13793 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:23:54.808610 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:54.770380 20228 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:54.807152 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:54.808794 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.810370 20078 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:23:54.817080 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:54.863132 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87413.3us
I20260709 12:23:54.768177 20117 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:23:54.868189 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 89727us
I20260709 12:23:54.774088 20034 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:54.872496 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:54.873596 13794 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:54.790110 20240 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:54.879391 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:54.817245 19967 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:54.893242 13793 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:23:54.893648 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:54.818137 20228 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:54.888695 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:54.900753 13795 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:23:54.905119 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:54.906502 13792 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:54.906975 13650 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:23:54.908136 19846 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:54.920675 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113639us
W20260709 12:23:54.910689 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93709.1us
I20260709 12:23:54.861929 20078 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:54.917570 13801 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:54.799659 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.928112 13793 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:23:54.879549 20227 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:54.921782 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:54.934607 13800 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:23:54.936298 13798 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:23:54.890606 20187 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:54.899084 20110 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:23:54.884799 20230 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:54.907372 20220 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:54.945245 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:54.950470 13644 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:23:54.958632 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:54.884435 20010 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:23:54.946888 19745 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:54.967882 19745 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.968564 19745 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:54.936647 19884 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:54.944180 20182 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:54.983179 20236 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:54.986091 20253 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.000161 20034 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:54.999970 20253 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:54.967751 19846 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:55.003105 20013 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:54.942327 20241 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:54.944877 20187 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:54.980764 19884 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:54.923707 20012 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:23:54.995958 19903 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:55.011709 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70907us
I20260709 12:23:55.005421 20228 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.017620 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:55.016093 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108386us
I20260709 12:23:54.943145 20117 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.026633 20228 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:55.011361 19884 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:55.011202 20187 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:54.973237 20230 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.035367 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:55.021113 20034 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:55.038795 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.040724 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.018648 20253 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:55.025118 20110 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:23:55.036448 14419 raft_consensus.cc:2376] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14. Re-sending same reply.
I20260709 12:23:55.031111 20023 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:55.054872 20241 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.057648 20110 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:55.036075 20078 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.058823 20078 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:55.058748 20110 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.018764 19967 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:23:55.063021 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.064090 20230 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:55.065366 20246 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:55.065866 20246 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:55.067727 20117 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.068567 19903 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:55.072561 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 149372us
W20260709 12:23:55.074120 14416 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.002s
I20260709 12:23:55.074606 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
W20260709 12:23:55.080060 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.089557 19903 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:55.090679 19903 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.095758 19903 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.029176 20234 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:23:55.013214 20236 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.024422 19587 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.040988 20034 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:55.101799 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 213188us
I20260709 12:23:55.108352 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.110338 14419 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:23:55.071424 19907 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:55.015447 20220 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.116029 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.115934 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:55.120667 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95136.3us
I20260709 12:23:55.122227 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:55.126467 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.112658 20034 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:55.121553 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:55.135402 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.137131 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.139565 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.144616 19587 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:55.146241 13725 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:55.154634 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:55.155720 20251 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:23:55.157585 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:55.159732 20252 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:23:55.160308 20246 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.162175 20246 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:55.166285 20223 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:55.167892 13647 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:55.177630 20016 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:55.184427 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:55.191550 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:55.192245 14420 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:23:55.196372 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.202438 13726 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:23:55.202936 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:55.203701 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.208325 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.177671 20236 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.191303 20234 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:55.195724 20075 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:55.221123 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.221904 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:23:55.211416 20138 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.002s
I20260709 12:23:55.225313 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.174925 19745 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.232203 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:23:55.234700 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113204us
I20260709 12:23:55.235252 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.235922 13725 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:55.239595 13650 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:55.219781 20075 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.219782 20234 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:55.218974 20110 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:55.240346 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:55.246127 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:55.241132 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:55.247674 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:55.231014 20138 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.253674 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.264386 13650 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:55.255188 20013 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:55.265938 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.266247 20013 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.266734 20013 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:55.266884 13649 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:55.243377 20011 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:55.268049 20011 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.268508 20011 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:55.244544 20234 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:55.278666 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.282307 20266 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.282840 20255 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:55.259313 20182 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:23:55.244889 20075 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:23:55.288580 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52724.1us
I20260709 12:23:55.290555 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:55.238976 20117 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:23:55.293463 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:55.302480 20248 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:23:55.303455 20264 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:23:55.253019 19907 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:55.253341 19876 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.250701 20012 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:23:55.309273 20261 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:23:55.318917 20259 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:55.329208 19884 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.331451 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:55.312872 20248 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:23:55.283319 20266 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.332722 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:55.321178 20262 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:55.319710 14417 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:23:55.338480 13727 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:23:55.361907 13727 pending_rounds.cc:85] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:23:55.362291 13727 pending_rounds.cc:107] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:23:55.362542 13727 raft_consensus.cc:2889] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:55.339601 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:55.363782 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47615.8us
I20260709 12:23:55.339679 20034 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.325995 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:55.344904 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.292042 20023 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.251328 19720 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:23:55.319185 19907 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:55.367012 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:55.329298 20257 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.302s	user 0.000s	sys 0.002s
I20260709 12:23:55.368927 13650 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:23:55.370563 19907 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:55.371953 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:55.373147 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:55.374883 20164 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.002s
I20260709 12:23:55.378281 14420 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
W20260709 12:23:55.379420 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 121981us
I20260709 12:23:55.379757 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.262527 20110 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.382232 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.388159 13794 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:55.395073 14416 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:23:55.403959 13725 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:23:55.365046 20248 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:23:55.380509 20231 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.005s
I20260709 12:23:55.424022 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:55.389173 20192 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.651s	user 0.006s	sys 0.010s
I20260709 12:23:55.428551 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:55.351713 19876 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.446158 13644 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:23:55.453004 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 272159us
I20260709 12:23:55.381850 20262 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:55.455787 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.457579 13645 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:55.463997 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:55.460227 13690 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.601s (SENT)
I20260709 12:23:55.471403 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:55.372028 20023 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.395645 20164 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.463129 20014 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:55.420627 20187 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.483031 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.450331 20266 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:55.478933 13793 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:23:55.493086 20014 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:23:55.500656 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48425.9us
I20260709 12:23:55.502396 13646 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:55.502633 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.506832 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.514029 13798 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:23:55.451236 20192 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.515571 13643 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:55.447235 20231 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.501920 20014 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:23:55.537477 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 83615.3us
W20260709 12:23:55.550887 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.553521 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:55.552923 20280 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:23:55.558081 20187 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:55.572111 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:55.573130 13645 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:23:55.513245 20012 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:55.574826 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.511171 20078 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:55.592760 13643 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:23:55.554441 20138 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:55.577328 20269 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:23:55.551875 13690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.725s (SENT) [suppressed 18 similar messages]
I20260709 12:23:55.610158 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.658047 13725 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:55.659291 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 107456us
W20260709 12:23:55.587018 20221 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.802s	user 0.015s	sys 0.000s
I20260709 12:23:55.617149 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.618613 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.622670 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.592554 20240 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:55.597321 20262 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.630379 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:55.631070 20275 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:23:55.665366 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.662453 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:55.715969 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.716253 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:23:55.724174 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61891.1us
I20260709 12:23:55.726609 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:55.731565 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.732575 13719 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:55.733006 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.733430 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.733796 13726 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:55.734656 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.735481 13726 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:55.718204 20248 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.734014 13719 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:23:55.739614 20228 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.004s
I20260709 12:23:55.740981 14422 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:23:55.742219 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:55.740119 13719 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:23:55.744180 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:55.745831 20248 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:55.746186 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.592355 20187 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:55.758278 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.772239 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:55.772845 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:55.774120 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.790822 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:55.792906 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.750452 20240 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.800402 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:55.842696 20110 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.778905 20187 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.771152 20034 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:55.843418 20241 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:55.790822 20136 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:55.867218 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59384.5us
I20260709 12:23:55.760171 20221 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:23:55.874629 20016 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.696s	user 0.013s	sys 0.000s
I20260709 12:23:55.877645 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:55.888011 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.889226 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.871483 20075 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.898125 13718 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:55.898801 20078 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.900202 13720 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:23:55.900884 20260 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.627s	user 0.004s	sys 0.000s
I20260709 12:23:55.903473 20231 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.904315 20164 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.906181 20023 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:55.907445 19745 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.908769 20187 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:55.917033 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:55.928922 13801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.010s	sys 0.036s
W20260709 12:23:55.961583 13801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.010s	sys 0.036s
W20260709 12:23:55.944000 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 187618us
I20260709 12:23:55.868629 20247 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.944842 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:55.956326 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:23:55.971292 13648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.021s	sys 0.050s
I20260709 12:23:55.975703 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:55.981365 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:55.982105 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:55.982733 14416 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:55.954883 20281 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.985707 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:55.986935 13792 raft_consensus.cc:2368] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
W20260709 12:23:55.979715 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.010s	sys 0.023s
I20260709 12:23:55.984963 13648 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
W20260709 12:23:55.984439 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.960160 20110 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:55.988458 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.010s	sys 0.023s
W20260709 12:23:55.989646 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 565727us
I20260709 12:23:55.964563 20136 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:55.979071 20187 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.000581 20187 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.002874 20187 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:55.983124 19745 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:56.003726 19745 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.005940 19745 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:55.990557 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:55.989939 13648 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:55.991636 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:55.989279 20110 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.977347 20164 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:55.929711 20241 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:55.954427 20078 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:56.008275 19846 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.006s	user 0.007s	sys 0.006s
I20260709 12:23:56.010376 20281 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:56.012694 20278 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:23:56.014415 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:56.015580 20011 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.747s	user 0.012s	sys 0.004s
I20260709 12:23:56.018733 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.058314 13724 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.020217 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.018646 13645 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:56.020633 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:55.983976 20247 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:55.948516 20221 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.027890 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.028790 14421 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:23:56.034418 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:56.037050 13801 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:56.066362 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:56.066939 13647 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:56.037050 14416 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
W20260709 12:23:56.039332 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.040432 20110 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:56.041185 19967 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:55.978454 19884 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:56.052966 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65406.1us
I20260709 12:23:56.056914 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:23:56.036525 20013 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.007s
I20260709 12:23:56.061266 13720 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 10.
I20260709 12:23:56.075094 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:56.032946 20012 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:56.063120 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:56.034554 20192 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:56.035416 20164 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:56.051123 20261 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.069072 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.070564 13792 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:56.073410 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.056774 19745 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.083099 19968 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.012s	sys 0.005s
I20260709 12:23:56.078086 19967 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.083680 20231 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:56.101684 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:56.102304 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.085495 13718 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.103503 13792 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:23:56.094600 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:56.105365 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:56.091405 13719 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:23:56.100128 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:56.105373 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.103459 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.111459 13724 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.105527 13800 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 10.
I20260709 12:23:56.113050 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.114418 19720 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:56.100893 20075 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.115777 20247 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.116060 20281 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.114533 20240 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.120064 13719 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:23:56.116967 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.072675 20016 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:23:56.061841 20011 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:56.098867 19968 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.013s	sys 0.005s
I20260709 12:23:56.128784 13725 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:23:56.129558 13718 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:23:56.100901 19967 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:56.090764 20164 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:56.130858 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.201798 13724 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
W20260709 12:23:56.203217 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 131494us
I20260709 12:23:56.205534 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.101267 20231 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:56.083866 20293 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:56.208210 13801 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:56.210623 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:56.180378 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.123426 19720 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:56.117067 20262 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:56.214219 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.218572 13646 raft_consensus.cc:2368] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:23:56.220286 13800 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:56.224360 20231 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.216624 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:56.226152 13792 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:56.226822 19720 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.229887 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.230759 13724 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.225507 13800 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:56.231817 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:56.232379 13724 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:56.098841 19745 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:56.212008 20014 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.000s
I20260709 12:23:56.236125 19745 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:56.237102 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.239147 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1026) took 1054 ms. Trace:
I20260709 12:23:56.239670 14422 rpcz_store.cc:276] 0709 12:23:55.184411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.223558 (+ 39147us) service_pool.cc:224] Handling call
0709 12:23:55.251497 (+ 27939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.741469 (+489972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.238840 (+497371us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54006be4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:56.239079 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124144768}
I20260709 12:23:56.243216 20078 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.246431 20235 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.247035 20235 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:56.248486 19907 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.003s
I20260709 12:23:56.249238 20235 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.249008 19907 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.250381 13651 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:56.250928 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3180) took 1121 ms. Trace:
I20260709 12:23:56.251354 13727 rpcz_store.cc:276] 0709 12:23:55.130100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.130335 (+   235us) service_pool.cc:224] Handling call
0709 12:23:55.142248 (+ 11913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.367700 (+225452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.250635 (+882935us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54001f0754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:56.250864 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449855616}
I20260709 12:23:56.252889 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.252954 19846 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:56.253616 19846 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.255581 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.256232 19846 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:56.256907 20192 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.257365 20192 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.256249 13795 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:23:56.264097 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:56.264452 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:56.264879 13792 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:23:56.213702 13725 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18. Re-sending same reply.
I20260709 12:23:56.267478 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:56.268159 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.025s	sys 0.003s
W20260709 12:23:56.271137 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.026s	sys 0.003s
I20260709 12:23:56.271445 13718 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:23:56.271904 13718 pending_rounds.cc:85] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 6
I20260709 12:23:56.272204 13718 pending_rounds.cc:107] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:23:56.084666 20013 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.268971 13799 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:23:56.254125 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:56.275542 13727 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:23:56.272490 13718 raft_consensus.cc:2889] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:56.276957 13725 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.278530 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.279938 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:56.209290 20293 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.302953 14422 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:23:56.303678 13795 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 12:23:56.301234 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:56.282757 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:56.101856 20012 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.309449 14424 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:56.307835 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:56.310415 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.298215 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.091044 20261 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.275688 20078 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.315573 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.324635 13719 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:23:56.315646 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:56.327203 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:56.333302 14419 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:56.277896 20294 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.338232 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55544.6us
I20260709 12:23:56.324004 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.332653 13795 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:56.305953 20117 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:23:56.282224 20016 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:23:56.354272 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:56.307376 20293 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:56.373754 13718 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.372301 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:56.362056 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.375787 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.376500 13792 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:23:56.236654 20014 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.376573 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:56.379935 13719 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:56.382565 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.384557 13725 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:56.281299 19745 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:23:56.276175 20262 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.387172 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:56.315927 20023 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:23:56.358660 20117 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.393101 13652 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:56.328159 20187 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 12:23:56.400646 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70432.3us
I20260709 12:23:56.400919 14417 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:56.347780 20284 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:23:56.404129 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.408248 13792 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:23:56.386128 20192 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.342227 20078 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.405129 13719 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:56.409017 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:56.418082 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:56.418875 13719 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:56.420203 13719 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:56.406760 20234 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.128s	user 0.011s	sys 0.007s
I20260709 12:23:56.416817 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:56.422022 13719 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:56.427870 13792 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:56.428668 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:56.428826 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.406214 20117 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:56.432804 14419 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:56.406530 20301 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:23:56.447765 13718 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:23:56.448349 13718 pending_rounds.cc:85] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:23:56.448585 13718 pending_rounds.cc:107] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:23:56.409710 20284 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:56.381412 20145 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:23:56.448850 13718 raft_consensus.cc:2889] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:56.459759 13792 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:23:56.441601 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:56.417793 20241 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.452143 13725 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:56.354238 20294 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.434963 20034 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:23:56.487808 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.488718 20014 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.468027 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1043) took 1015 ms. Trace:
I20260709 12:23:56.489454 20241 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.489620 20011 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:56.492388 20265 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.210s	user 0.004s	sys 0.000s
I20260709 12:23:56.489456 13800 rpcz_store.cc:276] 0709 12:23:55.448648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.231810 (+783162us) service_pool.cc:224] Handling call
0709 12:23:56.258660 (+ 26850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.258919 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:23:56.259227 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 12:23:56.463530 (+204303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.463596 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:23:56.463646 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.463721 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.463855 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10534,"prepare.queue_time_us":15510,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":41,"raft.queue_time_us":15159,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":21022,"thread_start_us":191071,"threads_started":1}
I20260709 12:23:56.454906 20307 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:56.493706 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1038) took 1388 ms. Trace:
I20260709 12:23:56.493829 20307 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.494459 13800 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:23:56.495091 20261 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.494073 14418 rpcz_store.cc:276] 0709 12:23:55.105605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.105800 (+   195us) service_pool.cc:224] Handling call
0709 12:23:55.123180 (+ 17380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:55.127480 (+  4300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.127505 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:55.142771 (+ 15266us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.237765 (+ 94994us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:23:55.239055 (+  1290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.239139 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:56.491488 (+1252349us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:56.493352 (+  1864us) raft_consensus.cc:1685] finished
0709 12:23:56.493373 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.493620 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9345,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":12002,"spinlock_wait_cycles":8821504,"thread_start_us":94475,"threads_started":1,"wal-append.queue_time_us":139405}
I20260709 12:23:56.469349 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:56.473006 20247 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:23:56.479321 20297 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:23:56.501686 20223 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.502195 20223 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:56.502911 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3568) took 1253 ms. Trace:
I20260709 12:23:56.503297 13797 rpcz_store.cc:276] 0709 12:23:55.249362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.250524 (+  1162us) service_pool.cc:224] Handling call
0709 12:23:55.913360 (+662836us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:23:55.913629 (+   269us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:23:55.914189 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.914222 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:23:55.948083 (+ 33861us) log.cc:844] Serialized 52 byte log entry
0709 12:23:56.276304 (+328221us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:56.276355 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.276433 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:56.502574 (+226141us) raft_consensus.cc:1685] finished
0709 12:23:56.502596 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.502842 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":546,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"spinlock_wait_cycles":13568384,"thread_start_us":296924,"threads_started":1,"wal-append.queue_time_us":386660}
I20260709 12:23:56.505757 13693 leader_election.cc:400] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:56.435097 20234 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.490006 20241 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:56.483301 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 71675.6us
W20260709 12:23:56.497529 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3571) took 1247 ms (client timeout 1602 ms). Trace:
I20260709 12:23:56.498025 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3569) took 1248 ms. Trace:
W20260709 12:23:56.494824 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.461181 20301 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:56.500500 13724 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:56.500995 20247 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:56.488734 13725 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
W20260709 12:23:56.517416 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75954.8us
I20260709 12:23:56.526047 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:56.506109 13693 leader_election.cc:403] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:23:56.498571 14423 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:23:56.526410 13794 rpcz_store.cc:269] 0709 12:23:55.250312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.254891 (+  4579us) service_pool.cc:224] Handling call
0709 12:23:56.497316 (+1242425us) spinlock_profiling.cc:243] Waited 81.5 ms on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:56.497434 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187493120}
I20260709 12:23:56.527096 13798 rpcz_store.cc:276] 0709 12:23:55.249698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.249830 (+   132us) service_pool.cc:224] Handling call
0709 12:23:55.263536 (+ 13706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.516157 (+252621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.497778 (+981621us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001ca7d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:56.497945 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553414016}
I20260709 12:23:56.456595 20284 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:56.535334 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:56.555722 13646 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:56.535072 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:23:56.557389 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:56.536513 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:23:56.544559 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:56.545408 13719 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:23:56.559967 13719 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:23:56.560248 13719 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:23:56.560631 13719 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:56.487828 20078 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.551244 13800 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:56.538162 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:56.567572 13727 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:23:56.567665 13725 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:56.550892 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:56.565475 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.565975 13794 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:23:56.507664 20262 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.511045 20307 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:56.530779 20227 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.649928 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 88293.2us
I20260709 12:23:56.497076 20164 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.550592 20110 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:56.544876 20014 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:56.651234 13798 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:23:56.557596 20145 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:56.564338 20231 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:56.660267 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:56.675103 13727 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:23:56.641628 20078 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:56.669646 20227 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.693396 20231 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.693957 20305 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:23:56.694461 20110 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:56.695398 20227 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.695905 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:56.703747 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.535s (SENT)
I20260709 12:23:56.704514 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:56.714092 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.673650 20014 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.679687 19968 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.667819 20259 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:23:56.722812 20262 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:56.739311 20014 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:56.659422 20247 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.535840 20301 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.679984 20145 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.694046 20231 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.753841 20309 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:56.801914 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 257431us
I20260709 12:23:56.776503 19884 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:56.757068 19968 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:56.822491 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 126650us
I20260709 12:23:56.817333 20110 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:56.820437 20234 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.827816 20110 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.814666 20192 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:23:56.835472 13693 leader_election.cc:400] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:56.835907 13693 leader_election.cc:403] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:23:56.816329 19884 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.840433 19884 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:56.843262 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:56.845203 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:56.848071 20240 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:23:56.848342 20078 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:23:56.849197 20321 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:23:56.850682 20311 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:23:56.809367 20309 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.855183 20309 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.857929 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2429) took 1237 ms. Trace:
I20260709 12:23:56.858554 13801 rpcz_store.cc:276] 0709 12:23:55.620313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.079542 (+459229us) service_pool.cc:224] Handling call
0709 12:23:56.857752 (+778210us) spinlock_profiling.cc:243] Waited 82.8 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:56.857860 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190434048}
W20260709 12:23:56.861258 20291 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.822s	user 0.001s	sys 0.001s
I20260709 12:23:56.858057 19968 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:56.866317 20117 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:56.870211 20117 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.873428 20245 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:56.874116 20245 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.874869 20145 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.876442 20245 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:56.890712 20310 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
W20260709 12:23:56.893319 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48207.9us
W20260709 12:23:56.896224 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66604us
I20260709 12:23:56.823030 20164 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:56.898845 13646 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:56.835086 20192 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:56.910602 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.907413 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:56.836174 20241 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:56.924587 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:56.948670 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.956910 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:56.897331 20023 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:56.960260 14423 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:56.960664 14418 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:56.957731 13801 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:23:56.911917 20192 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:23:56.964110 20023 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:56.985730 20023 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:23:56.987943 20286 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.916s	user 0.001s	sys 0.002s
I20260709 12:23:57.033975 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:56.999235 20231 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.008512 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.011651 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1043) took 1142 ms. Trace:
I20260709 12:23:57.012424 20241 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:23:57.013123 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 112998us
I20260709 12:23:57.013005 20322 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:23:57.001953 20223 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.015079 20307 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.015650 20220 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:57.020494 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2427) took 1518 ms (client timeout 1641 ms). Trace:
I20260709 12:23:57.019964 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.065692 13646 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
W20260709 12:23:57.021035 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.066020 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.057070 14417 rpcz_store.cc:276] 0709 12:23:55.869595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.869790 (+   195us) service_pool.cc:224] Handling call
0709 12:23:55.870161 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.404698 (+534537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.011330 (+606632us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54005965d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:57.011577 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220077952}
I20260709 12:23:57.066793 13648 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:57.034090 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.035562 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.062438 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.067970 13645 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:56.998526 20012 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:57.052325 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.695s (SENT)
I20260709 12:23:57.057675 20314 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:23:57.058245 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:57.026731 14418 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:23:57.056555 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:23:57.064788 13799 rpcz_store.cc:269] 0709 12:23:55.502540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.743848 (+241308us) service_pool.cc:224] Handling call
0709 12:23:57.020312 (+1276464us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:57.020430 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715305344}
I20260709 12:23:57.069972 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.071280 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.072638 13644 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:57.077332 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.080789 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.082369 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.059587 20241 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.085613 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.091789 13724 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:57.092572 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.093039 13693 leader_election.cc:400] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:23:57.093359 13693 leader_election.cc:403] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:57.095042 13718 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:23:57.098801 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.097712 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3190) took 1810 ms. Trace:
I20260709 12:23:57.079952 20247 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:57.100071 13651 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:57.100697 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.099575 13720 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:23:57.082691 20136 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:57.108412 13693 leader_election.cc:400] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:23:57.108888 13693 leader_election.cc:403] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:57.110137 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.094507 20241 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.111470 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.116947 13718 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:57.113121 13799 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:23:57.096619 20012 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:57.119793 20231 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:23:57.119891 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.120590 20231 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:57.098779 20331 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:23:57.125113 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.082993 20322 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:23:57.098711 20138 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:23:57.129993 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77720.5us
W20260709 12:23:57.133191 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.983s (SENT)
I20260709 12:23:57.133626 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.133761 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.136035 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.141027 20307 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.142194 20284 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.144532 20284 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.146311 20223 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.147511 20014 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:57.111465 20293 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.714s	user 0.007s	sys 0.007s
I20260709 12:23:57.156895 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.104748 20321 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:23:57.163918 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.167686 13727 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:23:57.100047 13723 rpcz_store.cc:276] 0709 12:23:55.294618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.294827 (+   209us) service_pool.cc:224] Handling call
0709 12:23:55.313006 (+ 18179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:55.313254 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:23:55.316484 (+  3230us) log.cc:844] Serialized 14 byte log entry
0709 12:23:56.892994 (+1576510us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b180038cec0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:23:57.097348 (+204354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.097407 (+    59us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:23:57.097455 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.097540 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.097652 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3589,"prepare.queue_time_us":5463,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":4035,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":2129,"spinlock_wait_cycles":48889216,"thread_start_us":181651,"threads_started":1}
I20260709 12:23:57.169963 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.172286 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.150131 20014 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.173795 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.174754 13649 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 11: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:23:57.176546 20289 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.134s	user 0.004s	sys 0.000s
I20260709 12:23:57.178583 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.179486 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.125532 20078 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:23:57.171432 20012 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:23:57.167528 20293 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.176250 20268 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:23:57.194180 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.195220 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3193) took 1815 ms. Trace:
I20260709 12:23:57.208194 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.212052 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.121912 20231 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.212842 13800 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:23:57.214901 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.202512 13722 rpcz_store.cc:276] 0709 12:23:55.387211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.419367 (+ 32156us) service_pool.cc:224] Handling call
0709 12:23:55.449184 (+ 29817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:55.455750 (+  6566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.455792 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:55.501275 (+ 45483us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.952144 (+450869us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:23:55.952185 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.952249 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.175831 (+1223582us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:57.191392 (+ 15561us) raft_consensus.cc:1685] finished
0709 12:23:57.191416 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.195119 (+  3703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14264064,"thread_start_us":368526,"threads_started":1,"wal-append.queue_time_us":518638}
I20260709 12:23:57.225935 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.226876 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.233908 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.237226 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.238459 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:23:57.242007 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.216449 20164 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:23:57.244570 20136 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.245170 20136 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.245582 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.247756 13792 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:23:57.252468 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.254609 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:57.256213 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.257162 13800 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:23:57.213959 20012 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:23:57.254492 20117 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:57.201924 20270 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:23:57.258473 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.268106 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.213207 20301 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:23:57.268360 13721 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:57.271059 13727 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:23:57.259140 20164 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:23:57.240477 20192 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.255719 20240 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:57.276881 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:57.282373 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.262609 20078 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:57.276705 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.283609 13727 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:57.291188 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:57.293569 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.293593 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.298194 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.303404 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.310122 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.284695 20182 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:23:57.347509 13727 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:57.348779 20182 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:57.295969 20187 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:23:57.380970 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.381856 13727 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:57.356897 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.382670 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.388178 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.429466 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.456284 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:57.459800 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148818us
I20260709 12:23:57.473717 20246 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:57.304949 20284 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:57.383801 20329 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:23:57.511770 13793 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2422) took 2234 ms (client timeout 2564 ms). Trace:
I20260709 12:23:57.511904 20329 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:57.512439 20329 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:23:57.512830 20342 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:23:57.295794 20117 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:23:57.484927 20345 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:23:57.485424 20138 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:57.486737 20341 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 12:23:57.487758 20340 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:23:57.488436 20136 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:57.488969 20344 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:23:57.489255 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 212447us
I20260709 12:23:57.489809 19907 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:57.490298 20336 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:23:57.494413 20347 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:57.545590 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.497620 13727 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:23:57.546715 13652 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.506465 20323 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:57.430548 20182 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.508253 20293 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.509263 20241 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.509733 20013 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.510305 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:57.512250 13793 rpcz_store.cc:269] 0709 12:23:55.264677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.372818 (+108141us) service_pool.cc:224] Handling call
0709 12:23:57.499248 (+2126430us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x7b54002cc4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:57.499380 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224913280}
W20260709 12:23:57.521132 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 172526us
I20260709 12:23:57.523599 14424 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:57.615625 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.615761 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.617548 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.617827 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.377995 20240 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:57.525235 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.624089 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.625584 13792 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 10: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:57.525843 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.309813 20023 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:23:57.531435 13797 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:23:57.532539 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 264500us
I20260709 12:23:57.349336 20229 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.533846 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.535812 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.537571 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.527827 20138 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:57.533859 20012 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:23:57.549001 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:57.555768 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.557732 19876 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.565613 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.576783 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.580600 20228 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:23:57.592222 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:57.595971 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.597065 13801 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:57.599115 20164 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:23:57.610097 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:57.618793 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83570.4us
I20260709 12:23:57.619769 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.527848 20270 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:23:57.634832 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.627017 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.635398 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.648730 13793 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:57.627799 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.629011 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:57.639149 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90286.6us
I20260709 12:23:57.639425 13720 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:57.650617 13645 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:57.631387 13798 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:57.631481 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:57.641290 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:57.640254 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2420) took 2391 ms (client timeout 1589 ms). Trace:
I20260709 12:23:57.652781 13643 raft_consensus.cc:2368] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12.
I20260709 12:23:57.652283 13792 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 13: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:57.642761 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:57.653962 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:57.640574 13724 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:23:57.640964 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.655747 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.641429 13800 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:57.652962 13796 rpcz_store.cc:269] 0709 12:23:55.248369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.248535 (+   166us) service_pool.cc:224] Handling call
0709 12:23:57.565393 (+2316858us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:23:57.640175 (+ 74782us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6436736}
I20260709 12:23:57.643170 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.647856 20013 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.662757 20013 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.648452 20182 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:57.667289 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:57.667513 20013 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.668112 20078 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:57.668684 13693 leader_election.cc:400] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:23:57.669589 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.610742 20323 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:57.630676 20345 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:23:57.679419 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:57.633976 20285 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.485s	user 0.009s	sys 0.003s
I20260709 12:23:57.631098 20023 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.687146 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:57.688222 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.640447 20241 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.639619 20247 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:23:57.691476 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64604.9us
I20260709 12:23:57.693089 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.640733 20228 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:57.694964 13724 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:57.664764 20295 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:23:57.684490 20304 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.214s	user 0.009s	sys 0.000s
I20260709 12:23:57.699945 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.689808 20023 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:57.707472 20329 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.633885 20240 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.689894 20284 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:23:57.713892 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.664321 19876 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:23:57.648518 20229 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.696753 20328 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:57.715176 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:23:57.722561 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 198370us
I20260709 12:23:57.675177 20012 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:23:57.731361 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:57.699699 20242 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.711747 20247 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:57.733381 13720 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:23:57.710877 20241 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:57.730751 20328 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.736799 20242 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.744652 20328 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:57.736068 20117 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:23:57.733136 20350 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:23:57.748292 20350 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:57.748719 20350 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.734095 13724 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:57.698230 20295 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:57.734951 13652 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:57.716881 19903 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:23:57.758688 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.759641 13719 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:57.760761 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.761503 13719 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:23:57.764896 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.765728 13724 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:23:57.766664 19903 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:23:57.733534 20284 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:57.768785 20284 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.738852 20228 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.771488 20228 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:57.749315 20268 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:23:57.740972 20138 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.720963 20301 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:23:57.736824 20294 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.754002 20295 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:57.815958 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 280945us
I20260709 12:23:57.736821 20323 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.821370 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.824102 13722 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 18 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:23:57.822818 20242 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.833853 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.840330 20242 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:57.777709 20268 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:23:57.738327 20229 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.843354 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:57.856946 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.860062 20242 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:23:57.813238 20034 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:57.861426 20034 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.863317 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.863750 20034 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:57.866526 20034 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:57.871608 20240 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:57.844477 20138 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:57.873791 14425 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:23:57.875815 20247 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:57.878428 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.879004 20292 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:57.885733 20292 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:57.886900 20292 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.878816 20295 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.889499 20331 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:57.890044 20331 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:57.891654 20295 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:57.899183 20329 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:57.904119 20296 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.862s	user 0.009s	sys 0.000s
I20260709 12:23:57.914947 20192 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:57.915246 20350 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:57.917663 20350 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:57.922032 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:57.924707 14421 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:57.879937 20355 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:57.880434 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:57.880905 20328 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.845692 20293 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:57.934072 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:57.949920 13725 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:57.935305 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:23:57.943624 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65414.8us
I20260709 12:23:57.951594 13727 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:57.948625 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3253) took 2272 ms. Trace:
I20260709 12:23:57.943783 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:57.971179 13649 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:23:57.974929 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.976931 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.920866 20023 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:57.952965 13726 rpcz_store.cc:276] 0709 12:23:55.666547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.737346 (+ 70799us) service_pool.cc:224] Handling call
0709 12:23:55.737655 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:55.755224 (+ 17569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:55.755272 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:55.757194 (+  1922us) log.cc:844] Serialized 26 byte log entry
0709 12:23:55.978359 (+221165us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:23:55.980516 (+  2157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:55.980634 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:57.553160 (+1572526us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:23:57.903673 (+350513us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b2400166a18. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b4609d 00007f539377810e 00007f539378f88c 00007f539378ebb6 00007f539378e7d8 00007f5393798221
0709 12:23:57.938927 (+ 35254us) raft_consensus.cc:1685] finished
0709 12:23:57.938963 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.939152 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17017,"raft.queue_time_us":17399,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":6672,"spinlock_wait_cycles":136509184,"thread_start_us":213413,"threads_started":1,"wal-append.queue_time_us":277200}
W20260709 12:23:57.988158 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 252231us
I20260709 12:23:57.991312 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:57.942960 20351 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:57.970376 20329 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:23:57.995957 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 262028us
I20260709 12:23:58.008386 13727 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:23:58.013877 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 344333us
I20260709 12:23:58.012439 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:57.951462 20350 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:23:58.053324 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:23:58.057008 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44661.8us
I20260709 12:23:58.062896 13800 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:23:57.970732 20322 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:23:57.990643 20300 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:58.070473 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:57.995025 20356 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:57.987784 20138 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 12:23:58.055363 20329 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.009037 20117 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:57.994302 20295 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:23:58.048832 20323 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:58.059818 19876 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:23:58.086702 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.962769 20355 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.087980 13725 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:23:58.049077 20012 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:23:58.095110 13795 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:23:58.010027 20220 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:23:58.065271 20255 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.987329 20240 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.083s
I20260709 12:23:58.104727 13650 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:58.102679 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:23:58.111511 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58304.4us
I20260709 12:23:58.050447 20351 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.082024 20034 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:58.133216 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.131448 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:58.134953 13651 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:58.136904 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:58.138839 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.090533 20329 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:58.083411 20182 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
W20260709 12:23:58.115008 20227 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.419s	user 0.006s	sys 0.023s
I20260709 12:23:58.113942 20368 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.115464 20138 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:58.134385 20322 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:23:58.161098 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.167271 13643 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:58.169103 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.162640 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:58.185144 13646 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
W20260709 12:23:58.188411 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 224255us
I20260709 12:23:58.139624 20294 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:58.175976 20138 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:58.201246 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:58.161646 20352 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:23:58.162313 20358 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:23:58.183836 20368 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.174177 20293 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:58.139700 20034 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.184054 20347 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:58.182026 20227 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.462s	user 0.007s	sys 0.023s
W20260709 12:23:58.208277 20354 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:23:58.174561 20362 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:58.234745 20138 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.236322 20288 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.236946 19069 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:23:58.240010 20368 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.128908 20255 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.230636 20293 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:58.233628 20294 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:58.264298 13798 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:23:58.261525 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50748.5us
I20260709 12:23:58.159140 20117 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.269367 14421 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:23:58.253760 20328 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:58.289764 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.291190 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.252736 20013 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:23:58.306223 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.307152 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:58.310173 13798 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:58.314529 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.258118 19907 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:23:58.316322 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.320392 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:58.306206 19720 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:58.266645 20369 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:58.273777 20138 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:58.299840 20023 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:58.328872 13798 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:58.332027 19720 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.338418 20034 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:58.343123 20355 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:58.319923 19907 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:58.349833 20241 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.004s
I20260709 12:23:58.350594 20240 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:58.351023 20351 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:58.351501 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82063.8us
I20260709 12:23:58.258038 20288 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.270872 20182 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:23:58.360710 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:58.372699 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.384835 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.386025 13725 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:58.387236 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:58.394768 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2442) took 1262 ms. Trace:
I20260709 12:23:58.396133 20182 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:58.396559 20182 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:58.361598 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 131557us
I20260709 12:23:58.397454 20182 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.399668 19720 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:58.361604 20241 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:58.411511 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:58.382397 13798 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:58.383483 19069 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:58.395262 13801 rpcz_store.cc:276] 0709 12:23:57.132857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.160170 (+ 27313us) service_pool.cc:224] Handling call
0709 12:23:58.394505 (+1234335us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:58.394690 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958038400}
I20260709 12:23:58.407248 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.401486 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:58.406724 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.413789 13721 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:58.418038 13725 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:23:58.383965 20240 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:58.363066 20322 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:58.423352 14416 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:23:58.425300 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.426692 19069 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.434857 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:58.361791 20377 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:23:58.439574 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:58.437538 20241 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:23:58.421002 20351 raft_consensus.cc:2764] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:23:58.457419 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:23:58.427990 20378 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.500566 20378 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.505316 20378 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:58.460700 20227 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:58.506453 20227 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:58.507731 20227 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.461555 20240 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.508767 20361 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:58.509107 20227 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:58.509348 20361 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:23:58.510540 20361 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.511834 20361 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:58.516635 20300 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:23:58.508431 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73513.6us
I20260709 12:23:58.520615 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:58.523798 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:23:58.415552 19907 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.475541 20110 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:58.437852 20295 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:58.479326 20322 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.422629 20355 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:23:58.549738 20023 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:23:58.448814 20220 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:23:58.488739 20255 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.552080 20255 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.552218 20023 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:58.552634 20255 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.487605 20288 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.421789 20376 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:23:58.555136 20376 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:23:58.555481 20376 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:58.556269 20376 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.553130 20295 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.557504 20376 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:58.492501 20331 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.002s
I20260709 12:23:58.558598 20331 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.559371 20292 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:23:58.559723 14424 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:23:58.560734 20331 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.433323 19720 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:23:58.432744 20136 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:23:58.561918 20136 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.562343 20136 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:58.566617 20288 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:58.567201 20288 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.505211 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.527837 13721 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:23:58.405270 20294 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.529762 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.527053 19903 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.450284 20365 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:58.531392 20110 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.531421 20382 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:58.440984 20247 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:23:58.585553 20365 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:58.586918 20365 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:23:58.590783 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61139us
I20260709 12:23:58.591650 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:23:58.593966 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.549494 19907 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:58.535838 20078 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:58.598429 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.609980 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.590783 19903 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.613775 19903 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.614600 20034 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:23:58.596982 20300 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:23:58.618644 20300 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:58.619163 20300 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:23:58.596688 20187 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:58.611596 13651 raft_consensus.cc:2376] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21. Re-sending same reply.
I20260709 12:23:58.585253 20247 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:58.595026 20288 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.626897 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.617620 20136 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.623306 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.634218 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.635947 13725 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:23:58.636722 13723 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:23:58.619891 20187 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.639755 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.640678 13721 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:58.639669 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.642617 13718 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:58.622407 20247 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:23:58.648753 13725 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:23:58.650195 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.652076 13720 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:58.625053 20034 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:58.584888 20329 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.631904 19876 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:23:58.656955 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.653707 20328 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.659495 20328 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.660797 20231 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:58.661458 20231 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.663591 20231 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.596535 20023 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.641287 20187 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:58.595372 20382 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.671288 13721 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:23:58.671581 13723 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:23:58.672663 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.610170 20293 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.674330 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.674981 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.678633 13721 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:58.680562 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.647840 20356 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:58.684481 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.647477 20255 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.688060 13726 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 11: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:58.689971 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.691488 13718 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 12:23:58.691370 13727 raft_consensus.cc:2376] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15. Re-sending same reply.
I20260709 12:23:58.698462 13724 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:23:58.699707 13794 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:23:58.704952 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.692073 20231 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:58.710770 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.711863 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:58.713814 13644 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:23:58.715426 20034 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:58.719151 20396 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:23:58.719378 14423 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:58.723397 20110 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:58.726500 20382 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:58.731243 20136 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:58.731560 20293 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:58.733752 20364 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:23:58.734673 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2430) took 1109 ms. Trace:
I20260709 12:23:58.803751 20364 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:58.803872 13649 rpcz_store.cc:276] 0709 12:23:57.625062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.627315 (+  2253us) service_pool.cc:224] Handling call
0709 12:23:58.734388 (+1107073us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:58.734565 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710229760}
I20260709 12:23:58.696395 20356 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:58.805181 20364 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.806401 20356 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:58.739526 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.807912 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:58.712756 20295 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:58.808776 13799 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:58.706120 20255 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.809410 20255 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.742990 20231 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:58.746701 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.811776 13718 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22. Re-sending same reply.
I20260709 12:23:58.747578 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.812671 13799 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:23:58.813097 13724 raft_consensus.cc:2368] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:58.748510 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.814349 13647 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:23:58.749640 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1031) took 1158 ms. Trace:
I20260709 12:23:58.815083 14425 rpcz_store.cc:276] 0709 12:23:57.591429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.591829 (+   400us) service_pool.cc:224] Handling call
0709 12:23:58.749385 (+1157556us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:23:58.749528 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233317632}
I20260709 12:23:58.751638 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.816339 13727 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22. Re-sending same reply.
W20260709 12:23:58.850283 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:58.753950 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.755909 20301 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:58.758692 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.844825 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:58.862864 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:23:58.759737 13652 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:23:58.771888 20282 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:23:58.772599 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60846.1us
I20260709 12:23:58.871981 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:58.873332 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:58.773356 20221 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:58.775698 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:58.789793 20379 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.903455 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:58.794049 14419 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:23:58.796424 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:58.797277 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.799481 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:58.801198 20247 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.802404 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:58.912050 13792 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:58.865116 13652 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:58.785975 20192 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:58.875299 14420 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:23:58.877573 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:58.878552 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 236660us
I20260709 12:23:58.883833 19069 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:58.887899 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:23:58.886726 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:58.866389 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.928309 13718 raft_consensus.cc:2410] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 12:23:58.928484 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:58.915987 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44455.6us
I20260709 12:23:58.922076 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.930080 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:58.929414 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:58.872174 20182 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:23:58.948786 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60969.8us
I20260709 12:23:58.870811 20282 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.932490 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.945283 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:58.959331 14416 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:23:58.960597 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:58.914958 13652 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.961778 13652 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:58.962070 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.951416 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:58.962903 13720 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:58.964293 20268 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:23:58.964784 20268 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.965292 20268 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:23:58.966485 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:58.966993 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.968410 20382 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:58.969019 20382 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:58.965306 20408 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:23:58.972259 20395 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:23:58.972918 20395 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:58.973326 20395 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:58.974180 14422 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:23:58.911571 20326 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:58.961238 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:23:58.921546 20385 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.871814 20220 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:23:59.004756 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:59.005846 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1033) took 1084 ms. Trace:
I20260709 12:23:58.902442 20331 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 12:23:59.012173 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.959710 20282 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:59.009428 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:58.993741 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:58.943202 20383 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:58.884047 20301 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.019797 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.981331 20262 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:23:59.037200 13648 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 10.
I20260709 12:23:59.006318 14424 rpcz_store.cc:276] 0709 12:23:57.920999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.950615 (+ 29616us) service_pool.cc:224] Handling call
0709 12:23:57.950985 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:58.560162 (+609177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.005594 (+445432us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54006be4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:23:59.005785 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1399424896}
I20260709 12:23:59.027657 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:58.944862 20192 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:58.996109 20284 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:23:58.925575 20379 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.000937 20361 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:23:59.019745 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.060340 13792 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:23:59.001221 20242 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:23:59.050967 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.063016 13727 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:23:59.063985 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:59.053954 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.055766 13800 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:23:59.018605 13726 raft_consensus.cc:2376] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15. Re-sending same reply.
I20260709 12:23:59.061722 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.066406 13725 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:23:59.067894 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.073493 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.052003 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.075028 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.075976 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.076637 13721 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:23:59.079339 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.080106 13721 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:23:59.081712 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:59.081732 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.082620 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:59.073750 13652 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 14. (term mismatch)
I20260709 12:23:59.084000 13652 pending_rounds.cc:85] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 13
I20260709 12:23:59.084280 13652 pending_rounds.cc:107] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 15.14
I20260709 12:23:59.023327 20382 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.017802 20326 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:59.063038 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.083650 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.091733 13722 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:23:59.099665 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.102102 13720 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:23:59.076691 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.092502 20284 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:23:59.103526 20391 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:23:59.082806 20281 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:23:59.103857 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.105193 13792 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:23:59.105327 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.090618 13652 raft_consensus.cc:2889] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:59.106999 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:59.018302 20385 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.111991 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:59.057410 20220 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 12:23:59.102911 20117 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.089295 20262 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:59.101939 20242 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:59.111189 20399 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:59.109117 13721 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:23:59.119899 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:59.123039 20281 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:23:59.123972 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47389.8us
I20260709 12:23:59.121542 19069 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:23:59.083303 20361 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.105911 20231 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:59.117074 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 187469us
I20260709 12:23:59.040984 20383 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:59.135700 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 249041us
I20260709 12:23:59.153698 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:59.138969 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.148757 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:59.139104 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.161151 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:59.162848 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.109709 20400 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.135540 13801 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:23:59.163918 13649 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:59.165537 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:59.166702 13649 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:23:59.166798 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:59.169354 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.170163 13649 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:23:59.171892 19903 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:23:59.177623 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.179019 13801 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:59.180271 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.182076 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.183549 13801 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:23:59.184718 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.177316 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.126122 20329 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:23:59.188041 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49167.4us
I20260709 12:23:59.133348 20075 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.148586 19069 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:59.244836 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:59.260547 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.259398 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:59.261695 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105756us
I20260709 12:23:59.150112 20328 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:59.186591 20367 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:23:59.264068 20367 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:59.146533 20281 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:59.267225 20281 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.268869 20281 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:23:59.271329 20231 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:59.254813 20407 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:59.282450 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129475us
W20260709 12:23:59.282913 20388 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:23:59.283759 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:59.285419 20413 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:23:59.287532 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.284909 20411 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:23:59.289197 20411 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.289615 20411 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:23:59.140264 20399 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:23:59.298015 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 152483us
W20260709 12:23:59.298130 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 150862us
W20260709 12:23:59.305569 13723 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.618s	user 0.014s	sys 0.006s
W20260709 12:23:59.254683 20365 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.667s	user 0.019s	sys 0.013s
W20260709 12:23:59.317991 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 140995us
I20260709 12:23:59.323201 13649 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: [18.15-20.16]
I20260709 12:23:59.337344 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:59.306210 13723 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:23:59.255261 20240 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:59.136557 20117 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.299417 20221 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:23:59.328986 20227 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.819s	user 0.021s	sys 0.006s
W20260709 12:23:59.328675 20376 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.771s	user 0.022s	sys 0.005s
I20260709 12:23:59.355037 13801 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:23:59.355633 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56429.6us
I20260709 12:23:59.254488 20292 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 12:23:59.291754 20247 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.323103 20231 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:23:59.382557 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 83238.9us
I20260709 12:23:59.383322 13647 raft_consensus.cc:2410] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:23:59.332507 20012 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:23:59.391471 13652 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
W20260709 12:23:59.399488 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67795.3us
I20260709 12:23:59.334729 20365 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.381261 20361 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.409682 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.382475 20323 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:59.411734 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 152451us
I20260709 12:23:59.312501 20326 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.421802 13617 leader_election.cc:400] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16.
I20260709 12:23:59.390745 20221 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:23:59.455122 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:59.445536 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.404337 19903 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:59.399689 20412 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:59.472956 20412 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.475838 20412 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:59.478482 20406 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 12:23:59.452560 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.450691 20255 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:23:59.459376 20410 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:23:59.463406 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.409705 20292 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:23:59.378118 20242 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.496958 20361 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:23:59.471011 19903 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.453717 20323 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.481357 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.354632 20284 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.482331 20379 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.434888 20192 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.420540 20328 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:23:59.400059 20231 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.487905 20364 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:23:59.492969 20255 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.451434 20301 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.444422 20329 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:23:59.517478 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.518463 20292 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.405210 20262 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.525965 20367 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.527542 20367 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:23:59.528829 20419 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:23:59.529331 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.529784 20385 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.530824 20383 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.502416 20421 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:23:59.401381 20399 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.501951 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.515663 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:23:59.534168 20403 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:23:59.556721 20270 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:23:59.558411 20270 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:23:59.557736 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76538.2us
I20260709 12:23:59.540884 20012 raft_consensus.cc:2764] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:23:59.559613 20231 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.541414 20351 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.544118 20383 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.561241 20365 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.561435 20383 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.559361 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.567418 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:23:59.577618 14422 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:59.576205 13797 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:23:59.564445 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:59.550514 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48652.2us
I20260709 12:23:59.559361 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.585059 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:59.586969 13693 leader_election.cc:400] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12.
I20260709 12:23:59.587538 13693 leader_election.cc:403] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:59.587402 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:23:59.549798 20187 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.880s	user 0.018s	sys 0.009s
I20260709 12:23:59.586555 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:23:59.592391 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.594163 20270 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.608193 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:23:59.578562 20351 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:59.608955 13797 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:23:59.539045 20421 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:59.609458 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.614194 20365 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:23:59.616338 20182 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.656s	user 0.014s	sys 0.000s
I20260709 12:23:59.616564 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.617524 13644 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:23:59.587011 19907 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:23:59.584072 20356 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.590160 20231 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:23:59.604594 20117 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.606045 20328 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:59.554235 20301 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:23:59.608022 20321 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:23:59.617976 20365 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:23:59.634861 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.605372 20418 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 11. Current term is 12.
I20260709 12:23:59.645191 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.605746 20377 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:23:59.621291 19907 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:59.607088 20187 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:59.641583 20300 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.022s	user 0.016s	sys 0.012s
I20260709 12:23:59.631385 20227 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:59.634477 20328 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.655475 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:23:59.602131 20023 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:23:59.641831 20270 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:23:59.649870 20301 consensus_queue.cc:1243] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:23:59.670498 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:23:59.687994 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.690626 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:23:59.664363 20268 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.001s
I20260709 12:23:59.694298 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.660382 20328 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.662676 20262 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:23:59.678730 20323 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:59.672825 20383 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.664736 20411 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.689246 20136 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:59.682601 19907 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.718808 20136 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.678800 20242 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:23:59.699200 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:59.699491 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:23:59.685539 20405 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 12:23:59.701924 13648 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:23:59.678412 20389 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.700733 20284 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:23:59.708705 20356 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:23:59.684337 20414 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 12:23:59.667371 20227 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.671181 20365 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:23:59.722455 20136 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.726752 13647 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:23:59.737912 20323 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:23:59.705487 20421 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:23:59.726096 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 156255us
I20260709 12:23:59.731123 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:23:59.727730 20262 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:23:59.735751 20284 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.723492 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.665393 20300 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.665110 19903 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.725660 20242 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:23:59.725124 20427 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:23:59.757439 13648 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:23:59.758064 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:23:59.752758 20423 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:23:59.760823 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.763108 13718 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:23:59.724530 20268 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.763859 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:23:59.785422 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.767038 13725 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:23:59.739300 20411 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.787213 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.771160 13727 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:23:59.771231 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47799.6us
I20260709 12:23:59.788599 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:59.771821 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.789588 13725 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:59.791152 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.793406 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:23:59.794358 13721 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:23:59.794405 13727 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:23:59.794945 13727 pending_rounds.cc:85] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 13
I20260709 12:23:59.795235 13727 pending_rounds.cc:107] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:23:59.795497 13727 raft_consensus.cc:2889] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:59.802148 13796 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
W20260709 12:23:59.802904 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44918.1us
W20260709 12:23:59.803746 20425 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:23:59.760874 20240 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:59.806389 20240 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:59.806980 20395 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.000s
I20260709 12:23:59.809162 20240 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.798666 20164 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:23:59.810702 20164 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:59.811124 20164 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.798868 20361 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:59.814819 20361 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.772455 20383 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.816994 20375 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:23:59.783157 20365 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:23:59.761274 20284 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:23:59.807531 20395 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.853758 13649 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:23:59.847190 20377 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:59.847678 20281 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.002s
I20260709 12:23:59.849009 20227 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:23:59.845866 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:23:59.844161 20361 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:23:59.854435 13649 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:23:59.858243 13719 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:23:59.819875 20412 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:23:59.860854 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:23:59.757014 20323 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:23:59.784138 19903 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.787678 20427 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:59.780241 20136 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.869068 13649 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:23:59.853893 20282 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.815s	user 0.009s	sys 0.005s
I20260709 12:23:59.826871 20182 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:23:59.880044 13721 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:23:59.903475 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 139694us
I20260709 12:23:59.907577 13649 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:23:59.882746 20412 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:59.917785 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:23:59.913367 13646 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:23:59.914327 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:23:59.880446 20301 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:23:59.884397 20365 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.883888 20427 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:23:59.858832 20383 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.869549 20377 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:23:59.921496 20412 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:23:59.919611 20187 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:23:59.925731 20411 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:23:59.902793 20136 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.900648 20282 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.958866 13721 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:59.967583 13723 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:23:59.968606 13643 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:23:59.902793 20182 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.936393 20228 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:23:59.937561 20301 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.882175 20242 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.957432 20012 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.993300 13646 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:23:59.938818 20365 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:23:59.950912 20326 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:23:59.944103 20331 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:23:59.989099 20361 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:00.030977 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.040370 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 114093us
I20260709 12:24:00.045564 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.072764 13725 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:00.074888 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.027243 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:00.043478 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:00.053773 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.076808 13719 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 12: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:00.054237 20136 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.057199 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:00.058212 20364 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.004s
I20260709 12:24:00.030125 20328 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.059319 20182 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:00.061108 20420 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.616s	user 0.001s	sys 0.004s
I20260709 12:24:00.062709 19903 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.064435 20361 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.002043 20164 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.066182 20282 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:00.071138 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:00.076053 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49016.8us
I20260709 12:24:00.015586 20012 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.087899 13720 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:00.090031 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.077235 20379 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.096611 13645 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:00.098227 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.105611 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.081115 13722 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:00.073912 20192 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.116773 14417 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:00.118552 20234 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:00.093860 20421 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:00.087664 20262 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.120424 13722 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:24:00.121860 13794 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:00.120424 13724 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:00.125258 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.129347 13724 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:24:00.122596 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 198719us
I20260709 12:24:00.135409 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.136025 13797 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:00.098462 20268 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.139134 13723 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:24:00.139534 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.105496 20395 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.106462 20136 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:00.149204 13720 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:00.152130 13649 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:00.172487 13797 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
W20260709 12:24:00.152681 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49713.8us
I20260709 12:24:00.161963 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:00.175176 13650 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:00.163254 20136 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.175609 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.133198 20262 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.174249 13720 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
W20260709 12:24:00.176692 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.175642 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:00.176469 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.180851 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:00.185806 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.187155 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.188462 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:00.188769 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.193084 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.180523 20227 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.133701 20234 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 12:24:00.128332 20379 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.204722 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.186014 20262 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.124466 20192 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.205370 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.210577 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3615) took 1324 ms. Trace:
I20260709 12:24:00.211221 14422 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:00.214183 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.189392 20367 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:00.211117 13793 rpcz_store.cc:276] 0709 12:23:58.885981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.886445 (+   464us) service_pool.cc:224] Handling call
0709 12:23:58.886796 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:58.887164 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:58.887194 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:58.904502 (+ 17308us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.328024 (+423522us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:23:59.328083 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.328187 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.964169 (+635982us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:00.202275 (+238106us) raft_consensus.cc:1685] finished
0709 12:24:00.202299 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.210489 (+  8190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4093,"raft.queue_time_us":17554,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":20530,"spinlock_wait_cycles":32352512,"thread_start_us":352387,"threads_started":1,"wal-append.queue_time_us":434361}
I20260709 12:24:00.228641 20136 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.230628 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.230957 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.231832 14420 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:00.234754 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.235733 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3324) took 1181 ms. Trace:
I20260709 12:24:00.236145 13726 rpcz_store.cc:276] 0709 12:23:59.053902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.069148 (+ 15246us) service_pool.cc:224] Handling call
0709 12:23:59.338976 (+269828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:59.339420 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.339457 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.340055 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:23:59.513872 (+173817us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:23:59.513926 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.514000 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:23:59.576082 (+ 62082us) raft_consensus.cc:1685] finished
0709 12:23:59.576578 (+   496us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.235311 (+658733us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400157cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:00.235576 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":962,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":14801,"spinlock_wait_cycles":619163008,"thread_start_us":113957,"threads_started":1,"wal-append.queue_time_us":167395}
I20260709 12:24:00.240144 20361 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.238538 20367 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:00.238713 13643 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:00.241178 20383 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.241619 20367 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.242405 20328 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.255057 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.255275 20383 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:00.256146 14417 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:00.256531 20383 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:00.256968 20383 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:00.257864 20383 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.259645 20383 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.260661 20301 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.260865 20284 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.262766 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3616) took 1196 ms. Trace:
I20260709 12:24:00.263237 13796 rpcz_store.cc:276] 0709 12:23:59.066013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.083180 (+ 17167us) service_pool.cc:224] Handling call
0709 12:23:59.104177 (+ 20997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.820330 (+716153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.262496 (+442166us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:00.262697 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604636032}
I20260709 12:24:00.264771 20379 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:00.265568 20075 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.272024 20075 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.195324 20448 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:00.273764 14420 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:00.267211 20412 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.268770 20300 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:00.267691 13543 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 10 to 4, term changed from 18 to 14, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65), VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66), VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67), VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 14 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } } pending_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:00.294523 20075 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:00.283118 20136 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.297353 20136 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.299760 20136 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.301453 20282 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:00.302150 20282 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.301311 13543 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:00.304838 20282 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:00.269793 20262 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.272390 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:00.273142 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.273672 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.273834 20192 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.266686 20220 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.200980 20227 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.348330 20412 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.348522 20227 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.349809 20427 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.352687 20427 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.354416 20284 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:00.197238 20281 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 12:24:00.184608 20012 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.363235 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.278641 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:00.308240 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.365823 13725 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:24:00.309860 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109381us
I20260709 12:24:00.364804 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:00.367678 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.288604 20361 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.327150 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.367100 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:00.328392 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.372529 13795 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:00.330935 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.331681 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:00.305811 13543 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 15, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:00.288832 20301 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:24:00.334864 20262 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:00.336347 20445 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:24:00.339977 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3682) took 1331 ms. Trace:
I20260709 12:24:00.380121 13644 rpcz_store.cc:276] 0709 12:23:59.008839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.009080 (+   241us) service_pool.cc:224] Handling call
0709 12:24:00.339712 (+1330632us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:00.339886 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376354560}
I20260709 12:24:00.381654 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.382946 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.341907 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:00.342582 13544 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 255) took 26363 ms (client timeout 15000 ms). Trace:
I20260709 12:24:00.383765 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.344071 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2444) took 1218 ms. Trace:
I20260709 12:24:00.384405 13645 rpcz_store.cc:276] 0709 12:23:59.125679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.163214 (+ 37535us) service_pool.cc:224] Handling call
0709 12:24:00.343865 (+1180651us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400115154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:00.343999 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135501184}
I20260709 12:24:00.325516 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.368723 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:00.369876 13650 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:24:00.371146 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92606us
I20260709 12:24:00.371407 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.330163 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.378695 13799 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:00.386323 13801 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:00.383466 13644 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:00.387686 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.388796 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.389375 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.390914 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:00.391345 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.392742 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.372097 20427 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.395201 13646 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:24:00.395280 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.374295 20227 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:00.332877 20282 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:24:00.398885 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:00.398787 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.396168 13650 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:00.400183 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.400682 13644 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:00.383057 20228 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:00.396534 13647 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:00.418908 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.363826 20220 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.402083 20164 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.464985 13796 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.440148 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.440860 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:00.472923 13795 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
W20260709 12:24:00.431849 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54552.9us
I20260709 12:24:00.413664 13799 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:00.476549 14420 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
W20260709 12:24:00.465770 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66968.7us
I20260709 12:24:00.466058 13647 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:00.438369 13648 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:00.468866 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:00.479274 13644 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:00.472808 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:00.440145 14422 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:00.406129 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.481496 13648 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:24:00.483119 14421 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:00.471766 20247 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:00.376480 20361 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.432152 20012 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:24:00.439016 19907 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:00.486449 20361 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.484118 20247 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:00.487864 20247 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.488319 20356 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:00.399633 20136 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:00.394467 20383 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:00.401489 20427 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.492817 20220 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.439538 20379 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:00.494558 20379 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:00.397328 20452 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:00.496466 20452 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.434844 20377 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.498512 20227 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:00.498971 20228 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:00.439952 20350 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:24:00.498977 20227 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:00.499958 20228 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.500586 20227 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:00.495522 20379 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.506806 13647 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:24:00.509984 20427 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.511087 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.399739 20240 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:00.509050 13647 pending_rounds.cc:85] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 8
I20260709 12:24:00.418948 20284 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:00.519773 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124448us
I20260709 12:24:00.529136 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.537585 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:00.541208 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.542057 13795 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:00.541237 13651 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:00.545377 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.521001 13647 pending_rounds.cc:107] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:24:00.514869 20361 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.548357 14417 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:00.551203 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:00.527477 20220 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:00.522290 20234 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:24:00.561471 20402 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.275s	user 0.000s	sys 0.005s
I20260709 12:24:00.565987 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:00.568542 13647 raft_consensus.cc:2889] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:00.551998 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.577977 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:00.384073 13544 rpcz_store.cc:269] 0709 12:23:33.978665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:33.981263 (+  2598us) service_pool.cc:224] Handling call
0709 12:24:00.330188 (+26348925us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:49.159402 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:49.160142 (+   740us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:49.160199 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:23:49.160217 (+    18us) tablet.cc:662] Decoding operations
0709 12:23:49.267178 (+106961us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:49.267285 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:49.267318 (+    33us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:23:49.534232 (+266914us) tablet.cc:701] Row locks acquired
0709 12:23:49.534301 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:23:49.534536 (+   235us) write_op.cc:270] Start()
0709 12:23:49.534765 (+   229us) write_op.cc:276] Timestamp: P: 1783599829534438 usec, L: 0
0709 12:23:49.534784 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:23:49.668940 (+134156us) log.cc:844] Serialized 11278 byte log entry
0709 12:23:50.120210 (+451270us) op_driver.cc:464] REPLICATION: finished
0709 12:23:50.512428 (+392218us) write_op.cc:301] APPLY: starting
0709 12:23:50.512591 (+   163us) tablet.cc:1367] starting BulkCheckPresence
0709 12:23:50.586575 (+ 73984us) tablet.cc:1370] finished BulkCheckPresence
0709 12:23:50.586716 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:23:51.399794 (+813078us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b180042d6c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:52.867108 (+1467314us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:23:52.982719 (+115611us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 12:23:52.982772 (+    53us) write_op.cc:312] APPLY: finished
0709 12:23:52.983376 (+   604us) log.cc:844] Serialized 252 byte log entry
0709 12:23:53.109284 (+125908us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:53.176849 (+ 67565us) write_op.cc:454] Released schema lock
0709 12:23:53.190770 (+ 13921us) write_op.cc:341] FINISH: Updating metrics
0709 12:23:53.692223 (+501453us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b180042d6c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:23:53.692722 (+   499us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:23:53.692871 (+   149us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2736522,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":345113,"apply.run_cpu_time_us":65871,"apply.run_wall_time_us":2762641,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125579,"num_ops":29,"prepare.queue_time_us":293063,"prepare.run_cpu_time_us":22066,"prepare.run_wall_time_us":678476,"raft.queue_time_us":182633,"raft.run_cpu_time_us":40517,"raft.run_wall_time_us":164366,"replication_time_us":585243,"spinlock_wait_cycles":38276736,"thread_start_us":657765,"threads_started":5,"wal-append.queue_time_us":450164,"wal-append.run_cpu_time_us":79000,"wal-append.run_wall_time_us":2415539}]]}
I20260709 12:24:00.579782 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.584424 14417 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:00.526600 20294 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:24:00.563783 20220 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:00.593640 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.598232 13724 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:00.580876 20448 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:00.569315 20452 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.569922 20450 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.613276 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.562089 20427 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:00.582342 20240 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:00.644377 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.582775 20192 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:24:00.562825 20361 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.677642 20361 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.571291 20164 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:24:00.660342 13647 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:24:00.644606 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:00.583225 20356 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:24:00.680464 20367 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:00.681269 20367 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:00.660687 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47504.9us
I20260709 12:24:00.582988 20328 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:24:00.583139 20451 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:24:00.636242 20457 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:00.693522 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 118415us
I20260709 12:24:00.695219 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:00.683238 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:00.680588 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:24:00.682400 20411 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:00.702690 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.703892 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.654042 20450 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:00.706357 13793 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:00.701192 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:00.661885 20240 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:00.719949 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.701184 13796 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:00.734699 13794 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:00.704339 20411 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.760632 20411 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:00.761050 20365 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.763445 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:00.772526 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89379.7us
I20260709 12:24:00.775090 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:00.776820 13793 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:00.664779 20456 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:00.781165 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:00.784746 19069 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:00.785909 19069 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:00.783923 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:00.776466 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:00.735910 20420 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:24:00.798211 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.792407 13793 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:00.704124 20136 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:00.735633 20450 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.796161 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.801905 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.858132 14422 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:00.692812 20294 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:00.819597 20247 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.779503 20282 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:00.803043 20350 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:00.782711 20457 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:24:00.830468 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.830623 19069 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.706106 20361 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 12:24:00.835866 13796 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:24:00.839126 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.739933 20240 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.842689 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.883780 14421 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:00.843875 20399 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:00.847702 13726 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:00.886590 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.848378 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.848546 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.849426 13723 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:24:00.889326 13720 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:00.850028 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.890417 13722 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:00.770112 20227 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.855239 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.891633 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:00.789232 20434 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:00.860442 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.860800 13645 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:24:00.860332 20234 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:00.863875 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67874.4us
I20260709 12:24:00.907807 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.862493 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
W20260709 12:24:00.875623 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.786237 20456 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:00.859843 20323 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:00.843484 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:00.880548 20255 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:00.889181 13727 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:00.691452 20220 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:00.861589 20294 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:00.933364 13719 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:00.900432 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.910459 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.910897 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:00.912355 13794 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:00.883157 20350 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:00.685623 20367 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:00.923386 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 117183us
W20260709 12:24:00.924494 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121467us
I20260709 12:24:00.880661 20282 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:00.937726 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:00.939388 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 236780us
I20260709 12:24:00.945109 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:00.947497 20282 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.951373 20075 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:00.952498 20023 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:00.953480 20075 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:00.955857 20227 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:00.956779 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:00.957724 20023 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:00.958158 20023 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.915355 20422 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:00.959173 20023 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:00.960705 20242 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:00.879802 20329 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:24:00.970301 13722 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:00.971692 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.972779 13794 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:00.918002 20323 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:00.974838 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:00.976504 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:00.981760 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.983119 13722 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:00.984434 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.963972 20411 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.982407 20323 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:00.991008 14417 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:24:00.992517 20428 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.993116 20428 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.994004 20262 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:00.995155 20262 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:00.995553 20262 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:00.914045 20456 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:00.996500 20262 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.997984 20262 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:01.000172 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:01.001137 13719 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:00.954106 20075 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:01.004172 20075 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.004259 20350 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.007433 20234 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.009121 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:01.010538 20464 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:01.011485 20464 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.011960 20451 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:01.012015 20464 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:01.012277 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2446) took 1112 ms. Trace:
I20260709 12:24:00.933364 20383 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:01.012741 13645 rpcz_store.cc:276] 0709 12:23:59.899760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.390620 (+490860us) service_pool.cc:224] Handling call
0709 12:24:01.012014 (+621394us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:01.012209 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069534592}
I20260709 12:24:01.013176 20383 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:00.969920 20395 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:24:01.015044 20467 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:24:01.015666 20383 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:00.977317 20377 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:24:01.016793 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:00.943687 20300 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:24:01.017011 20262 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:01.017567 13725 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:01.017509 20262 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:01.019598 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49867.4us
I20260709 12:24:01.021472 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.028841 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:01.031170 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:01.031539 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.031580 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:01.033733 20395 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:01.039434 20395 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.048400 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:01.052969 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:01.062300 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.011765 20281 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:01.067703 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:01.071214 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:00.982961 20220 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:01.066078 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:01.072870 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:01.073540 13649 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:01.074699 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:01.076202 13645 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:01.076012 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92936.7us
I20260709 12:24:01.078063 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.079221 13725 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:24:01.083225 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:01.087010 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.088501 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:01.090248 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:01.097506 13652 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:24:01.099028 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:01.108690 13650 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:01.018748 20262 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.117601 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.120558 13719 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:01.070508 20281 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:01.078225 20383 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:01.108040 19907 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:01.132952 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123945us
I20260709 12:24:01.096640 20227 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:01.139822 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.157783 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:01.161064 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77968.4us
I20260709 12:24:01.163853 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:01.076212 20422 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:24:01.080811 20413 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:24:01.134497 20301 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:01.179955 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.180814 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.096560 20255 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:24:01.134124 20361 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.188058 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.190655 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:01.141353 20281 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:01.113518 20328 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:24:01.113627 20444 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:24:01.183436 20434 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:01.206908 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.107045 20472 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:24:01.121825 20164 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 12:24:01.213268 13721 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:01.174837 20262 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:01.193259 20449 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:24:01.119017 20329 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 12:24:01.243513 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 128753us
I20260709 12:24:01.193154 20247 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:01.200183 19907 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:01.193730 20192 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:01.321730 19069 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.185971 20383 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:01.326136 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 162360us
I20260709 12:24:01.345217 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:01.180665 20464 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.322079 20427 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.339587 20227 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:01.397830 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 307670us
I20260709 12:24:01.215240 20483 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.177s
I20260709 12:24:01.398495 19069 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.217286 20361 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.370898 20479 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:01.369979 20457 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:01.377497 19907 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.409930 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:01.413179 20377 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.483515 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:01.484841 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75197.8us
I20260709 12:24:01.496095 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:01.499035 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.495493 14417 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:24:01.432737 20270 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:01.441515 19907 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.428519 20192 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:01.514986 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 126556us
I20260709 12:24:01.514447 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:01.528178 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:01.513881 19069 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.525193 20270 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.530247 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:01.531600 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:01.432545 20383 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:24:01.414486 20427 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.532018 20270 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.444439 20457 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.436544 20479 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.520757 20164 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:01.508617 20377 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.537614 20472 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:01.537694 20227 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:24:01.591528 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46551.7us
I20260709 12:24:01.560326 20270 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:01.607024 20476 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:24:01.599064 20480 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.339s	user 0.000s	sys 0.002s
I20260709 12:24:01.593520 20301 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:01.607811 20377 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:01.607815 20471 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:24:01.591365 20499 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:01.611931 20478 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 12:24:01.601178 20227 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:01.599471 20498 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.579324 20164 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:01.660497 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.622367 20484 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
W20260709 12:24:01.711117 20485 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:24:01.711935 20428 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:01.712846 20473 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:24:01.705283 20464 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:01.715126 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:01.715873 13720 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.718415 20498 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.720333 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1406) took 1130 ms. Trace:
I20260709 12:24:01.723918 13718 rpcz_store.cc:276] 0709 12:24:00.589841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.590031 (+   190us) service_pool.cc:224] Handling call
0709 12:24:00.590455 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.590982 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.591031 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.597663 (+  6632us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.060030 (+462367us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:01.060097 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.060210 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:01.720063 (+659853us) raft_consensus.cc:1685] finished
0709 12:24:01.720099 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.720255 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1070,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":3074,"thread_start_us":394538,"threads_started":1,"wal-append.queue_time_us":859933}
I20260709 12:24:01.609429 20192 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.726356 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:01.726810 20506 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:01.727196 13726 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:01.727643 20506 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.728030 20294 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.728636 20361 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.702855 20227 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:01.737040 20294 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:01.737695 20457 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.726015 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.612298 20497 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.625110 20242 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.013s
I20260709 12:24:01.617518 20472 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:01.706987 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.016s	sys 0.032s
W20260709 12:24:01.757054 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.016s	sys 0.032s
I20260709 12:24:01.659509 20377 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:01.704449 20474 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
W20260709 12:24:01.757589 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 652273us
W20260709 12:24:01.723259 20481 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:24:01.729411 20361 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.774094 13800 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:01.774129 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:01.751484 20480 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.099s	user 0.000s	sys 0.010s
I20260709 12:24:01.749106 20499 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.757532 20475 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.328s	user 0.000s	sys 0.002s
W20260709 12:24:01.752048 20490 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:24:01.776804 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.742374 20379 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:01.808410 13720 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:01.750622 20497 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.832711 20497 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.833825 20479 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.783483 20242 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:01.845005 20242 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:01.845598 20242 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:01.835237 20506 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.786394 20487 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:01.835249 20498 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:01.839150 20456 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.843s	user 0.011s	sys 0.003s
I20260709 12:24:01.840471 20427 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.779237 20377 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:01.762781 20450 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.884s	user 0.008s	sys 0.005s
I20260709 12:24:01.871503 13719 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [13.12-15.13]
W20260709 12:24:01.874598 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 343058us
I20260709 12:24:01.842732 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:01.876418 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:01.843585 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:01.818135 20499 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:01.852596 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45383.1us
I20260709 12:24:01.854652 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:01.851191 20487 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.856705 13723 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:01.857151 20452 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:24:01.858604 20023 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.899s	user 0.006s	sys 0.009s
W20260709 12:24:01.860236 20281 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.001s
W20260709 12:24:01.862411 20323 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.871s	user 0.011s	sys 0.003s
I20260709 12:24:01.863129 20136 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:01.864418 20411 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:01.865447 20240 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.962s	user 0.010s	sys 0.002s
I20260709 12:24:01.783727 20361 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:01.867132 20482 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.266s	user 0.002s	sys 0.002s
I20260709 12:24:01.842952 20164 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:01.878554 13721 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:01.901801 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.883819 20486 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.872416 20220 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.791s	user 0.009s	sys 0.005s
I20260709 12:24:01.907343 20487 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.911546 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.912967 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.913661 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:01.915151 20443 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:01.915616 20443 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:01.916628 20443 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.917783 20443 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:01.921196 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.922056 13793 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:24:01.888425 20023 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.924885 20023 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:01.926493 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:01.874912 20450 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.928974 20227 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:01.929577 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3626) took 1824 ms (client timeout 2051 ms). Trace:
I20260709 12:24:01.876087 19907 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:01.931722 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.931917 19907 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:01.933202 20500 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.213s	user 0.002s	sys 0.001s
I20260709 12:24:01.879644 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:01.934787 20284 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:24:01.934934 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55377.7us
I20260709 12:24:01.935335 20284 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:01.936251 20284 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.938457 13795 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:01.939105 13794 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:24:01.939222 13718 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:24:01.942381 20187 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:01.942667 20448 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:01.942852 20187 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:01.943151 20448 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:01.943920 20187 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.944108 20448 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.944979 20187 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.945187 20448 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:01.947001 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:01.948534 20433 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:01.949060 20433 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:01.950373 20227 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:01.950595 13645 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:24:01.951001 13645 pending_rounds.cc:85] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 9
I20260709 12:24:01.951217 13645 pending_rounds.cc:107] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:24:01.951434 13645 raft_consensus.cc:2889] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:01.952653 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:01.953642 13649 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:24:01.954556 13644 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:01.954859 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:01.957573 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:01.958662 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:01.959620 13648 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:01.888967 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:01.882342 20472 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:01.890859 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.891166 13720 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:24:01.890110 20136 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:01.895135 20510 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.878471 20479 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:24:01.981038 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:01.982004 13793 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:01.894882 20240 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.897603 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3278) took 1010 ms. Trace:
I20260709 12:24:01.898500 20361 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.892520 20323 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.888674 20281 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.899492 20377 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.879993 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:01.887262 20350 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.003278 13798 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:01.852033 20456 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:01.910643 13721 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:24:01.961143 13723 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:01.961872 13644 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:02.010532 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:01.929993 13800 rpcz_store.cc:269] 0709 12:24:00.104673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.105218 (+   545us) service_pool.cc:224] Handling call
0709 12:24:01.929380 (+1824162us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:01.929516 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3785511680}
I20260709 12:24:01.965174 14421 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:01.965530 13643 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:01.967763 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.970736 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.978195 20510 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.982224 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1084) took 1348 ms. Trace:
I20260709 12:24:01.984922 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.020642 13799 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:24:01.984671 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:01.965444 20487 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:01.984962 20220 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.997781 20240 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.998596 20379 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:01.969312 20365 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.047956 13724 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:01.999135 20385 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:02.001111 20281 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:02.001434 20479 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:01.908782 13719 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:24:01.907621 20486 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.004272 13722 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:02.001786 20136 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.095796 20136 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.096897 20486 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:01.984932 13727 rpcz_store.cc:276] 0709 12:24:00.886976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.962482 (+ 75506us) service_pool.cc:224] Handling call
0709 12:24:00.962914 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.963348 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.963379 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.963972 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.338594 (+374622us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:01.338659 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.338761 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:01.866411 (+527650us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:01.897285 (+ 30874us) raft_consensus.cc:1685] finished
0709 12:24:01.897309 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.897535 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":541,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"spinlock_wait_cycles":10851072,"thread_start_us":374120,"threads_started":1,"wal-append.queue_time_us":571362}
I20260709 12:24:02.099045 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:02.099661 13727 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:02.019474 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:02.107034 13643 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:24:02.019675 13795 rpcz_store.cc:276] 0709 12:24:00.634307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.634531 (+   224us) service_pool.cc:224] Handling call
0709 12:24:00.634986 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.938901 (+1303915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.981874 (+ 42973us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003d45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:01.982136 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2997236480}
I20260709 12:24:01.997308 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:02.111654 20501 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.393s	user 0.005s	sys 0.000s
I20260709 12:24:02.042631 19069 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:02.044764 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3354) took 1015 ms. Trace:
I20260709 12:24:02.135746 13723 rpcz_store.cc:276] 0709 12:24:01.029483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.882869 (+853386us) service_pool.cc:224] Handling call
0709 12:24:01.960938 (+ 78069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.020684 (+ 59746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.044420 (+ 23736us) spinlock_profiling.cc:243] Waited 77.7 ms on lock 0x7b54002f6550. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:02.044664 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178802944}
I20260709 12:24:02.046221 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:02.059281 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
I20260709 12:24:02.071254 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.006994 20294 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.072502 13645 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
W20260709 12:24:02.004237 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 124324us
I20260709 12:24:02.143627 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:02.075589 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.019383 20361 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:02.020226 20350 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.079349 20510 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.047580 20491 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.047333 20164 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.059023 20498 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.071795 20379 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:01.949957 20433 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.071615 20365 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.082675 20505 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.048683 20220 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.084934 20364 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.085631 20117 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.038782 20487 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.111510 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:02.035966 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:02.130523 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.060317 20497 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.075062 20479 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:02.077085 20427 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:02.135103 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58603.1us
I20260709 12:24:02.032531 20411 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.197151 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3285) took 1017 ms. Trace:
W20260709 12:24:02.197484 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50326.7us
W20260709 12:24:02.270409 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.195827 20472 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.203589 14417 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:24:02.209547 13648 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:02.202553 20510 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.146328 20240 raft_consensus.cc:2764] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:24:02.194582 20294 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:02.270967 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.261962 20499 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.275784 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:02.200678 20242 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.203589 13727 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
W20260709 12:24:02.203366 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59860.2us
I20260709 12:24:02.298218 13718 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:02.286957 20510 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.282912 20472 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.296730 13618 leader_election.cc:400] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:24:02.303880 13618 leader_election.cc:403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:02.298826 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:02.295349 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:02.269719 13721 rpcz_store.cc:276] 0709 12:24:01.179449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.145364 (+965915us) service_pool.cc:224] Handling call
0709 12:24:02.196599 (+ 51235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.196816 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.196842 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:02.196886 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:02.196929 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.196985 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.197090 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:02.230428 20498 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.309235 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:02.209657 20164 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.258797 20364 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.316696 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:02.313167 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:02.258136 20117 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.209385 20491 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.324391 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:02.275195 20497 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.257792 20505 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.278602 20427 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.278508 20411 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.335693 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:02.299373 19907 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:02.329613 20262 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:02.358134 13794 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:02.328506 20361 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:02.316545 20499 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:02.331358 20323 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:02.330461 20240 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:02.333312 20270 raft_consensus.cc:2764] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:24:02.363054 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:02.336946 20456 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.352321 19907 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.349153 20385 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.344236 20512 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:02.389695 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 348664us
I20260709 12:24:02.392396 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:02.390832 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:02.407630 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:02.408651 14417 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:02.413455 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.414402 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:02.306579 20479 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.422596 13796 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:24:02.400125 19907 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:02.425918 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:02.429939 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.410588 20227 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:02.401445 20499 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:02.485339 20512 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.487154 20450 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.487337 20365 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.489969 20294 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.492414 20427 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.493561 20497 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.494001 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:02.495085 20498 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.495200 20444 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:02.495759 20444 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.495846 20164 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.496181 20444 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:02.496909 20505 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:02.498047 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.501109 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:02.501618 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.493646 19069 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:24:02.516275 19069 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:24:02.524128 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:02.526525 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:02.446748 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.411566 20240 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:02.412304 20385 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.454552 20451 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:02.458763 13644 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 12:24:02.463174 20187 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.004s
W20260709 12:24:02.465294 20515 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.186s	user 0.001s	sys 0.001s
I20260709 12:24:02.465691 20379 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.475723 20255 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:02.523089 20516 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:24:02.448848 13643 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:02.492411 20227 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.478797 20472 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.482375 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3338) took 1511 ms. Trace:
I20260709 12:24:02.482621 20411 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:02.417119 20399 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:02.424453 20361 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:24:02.532970 13799 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:02.535681 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:02.537339 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:02.544462 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:02.546062 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:02.688895 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 151698us
I20260709 12:24:02.546733 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:02.476020 20301 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:02.525283 20522 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:24:02.548336 20451 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:02.477596 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3349) took 1504 ms. Trace:
I20260709 12:24:02.512132 20505 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.550664 20456 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.509997 20350 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.701254 13719 rpcz_store.cc:276] 0709 12:24:00.973647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.873495 (+899848us) service_pool.cc:224] Handling call
0709 12:24:01.873863 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.075209 (+201346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.477348 (+402139us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b5400173754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:02.477532 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75549952}
I20260709 12:24:02.655143 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:02.670818 20227 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:02.707528 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:02.709049 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.521799 20294 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 12:24:02.485996 20512 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.534541 20228 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:02.553766 13722 rpcz_store.cc:276] 0709 12:24:00.970512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.984165 (+ 13653us) service_pool.cc:224] Handling call
0709 12:24:02.482139 (+1497974us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003fedd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:02.482250 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2343440256}
W20260709 12:24:02.680126 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.147s (SENT) [suppressed 20 similar messages]
I20260709 12:24:02.637662 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:02.687671 14422 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:02.667374 20242 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:02.520776 20517 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 12:24:02.676832 20136 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.580s	user 0.014s	sys 0.000s
I20260709 12:24:02.666944 20472 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:02.705118 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:02.685823 20399 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.705313 20456 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.707675 13643 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.533782 20240 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:02.554616 20187 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:02.711263 13719 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:02.755246 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.756574 13719 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
W20260709 12:24:02.758579 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.723737 20228 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:02.757943 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:02.716881 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 343742us
I20260709 12:24:02.677047 20506 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.638015 20075 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
W20260709 12:24:02.726442 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 177949us
I20260709 12:24:02.727836 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:02.695081 20329 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:02.729926 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 183275us
W20260709 12:24:02.731928 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94386.9us
I20260709 12:24:02.736603 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.706527 20510 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.777436 20512 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:02.774554 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:02.834914 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:02.835418 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61000.9us
I20260709 12:24:02.835530 20510 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:02.835700 13722 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:02.836082 20510 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.836459 20510 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.836771 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.837888 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.845702 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.853583 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.855365 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.857997 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:02.733777 20448 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.784s	user 0.015s	sys 0.001s
I20260709 12:24:02.862628 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.876464 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.878090 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.782838 13719 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:02.735764 20411 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:02.882753 13795 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:02.883181 20411 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:02.883633 20411 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.737331 20234 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:02.803082 20228 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.805505 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3283) took 1626 ms. Trace:
I20260709 12:24:02.751797 20399 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:02.886641 13726 rpcz_store.cc:276] 0709 12:24:01.178851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.961652 (+782801us) service_pool.cc:224] Handling call
0709 12:24:01.961952 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.990804 (+ 28852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.990855 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.991557 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.387425 (+395868us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:02.387487 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.387578 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.805126 (+417548us) raft_consensus.cc:1685] finished
0709 12:24:02.805162 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.805412 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20327,"raft.queue_time_us":30828,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"thread_start_us":340504,"threads_started":1,"wal-append.queue_time_us":783520}
I20260709 12:24:02.888861 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.705135 20505 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:02.889992 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:02.891479 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 211395us
I20260709 12:24:02.809727 13647 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:02.816692 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.892068 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.886214 20228 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:02.890399 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:02.753001 20240 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:02.892624 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.892558 13792 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:02.831324 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3282) took 1652 ms. Trace:
I20260709 12:24:02.963089 13643 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:24:02.961555 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 128284us
I20260709 12:24:02.979650 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:02.893450 13726 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:02.980428 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:02.878903 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.939013 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.982344 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:02.982999 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.978420 13725 rpcz_store.cc:276] 0709 12:24:01.178481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.924102 (+745621us) service_pool.cc:224] Handling call
0709 12:24:01.924468 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.924855 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.924890 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.925441 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.300908 (+375467us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:02.300949 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.301021 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.830913 (+529892us) raft_consensus.cc:1685] finished
0709 12:24:02.830981 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.831245 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1752,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"thread_start_us":372973,"threads_started":1,"wal-append.queue_time_us":431984}
W20260709 12:24:02.812076 20518 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:24:02.754683 20472 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.023037 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:03.023988 13726 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:03.025427 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:02.907315 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73515.9us
W20260709 12:24:02.907019 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73979.6us
I20260709 12:24:02.924784 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.018162 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.893795 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:02.923807 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 197525us
I20260709 12:24:03.033252 13726 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:24:03.033799 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.035725 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:02.779112 20136 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.007344 20301 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.044202 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:03.053975 13647 raft_consensus.cc:2368] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:24:02.817356 20539 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:03.021224 20490 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:24:02.924229 20192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.006s	sys 0.005s
I20260709 12:24:03.012089 20356 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:24:03.063110 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80878.7us
I20260709 12:24:03.079250 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.068222 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:02.895030 20512 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:03.080242 13647 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:03.083679 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:03.084910 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:02.892108 20505 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.073447 13693 leader_election.cc:400] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:24:03.008404 20234 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:02.906490 20543 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.047323 20485 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:24:03.053808 20262 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.681s	user 0.012s	sys 0.000s
I20260709 12:24:03.066861 20539 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:03.073210 20192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.007s	sys 0.006s
I20260709 12:24:03.093407 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:03.094831 13693 leader_election.cc:403] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:24:03.112562 13725 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:24:03.087585 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3649) took 1192 ms. Trace:
I20260709 12:24:02.927524 20506 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.079408 20413 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.105877 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:03.109938 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:03.119603 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139559us
I20260709 12:24:03.078732 20356 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.070187 20385 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.086414 20301 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.115453 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85408.6us
I20260709 12:24:03.113332 13801 rpcz_store.cc:276] 0709 12:24:01.894854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.895593 (+   739us) service_pool.cc:224] Handling call
0709 12:24:01.895987 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.940047 (+ 44060us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:01.940114 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.940647 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.314067 (+373420us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:02.314110 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.314177 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.087254 (+773077us) raft_consensus.cc:1685] finished
0709 12:24:03.087286 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.087500 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36536,"raft.queue_time_us":37814,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":373054,"threads_started":1,"wal-append.queue_time_us":613499}
I20260709 12:24:03.061846 20136 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.087200 20546 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.129966 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.124028 13647 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:03.130059 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3270) took 2273 ms (client timeout 2955 ms). Trace:
W20260709 12:24:03.134492 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55357.3us
W20260709 12:24:03.133275 13724 rpcz_store.cc:269] 0709 12:24:00.851073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.860184 (+  9111us) service_pool.cc:224] Handling call
0709 12:24:03.124340 (+2264156us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002f7454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:03.124455 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651098496}
I20260709 12:24:03.136974 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.137132 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:03.139235 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.117970 20262 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:24:03.141193 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.142251 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:03.146226 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.125597 20356 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:03.149398 13722 raft_consensus.cc:2410] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:24:03.152655 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:03.153417 13725 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:24:03.153865 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:03.121403 20555 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:03.091890 20512 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.141667 20451 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:03.147505 20350 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:03.106705 20543 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.197172 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1107) took 1192 ms. Trace:
I20260709 12:24:03.126335 20413 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.160413 20533 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:03.197576 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.197760 13650 rpcz_store.cc:276] 0709 12:24:02.004844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.332282 (+327438us) service_pool.cc:224] Handling call
0709 12:24:02.355517 (+ 23235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.546647 (+191130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.546684 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.646955 (+100271us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.746808 (+ 99853us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:02.746861 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.746969 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.196917 (+449948us) raft_consensus.cc:1685] finished
0709 12:24:03.196941 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.197100 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68184,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":12552,"thread_start_us":271074,"threads_started":2,"wal-append.queue_time_us":455788}
I20260709 12:24:03.201825 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:03.132105 20301 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.199440 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.205442 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:03.211274 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:03.212224 13651 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:03.213310 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:03.214709 20534 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:24:03.186553 20234 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.197096 20385 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:03.160674 20546 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:24:03.218233 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1428) took 1223 ms. Trace:
I20260709 12:24:03.251883 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.230912 13651 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: [14.9-15.10]
I20260709 12:24:03.250686 14422 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:03.255203 13727 rpcz_store.cc:276] 0709 12:24:01.995027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.217207 (+1222180us) service_pool.cc:224] Handling call
0709 12:24:03.217625 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.217843 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.217875 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.217930 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:03.217971 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.218031 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.218161 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.247430 20136 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.212224 20512 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.264220 13651 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:03.293560 13794 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:03.261472 13792 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:24:03.271811 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.298135 13795 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:03.263372 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.271554 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:03.201220 20533 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:03.249089 20444 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.000s
I20260709 12:24:03.299669 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.300036 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:03.209826 20536 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.403s	user 0.001s	sys 0.001s
I20260709 12:24:03.377554 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:03.378182 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.378902 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.379356 13792 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:03.379561 13796 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:03.302646 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:03.380719 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.304994 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.306330 20532 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:03.385206 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 85265.3us
I20260709 12:24:03.387333 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:03.307370 20543 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:03.270428 19907 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.791s	user 0.014s	sys 0.001s
I20260709 12:24:03.308605 20350 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:03.396994 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:03.310617 20443 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.392s	user 0.009s	sys 0.010s
I20260709 12:24:03.225914 20240 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.312651 13651 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:03.200695 20560 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.108s
I20260709 12:24:03.165917 20539 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.318252 20506 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.318974 20399 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:03.321120 20531 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:24:03.321070 20356 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:03.324489 20492 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.603s	user 0.009s	sys 0.000s
I20260709 12:24:03.324937 20413 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:03.326864 20535 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.572s	user 0.000s	sys 0.001s
I20260709 12:24:03.330639 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:03.338186 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:03.339644 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.340960 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:03.341428 20367 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:03.341722 20328 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:24:03.343926 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.221589 20364 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.347659 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:03.279881 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:03.220072 20491 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.377337 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.380947 13794 raft_consensus.cc:2368] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:24:03.241308 20301 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.390538 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.391902 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.392516 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:03.244576 20472 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.399657 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:03.397845 20565 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.392019 20270 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:03.224584 20510 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.377068 20533 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:03.405305 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 264342us
I20260709 12:24:03.405432 14420 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:03.440791 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:03.391862 20444 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:03.505031 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 299842us
I20260709 12:24:03.543483 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:03.520076 13795 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
W20260709 12:24:03.533324 20545 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:03.484668 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1427) took 1494 ms. Trace:
I20260709 12:24:03.547802 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:03.479574 13721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.012s	sys 0.008s
W20260709 12:24:03.548695 13721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.014s	sys 0.013s
W20260709 12:24:03.521579 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 241834us
I20260709 12:24:03.549459 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3293) took 1571 ms. Trace:
I20260709 12:24:03.521382 13727 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:03.524243 13796 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
W20260709 12:24:03.400872 20227 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.670s	user 0.015s	sys 0.000s
I20260709 12:24:03.520884 14424 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:03.522671 20541 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 12:24:03.531152 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 190279us
I20260709 12:24:03.532012 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3297) took 1446 ms. Trace:
I20260709 12:24:03.397603 20350 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.566915 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:03.480033 20536 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:24:03.540844 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 384890us
W20260709 12:24:03.529029 20531 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:03.570709 20350 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:03.520520 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:03.463737 20240 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.398918 19907 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:03.598146 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.600026 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.526980 20451 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:03.519899 20328 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:03.548440 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:03.522200 13722 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:24:03.607087 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:03.549854 13721 rpcz_store.cc:276] 0709 12:24:01.978431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.317384 (+338953us) service_pool.cc:224] Handling call
0709 12:24:02.346164 (+ 28780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.346481 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:02.693946 (+347465us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.822856 (+128910us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.549043 (+726187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.549139 (+    96us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:03.549184 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.549266 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.549390 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31502,"prepare.queue_time_us":667222,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":150793,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":134,"thread_start_us":1176498,"threads_started":3}
I20260709 12:24:03.607589 13795 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:03.548077 13725 rpcz_store.cc:276] 0709 12:24:01.990715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.184126 (+1193411us) service_pool.cc:224] Handling call
0709 12:24:03.207928 (+ 23802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.208264 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:03.267562 (+ 59298us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.484237 (+216675us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.484298 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:03.484348 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.484431 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.484584 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81053,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":18374,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":15628,"thread_start_us":216221,"threads_started":1}
W20260709 12:24:03.526358 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.608548 13722 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:03.609632 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:03.530174 20361 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.552343 13643 raft_consensus.cc:2368] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:24:03.530177 20472 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.559492 13718 rpcz_store.cc:276] 0709 12:24:02.085233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.530973 (+1445740us) service_pool.cc:224] Handling call
0709 12:24:03.531320 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.531527 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.531567 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.531626 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:03.531691 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.531770 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.531933 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.538607 20192 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.572759 14421 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
W20260709 12:24:03.573666 20228 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.006s
I20260709 12:24:03.619083 13718 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:03.637867 14423 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:03.632262 20510 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.619220 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3299) took 1476 ms. Trace:
I20260709 12:24:03.620941 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:03.621912 20568 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:03.621766 20472 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.656015 20472 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.617331 20240 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.656476 20506 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:03.657063 20240 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.622743 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1429) took 1175 ms. Trace:
I20260709 12:24:03.659868 13725 rpcz_store.cc:276] 0709 12:24:02.446897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.608580 (+1161683us) service_pool.cc:224] Handling call
0709 12:24:03.622189 (+ 13609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.622399 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.622426 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.622473 (+    47us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:03.622513 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.622569 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.622681 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.621801 20328 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.623294 20361 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.624138 20562 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:03.624756 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3298) took 1516 ms. Trace:
I20260709 12:24:03.627948 19907 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.628010 20541 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:03.628556 20565 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:03.628347 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:03.622851 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.629704 20444 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.568782 20227 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.638880 13722 rpcz_store.cc:276] 0709 12:24:02.142968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.607878 (+1464910us) service_pool.cc:224] Handling call
0709 12:24:03.608331 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.618110 (+  9779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.619159 (+  1049us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.678874 13722 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:03.663847 13727 rpcz_store.cc:276] 0709 12:24:02.108290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.596779 (+1488489us) service_pool.cc:224] Handling call
0709 12:24:03.624209 (+ 27430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.624418 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.624443 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.624491 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:03.624538 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.624592 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.624696 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:03.682193 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53954.8us
W20260709 12:24:03.650779 20411 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.766s	user 0.016s	sys 0.000s
I20260709 12:24:03.687650 13722 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:24:03.688225 13722 pending_rounds.cc:85] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 9
I20260709 12:24:03.688665 13722 pending_rounds.cc:107] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:24:03.689030 13722 raft_consensus.cc:2889] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:03.684094 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:03.691689 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.650604 20555 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16.
I20260709 12:24:03.618299 20187 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:24:03.666206 20562 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.698138 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:03.665061 20568 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:03.677507 20328 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:03.716796 13725 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.715643 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.690492 19907 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:03.732753 13718 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:24:03.733882 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 161714us
I20260709 12:24:03.735617 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3365) took 1288 ms. Trace:
I20260709 12:24:03.736029 13718 rpcz_store.cc:276] 0709 12:24:02.447331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.618572 (+1171241us) service_pool.cc:224] Handling call
0709 12:24:03.618932 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.735334 (+116402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.735554 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.714493 20562 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:03.734668 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.713819 20506 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.721382 20364 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:03.766445 13725 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:24:03.769646 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:03.772310 20468 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:03.740484 20301 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:03.778514 20472 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:03.782919 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45215us
I20260709 12:24:03.742161 20356 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.729153 20240 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.694499 20411 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.788022 19907 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:03.787945 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 225365us
I20260709 12:24:03.773746 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:03.760417 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:03.726372 20560 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:24:03.788623 20451 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:03.793000 20326 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:03.802632 14419 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:03.806470 13725 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:24:03.806973 20572 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:24:03.789495 20468 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:03.748119 20555 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:03.788812 20506 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.798802 20472 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.843108 20472 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:03.843223 20569 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:24:03.801477 20240 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.811776 20364 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:03.815771 20326 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.871222 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:03.874773 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:03.844173 20472 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.874311 20326 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:03.878183 20361 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.849787 20555 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.872515 20240 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.813400 20568 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:03.896539 20411 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:03.910079 20227 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.909466 20451 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.918833 20361 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:03.894174 20472 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:03.976716 20479 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:03.990628 20487 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:03.975286 20506 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:03.984386 20361 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:04.060091 20533 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.004s
I20260709 12:24:04.024093 20468 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:04.050241 20456 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.121436 20399 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.113700 20468 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:04.060927 20533 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:04.131016 20350 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.006s
I20260709 12:24:04.158576 20350 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:03.803831 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43563.2us
I20260709 12:24:04.258136 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:04.410864 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 153225us
I20260709 12:24:04.411413 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:04.260396 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.412969 20350 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.276641 13644 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:04.414659 20541 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:04.415141 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3757) took 1058 ms. Trace:
I20260709 12:24:04.415530 20571 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:24:04.276870 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2472) took 1438 ms (client timeout 1657 ms). Trace:
I20260709 12:24:04.416288 20571 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:04.416893 20571 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:04.289345 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3288) took 2323 ms. Trace:
W20260709 12:24:04.298125 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 423252us
I20260709 12:24:04.417832 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:04.419180 13650 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:24:04.420945 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:04.038360 20220 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:04.319561 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.321203 20323 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:04.422318 20564 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:04.322074 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3294) took 2343 ms. Trace:
I20260709 12:24:04.417531 13720 rpcz_store.cc:276] 0709 12:24:01.965668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.111118 (+145450us) service_pool.cc:224] Handling call
0709 12:24:04.288668 (+2177550us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:04.289002 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92001536}
I20260709 12:24:04.322687 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3305) took 1070 ms. Trace:
W20260709 12:24:04.325974 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.157159 20367 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.629s	user 0.012s	sys 0.002s
I20260709 12:24:04.177464 20472 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.431573 20294 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:04.162209 20539 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:04.197037 20533 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.433061 20579 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:04.343380 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.427822 13719 rpcz_store.cc:276] 0709 12:24:03.226472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.780031 (+553559us) service_pool.cc:224] Handling call
0709 12:24:03.780471 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.838675 (+ 58204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.838740 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.838813 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:03.838868 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.296299 (+457431us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b54001a4d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.296384 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.296715 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132951936}
I20260709 12:24:04.415621 13644 rpcz_store.cc:276] 0709 12:24:03.357059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.274682 (+917623us) service_pool.cc:224] Handling call
0709 12:24:04.275658 (+   976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.414762 (+139104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.415049 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:04.339744 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1295) took 1828 ms. Trace:
W20260709 12:24:04.440352 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.149839 20551 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.054s	user 0.000s	sys 0.005s
I20260709 12:24:04.442221 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:04.129762 20555 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.344607 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:04.151053 20576 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.270s	user 0.000s	sys 0.005s
W20260709 12:24:04.359207 13726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.001s	sys 0.012s
W20260709 12:24:04.445814 13726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.001s	sys 0.012s
W20260709 12:24:04.445848 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.446350 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1106) took 2456 ms. Trace:
I20260709 12:24:04.444553 20589 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:04.448398 20589 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.448900 20589 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:04.370062 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.371380 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1105) took 2381 ms. Trace:
I20260709 12:24:04.372334 20361 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.415205 20541 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.429838 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:04.401717 20117 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.452940 20541 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:04.404992 20262 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:04.416474 13643 rpcz_store.cc:269] 0709 12:24:02.838052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.979348 (+141296us) service_pool.cc:224] Handling call
0709 12:24:04.276186 (+1296838us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54001473d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:04.276556 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315054464}
I20260709 12:24:04.393313 20570 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:04.458818 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3753) took 1137 ms. Trace:
I20260709 12:24:04.459299 13650 rpcz_store.cc:276] 0709 12:24:03.321670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.374028 (+ 52358us) service_pool.cc:224] Handling call
0709 12:24:04.458522 (+1084494us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:04.458732 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2058065280}
W20260709 12:24:04.431655 14421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1299) took 1736 ms (client timeout 1668 ms). Trace:
W20260709 12:24:04.460549 14421 rpcz_store.cc:269] 0709 12:24:02.695052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.695244 (+   192us) service_pool.cc:224] Handling call
0709 12:24:04.431067 (+1735823us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54006be4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:04.431353 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920839424}
I20260709 12:24:04.461498 20494 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:04.462088 20494 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:04.439611 14417 rpcz_store.cc:276] 0709 12:24:02.511246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.511379 (+   133us) service_pool.cc:224] Handling call
0709 12:24:02.521533 (+ 10154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.671232 (+149699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.671264 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:02.671309 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:02.671345 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.338969 (+1667624us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b5400600254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.339078 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.339423 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78185728}
I20260709 12:24:04.343684 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.462574 20494 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:04.426519 13723 rpcz_store.cc:276] 0709 12:24:01.978722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.339795 (+361073us) service_pool.cc:224] Handling call
0709 12:24:02.340263 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.384852 (+ 44589us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:02.677440 (+292588us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.677730 (+   290us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:02.677798 (+    68us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:02.677852 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.321382 (+1643530us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b5400175554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.321477 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.321828 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":432920,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":156692,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":101782912,"thread_start_us":265896,"threads_started":1}
I20260709 12:24:04.446900 13648 rpcz_store.cc:276] 0709 12:24:01.989668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.278072 (+288404us) service_pool.cc:224] Handling call
0709 12:24:02.278533 (+   461us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:02.284198 (+  5665us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:02.540944 (+256746us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.742896 (+201952us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:02.743914 (+  1018us) log.cc:844] Serialized 78 byte log entry
0709 12:24:02.744124 (+   210us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:04.149353 (+1405229us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b2400196ef8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b4609d 00007f539377810e 00007f539378f88c 00007f539378ebb6 00007f539378e7d8 00007f5393798221
0709 12:24:04.342648 (+193295us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.390859 (+ 48211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.391059 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.445821 (+ 54762us) raft_consensus.cc:1685] finished
0709 12:24:04.445845 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.446095 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1642690,"prepare.queue_time_us":1725291,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":98,"raft.queue_time_us":68145,"raft.run_cpu_time_us":544,"raft.run_wall_time_us":17197,"spinlock_wait_cycles":4470656,"thread_start_us":293458,"threads_started":2,"wal-append.queue_time_us":412818}
I20260709 12:24:04.444635 20323 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:04.451555 20294 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.466652 20323 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.467999 13723 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:04.443090 20472 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:04.468833 20294 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.462632 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3303) took 1236 ms. Trace:
I20260709 12:24:04.469856 13722 rpcz_store.cc:276] 0709 12:24:03.225706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.683737 (+458031us) service_pool.cc:224] Handling call
0709 12:24:03.684017 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.689668 (+  5651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.462040 (+772372us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b54000f7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.462418 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7392512}
I20260709 12:24:04.472606 13722 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:04.473275 20240 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:04.481168 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51586.5us
I20260709 12:24:04.469230 20472 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:04.360023 13691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.005s	sys 0.012s
W20260709 12:24:04.518605 13691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.007s	sys 0.012s
I20260709 12:24:04.434005 20579 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:04.451297 13645 rpcz_store.cc:276] 0709 12:24:01.989298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.143540 (+154242us) service_pool.cc:224] Handling call
0709 12:24:02.143911 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.147406 (+  3495us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:02.267579 (+120173us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.637942 (+370363us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.638157 (+   215us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.638792 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.638998 (+   206us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:03.310626 (+671628us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b18002e81c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:03.339369 (+ 28743us) spinlock_profiling.cc:243] Waited 7.91 ms on lock 0x7b18002e81c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:04.330815 (+991446us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b18002e81c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:24:04.332186 (+  1371us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.370405 (+ 38219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.370566 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.370634 (+    68us) raft_consensus.cc:1685] finished
0709 12:24:04.370657 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.371041 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1893575,"prepare.queue_time_us":1934380,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":139,"raft.queue_time_us":123432,"raft.run_cpu_time_us":745,"raft.run_wall_time_us":59568,"spinlock_wait_cycles":35706752,"thread_start_us":262024,"threads_started":1,"wal-append.queue_time_us":442660}
I20260709 12:24:04.534941 20427 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.551558 20377 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:04.549883 20294 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:04.577282 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3308) took 1349 ms. Trace:
I20260709 12:24:04.577471 20301 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.578425 20301 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.583604 20472 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.589046 20472 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.589716 20361 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.589936 20472 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.590572 20361 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:04.591776 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.593149 20361 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.592725 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.593680 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.593946 20427 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:04.594521 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:04.595762 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:04.596909 14418 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:04.577881 13726 rpcz_store.cc:276] 0709 12:24:03.227488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.809639 (+582151us) service_pool.cc:224] Handling call
0709 12:24:03.809993 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.810223 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:03.810507 (+   284us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.521718 (+711211us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.568451 (+ 46733us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.568540 (+    89us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:04.568630 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.568729 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.577198 (+  8469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36600,"prepare.queue_time_us":9154,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":39017,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":2964,"thread_start_us":709846,"threads_started":2,"wal-append.queue_time_us":576849}
I20260709 12:24:04.604223 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3742) took 2118 ms. Trace:
I20260709 12:24:04.605971 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:04.606285 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1100) took 2631 ms. Trace:
I20260709 12:24:04.606559 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.606763 13649 rpcz_store.cc:276] 0709 12:24:01.975419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.976317 (+   898us) service_pool.cc:224] Handling call
0709 12:24:01.976722 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.534487 (+2557765us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:04.539724 (+  5237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.539797 (+    73us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:04.544329 (+  4532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.605865 (+ 61536us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.605913 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.606081 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75804,"raft.queue_time_us":63531,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":12929,"spinlock_wait_cycles":5648543488}
I20260709 12:24:04.607429 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:04.608624 14421 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:24:04.573391 13727 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.003s	sys 0.009s
I20260709 12:24:04.581889 20301 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:04.615006 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50134.9us
W20260709 12:24:04.622720 13727 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.003s	sys 0.010s
I20260709 12:24:04.628118 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:04.630291 13718 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.009s	sys 0.005s
W20260709 12:24:04.630788 13718 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.010s	sys 0.005s
I20260709 12:24:04.604872 13652 rpcz_store.cc:276] 0709 12:24:02.485932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.704705 (+218773us) service_pool.cc:224] Handling call
0709 12:24:02.705129 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.705375 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:04.357759 (+1652384us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.558526 (+200767us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.597614 (+ 39088us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b180025d200. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:04.602829 (+  5215us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.603161 (+   332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.603246 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:04.603375 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.603537 (+   162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.603941 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1678030,"prepare.queue_time_us":1695914,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":92,"raft.queue_time_us":49590,"spinlock_wait_cycles":3485568,"thread_start_us":131597,"threads_started":1}
I20260709 12:24:04.638417 20377 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.641613 20505 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:04.644062 20562 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.856s	user 0.013s	sys 0.008s
I20260709 12:24:04.644742 20385 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:04.644830 20562 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.645912 20567 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:04.646553 20385 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:04.647084 20385 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:04.648438 20385 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.650228 20385 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:04.653851 20592 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.002s
W20260709 12:24:04.676565 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 72002.7us
W20260709 12:24:04.665743 13721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.013s	sys 0.017s
W20260709 12:24:04.686705 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 266123us
W20260709 12:24:04.694852 13721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.013s	sys 0.018s
I20260709 12:24:04.703507 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:04.714754 20428 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:04.715677 20428 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:04.716614 20328 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.989s	user 0.015s	sys 0.009s
I20260709 12:24:04.718842 20428 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.720762 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1105) took 1585 ms. Trace:
I20260709 12:24:04.721328 13801 rpcz_store.cc:276] 0709 12:24:03.136845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.137157 (+   312us) service_pool.cc:224] Handling call
0709 12:24:03.137507 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:03.137922 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.137962 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:03.138509 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:24:04.299903 (+1161394us) spinlock_profiling.cc:243] Waited 910 us on lock 0x7b180019a5c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:24:04.370378 (+ 70475us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:04.370467 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.370579 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.720283 (+349704us) raft_consensus.cc:1685] finished
0709 12:24:04.720311 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.720542 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":16415,"spinlock_wait_cycles":2092416,"thread_start_us":67762,"threads_started":1,"wal-append.queue_time_us":1413365}
I20260709 12:24:04.724325 20562 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.661988 20301 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:04.727511 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3302) took 1502 ms. Trace:
I20260709 12:24:04.727967 13727 rpcz_store.cc:276] 0709 12:24:03.225429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.681930 (+456501us) service_pool.cc:224] Handling call
0709 12:24:03.682202 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.682445 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:03.700442 (+ 17997us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.660969 (+960527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.695907 (+ 34938us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:04.727118 (+ 31211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.727244 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.727441 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17800,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":42720,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":922411,"threads_started":2,"wal-append.queue_time_us":187489}
W20260709 12:24:04.729676 20479 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.009s
I20260709 12:24:04.731021 13725 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:24:04.731443 13725 pending_rounds.cc:85] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 7
I20260709 12:24:04.731715 13725 pending_rounds.cc:107] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:24:04.732013 13725 raft_consensus.cc:2889] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:04.733970 20427 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:04.734750 20234 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:04.724388 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:04.740988 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50494.2us
I20260709 12:24:04.753208 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:04.757085 13649 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:04.679538 20294 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:04.778697 20601 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:04.790755 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 146713us
I20260709 12:24:04.793896 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:04.779981 20328 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:04.806346 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3356) took 2362 ms. Trace:
I20260709 12:24:04.813390 13650 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:04.798965 20301 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:04.792387 20427 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:04.786999 20234 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.812325 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:04.819192 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.058s	sys 0.070s
I20260709 12:24:04.782086 20601 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:04.803556 20579 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:04.833503 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.059s	sys 0.071s
W20260709 12:24:04.846845 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.818215 20301 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.851107 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:04.852236 13794 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:04.855410 20564 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:04.856142 20328 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.858944 20328 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.863747 20234 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.867619 13800 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:04.811121 13721 rpcz_store.cc:276] 0709 12:24:02.443872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.608117 (+1164245us) service_pool.cc:224] Handling call
0709 12:24:03.659197 (+ 51080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.659708 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:03.676333 (+ 16625us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.766722 (+1090389us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.772380 (+  5658us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:04.780983 (+  8603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.794953 (+ 13970us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.806229 (+ 11276us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17935,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":162218,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":8143,"thread_start_us":1018556,"threads_started":2,"wal-append.queue_time_us":247599}
W20260709 12:24:04.879112 20606 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.129s	user 0.000s	sys 0.002s
I20260709 12:24:04.879805 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:04.881654 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.821928 20598 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.844452 13796 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:04.884439 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.853987 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3295) took 2864 ms. Trace:
W20260709 12:24:04.889181 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.897523 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:04.848344 20301 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:04.894462 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61116us
W20260709 12:24:04.909231 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 191131us
W20260709 12:24:04.910421 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.06635e+06us
I20260709 12:24:04.923171 13799 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
W20260709 12:24:04.942818 20468 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.776s	user 0.013s	sys 0.012s
I20260709 12:24:04.942888 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:04.946723 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:04.951337 20611 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:24:04.954566 20326 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.047s	user 0.014s	sys 0.015s
I20260709 12:24:04.955240 20326 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:04.956544 14418 raft_consensus.cc:2368] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:04.957747 20326 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:04.959157 20608 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:24:04.961210 20294 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:04.961238 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3755) took 1617 ms. Trace:
I20260709 12:24:04.962705 13651 rpcz_store.cc:276] 0709 12:24:03.322180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.815065 (+492885us) service_pool.cc:224] Handling call
0709 12:24:03.815571 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.255243 (+439672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.255339 (+    96us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.256669 (+  1330us) log.cc:844] Serialized 26 byte log entry
0709 12:24:04.519544 (+262875us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:04.519591 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.916641 (+397050us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b5400096d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.916700 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.916761 (+    61us) raft_consensus.cc:1685] finished
0709 12:24:04.916784 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.939729 (+ 22945us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":62728,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":404,"spinlock_wait_cycles":1007416576,"thread_start_us":262153,"threads_started":1,"wal-append.queue_time_us":415468}
I20260709 12:24:04.967011 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:04.888113 13724 rpcz_store.cc:276] 0709 12:24:01.989949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.146404 (+1156455us) service_pool.cc:224] Handling call
0709 12:24:03.146715 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.146935 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:03.161520 (+ 14585us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.325679 (+164159us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b180031f180. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:04.293151 (+967472us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b180031f180. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:04.778177 (+485026us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.797632 (+ 19455us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.821472 (+ 23840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.827030 (+  5558us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:04.842915 (+ 15885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.853628 (+ 10713us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.853806 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3397,"prepare.queue_time_us":302651,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":108,"raft.queue_time_us":19108,"spinlock_wait_cycles":46370560,"thread_start_us":246215,"threads_started":2,"wal-append.queue_time_us":1554524}
I20260709 12:24:04.936887 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:04.975725 13796 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:04.977285 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:04.980031 13801 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:04.979789 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3309) took 1752 ms. Trace:
W20260709 12:24:04.980759 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53563.4us
I20260709 12:24:04.923827 20472 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:04.983592 13725 rpcz_store.cc:276] 0709 12:24:03.227813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.835400 (+607587us) service_pool.cc:224] Handling call
0709 12:24:03.835831 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.732674 (+896843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.967293 (+234619us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b54002f6f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:04.979695 (+ 12402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2056961920}
I20260709 12:24:04.982141 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:04.998464 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:05.004367 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.005254 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:04.914448 20598 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.008788 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:04.983305 20234 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:05.017665 13644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.003s	sys 0.022s
W20260709 12:24:05.018247 13644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.003s	sys 0.022s
W20260709 12:24:05.020398 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 284896us
W20260709 12:24:05.027870 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50755.1us
I20260709 12:24:05.030701 20600 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.031579 20600 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:05.019982 20541 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.009s
I20260709 12:24:05.018625 20564 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:05.013769 20262 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:05.029404 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:05.046631 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:05.051927 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.053196 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:05.055994 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.010332 20601 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.052690 20539 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:05.051272 13649 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:05.053689 20262 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:05.067010 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:05.046922 20541 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.043418 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:05.069494 20613 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.111s	user 0.003s	sys 0.000s
I20260709 12:24:05.072851 20377 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.073793 13645 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:05.074158 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
W20260709 12:24:05.074807 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51209.7us
W20260709 12:24:05.063751 20494 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.007s
I20260709 12:24:05.087311 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.083395 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.094091 13651 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:05.034215 20600 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.076794 20539 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.102348 13647 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:24:05.079262 20472 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:05.151288 20600 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:05.115010 20539 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:05.192546 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:05.202920 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:05.204684 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137885us
I20260709 12:24:05.206559 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:05.209591 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 101252us
I20260709 12:24:05.213155 13767 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:24:05.213644 13767 leader_election.cc:403] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 12:24:05.216805 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.098623 20494 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.180699 20564 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:05.190188 20377 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:05.234999 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.236340 20479 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:05.237004 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 180126us
I20260709 12:24:05.220044 20468 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.273509 20350 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:05.274490 20600 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:05.276134 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69700us
I20260709 12:24:05.278465 13652 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:05.278591 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.224721 20117 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:05.190994 20294 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 12:24:05.289829 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.291850 20614 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.331s	user 0.009s	sys 0.001s
I20260709 12:24:05.302204 20603 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
W20260709 12:24:05.303140 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 165668us
I20260709 12:24:05.304232 13800 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:05.301925 20117 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:05.306125 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.306398 20427 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:05.307037 20427 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:05.307159 14420 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:05.307521 14417 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:05.305809 20117 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.308903 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.309793 14419 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:05.311034 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.278791 20564 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:05.211907 20472 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.296671 20377 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.329650 20377 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.330070 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.330924 13646 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:05.331317 20323 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:05.331949 20603 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:05.332926 20262 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.328727 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.326246 20468 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.334828 13644 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:24:05.336694 20468 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.307479 20427 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:05.332216 20377 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.319375 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:05.296545 20505 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:05.338665 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.344137 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.369254 20505 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.369809 13647 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:05.309849 20479 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:05.371052 20615 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.426s	user 0.000s	sys 0.004s
I20260709 12:24:05.350589 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:05.371831 20505 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.372517 20505 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:05.372869 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1439) took 1737 ms. Trace:
I20260709 12:24:05.371853 20479 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.373284 13720 rpcz_store.cc:276] 0709 12:24:03.635550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.471015 (+835465us) service_pool.cc:224] Handling call
0709 12:24:04.471755 (+   740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.489410 (+ 17655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.489481 (+    71us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.555630 (+ 66149us) log.cc:844] Serialized 26 byte log entry
0709 12:24:04.887517 (+331887us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:04.887591 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.887681 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:05.370343 (+482662us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:05.372512 (+  2169us) raft_consensus.cc:1685] finished
0709 12:24:05.372536 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.372790 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33435,"raft.queue_time_us":12938,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":35602,"spinlock_wait_cycles":84046208,"thread_start_us":225438,"threads_started":1,"wal-append.queue_time_us":317564}
I20260709 12:24:05.375484 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.340236 20350 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.381042 20350 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.382324 20564 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.378183 20505 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:24:05.351593 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.352906 20600 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.353147 20472 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.354161 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.360013 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.361934 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:05.362891 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.366791 20541 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:05.396235 20589 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.947s	user 0.018s	sys 0.000s
I20260709 12:24:05.387223 20494 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.401887 20598 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.388492 13652 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:05.451489 13651 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:05.456070 13615 leader_election.cc:400] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:05.456270 13650 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:24:05.440783 20571 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.023s	user 0.003s	sys 0.014s
I20260709 12:24:05.460332 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.470428 13800 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:05.472817 20323 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:05.472962 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.473799 20323 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.470454 20571 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.475150 19069 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:05.476593 20323 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:05.478392 13649 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:05.478724 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:05.480365 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:05.480557 19069 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:24:05.477035 20571 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.487025 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.488415 14418 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:05.493505 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:05.450497 20468 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:05.459802 20234 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:05.515034 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:05.427341 20301 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.530616 20468 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.543262 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.546660 20367 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.547624 20367 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.501574 20598 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:05.505434 20234 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:05.559890 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 105093us
I20260709 12:24:05.444882 20589 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.568233 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:05.558494 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51935.5us
I20260709 12:24:05.572440 20494 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.573258 20494 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:05.575620 20619 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.516s	user 0.008s	sys 0.001s
I20260709 12:24:05.576190 20494 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.576857 20472 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:05.577054 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.577351 20472 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:05.577852 20472 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:05.577981 13798 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.580096 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:05.580895 13793 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:05.581589 20377 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.582367 20377 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.584553 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:05.584895 20377 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.555294 20234 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:05.587914 13649 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:05.586126 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.572925 20598 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:05.591971 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1443) took 1318 ms. Trace:
I20260709 12:24:05.602022 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.597720 13723 rpcz_store.cc:276] 0709 12:24:04.274102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.531163 (+257061us) service_pool.cc:224] Handling call
0709 12:24:04.562606 (+ 31443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.683314 (+120708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.683350 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.759669 (+ 76319us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.007588 (+247919us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:05.007655 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.007753 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:05.574701 (+566948us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:05.588447 (+ 13746us) raft_consensus.cc:1685] finished
0709 12:24:05.588480 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.591906 (+  3426us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5028,"raft.queue_time_us":33699,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":61649,"spinlock_wait_cycles":71130368,"thread_start_us":207716,"threads_started":1,"wal-append.queue_time_us":269469}
I20260709 12:24:05.607092 14422 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:05.565227 20468 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.610527 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:05.609599 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.614292 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.618728 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3304) took 2392 ms. Trace:
I20260709 12:24:05.619388 13718 rpcz_store.cc:276] 0709 12:24:03.226021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.736922 (+510901us) service_pool.cc:224] Handling call
0709 12:24:03.737273 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.737485 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:03.742941 (+  5456us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.929328 (+1186387us) log.cc:844] Serialized 14 byte log entry
0709 12:24:05.601042 (+671714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.608467 (+  7425us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:05.618226 (+  9759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.618420 (+   194us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.618649 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6475,"prepare.queue_time_us":123357,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":3644,"raft.queue_time_us":91345,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":186,"thread_start_us":1597215,"threads_started":5,"wal-append.queue_time_us":455802,"wal-append.run_cpu_time_us":1267,"wal-append.run_wall_time_us":1001275}
I20260709 12:24:05.594945 20598 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:05.625418 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:05.630630 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:05.634227 13652 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:05.642549 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:05.644786 20598 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.645416 20262 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:05.613481 20600 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.647720 20600 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.654899 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.656800 20367 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.661662 20539 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:05.673882 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 380764us
I20260709 12:24:05.678474 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.690430 13798 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:05.694823 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.702643 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.703650 13792 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:05.685649 20328 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.704403 13643 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:24:05.705444 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 416181us
I20260709 12:24:05.705291 20294 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:05.718133 20294 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:05.718566 20294 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:05.719823 20294 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.721361 20294 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.706205 20589 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.706854 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.707516 20301 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:05.688622 20600 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.710641 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:05.731462 14421 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:05.711476 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:05.732477 14419 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:05.731479 20640 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:24:05.724604 20328 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.740911 20328 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:05.742247 20323 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:05.746378 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:05.747220 13796 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:05.748240 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.749341 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.750801 13796 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:05.688788 20598 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:05.696117 20468 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.716238 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.754313 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.754963 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:05.756291 20234 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.758670 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.758046 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:05.759528 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:05.766486 14422 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:05.711766 20539 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.724023 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:05.770952 14419 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:05.775547 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:05.779541 14421 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:05.786520 20468 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.790010 13650 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:05.791024 14418 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:24:05.793666 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:05.790283 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66779.5us
I20260709 12:24:05.796643 20579 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:05.797120 20579 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:05.798617 20579 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:05.799003 20427 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:05.800650 20468 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:05.801282 20427 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:05.803785 13650 pending_rounds.cc:85] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 16
I20260709 12:24:05.804741 13650 pending_rounds.cc:107] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 12:24:05.805051 13650 raft_consensus.cc:2889] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:05.804181 13648 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:24:05.809053 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.816656 13793 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:05.820277 13792 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:05.842460 20567 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:24:05.855332 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61798.6us
I20260709 12:24:05.858547 20367 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:05.859444 20367 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:05.860668 20367 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.864351 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:05.864606 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.865410 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:05.865906 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.866714 13643 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:05.866918 14421 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:05.868431 20494 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:05.869179 20117 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:05.869642 20117 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.869974 20117 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:05.867501 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.881886 13643 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:05.883714 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.888595 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133710us
I20260709 12:24:05.883240 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:05.904284 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.917268 13643 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:05.869226 20494 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:05.923188 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:05.792289 20640 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 12:24:05.937508 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:05.950583 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:05.952313 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:05.950002 20600 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.957479 13721 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:05.930141 20494 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:05.963725 13726 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:05.934700 20505 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.960721 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:05.929759 20323 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.934930 20262 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:05.955425 20603 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:05.969067 13643 raft_consensus.cc:2368] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:24:05.984158 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1115) took 1197 ms. Trace:
I20260709 12:24:05.988610 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:05.996227 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113073us
I20260709 12:24:05.973767 20603 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:05.974989 20427 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:06.073690 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45749.5us
I20260709 12:24:05.988750 13795 rpcz_store.cc:276] 0709 12:24:04.624141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.640414 (+ 16273us) service_pool.cc:224] Handling call
0709 12:24:05.075478 (+435064us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:24:05.095121 (+ 19643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.188798 (+ 93677us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:05.608848 (+420050us) log.cc:844] Serialized 14 byte log entry
0709 12:24:05.771319 (+162471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.771397 (+    78us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:05.771473 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.771955 (+   482us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.772125 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23492,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":225,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":19796,"spinlock_wait_cycles":69615488,"thread_start_us":531084,"threads_started":2,"wal-append.queue_time_us":162536}
I20260709 12:24:06.076059 13721 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:06.077522 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 110591us
W20260709 12:24:06.080536 13800 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.007s
I20260709 12:24:05.987828 20600 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.033749 20603 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:05.984454 20323 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:05.993572 20505 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.011664 20262 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:06.088883 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129800us
I20260709 12:24:06.072921 20564 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.013404 20472 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.094026 13797 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:06.094777 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.096336 13725 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:06.097589 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.087321 13800 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:24:06.100761 20643 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:24:06.101801 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.102463 20564 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.104449 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.105388 20629 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.165s
I20260709 12:24:06.107858 20505 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.108594 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.076256 20328 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:24:06.090633 20427 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:06.101697 20645 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:06.072649 20579 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.086560 20234 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:06.146420 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:06.129906 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.132055 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.133502 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.150581 20625 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:06.150981 20646 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:06.151407 20625 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.151923 20472 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.152100 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.153546 13797 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:24:06.153393 13719 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:06.146059 20589 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.148100 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:06.155174 20649 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:24:06.147783 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.156560 20262 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.157505 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.141248 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.158196 13718 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:06.158674 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:06.145022 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59724.9us
I20260709 12:24:06.121100 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.150720 13722 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:24:06.130154 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121592us
I20260709 12:24:06.158083 13723 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:24:06.166282 20658 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:24:06.168154 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76228us
I20260709 12:24:06.171947 20630 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:06.172564 20630 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:06.178618 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.184445 14418 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:24:06.184715 20648 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:06.186766 20564 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.186637 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.187920 13724 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:06.190567 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.192385 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.193017 13794 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:06.194542 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:06.196512 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:06.212672 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:06.160116 20625 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.218712 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.217465 13721 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:06.223652 13794 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:06.225004 13800 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:06.226691 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:06.227384 13648 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:06.227661 20023 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 12:24:06.227842 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:06.155236 20589 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.228721 13650 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:06.230767 20589 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.235354 20023 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:06.235898 20023 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:06.236966 20023 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.244095 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:06.222852 20654 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:24:06.222487 20579 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.245340 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:06.246379 20655 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
W20260709 12:24:06.247701 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51335.9us
I20260709 12:24:06.248175 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.238735 20630 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.249102 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:06.249716 20630 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:06.250622 20647 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:06.239596 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.240998 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.260015 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.240321 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.241667 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.250561 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:06.252717 20647 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:06.262445 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:06.241236 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:06.270442 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 511252us
I20260709 12:24:06.211050 20629 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:24:06.275882 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.243608 20625 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:06.314055 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.301409 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:06.302088 13721 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:06.225859 20603 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.316962 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.301985 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:06.319301 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:06.283622 20428 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:06.284726 20598 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.321523 13727 raft_consensus.cc:2376] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23. Re-sending same reply.
I20260709 12:24:06.353329 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.352288 14418 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:06.355192 20626 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.355571 20427 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.356012 20626 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.320582 20023 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:06.364372 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.368031 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:06.372748 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 156287us
I20260709 12:24:06.365170 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:06.373574 20234 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:06.378767 20234 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.379289 20234 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:06.383307 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 111542us
I20260709 12:24:06.390376 20598 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.392858 20598 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:06.395172 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.394821 20234 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.395972 20468 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:06.396519 20600 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:06.399659 20294 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.678s	user 0.012s	sys 0.000s
I20260709 12:24:06.401223 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.401571 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.401939 13726 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 11.
I20260709 12:24:06.396551 20468 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:06.403545 20468 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.404632 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.404857 20468 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:06.396059 20647 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:06.405522 13720 raft_consensus.cc:2410] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:24:06.405833 20647 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:06.406627 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.407465 13720 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:24:06.401974 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.409198 13764 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
W20260709 12:24:06.409868 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 90703.9us
I20260709 12:24:06.412237 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.412916 13800 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:06.327700 20625 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.413326 20640 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:06.427702 13725 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:06.430670 13721 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:06.430729 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:06.432547 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47968.8us
W20260709 12:24:06.432857 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 151714us
I20260709 12:24:06.436612 20494 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.432138 13720 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:06.449311 20600 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:06.450088 20600 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.452487 20600 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.407364 20647 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.456678 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:06.459026 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:06.433346 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:06.462245 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.465481 20625 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:06.466426 20664 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:24:06.431660 20640 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.469339 20640 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:06.466862 20603 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.479260 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:06.487305 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:06.488054 13649 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.489008 20603 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:06.489454 20603 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.489843 20603 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:06.491164 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:06.494148 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60896.6us
I20260709 12:24:06.495334 20234 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:06.497776 20671 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:06.498443 20671 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.491259 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.500418 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.504621 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.507196 13723 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:06.511101 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.511894 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.512640 13727 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.519063 20326 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.519953 20326 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.524228 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.486445 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.531179 14422 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:06.539146 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.540565 13724 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:24:06.542397 13652 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:06.543205 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.541821 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:06.529064 13718 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:06.546298 13725 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:06.548224 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.549389 13725 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
W20260709 12:24:06.558821 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 90080.6us
I20260709 12:24:06.564457 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.561832 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:06.569136 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.575420 20671 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.544703 20600 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:06.578886 13796 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:06.580209 20567 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:06.580848 20567 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:06.581911 20567 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.584563 20626 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.587142 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.585928 20589 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:24:06.587935 13718 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:06.588775 20589 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:06.589346 20589 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:06.587229 20600 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:06.590896 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:06.591102 20600 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.591572 13544 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 21, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 21 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:06.593957 13792 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:06.594815 13797 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:06.595784 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:06.597658 13723 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:06.605760 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:06.616925 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:06.609112 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56113.7us
I20260709 12:24:06.620777 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:06.608249 13649 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:06.631711 20328 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:06.632202 20328 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:06.632658 20328 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:06.633277 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:06.634531 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.643033 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:06.645424 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:06.646129 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52200.5us
I20260709 12:24:06.644289 20326 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:06.650920 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:06.602488 20625 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:06.632431 20675 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:06.651809 13645 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:06.629585 20671 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:06.635303 20630 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:06.633814 20655 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:06.666304 13721 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:06.659348 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:06.644758 20539 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.686702 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:06.688604 20472 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:06.661182 20579 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:06.698151 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.689455 20678 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:06.707072 20598 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:06.707278 20628 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:24:06.707706 20598 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.708252 20598 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.708978 20365 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.674384 20655 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:06.593400 20589 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.705317 20678 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:06.712227 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.712370 20678 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:06.713014 13726 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.709559 20365 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.655609 20640 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.680544 20630 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:06.644953 13652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48612be3c56f482c81533c30abebe60b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:06.729230 13652 raft_consensus.cc:606] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:06.710180 20644 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.745s	user 0.006s	sys 0.000s
I20260709 12:24:06.667418 20505 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:24:06.724520 20668 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:06.723869 20323 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:06.724062 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:06.704022 20472 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.684337 20539 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.742681 20579 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.719352 20655 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:06.747443 13726 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:06.751530 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.751112 20630 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.752581 13796 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:06.751327 20262 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:06.753104 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.756910 13722 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:06.759608 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:06.769286 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:06.742930 20427 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:24:06.770102 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:06.742992 20668 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.753249 20472 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:06.770081 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:06.759662 20625 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:06.787002 13801 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:24:06.756047 20262 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:06.792616 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:06.778283 20589 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:06.801198 14417 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:06.813854 20662 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.814085 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:06.816447 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:06.821399 20494 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.785061 20656 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:06.773063 20579 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:06.809576 20117 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.892s	user 0.010s	sys 0.003s
I20260709 12:24:06.808001 20539 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:06.812108 20262 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.825371 13727 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:06.790354 20023 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:06.810115 20641 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.032s	user 0.009s	sys 0.000s
I20260709 12:24:06.831035 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:06.895566 13718 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:06.835187 20640 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:06.786688 20626 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:06.783910 20668 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.791937 20625 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:06.868000 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 130974us
I20260709 12:24:06.898990 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:06.902436 13719 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:06.841466 20656 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.902683 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1130) took 1259 ms. Trace:
I20260709 12:24:06.906703 20262 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:06.915512 13718 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:06.906772 20625 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:06.899039 20023 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:06.923344 20023 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:06.903343 20626 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:06.925616 20626 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:06.926900 20682 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:24:06.932029 13722 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:06.940802 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.941586 13800 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:06.942842 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:06.943486 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:06.942840 13719 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:06.943969 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:06.944108 13645 raft_consensus.cc:606] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:06.944550 13645 raft_consensus.cc:614] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:06.956064 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:06.965243 13799 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:06.872751 20294 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:24:06.909560 20656 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:06.875290 20562 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 12:24:06.911873 13646 rpcz_store.cc:276] 0709 12:24:05.642986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.651248 (+  8262us) service_pool.cc:224] Handling call
0709 12:24:05.651596 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:05.651961 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.651996 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:05.752424 (+100428us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:24:05.752933 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.778359 (+ 25426us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:05.806567 (+ 28208us) log.cc:844] Serialized 14 byte log entry
0709 12:24:05.853847 (+ 47280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.853939 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.542056 (+688117us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:06.902284 (+360228us) raft_consensus.cc:1685] finished
0709 12:24:06.902319 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.902593 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47000,"prepare.queue_time_us":28042,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":47649,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":641,"spinlock_wait_cycles":42290688,"thread_start_us":24658,"threads_started":1,"wal-append.queue_time_us":166888}
I20260709 12:24:06.864449 20662 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:06.868762 20117 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:06.969726 20652 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:06.966101 20262 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:06.995877 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:24:06.998796 20681 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:24:07.000763 20695 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:24:07.006305 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 238195us
I20260709 12:24:07.007491 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:06.945960 20539 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 12:24:07.013046 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57012.6us
W20260709 12:24:07.013819 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133023us
I20260709 12:24:07.016403 14422 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:06.991066 20262 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.045985 13800 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:07.049077 13800 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
W20260709 12:24:07.050815 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 159863us
I20260709 12:24:07.055598 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.050123 20262 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:07.079432 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:07.086081 13645 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:07.080395 13719 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:24:07.083777 13799 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
W20260709 12:24:07.087116 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.086849 13719 pending_rounds.cc:85] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:24:07.092895 13719 pending_rounds.cc:107] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:24:07.093227 13719 raft_consensus.cc:2889] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:07.032012 20427 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:24:07.097431 20700 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:07.098007 20700 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:07.098405 20700 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:07.098536 20689 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:24:07.098912 20603 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.609s	user 0.015s	sys 0.005s
I20260709 12:24:07.099431 20603 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:07.103758 20683 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:24:07.049290 20294 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 12:24:07.123041 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115666us
I20260709 12:24:07.124107 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:07.126596 20708 raft_consensus.cc:2764] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:24:07.122611 20695 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:07.141096 13693 leader_election.cc:400] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14.
I20260709 12:24:07.146621 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:07.153568 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:07.154399 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:07.155499 20653 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:07.158593 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:07.160430 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.161360 13644 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:07.161888 20692 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.164408 20692 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.166939 20692 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:07.162451 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.162626 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.171321 20662 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.112156 20625 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.176390 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.127848 20579 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:07.178282 20494 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:07.190054 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.192355 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.174134 20625 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.191618 20598 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.223483 20598 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.231168 20695 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.243302 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.244285 13649 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:07.253387 13726 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:07.254839 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76658.1us
I20260709 12:24:07.255836 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:07.255985 13726 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:07.257407 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:07.258535 20629 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:07.259310 20629 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:07.260318 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.261070 13644 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:24:07.261761 20117 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.264372 20365 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:07.265458 20328 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.609s	user 0.005s	sys 0.008s
I20260709 12:24:07.194804 20668 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.267390 13723 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:07.268360 20668 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.268283 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:07.269250 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:07.271037 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:07.195894 20603 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:07.273554 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.274356 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:07.275401 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:07.278463 20655 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.003s
I20260709 12:24:07.280470 13645 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:07.281776 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:07.282428 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:07.283140 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:07.284129 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:07.285717 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57494.1us
I20260709 12:24:07.287446 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.198235 20626 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.273991 20472 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.278991 20655 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.310639 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:07.312369 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:07.266146 20328 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.310464 19069 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:24:07.319186 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.314586 20262 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.327227 20472 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:07.329445 20678 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.001s
I20260709 12:24:07.331527 20656 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.286347 20707 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:07.341523 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:07.343037 19069 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:07.343160 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:07.345432 13727 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:07.349485 14419 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:07.300709 20117 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:07.298122 20323 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:07.307296 20365 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:07.355105 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.352660 13798 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:07.355619 20603 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:07.355944 13798 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:07.343791 20326 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:07.356567 20326 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:07.356770 20494 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:07.353395 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:07.344740 20505 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:07.356598 13652 raft_consensus.cc:2368] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:07.355239 20365 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:07.356364 20603 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:07.402150 14419 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
W20260709 12:24:07.362042 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2111687130000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 12:24:07.404822 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2012687129999904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 12:24:07.405208 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1283687129998725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:24:07.406450 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1112687129998449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 12:24:07.406849 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0914687129998129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
W20260709 12:24:07.407207 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0419687130001876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
I20260709 12:24:07.355278 20323 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:07.409457 19069 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:07.409888 19069 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:07.403846 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:07.410952 20323 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:07.412353 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.412132 20323 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.386415 20365 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:07.415140 13797 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:24:07.416106 20262 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.385448 20427 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:07.423091 14422 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:07.425453 20674 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:07.427333 20655 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:07.349112 20700 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.429931 20700 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:07.358640 20707 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.431443 20626 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:07.433092 20686 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.377s	user 0.004s	sys 0.000s
I20260709 12:24:07.434386 13792 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:07.357707 20326 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.426132 20674 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:07.435920 20656 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:07.438890 19069 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.439764 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:07.449157 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.449735 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:07.347144 20678 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.312214 20713 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:24:07.441187 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79636.1us
I20260709 12:24:07.450830 14419 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:07.455621 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:07.460752 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.442348 20505 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:07.473459 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:07.483264 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3333) took 1065 ms. Trace:
I20260709 12:24:07.473037 20326 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:07.490864 20505 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.495440 20626 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:07.495947 20626 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.496289 20626 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:07.470193 20494 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:07.502934 20674 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.459641 20385 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:24:07.517342 20365 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.458175 20700 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:24:07.483949 13725 rpcz_store.cc:276] 0709 12:24:06.417788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.652564 (+234776us) service_pool.cc:224] Handling call
0709 12:24:06.670360 (+ 17796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.670825 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.670858 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.671508 (+   650us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.836341 (+164833us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:06.930677 (+ 94336us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.955120 (+ 24443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.955229 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:07.430693 (+475464us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b18002d3880. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db
0709 12:24:07.432520 (+  1827us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:07.482916 (+ 50396us) raft_consensus.cc:1685] finished
0709 12:24:07.482951 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:07.483170 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55706,"prepare.queue_time_us":88608,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":141077,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":25362,"spinlock_wait_cycles":14331008,"thread_start_us":141277,"threads_started":2,"wal-append.queue_time_us":280057}
W20260709 12:24:07.523665 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47043.8us
I20260709 12:24:07.530673 13645 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 18. (term mismatch)
I20260709 12:24:07.531210 13645 pending_rounds.cc:85] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 17
I20260709 12:24:07.531491 13645 pending_rounds.cc:107] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 21.18
I20260709 12:24:07.531597 13649 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:07.531747 13645 raft_consensus.cc:2889] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:07.530560 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57820.4us
I20260709 12:24:07.532850 20365 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.533246 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.540036 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:07.540710 13643 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:07.553328 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.552613 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.553612 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.560887 13795 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:07.561582 13799 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:07.553164 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.562963 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.562105 20472 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:07.571771 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:07.573191 20472 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:07.573284 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.528415 20494 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:07.575201 13796 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:07.574609 20472 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:07.575924 20494 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.577204 20707 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:07.579262 20678 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:07.582638 20567 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:07.589999 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:07.592583 20675 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:07.592849 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.593115 20675 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:07.592588 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.595876 20675 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.599818 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3335) took 1136 ms. Trace:
I20260709 12:24:07.600242 13727 rpcz_store.cc:276] 0709 12:24:06.463662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.887401 (+423739us) service_pool.cc:224] Handling call
0709 12:24:06.887802 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:07.349844 (+462042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:07.599566 (+249722us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54000f5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:07.599756 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051842944}
W20260709 12:24:07.459708 20669 consensus_queue.cc:1175] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dafe08a4f404502822a0732fe570de4" responder_term: 24 status { last_received { term: 24 index: 18 } last_committed_idx: 18 last_received_current_leader { term: 24 index: 18 } }
I20260709 12:24:07.601030 20675 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:07.602793 20671 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:07.603232 20671 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:07.604172 20671 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.594874 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:07.608866 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:07.610589 14420 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:24:07.617727 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:07.634465 13727 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:07.640196 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.635018 20714 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.647305 20505 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:07.647706 20714 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:07.642786 13643 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:07.644603 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:07.645910 20698 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:07.646584 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:07.674451 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:07.677982 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:07.635710 20630 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:07.706616 13643 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:07.718732 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44466.7us
I20260709 12:24:07.721087 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:07.717321 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:07.732002 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:07.735600 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:07.738543 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 95336.1us
I20260709 12:24:07.739461 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.712630 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:07.743168 13645 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:07.660221 20579 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:07.883435 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.731647 20567 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:07.731815 20678 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.731499 20603 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:07.733911 20714 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:07.894348 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 154978us
I20260709 12:24:07.916311 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:07.892550 20579 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.884657 20630 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:07.738469 20323 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.184s
I20260709 12:24:07.914415 20700 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:07.926591 20603 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:07.929733 20579 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:07.915458 20626 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:07.965355 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 221593us
I20260709 12:24:07.922209 20678 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:07.969449 20678 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:07.977702 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:07.972730 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 353723us
I20260709 12:24:07.963107 20472 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:07.954969 20659 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 12:24:07.993206 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106314us
I20260709 12:24:07.999459 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.000447 13649 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:24:08.006667 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:08.016055 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:08.022162 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1460) took 1600 ms. Trace:
I20260709 12:24:08.022745 13724 rpcz_store.cc:276] 0709 12:24:06.421463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.653402 (+231939us) service_pool.cc:224] Handling call
0709 12:24:06.677516 (+ 24114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.682048 (+  4532us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:06.766708 (+ 84660us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.961583 (+194875us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.961653 (+    70us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:06.961722 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.021810 (+1060088us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:08.021914 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:08.022094 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7210,"prepare.queue_time_us":28059,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":143214,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":18827,"spinlock_wait_cycles":9819008,"thread_start_us":254686,"threads_started":3,"wal-append.queue_time_us":185082}
I20260709 12:24:08.018343 20628 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:08.035557 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:08.047823 13544 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 14 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 16 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:24:08.050398 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 318077us
I20260709 12:24:08.032173 20713 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:08.048669 20567 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:07.997781 20630 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.030205 20628 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.028366 20731 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:24:08.034724 20579 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:24:08.054528 20713 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.036094 20323 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
W20260709 12:24:08.068940 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.067886 13544 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:08.060139 20716 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:08.030494 20603 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.086612 20716 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.087073 20716 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:08.087177 13544 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 14 to 16, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 16 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.089032 13544 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 17, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 17 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:08.090652 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 174471us
I20260709 12:24:08.096295 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:08.127501 14422 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:24:08.130553 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.131997 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.133157 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.090849 13544 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: config changed from index 4 to 10, term changed from 14 to 22, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to <none>, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65), VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66), VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67), VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 22 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.135694 20700 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:08.136543 20700 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.136591 13544 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.139103 13544 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.142103 20698 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:08.142927 13544 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 18, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 18 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:08.144294 20472 raft_consensus.cc:2764] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:24:08.146548 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.146981 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:08.147727 14422 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.147842 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42338 (request call id 87) took 1152 ms. Trace:
I20260709 12:24:08.148336 13798 rpcz_store.cc:276] 0709 12:24:06.995448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.995720 (+   272us) service_pool.cc:224] Handling call
0709 12:24:08.147604 (+1151884us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54003d4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bee853 00007f5398bbc60a 00007f53921427d4 00007f5392142762 00007f5392142691 00007f539214260c 00007f53921418c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:08.147748 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818681216}
I20260709 12:24:08.149979 20694 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:08.149842 13544 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.152179 20694 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:08.152593 20694 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:08.153009 20694 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:08.158378 13543 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 232) took 53220 ms (client timeout 15000 ms). Trace:
I20260709 12:24:08.145283 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:08.144977 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62199.5us
I20260709 12:24:08.151932 13544 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 12, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 12 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.162945 13544 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.164073 20567 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.165043 13544 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:08.166772 20117 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:08.167145 20668 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.167474 20117 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.169346 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:08.170351 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.083302 20713 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:08.172534 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.174757 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.175066 20472 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:08.177706 13544 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 17, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 17 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.145807 20626 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:08.141377 20729 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:08.082158 20724 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.180599 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 85620.8us
I20260709 12:24:08.187701 13544 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 15, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 15 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.180406 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.198356 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.201472 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.170120 20117 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.212895 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:08.226861 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 195485us
I20260709 12:24:08.205628 20365 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:08.184465 20562 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:24:08.204973 20626 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:08.208913 20698 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:08.215925 20727 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:08.206705 20472 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
W20260709 12:24:08.253197 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65606.2us
I20260709 12:24:08.259469 14422 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:08.265734 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.266201 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:08.242712 20740 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:08.267735 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66350.6us
I20260709 12:24:08.199993 20724 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.271413 13544 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 20, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 20 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.266829 20472 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:08.264495 20628 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.251219 20626 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.272567 20698 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:08.286243 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.283785 13544 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 18, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 18 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.270478 20731 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
W20260709 12:24:08.158941 13543 rpcz_store.cc:269] 0709 12:23:14.937532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:14.948349 (+ 10817us) service_pool.cc:224] Handling call
0709 12:24:08.158040 (+53209691us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:01.399877 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:01.400607 (+   730us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:01.400665 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:24:01.400685 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:01.401408 (+   723us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:01.401530 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:01.401573 (+    43us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:24:01.402185 (+   612us) tablet.cc:701] Row locks acquired
0709 12:24:01.402248 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:24:01.402886 (+   638us) write_op.cc:270] Start()
0709 12:24:01.403097 (+   211us) write_op.cc:276] Timestamp: P: 1783599841402793 usec, L: 0
0709 12:24:01.403148 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 12:24:01.404142 (+   994us) log.cc:844] Serialized 1602 byte log entry
0709 12:24:01.932034 (+527892us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:02.480716 (+548682us) op_driver.cc:464] REPLICATION: finished
0709 12:24:02.829715 (+348999us) write_op.cc:301] APPLY: starting
0709 12:24:02.947819 (+118104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:03.313913 (+366094us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:03.313957 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:03.330029 (+ 16072us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:03.330599 (+   570us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:24:03.344390 (+ 13791us) write_op.cc:312] APPLY: finished
0709 12:24:03.344880 (+   490us) log.cc:844] Serialized 41 byte log entry
0709 12:24:03.384617 (+ 39737us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:03.400548 (+ 15931us) write_op.cc:454] Released schema lock
0709 12:24:03.408094 (+  7546us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:03.507298 (+ 99204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:03.527839 (+ 20541us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5428355,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":293349,"apply.run_cpu_time_us":11206,"apply.run_wall_time_us":672798,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16691,"num_ops":3,"prepare.queue_time_us":627620,"prepare.run_cpu_time_us":40601,"prepare.run_wall_time_us":193998,"raft.queue_time_us":465952,"raft.run_cpu_time_us":16106,"raft.run_wall_time_us":269372,"replication_time_us":1077361,"spinlock_wait_cycles":66838144,"thread_start_us":1024996,"threads_started":4,"wal-append.queue_time_us":209769,"wal-append.run_cpu_time_us":27178,"wal-append.run_wall_time_us":2748666}]]}
I20260709 12:24:08.264640 20365 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.297997 20716 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.299034 20695 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:08.299443 20695 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:08.300436 20695 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.300943 20716 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:08.283401 20668 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:08.308933 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:08.295589 20603 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:24:08.309648 20745 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:08.312249 20737 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:24:08.313735 20365 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.312281 20745 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.315553 20716 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.316380 20716 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.316936 20745 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:08.296171 20740 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:08.308111 13544 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 23, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 23 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.265939 20727 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.331257 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.297349 20698 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:08.300969 20724 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:08.307888 20695 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:08.340613 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54682.2us
I20260709 12:24:08.334538 13544 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:08.344507 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 116818us
I20260709 12:24:08.325340 20626 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.345880 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:08.347193 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:08.354455 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.329907 20707 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:08.354871 13800 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:08.349511 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:08.315495 20694 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.296964 20741 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:24:08.357573 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:08.364280 20323 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:08.365214 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:08.365885 13795 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.374145 20716 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.374817 20740 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:08.375663 20564 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:08.364732 20698 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.341388 20700 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:08.379912 14422 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:08.383359 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:08.393784 13800 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:08.393437 13544 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.332780 20713 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.360323 20668 raft_consensus.cc:2764] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:24:08.435711 20747 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:08.436324 20747 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.436635 20734 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:08.437168 20734 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:08.437392 20713 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.376219 20564 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:08.435294 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85851.2us
I20260709 12:24:08.457805 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.403260 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:08.472121 13724 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:24:08.453006 13801 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:24:08.438565 13797 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:24:08.474989 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.478770 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.479221 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:08.479818 13719 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:08.480950 13643 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:08.450307 13544 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 20, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 20 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.457804 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:08.438817 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81311.4us
I20260709 12:24:08.483349 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:08.483839 13693 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:08.484192 13693 leader_election.cc:403] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:08.450446 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:08.443813 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.364799 20323 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:08.468001 13795 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:08.486406 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:08.490068 13723 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:08.398524 20472 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.398712 20740 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:08.499455 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.501458 13795 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:08.455142 20716 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:08.508234 13544 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 19, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 19 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.432500 20328 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:08.485716 20713 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:08.436697 20747 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:08.439890 20727 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:08.505739 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:08.509305 13793 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:08.495265 20700 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.490828 20741 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:08.512328 13544 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.496007 20647 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:08.441013 20694 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:08.522724 20647 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.527000 20647 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:08.530489 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.533015 13544 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 14, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.534225 19069 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:08.509439 20472 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.633152 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:08.641310 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:08.642338 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:08.646529 13794 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:08.541883 20626 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:24:08.625576 20759 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:08.518932 20716 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.663887 13792 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
W20260709 12:24:08.669698 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 139284us
I20260709 12:24:08.682330 13792 pending_rounds.cc:85] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 12
I20260709 12:24:08.682662 13792 pending_rounds.cc:107] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:08.670315 13544 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 21, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 21 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.528950 20741 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.662298 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:08.682685 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.518510 20713 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.691493 20262 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:08.693856 13719 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:08.694437 20746 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:08.699044 20117 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:08.699719 20117 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.670047 20628 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:24:08.682960 13792 raft_consensus.cc:2889] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:08.667724 20626 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:08.438191 20734 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.725993 13544 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.702093 20117 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:08.657261 20630 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:24:08.755584 20714 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:08.715485 20668 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:08.758456 13544 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.636193 20323 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.721031 20494 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:24:08.764034 20564 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:08.760185 20755 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:24:08.760356 20714 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:08.763810 20323 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:08.773418 20713 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:08.761436 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71195.3us
I20260709 12:24:08.728976 20626 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.767915 20564 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:08.781046 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.772480 20714 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.783277 13719 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:24:08.784627 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.785982 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.795682 13722 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:08.786757 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.786831 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.796814 13726 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:08.797082 13721 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:24:08.789122 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.791553 20741 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:08.802656 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.813514 13718 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:08.816006 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 321646us
I20260709 12:24:08.816751 20262 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:08.813500 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:08.817234 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.820991 13544 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.822129 13544 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 23, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 23 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:08.823829 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.825995 13544 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 9, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 9 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:08.827435 13544 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 10, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 10 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:08.829447 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:08.761325 20365 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:24:08.830178 13544 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:08.829921 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:08.833384 13722 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:08.834729 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.834836 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:08.824680 13719 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24. Re-sending same reply.
I20260709 12:24:08.835624 13721 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:08.837141 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:08.837996 13726 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.782166 20564 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:08.841065 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.841362 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.847440 20714 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:08.850235 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.855105 13719 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:08.855449 20734 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:08.857659 13719 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:24:08.859548 20659 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:08.860045 20659 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:08.861033 20659 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.863055 20262 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:08.871030 20659 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:08.873199 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56092.7us
W20260709 12:24:08.874049 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.874825 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.876223 20763 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:24:08.845731 20750 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 12:24:08.819237 20669 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:24:08.890455 14420 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:08.910737 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.912681 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:08.913506 14418 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:08.915562 13726 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:08.918202 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:08.918943 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:08.919790 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:08.920372 13643 raft_consensus.cc:606] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:08.920852 13643 raft_consensus.cc:614] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:08.937208 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.942622 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:08.943429 13727 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:08.946830 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:08.899271 20731 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.967936 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:08.918908 20472 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:08.974691 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:08.939791 20695 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.000s
I20260709 12:24:08.976305 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.978000 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.931991 20117 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:08.978979 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.983301 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:08.990176 13645 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:08.991606 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:08.994217 13649 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:08.996701 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:08.997439 13643 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.004612 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:09.005654 13725 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:09.006758 20715 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:09.008795 13643 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:09.009305 13725 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:09.009868 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:09.021064 13725 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:09.022432 13725 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:09.024811 13725 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:09.024889 20626 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.026399 13718 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:09.027184 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.028431 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:09.028972 13651 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.029296 13651 raft_consensus.cc:614] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:09.030501 20762 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:08.988461 20117 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.031440 20117 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:08.920693 20640 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.060s
I20260709 12:24:09.037684 13718 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:09.021030 13722 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:09.040241 13719 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: [18.12-19.13]
I20260709 12:24:09.060959 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.061774 13651 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:09.062598 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.063208 13644 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.063895 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:09.076927 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:08.979168 20716 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.085947 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:09.093573 13645 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:09.093400 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.094353 13649 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:08.970405 20731 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:08.973249 20703 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 12:24:09.100433 13693 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:09.101418 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.101898 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:09.102450 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:09.111145 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.017769 20647 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.092108 20695 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.117178 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:09.119056 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:09.122447 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:09.116935 20746 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.132066 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.153435 13647 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:09.114645 20472 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:09.147146 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45336.4us
I20260709 12:24:09.116276 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.151032 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:09.165912 13649 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:09.148494 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.163976 13651 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:09.139437 13644 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:09.171931 13643 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:09.147588 13723 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:09.172458 13723 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:09.146669 13724 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 12:24:09.146826 13794 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:09.173732 13724 pending_rounds.cc:85] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:24:09.138789 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.173992 13724 pending_rounds.cc:107] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:24:09.124974 13722 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:09.174295 13724 raft_consensus.cc:2889] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:09.179109 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.123294 20713 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:24:09.186940 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70743.6us
I20260709 12:24:09.188235 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:09.193809 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:09.194851 13722 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.173218 13723 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:09.134997 20700 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.005s
I20260709 12:24:09.197120 13723 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:09.154239 20775 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:24:09.138195 20603 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:09.208313 13796 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:09.196380 20703 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:09.210357 20764 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:09.205869 20700 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:09.142293 20494 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:09.142578 20626 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:09.151731 20628 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.155025 20365 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.134153 20731 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.245044 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.247581 14422 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:09.219153 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:09.136662 20647 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:09.193156 20746 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:09.251416 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:09.239315 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53828.2us
W20260709 12:24:09.253985 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61477.3us
I20260709 12:24:09.134899 20695 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.263388 13644 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:09.211704 20603 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:09.229614 20747 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.705s	user 0.000s	sys 0.015s
I20260709 12:24:09.267910 13644 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:24:09.276971 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.271751 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52681.6us
I20260709 12:24:09.282842 13723 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 12:24:09.283485 13723 pending_rounds.cc:85] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 17
I20260709 12:24:09.283780 13723 pending_rounds.cc:107] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 12:24:09.284089 13723 raft_consensus.cc:2889] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:09.289495 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.223013 20494 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:09.270902 20766 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:09.217505 20764 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.191166 20748 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:24:09.222218 20713 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:09.325035 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:09.333127 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54908.6us
I20260709 12:24:09.393903 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:09.373778 13792 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
W20260709 12:24:09.374881 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 123559us
I20260709 12:24:09.375717 20735 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.399752 20735 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.252282 20472 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:09.402535 20472 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.403128 20472 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.407877 20735 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:09.389382 20674 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.410249 20659 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:09.411003 20674 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.413460 20674 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:09.415030 20780 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:24:09.275341 20731 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:09.421108 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.421277 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:09.430192 13651 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:09.423692 20784 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:09.425001 20494 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:09.380371 20734 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.003s
I20260709 12:24:09.441406 13794 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:09.445120 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:09.267750 20505 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.158s
I20260709 12:24:09.331700 20662 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:09.252640 20628 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.391222 20764 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:09.264916 20365 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:09.441960 20323 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.000s
I20260709 12:24:09.455550 20674 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:09.458691 20674 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:09.459087 20674 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:09.459920 20695 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.451005 20603 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.329478 20747 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.433252 20731 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:09.441570 20780 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:09.387142 13720 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:09.490164 13722 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:09.545598 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.546469 13645 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:09.448205 20713 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:09.452688 20494 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:09.269286 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81191.3us
I20260709 12:24:09.578619 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.580138 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:09.581043 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130522us
I20260709 12:24:09.581667 13693 leader_election.cc:400] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
W20260709 12:24:09.583783 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.584262 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.537528 20747 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:09.545030 20780 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:09.583395 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:09.543931 20731 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:09.570974 20505 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.420120 20766 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.596621 13645 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:24:09.597640 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.609407 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:09.582707 20746 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:09.582523 20695 raft_consensus.cc:2764] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:24:09.618763 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 221707us
I20260709 12:24:09.456125 20662 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:09.596269 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.513s (SENT) [suppressed 21 similar messages]
W20260709 12:24:09.647104 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 390723us
I20260709 12:24:09.604350 20780 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:09.651698 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.609436 20786 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:09.661043 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:09.671669 20365 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.658397 20716 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:09.675235 20628 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.675585 20695 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:09.676180 20695 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:09.659056 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62828.9us
W20260709 12:24:09.680363 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1736936680003964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 12:24:09.680837 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1393847140002435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 12:24:09.678973 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:09.679066 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.683790 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:09.684837 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:09.685086 13726 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:09.686491 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:09.681293 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0542659890002142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 12:24:09.687292 13723 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:09.687436 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.653151 20626 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:09.692775 13646 raft_consensus.cc:2368] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:09.607952 20745 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:09.695375 20745 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:09.692253 20700 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:09.699033 20700 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:09.700410 20700 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.702986 20695 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.698511 20745 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.704794 20603 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:09.705750 20745 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:09.706624 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.651710 20678 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:24:09.707858 20472 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.710062 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:09.711110 20765 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:09.712572 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1092) took 1447 ms. Trace:
I20260709 12:24:09.712781 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.713186 14421 rpcz_store.cc:276] 0709 12:24:08.265323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.265472 (+   149us) service_pool.cc:224] Handling call
0709 12:24:09.712255 (+1446783us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:09.712499 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257775360}
I20260709 12:24:09.713898 20765 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:09.710065 20736 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:09.714742 20765 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:09.715077 20736 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:09.715178 20765 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:09.699189 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:09.720028 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.717934 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.630663 20747 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 12:24:09.693219 20714 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.840s	user 0.016s	sys 0.000s
I20260709 12:24:09.726528 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.724215 20365 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.615517 20505 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.730926 20789 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:09.724551 20675 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:09.737344 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:09.740563 13693 leader_election.cc:400] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:09.743696 13693 leader_election.cc:403] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:09.746115 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:09.749754 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:09.750741 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 102155us
I20260709 12:24:09.743075 20628 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.692585 20716 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:24:09.754771 20628 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.761401 20628 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.743321 20630 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.762651 20630 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.745462 20675 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:09.742976 20365 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.764957 20630 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.765833 20365 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.743791 20790 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:09.768164 20790 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.768566 20790 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:09.769101 13792 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:09.749732 20746 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:09.762496 20789 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.772485 20789 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:09.762012 20628 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:09.780454 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:09.723941 20736 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.784225 20736 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:09.795113 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:09.816058 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 160215us
W20260709 12:24:09.825250 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73811.5us
W20260709 12:24:09.820209 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.832258 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112351us
W20260709 12:24:09.833428 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 89703us
I20260709 12:24:09.836156 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.836364 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3414) took 1054 ms. Trace:
I20260709 12:24:09.836906 13721 rpcz_store.cc:276] 0709 12:24:08.781713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.845194 (+ 63481us) service_pool.cc:224] Handling call
0709 12:24:08.884810 (+ 39616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.710334 (+825524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.710368 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.710415 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:09.710447 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.836005 (+125558us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b54003fd4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:09.836088 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.836279 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897866112}
I20260709 12:24:09.820729 20746 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:24:09.837984 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.841266 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.838887 13726 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:09.843694 13796 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
W20260709 12:24:09.829190 20117 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.778s	user 0.011s	sys 0.004s
I20260709 12:24:09.844899 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3404) took 1060 ms. Trace:
I20260709 12:24:09.846302 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:09.845850 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:09.853346 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.857735 20662 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:09.852021 13720 rpcz_store.cc:276] 0709 12:24:08.777849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.802785 (+ 24936us) service_pool.cc:224] Handling call
0709 12:24:09.838316 (+1035531us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54003bf454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:09.838450 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972552448}
I20260709 12:24:09.826958 20786 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:09.793674 20764 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.864478 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1140) took 1670 ms. Trace:
I20260709 12:24:09.922916 13797 rpcz_store.cc:276] 0709 12:24:08.194004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.197836 (+  3832us) service_pool.cc:224] Handling call
0709 12:24:08.198389 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.475295 (+276906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.861847 (+1386552us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b5400287dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:09.864380 (+  2533us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551648000}
I20260709 12:24:09.925264 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:09.926746 20668 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.927433 20668 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.932308 20764 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.934460 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:09.934888 20505 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.937031 20674 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.938726 20800 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:09.952629 20674 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:09.952234 20765 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.955253 20786 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.958415 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:09.962671 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3362) took 1021 ms. Trace:
I20260709 12:24:09.963156 13719 rpcz_store.cc:276] 0709 12:24:08.941365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.941498 (+   133us) service_pool.cc:224] Handling call
0709 12:24:09.039959 (+ 98461us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:09.049292 (+  9333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.962432 (+913140us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b54000f6750. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:09.962595 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159323520}
I20260709 12:24:09.964339 20746 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:09.964906 20746 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.966830 20780 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.969787 20780 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:09.973776 20740 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:09.974725 20740 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.861223 20117 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:09.977732 20740 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:09.958570 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:10.017982 13796 raft_consensus.cc:2368] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:10.019322 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.020534 13796 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:10.020733 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:10.019354 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:10.022785 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:09.839666 20628 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.165s
I20260709 12:24:10.028695 20765 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.029263 20564 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:09.930024 20668 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.033058 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:10.030906 14422 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:24:10.038849 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 205574us
I20260709 12:24:10.039646 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:10.040856 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.042085 13795 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:10.015494 20468 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.015990 20262 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:09.967581 20746 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.051692 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:10.025158 20731 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.053190 20468 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.089401 20468 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:10.076622 20262 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.086783 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.051766 20564 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:10.045071 20765 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.112884 20746 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:10.113647 20746 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.115573 20746 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.136142 20671 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:10.139544 20675 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.141131 20675 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:10.148543 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.089423 20731 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:10.163137 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:10.164361 13799 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:10.164803 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:10.165452 13723 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:10.166070 13797 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:24:10.166760 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:10.167182 13720 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:10.167308 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.175931 13796 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:10.177870 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:10.174819 13723 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:24:10.179574 13797 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:10.180707 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:10.181743 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:10.186532 13719 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:24:10.184873 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139256us
I20260709 12:24:10.189011 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:10.190868 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.193102 13797 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:10.194607 20765 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:10.195143 20765 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:10.195537 20765 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.198040 14420 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 16. Ops: []
I20260709 12:24:10.200654 20736 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:10.205426 13796 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:10.208552 13720 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:10.210634 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:10.211200 13797 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:10.219332 13797 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:10.223109 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3752) took 1077 ms. Trace:
I20260709 12:24:10.228091 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:10.223668 13794 rpcz_store.cc:276] 0709 12:24:09.145565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.203988 (+ 58423us) service_pool.cc:224] Handling call
0709 12:24:09.227251 (+ 23263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.443975 (+216724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.222814 (+778839us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:10.223040 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472345856}
I20260709 12:24:10.178301 20117 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:10.284932 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74412.5us
W20260709 12:24:10.285223 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.289069 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.208917 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:10.292578 13795 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 13: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:10.299162 13723 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
W20260709 12:24:10.301239 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.296969 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:10.302790 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 74855.9us
I20260709 12:24:10.298758 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:10.302165 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93381.8us
I20260709 12:24:10.303593 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:10.307753 14420 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:24:10.231880 20763 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.314952 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3743) took 1800 ms. Trace:
I20260709 12:24:10.213008 20628 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:24:10.297914 20468 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.172318 20790 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.294818 20745 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.315284 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:10.294620 20698 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:10.302691 20714 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:10.307149 20117 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:10.278697 20789 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:24:10.322618 20468 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.315414 13793 rpcz_store.cc:276] 0709 12:24:08.515029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.516305 (+  1276us) service_pool.cc:224] Handling call
0709 12:24:08.529489 (+ 13184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.729127 (+199638us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:08.834771 (+105644us) log.cc:844] Serialized 14 byte log entry
0709 12:24:08.921836 (+ 87065us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:08.921905 (+    69us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:08.921960 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.314726 (+1392766us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54001608d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:10.314781 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.314894 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28323,"prepare.queue_time_us":25640,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":166,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":16559,"spinlock_wait_cycles":458517760,"thread_start_us":177764,"threads_started":2,"wal-append.queue_time_us":265611,"wal-append.run_cpu_time_us":1427,"wal-append.run_wall_time_us":1026202}
I20260709 12:24:10.351207 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:10.349045 20472 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.349588 20468 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.341027 20698 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.359731 20472 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:10.360437 20472 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.361567 20698 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.320091 20763 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.331986 20603 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:24:10.322150 20746 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:24:10.343761 20117 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:10.333668 20745 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:10.378670 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67193.1us
I20260709 12:24:10.364867 20714 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:10.384153 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:10.468340 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84325.5us
I20260709 12:24:10.470064 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:10.473997 13723 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:10.478511 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.513609 13646 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:10.517594 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3846) took 1418 ms. Trace:
I20260709 12:24:10.518160 13643 rpcz_store.cc:276] 0709 12:24:09.098822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.099011 (+   189us) service_pool.cc:224] Handling call
0709 12:24:09.171726 (+ 72715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.172326 (+   600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.517331 (+1345005us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b5400198050. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:10.517518 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166378240}
I20260709 12:24:10.524029 13646 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:10.564386 13651 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:10.568846 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3412) took 1779 ms. Trace:
I20260709 12:24:10.569530 13718 rpcz_store.cc:276] 0709 12:24:08.781171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.820644 (+ 39473us) service_pool.cc:224] Handling call
0709 12:24:10.560779 (+1740135us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001f0754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:10.561000 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438659968}
I20260709 12:24:10.464108 20787 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.197s
I20260709 12:24:10.392252 20472 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.592684 13800 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:10.594748 20806 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:24:10.595383 20766 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.595950 20727 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:10.596866 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:10.597278 20727 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:10.598214 20745 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:10.600700 20764 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:10.602056 20262 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.598248 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:10.605430 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:10.608232 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.608942 20262 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.610272 20715 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.610921 20715 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.612094 20262 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:10.613183 20715 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.614053 20262 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:10.614841 20262 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:10.615553 20790 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.615387 20727 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.514907 20668 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.616560 20727 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.615752 20262 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.616631 20668 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:10.609131 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.619077 20668 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:10.619357 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.620046 20763 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.566604 20505 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:10.621474 20505 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:10.622174 20494 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:10.585698 20468 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.628302 20468 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:10.566917 20695 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:10.630729 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:10.630475 20695 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.631361 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:10.632153 13794 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:10.464929 20698 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:10.635277 20808 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.300s	user 0.003s	sys 0.000s
I20260709 12:24:10.635746 20698 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.511932 20786 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.639228 20647 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:10.639040 20786 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.639715 20647 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:10.640286 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.640693 20647 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.622397 20505 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:24:10.656252 14391 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.12.45.126:44051, user_credentials={real_user=slave}} blocked reactor thread for 80020.8us
I20260709 12:24:10.647655 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:10.674610 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:10.657160 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.702500 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 11.
I20260709 12:24:10.702824 13723 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:10.656008 13650 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:24:10.642652 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:10.613598 20117 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.714948 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.724984 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:10.752985 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:10.730906 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60056.9us
I20260709 12:24:10.649212 20262 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:10.658787 14450 heartbeater.cc:646] Failed to heartbeat to 127.12.45.126:44051 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.45.126:44051 timed out after 15.000s (SENT)
W20260709 12:24:10.786759 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.772078 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.641775 20647 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:10.511785 20795 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.244s
W20260709 12:24:10.832115 20671 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.004s
W20260709 12:24:10.802780 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65858us
I20260709 12:24:10.566283 20628 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:10.796198 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.815659 13648 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
W20260709 12:24:10.807170 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.642108 20786 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:10.826040 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:10.844199 13798 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:10.818782 13646 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:10.845896 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:10.644516 20494 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:10.847518 13797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.023s	sys 0.059s
I20260709 12:24:10.820400 20698 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:10.848483 13797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.023s	sys 0.059s
I20260709 12:24:10.822608 20739 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:10.828328 20695 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:10.774082 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.569816 20714 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:10.764204 20656 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.817229 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:10.843005 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.029s	sys 0.052s
W20260709 12:24:10.874073 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.029s	sys 0.052s
W20260709 12:24:10.875087 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 559838us
I20260709 12:24:10.819408 20687 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:10.711696 20505 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:10.848552 20688 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.847026 20494 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:10.854156 13796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.055s	sys 0.087s
I20260709 12:24:10.866358 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:10.843328 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.036s	sys 0.043s
W20260709 12:24:10.895200 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.036s	sys 0.044s
W20260709 12:24:10.895814 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 575357us
I20260709 12:24:10.868700 20784 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:10.901031 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:10.901993 13718 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:10.903400 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.906097 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:10.850935 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:10.906947 13723 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:10.887626 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:10.907902 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:10.854636 20736 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:10.872431 20671 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:10.850018 14422 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
W20260709 12:24:10.894701 13796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.055s	sys 0.087s
W20260709 12:24:10.900889 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1146) took 2555 ms (client timeout 2917 ms). Trace:
W20260709 12:24:10.911386 13798 rpcz_store.cc:269] 0709 12:24:08.338744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.338900 (+   156us) service_pool.cc:224] Handling call
0709 12:24:10.894308 (+2555408us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b54002cb5d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:10.894453 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5727418240}
I20260709 12:24:10.905889 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.855381 20790 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.915436 13649 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:10.920714 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:10.843552 13795 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.020s	sys 0.058s
W20260709 12:24:10.923117 13795 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.020s	sys 0.058s
I20260709 12:24:10.868281 20669 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:24:10.924146 20326 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:10.924963 20669 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:10.872738 20733 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:24:10.925176 20795 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:24:10.909732 20765 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.000s
I20260709 12:24:10.938027 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.938915 13646 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:10.901758 20784 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:10.902523 20695 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:10.949764 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:10.925359 20326 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:10.912150 20796 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.881237 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.955197 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:10.888816 20687 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.886278 20656 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:10.951436 20564 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.853s	user 0.005s	sys 0.007s
I20260709 12:24:10.961524 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:10.924964 20800 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:10.965409 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:10.971927 13726 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:24:10.926023 20671 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.925408 20669 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:10.898972 20688 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.956995 20795 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:10.926602 20790 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:10.957295 20784 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.987107 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:10.991523 13727 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:10.987628 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:10.994860 13649 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:10.996551 20790 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.963840 20326 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:10.998042 20326 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:10.999404 20326 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:10.965292 20695 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.004895 20695 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:11.005533 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3756) took 1164 ms. Trace:
I20260709 12:24:10.986740 20800 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.006053 13796 rpcz_store.cc:276] 0709 12:24:09.841241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.206634 (+365393us) service_pool.cc:224] Handling call
0709 12:24:10.207042 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.207347 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.207385 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.230350 (+ 22965us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.911420 (+681070us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:10.911466 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.911533 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.005300 (+ 93767us) raft_consensus.cc:1685] finished
0709 12:24:11.005323 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.005465 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":603881,"threads_started":1,"wal-append.queue_time_us":692185}
I20260709 12:24:11.008224 20800 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.009217 20494 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:10.951583 20765 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.011489 20765 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.012144 13643 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:10.994938 20809 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:11.015329 13723 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:10.973655 20505 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.018464 13796 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:24:11.026276 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:11.031598 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:11.032411 13720 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:11.034438 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:11.035347 13720 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:24:10.989569 20675 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.848s	user 0.015s	sys 0.002s
I20260709 12:24:10.986351 20799 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.014150 20687 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:11.057142 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.058764 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.062055 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:11.062778 13720 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:11.060205 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.063925 13794 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:11.049301 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:11.071959 14420 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:10.974045 20796 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.074613 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:11.081233 13720 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
W20260709 12:24:11.082394 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.042850 20731 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.962363 20714 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:11.046243 20804 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.000s	sys 0.016s
I20260709 12:24:11.104429 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:11.117786 13727 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:11.072731 20505 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:11.137715 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:24:11.130916 20804 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.002s	sys 0.016s
I20260709 12:24:11.138250 13718 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:11.122116 13649 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
W20260709 12:24:11.154861 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.156467 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.158133 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.158996 13796 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:11.160614 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.115321 20800 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:11.163085 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.135701 20790 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:24:11.163889 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.163547 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 490416us
W20260709 12:24:11.178187 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66233.6us
I20260709 12:24:11.064977 20799 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.175401 13798 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:11.182442 20800 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:11.186676 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:11.206983 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.190673 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.191973 20698 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:11.193193 20786 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.196563 20740 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:11.174377 20700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.022s	sys 0.041s
I20260709 12:24:11.198781 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:11.199119 20671 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:11.161298 20669 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.200698 20656 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:11.208969 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.213584 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:11.180797 20790 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:11.227073 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47992.3us
I20260709 12:24:11.253883 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:11.124624 20731 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.168226 20739 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.234872 20700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.022s	sys 0.041s
I20260709 12:24:11.257542 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:11.215106 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:11.275815 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.264809 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:11.277918 13617 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.017s	sys 0.022s
W20260709 12:24:11.278723 13617 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.017s	sys 0.023s
W20260709 12:24:11.279008 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 555474us
I20260709 12:24:11.286937 13721 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:11.287297 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:11.258783 20790 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:11.307081 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:11.319181 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:11.339227 20796 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.341717 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:11.342010 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:11.376358 20799 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.379236 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:11.381732 13643 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:11.381717 13796 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:11.384747 20799 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:11.228752 20698 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:11.396241 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.230086 20786 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.410173 20468 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.468818 20468 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.470214 20786 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.471879 20468 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:11.161007 20795 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.294793 20757 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:11.297305 20328 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:11.241317 20740 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.319515 20768 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:11.396602 20736 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:11.288079 20814 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.199s
I20260709 12:24:11.480353 20781 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.279513 20739 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:11.502853 13797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.019s	sys 0.011s
W20260709 12:24:11.505445 13797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.019s	sys 0.011s
I20260709 12:24:11.304379 20813 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.190s
W20260709 12:24:11.497661 20262 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.002s
I20260709 12:24:11.488036 20757 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.491364 20328 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.515733 13650 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:11.512876 20781 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.555199 20781 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:11.555831 20739 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:11.558434 20784 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.003s
I20260709 12:24:11.558920 20784 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.560091 14417 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:11.561194 20784 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.530139 20757 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:11.514133 20768 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:11.534530 20328 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:11.563207 20326 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
W20260709 12:24:11.585354 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.485177 20800 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:11.637835 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.629542 20117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.012s	sys 0.023s
I20260709 12:24:11.673558 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3378) took 1511 ms. Trace:
I20260709 12:24:11.674549 13721 rpcz_store.cc:276] 0709 12:24:10.162100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.162319 (+   219us) service_pool.cc:224] Handling call
0709 12:24:10.177807 (+ 15488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.287740 (+1109933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.673134 (+385394us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54000f5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:11.673458 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2549961984}
I20260709 12:24:11.640166 20814 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:11.604803 20736 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:11.750300 13801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.008s	sys 0.082s
W20260709 12:24:11.747164 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.023s	sys 0.060s
W20260709 12:24:11.732527 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.001s	sys 0.042s
W20260709 12:24:11.748847 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.000s	sys 0.047s
W20260709 12:24:11.717533 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.649775 20117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.013s	sys 0.023s
W20260709 12:24:11.820302 13801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.015s	sys 0.082s
W20260709 12:24:11.821602 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.005s	sys 0.042s
W20260709 12:24:11.822077 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 956392us
W20260709 12:24:11.820693 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.025s	sys 0.066s
W20260709 12:24:11.820997 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.000s	sys 0.048s
I20260709 12:24:11.823433 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:11.824524 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 936935us
I20260709 12:24:11.824819 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3768) took 1196 ms. Trace:
I20260709 12:24:11.825747 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:11.721345 20659 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.021s	sys 0.064s
W20260709 12:24:11.828270 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.014s	sys 0.082s
W20260709 12:24:11.711932 20647 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.862s	user 0.013s	sys 0.001s
W20260709 12:24:11.752470 20811 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.822s	user 0.002s	sys 0.004s
W20260709 12:24:11.828734 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.014s	sys 0.082s
I20260709 12:24:11.698698 20326 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:11.860106 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.721979 20639 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.853315 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 546333us
I20260709 12:24:11.840147 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:11.813880 20700 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:11.814862 20786 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:11.722294 20731 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:11.884081 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1162) took 1656 ms. Trace:
I20260709 12:24:11.868703 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:11.856238 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:11.870100 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:11.873721 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:11.891819 13651 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:11.876952 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:11.820638 20472 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:11.857765 20790 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:11.834794 20659 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.021s	sys 0.066s
I20260709 12:24:11.813961 20814 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:11.883324 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:11.825632 20671 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:11.891325 14418 raft_consensus.cc:2410] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:24:11.868263 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:11.891398 13643 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
W20260709 12:24:11.914045 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45924.5us
I20260709 12:24:11.759493 20698 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:11.894609 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:11.896317 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.014s	sys 0.016s
W20260709 12:24:11.924041 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.015s	sys 0.016s
I20260709 12:24:11.924860 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:11.898761 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:11.909808 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:11.677351 20768 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.861325 20740 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.889951 20753 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:11.929407 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.894755 13795 rpcz_store.cc:276] 0709 12:24:10.227578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.317172 (+ 89594us) service_pool.cc:224] Handling call
0709 12:24:10.317422 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.317613 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:10.937090 (+619477us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.882053 (+944963us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.882190 (+   137us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:11.883704 (+  1514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.883871 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.884018 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134223,"prepare.queue_time_us":668935,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":155759,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5821,"thread_start_us":1397830,"threads_started":3,"wal-append.queue_time_us":711159}
W20260709 12:24:11.975853 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.978006 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:11.978787 13718 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:11.928517 20698 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:11.980824 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:11.882887 20326 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:11.985141 13727 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:11.926050 20262 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:11.927889 20841 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:11.883293 20639 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.894002 20786 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.004089 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3383) took 1831 ms. Trace:
I20260709 12:24:11.943719 20628 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:11.974026 20468 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:11.982208 20740 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:11.896670 20700 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:11.908551 20472 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.013044 13722 rpcz_store.cc:276] 0709 12:24:10.172252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.172376 (+   124us) service_pool.cc:224] Handling call
0709 12:24:10.214566 (+ 42190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.215090 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:10.215131 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.215774 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.378217 (+162443us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:10.378281 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.378350 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.566043 (+1187693us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:11.989999 (+423956us) raft_consensus.cc:1685] finished
0709 12:24:11.990033 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.003986 (+ 13953us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4244,"raft.queue_time_us":959,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":56452,"spinlock_wait_cycles":14830336,"thread_start_us":73628,"threads_started":1,"wal-append.queue_time_us":587600}
W20260709 12:24:12.021231 13794 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.001s
I20260709 12:24:11.929641 20494 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:24:12.020147 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:12.027506 13794 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.005s	sys 0.001s
I20260709 12:24:12.038017 20740 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:12.019944 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 175079us
I20260709 12:24:12.039562 20740 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.043758 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:12.018934 20628 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.015966 20801 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:11.990825 20781 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:24:12.021672 20468 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:12.058773 20833 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 12:24:12.062894 20838 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 12:24:12.063803 20836 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:24:12.017989 20841 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.065629 20326 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:11.993904 20698 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.070644 20688 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.066675 20472 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:12.072394 20834 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:24:12.080335 20786 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.113185 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3766) took 2084 ms. Trace:
I20260709 12:24:12.113438 20786 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.113962 20786 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.068871 20468 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:12.072618 20812 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:12.114856 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1168) took 1599 ms. Trace:
I20260709 12:24:12.072228 20781 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.117552 20786 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.096035 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.118495 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2524) took 1385 ms. Trace:
I20260709 12:24:12.118602 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:12.114692 20117 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.049379 20628 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.119503 20117 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.122717 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.119057 13650 rpcz_store.cc:276] 0709 12:24:10.733430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.733625 (+   195us) service_pool.cc:224] Handling call
0709 12:24:12.118247 (+1384622us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400097c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:12.118411 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1560662784}
I20260709 12:24:12.125694 20472 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.126345 20472 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.127106 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.066944 20801 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:12.130611 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 276256us
I20260709 12:24:12.132201 20801 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:12.133303 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 309988us
I20260709 12:24:11.825901 13801 rpcz_store.cc:276] 0709 12:24:10.628216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.628365 (+   149us) service_pool.cc:224] Handling call
0709 12:24:10.628733 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.628978 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:11.103689 (+474711us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.822067 (+718378us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.823865 (+  1798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:11.823939 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:11.824016 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.824130 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.824303 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":374101,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":1706,"spinlock_wait_cycles":8553856,"thread_start_us":1184692,"threads_started":2}
W20260709 12:24:12.135068 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.114765 20494 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.116127 20630 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:12.115401 13652 rpcz_store.cc:276] 0709 12:24:10.515940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.525812 (+  9872us) service_pool.cc:224] Handling call
0709 12:24:10.573501 (+ 47689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.573782 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:24:11.137633 (+563851us) log.cc:844] Serialized 14 byte log entry
0709 12:24:12.114241 (+976608us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b1800482860. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:24:12.114448 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:12.114513 (+    65us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:12.114565 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.114669 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.114797 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":738863,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":51623,"raft.queue_time_us":19475,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":3904768,"thread_start_us":727754,"threads_started":2}
W20260709 12:24:12.095283 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51654.4us
W20260709 12:24:12.106963 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 164095us
I20260709 12:24:12.121850 20117 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.113668 13797 rpcz_store.cc:276] 0709 12:24:10.028520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.221544 (+193024us) service_pool.cc:224] Handling call
0709 12:24:10.221901 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.848951 (+627050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.848986 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.880425 (+ 31439us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.505888 (+625463us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:11.505936 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.506012 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.112878 (+606866us) raft_consensus.cc:1685] finished
0709 12:24:12.112903 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.113117 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126654,"raft.queue_time_us":113553,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":5718,"thread_start_us":1142153,"threads_started":2,"wal-append.queue_time_us":1102440}
W20260709 12:24:12.161911 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 229779us
I20260709 12:24:12.136278 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:12.160553 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:12.115497 20688 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:12.170962 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:12.174770 13693 leader_election.cc:400] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:12.181537 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:12.041410 20814 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.192967 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:12.195120 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:12.196134 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:12.156014 20801 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:12.169951 20838 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:12.146692 20472 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:12.199748 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.207593 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:12.178184 20647 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.212927 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:12.214010 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3397) took 1171 ms. Trace:
I20260709 12:24:12.236912 13719 rpcz_store.cc:276] 0709 12:24:11.042991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.043237 (+   246us) service_pool.cc:224] Handling call
0709 12:24:11.043615 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.043988 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.044018 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.055494 (+ 11476us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.499317 (+443823us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:11.499384 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.499468 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.213767 (+714299us) raft_consensus.cc:1685] finished
0709 12:24:12.213795 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.213939 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":506,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":13181,"thread_start_us":352478,"threads_started":1,"wal-append.queue_time_us":737018}
W20260709 12:24:12.215245 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.215718 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:12.184530 20786 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:12.159624 20494 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.222615 20688 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:12.224356 20714 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:12.225695 13652 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:12.256599 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:12.225941 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.257566 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:12.259747 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:12.260706 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:12.241295 20757 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.005s
I20260709 12:24:12.218914 20564 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.266427 20671 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:24:12.270972 20832 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:12.271418 20790 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.217829 20472 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:12.250005 20812 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:12.269852 20628 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:12.327934 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68237.9us
W20260709 12:24:12.297369 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126514us
I20260709 12:24:12.244948 20262 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:12.228039 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:12.210487 20801 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.361730 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:12.257288 20714 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.277552 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:12.386973 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158113us
I20260709 12:24:12.279035 13645 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:12.276144 20786 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:12.282168 20695 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:24:12.348155 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97239.4us
W20260709 12:24:12.352550 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.513s (SENT)
I20260709 12:24:12.293208 20671 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.247462 20647 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.263255 20853 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:12.356559 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:12.302022 20117 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:12.304093 20639 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:12.347589 20328 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.749s	user 0.013s	sys 0.001s
W20260709 12:24:12.393074 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 165248us
I20260709 12:24:12.277839 20757 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.403163 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:12.442519 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90019.7us
I20260709 12:24:12.404481 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:12.393741 20714 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:12.431330 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:12.440457 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:12.440722 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:12.359280 20800 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:12.412163 20671 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:12.405710 20829 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.449265 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92803.6us
I20260709 12:24:12.467880 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:12.401988 20865 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:12.337757 20564 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.417647 20739 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:12.477289 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:12.428241 20847 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:12.437376 20857 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:12.488547 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:12.448808 20853 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:12.361681 20812 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.480211 20786 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:12.481771 20736 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.487246 20472 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:12.487571 20847 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:12.496129 20790 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:12.448614 20328 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:12.609694 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.535246 20748 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:12.562230 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.619392 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.592752 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 315278us
I20260709 12:24:12.621172 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:12.498674 20866 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.437129 20688 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.544782 20873 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:12.492789 20857 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:12.473117 20800 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.606477 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:12.437964 20117 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.658943 20639 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:12.659504 20639 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.661060 20117 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.641577 20748 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:12.663591 20748 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:12.624501 20876 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:24:12.640978 20659 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:12.666826 20659 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:12.667934 20687 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:12.667877 20659 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.543704 20768 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:24:12.610200 20875 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:24:12.622727 20866 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:12.623636 20851 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:24:12.480245 20829 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.591904 20847 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:12.557235 20736 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.485253 20865 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.635227 13797 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:12.649010 20879 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:24:12.650795 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:12.653538 20647 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.654764 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1336) took 1455 ms. Trace:
W20260709 12:24:12.655049 20850 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 12:24:12.656637 20874 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:24:12.608778 20880 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:12.650877 20656 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:12.657303 20853 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:12.658716 20871 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:24:12.668402 20687 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:12.539985 20786 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:12.778704 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:12.675019 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:12.644979 20740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.006s
W20260709 12:24:12.679277 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 275069us
I20260709 12:24:12.615597 20472 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:12.670210 20630 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.509s	user 0.004s	sys 0.004s
I20260709 12:24:12.755180 20829 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:12.647506 20857 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:12.738422 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:12.753196 14417 rpcz_store.cc:276] 0709 12:24:11.198926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.199140 (+   214us) service_pool.cc:224] Handling call
0709 12:24:11.199479 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:11.560566 (+361087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.654364 (+1093798us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400498c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:12.654684 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828709120}
W20260709 12:24:12.755677 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 312532us
W20260709 12:24:12.681906 20781 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.000s
W20260709 12:24:12.826567 20882 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:12.826733 20781 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:12.827965 20868 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.828567 20868 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:12.829319 20878 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:24:12.830672 13800 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:12.830812 20889 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:12.831449 20889 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.832151 20849 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:12.832674 20849 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.833891 20889 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:12.769025 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 162623us
I20260709 12:24:12.845412 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:12.707127 20328 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.500370 20262 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.783025 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:12.840266 20757 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.853344 20868 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:12.616816 20852 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.332s	user 0.002s	sys 0.008s
W20260709 12:24:12.773213 20875 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:24:12.800290 20669 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:12.801245 20630 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:12.774655 20880 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.802951 20854 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:12.780269 20639 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.805882 20736 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.806603 20564 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.807871 20866 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:12.809301 20505 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.655s	user 0.014s	sys 0.004s
I20260709 12:24:12.809551 20865 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:12.816543 20873 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:24:12.910921 20873 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:12.911586 20873 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:12.787108 20468 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.001s
I20260709 12:24:12.914232 20468 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.915530 20880 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.907976 20854 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.918176 20468 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:12.918619 20854 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.919050 20727 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.919621 20727 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:12.923792 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.770174 20855 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:24:12.835333 20800 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.835445 20628 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.783915 20786 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:12.836437 20848 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:12.837109 20781 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.838176 13724 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:24:12.839421 20801 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.839790 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.839439 13650 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:12.840569 20884 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.783711 20740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.000s	sys 0.006s
I20260709 12:24:12.841310 20893 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.842168 20862 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.020088 20893 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.020221 20862 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:12.843106 20860 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.022570 20860 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.023408 20893 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.025192 20860 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.833923 20790 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:12.843871 20734 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.026391 20734 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.026309 20790 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.028851 20734 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.830854 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.845956 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:12.840159 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:12.846894 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:12.764760 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.845969 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:13.030689 13796 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:13.031253 13726 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:13.031159 20862 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:12.717568 20494 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:12.872305 20868 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:13.032390 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.032143 20868 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.033106 13726 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:12.898593 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.034576 13725 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:13.035977 20868 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:12.819945 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.940739 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 316358us
W20260709 12:24:13.013770 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.879s (SENT)
I20260709 12:24:13.038494 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.018383 20884 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.030592 13649 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:12.851720 13719 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:13.039908 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.044679 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.045291 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:13.045920 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 200064us
I20260709 12:24:13.047068 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.036154 13725 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:12.906147 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.048261 13722 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:13.034085 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:13.037264 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.042316 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:13.049160 13726 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.050892 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.049036 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:13.051759 13724 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:13.052309 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:13.054585 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.055359 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:13.055364 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.057646 13652 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:13.061403 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:13.064059 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.068881 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.074031 20891 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:13.075086 20472 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:13.075636 20472 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.075753 20867 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:13.076117 20472 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:13.065835 20860 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:13.079160 20860 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.079317 20839 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.079725 20860 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:13.079991 20839 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.080502 20839 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:13.081310 20887 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 12:24:13.059659 20669 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.068750 20829 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:24:13.074998 20891 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:24:13.063411 20695 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.000s
I20260709 12:24:13.060930 20734 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.064034 20893 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.052584 20853 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.096488 13652 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:13.007038 20781 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.101297 13726 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:13.098647 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.103231 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.112095 13722 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:13.011654 20714 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.100286 13801 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:24:13.115315 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.113111 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.060391 20740 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:24:13.102339 20695 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:13.101900 20829 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.104924 20893 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:13.110584 20659 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:13.175522 20854 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:13.177790 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.099315 20669 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.007413 20848 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:13.182318 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110075us
I20260709 12:24:13.191641 13649 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:13.213614 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:13.175262 20849 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:13.217375 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.217784 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.218927 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:13.219800 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.217422 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:13.221658 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.221956 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.222313 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.228399 14421 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:13.186677 20671 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.706s	user 0.006s	sys 0.006s
I20260709 12:24:13.236567 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.200187 20893 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.243247 14416 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:13.246716 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.232318 20695 raft_consensus.cc:2764] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:24:13.256727 20428 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:13.262171 20428 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.262437 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.264407 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.264516 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.265218 13795 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:13.265492 13792 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:13.190464 20829 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:13.252470 20899 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:24:13.223829 20659 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.281615 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.283345 14419 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:13.285508 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.239518 20671 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.288163 20669 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.250123 20328 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:24:13.299502 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:13.309180 20903 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:13.298400 20262 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.312006 20262 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.314602 20688 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:13.314872 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.315420 20688 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:13.315686 13648 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 11.
I20260709 12:24:13.316046 20829 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.316356 20688 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.317803 20688 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.319428 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:13.321527 20700 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.259714 20857 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.325227 20857 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.326143 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3889) took 1100 ms. Trace:
I20260709 12:24:13.326597 13645 rpcz_store.cc:276] 0709 12:24:12.225563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.225694 (+   131us) service_pool.cc:224] Handling call
0709 12:24:13.325968 (+1100274us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:13.326081 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46801280}
W20260709 12:24:13.327553 20886 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.567s	user 0.000s	sys 0.002s
I20260709 12:24:13.317706 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.329605 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:13.330727 20698 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:13.331674 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:13.332834 20904 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:13.332924 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.333499 20790 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:13.333658 13720 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:13.338318 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.338832 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.339023 13718 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:13.340086 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.341236 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.339109 20472 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.340090 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.344404 20727 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.344658 13726 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:13.342510 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.345927 13649 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:13.347911 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.348013 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.348645 13719 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:13.351900 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:13.352902 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.352734 13722 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:13.302131 20847 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.001s
W20260709 12:24:13.314791 20897 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:24:13.378131 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.378197 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.378831 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:13.378980 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.376076 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.376076 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:13.376433 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.380417 13718 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.380584 13719 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:13.382041 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:13.373967 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113821us
I20260709 12:24:13.376806 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:13.352944 20884 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.389714 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:13.394212 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:13.393765 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.445818 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.440404 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:13.417249 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.447685 14416 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:13.419425 20472 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:13.420605 20262 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.423954 20786 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.401820 13722 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:13.469944 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.470340 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.422407 20671 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.446182 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.394982 13719 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:13.405840 20891 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:13.422904 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.401759 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.452217 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.455403 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.475348 13725 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:13.474351 13718 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:13.476297 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:13.476583 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.476680 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:13.477038 13725 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:13.477698 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.456544 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.379328 20847 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.457890 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.482357 13644 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:13.389945 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:13.464306 14422 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:13.464919 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:13.467664 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.483399 14419 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:13.460393 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:13.483743 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.484602 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.431053 20727 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.478857 13643 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:13.479974 13648 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:24:13.440892 20781 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.484870 13727 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:13.474867 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:13.494072 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.496718 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.497071 20731 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:13.497366 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.480093 20630 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:13.497980 20731 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:13.498061 13725 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:13.483678 20842 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:13.499264 20731 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.502398 20727 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.502954 20727 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.503803 20472 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:13.504355 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.505188 20472 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:13.505596 20472 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.505996 20472 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:13.507357 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.508793 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.509110 13720 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:13.512671 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122793us
I20260709 12:24:13.499519 13726 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:24:13.520512 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:13.516218 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.521261 13722 raft_consensus.cc:2376] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14. Re-sending same reply.
I20260709 12:24:13.521950 13800 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:13.523651 20842 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:13.524008 13719 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:13.524632 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:13.527979 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.529043 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.473292 20740 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.531212 13794 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:13.532090 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.534168 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.517313 20790 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:13.535955 13649 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:13.537017 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:13.525235 20905 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:24:13.543699 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:13.543973 14420 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:13.537708 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:13.525596 20912 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:13.554160 20912 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.554752 20912 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:13.545490 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.510797 20891 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.549782 20734 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:13.563373 13797 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:24:13.563434 20847 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.561112 20893 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.564078 20891 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.564396 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:13.562026 20873 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.650s	user 0.001s	sys 0.014s
I20260709 12:24:13.564666 20472 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.565281 13649 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.565418 20873 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.535657 20842 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:13.547462 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:13.566143 20117 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.570518 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:13.571902 13727 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:13.548182 20262 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:13.567677 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:13.538561 20781 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.525184 20890 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:24:13.575770 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:13.556613 20840 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:13.589205 13617 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:13.590308 13649 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:13.591737 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.592006 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.593259 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:13.596493 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.536566 20740 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.572918 20739 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:13.602734 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.599409 13727 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:13.586871 20428 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.575093 20262 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:13.595508 20840 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.614607 20840 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:13.589605 20842 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:13.608156 20428 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.618911 20842 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.619076 20428 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.619645 20842 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.612592 20893 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.596293 20891 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:24:13.622290 20891 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:13.622694 20891 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:13.623615 20891 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.624986 20891 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:13.601967 20727 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.627718 20873 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.629678 20727 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:13.630483 20727 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:13.633018 20814 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:13.637945 20740 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.638983 13721 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:13.639801 20472 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:13.641795 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 3873) took 2687 ms (client timeout 2948 ms). Trace:
I20260709 12:24:13.598726 20870 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:13.642501 20870 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.642906 20870 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.643078 20428 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.645646 20428 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.594539 20630 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:13.597378 20117 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:13.607980 20765 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:13.649351 13727 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 12:24:13.649433 13646 rpcz_store.cc:269] 0709 12:24:10.954386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.954776 (+   390us) service_pool.cc:224] Handling call
0709 12:24:13.641528 (+2686752us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b5400112954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:13.641702 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4601431296}
I20260709 12:24:13.575440 20839 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.657205 20839 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.653896 20630 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:13.654114 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:13.646356 20262 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.589867 20781 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:13.654002 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.659255 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:13.660122 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:13.660941 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58298.5us
W20260709 12:24:13.668510 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 98717us
I20260709 12:24:13.662617 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:13.661481 13794 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:13.672943 13649 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:13.660976 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:13.679087 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:13.683795 20781 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:13.687891 20860 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.607s	user 0.004s	sys 0.012s
I20260709 12:24:13.689949 13718 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:13.691867 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.692071 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.692770 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:13.693312 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.693790 13798 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:13.694768 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:24:13.695438 13801 raft_consensus.cc:606] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.695802 13801 raft_consensus.cc:614] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:13.694911 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:13.696743 13798 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:13.703792 13793 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:13.710632 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.711796 13799 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:13.715421 13795 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:13.715247 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:13.716588 13793 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:13.712932 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.718952 13799 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:13.720819 13649 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:24:13.723807 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.669358 20659 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:13.683919 20918 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:13.714412 20914 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:24:13.748145 13794 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:13.748435 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.713004 20687 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.750815 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:13.753335 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50458.8us
I20260709 12:24:13.710697 20727 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:13.760519 13801 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:13.764341 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.714202 20881 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.761319 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.768916 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:13.779719 20916 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:24:13.769084 20630 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.784960 20840 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.777987 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:13.803836 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:13.788085 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.787052 13798 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:13.810457 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:13.790659 14419 raft_consensus.cc:2368] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:24:13.819226 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.819921 13801 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:13.820722 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.820966 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.707080 20860 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.852674 13793 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:13.822424 13801 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:13.822880 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:13.821911 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:13.864027 13798 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:13.864559 13793 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 12:24:13.870373 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60049.5us
I20260709 12:24:13.762558 20884 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:13.832144 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:13.875944 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43949.3us
I20260709 12:24:13.884811 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:13.773860 20731 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:13.764557 20848 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:13.900111 13796 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:13.902484 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:13.808297 20748 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:13.834754 20659 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.764662 20918 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.862812 20468 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.907739 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:13.902029 20872 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:24:13.911505 20862 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:13.879904 20781 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:13.788193 20893 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
W20260709 12:24:13.931612 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54750.5us
I20260709 12:24:13.858927 20262 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.847453 20912 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:13.789606 20881 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.918443 20848 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:13.910405 20748 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:13.947669 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.940938 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:13.934746 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:13.916603 20659 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.939646 20328 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:13.918924 20870 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:13.938879 20862 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:14.022800 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88157us
W20260709 12:24:14.033043 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92226.2us
I20260709 12:24:14.047595 13795 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:14.100733 20740 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:13.994079 20891 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:14.167438 20860 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:14.173512 20814 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 12:24:14.174351 20714 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:14.174954 20840 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.177071 20790 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.177435 20870 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.179906 20870 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:14.178015 20790 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:14.031750 20659 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.034005 20328 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.937166 20468 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.184780 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1516) took 1420 ms. Trace:
I20260709 12:24:14.194314 13723 rpcz_store.cc:276] 0709 12:24:12.764126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.764263 (+   137us) service_pool.cc:224] Handling call
0709 12:24:14.184587 (+1420324us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:14.184714 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713445248}
I20260709 12:24:14.181684 20787 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:14.184765 20912 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:13.938987 20842 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.027824 20748 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:14.023252 20848 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.022545 20921 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:14.233821 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.032120 20884 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:14.137171 20862 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:14.190912 20328 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.260680 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.262656 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:14.263388 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.280323 13798 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:24:14.280521 13793 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:24:14.281527 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.282677 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:14.283728 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:14.266045 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.284670 13799 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:14.287468 20262 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.287735 20842 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.288208 20262 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:14.257045 20848 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:14.291039 20472 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.291615 20472 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.294078 20472 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.296314 20890 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:14.296854 20890 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:14.300040 20920 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
I20260709 12:24:14.306324 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.307875 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:14.293990 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:14.310175 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:14.310411 13693 leader_election.cc:400] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:24:14.310768 13693 leader_election.cc:403] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.317371 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:14.255370 20787 consensus_queue.cc:1243] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:24:14.205042 20731 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 12:24:14.318992 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.282004 20630 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:14.199352 20740 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:14.251521 20891 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:14.324955 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 191803us
I20260709 12:24:14.325215 13722 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:14.299140 20890 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.282896 20884 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:14.323541 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.307744 20870 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:24:14.287477 20881 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:14.203783 20812 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.333630 13798 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:14.337116 13719 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:14.337509 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.334896 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:14.339051 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.339362 13721 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:14.341436 13726 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:14.342046 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.309185 20472 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:14.342957 13798 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:14.327878 20787 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:14.276149 20921 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.353466 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.357501 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.359992 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:14.360927 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:14.361632 13727 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:14.347301 20890 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:24:14.366386 13793 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
W20260709 12:24:14.371022 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 231794us
I20260709 12:24:14.352744 20787 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:14.369516 20926 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.429s	user 0.001s	sys 0.000s
I20260709 12:24:14.381281 13721 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:14.347504 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:14.351310 20881 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:24:14.402504 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:14.406997 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59584.6us
I20260709 12:24:14.408310 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:14.413100 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:14.415194 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:14.417390 13719 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:24:14.352622 20472 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:14.498498 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 188156us
I20260709 12:24:14.522481 20787 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:14.525485 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:14.379522 20927 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:24:14.531203 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123071us
I20260709 12:24:14.541987 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:14.557683 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:14.543013 13722 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:14.555485 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 145856us
I20260709 12:24:14.567169 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:14.576843 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:14.588927 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:14.589926 13726 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:14.427907 20812 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:14.613734 20940 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:14.614859 20472 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.617296 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3435) took 1381 ms. Trace:
I20260709 12:24:14.623884 13727 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:14.527489 20796 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:14.636703 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:14.632682 20862 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.591717 20781 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:14.584231 20860 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:14.557053 20927 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:14.623795 13725 rpcz_store.cc:276] 0709 12:24:13.235852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.498840 (+262988us) service_pool.cc:224] Handling call
0709 12:24:14.273980 (+775140us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:24:14.274216 (+   236us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:14.274839 (+   623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.274881 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:14.275680 (+   799us) log.cc:844] Serialized 52 byte log entry
0709 12:24:14.438691 (+163011us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:14.438757 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.438862 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.616865 (+178003us) raft_consensus.cc:1685] finished
0709 12:24:14.616907 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.617209 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":167,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":207,"spinlock_wait_cycles":43134464,"thread_start_us":135567,"threads_started":1,"wal-append.queue_time_us":333749}
I20260709 12:24:14.562726 20890 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:14.584837 20787 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:14.592885 20918 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.641211 20796 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:14.647663 20727 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.880s	user 0.012s	sys 0.001s
I20260709 12:24:14.575510 20790 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.636181 20117 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:14.625384 20841 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.640659 20472 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:14.663652 20927 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:14.668771 20727 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:14.676432 20935 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:24:14.677137 20786 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.427017 20884 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.657122 20781 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:14.679113 20929 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:24:14.668969 20890 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.637529 20940 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:14.669606 20860 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:14.681821 20841 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:14.687880 20781 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:14.686683 20891 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.644446 20748 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:14.704854 20790 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.705448 20921 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.707804 20860 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:14.709870 20944 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.097s	user 0.003s	sys 0.000s
W20260709 12:24:14.709877 20765 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.049s	user 0.013s	sys 0.000s
I20260709 12:24:14.701897 20841 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.710529 20848 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:14.879539 20796 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.880504 20890 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.879822 20841 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:14.886651 20848 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:14.887274 20940 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:14.856436 20862 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:14.907662 20927 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:14.955407 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.009s	sys 0.009s
I20260709 12:24:14.680909 20472 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:14.956449 13616 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.010s	sys 0.067s
I20260709 12:24:14.787838 20765 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:14.658015 13726 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:14.964193 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2540) took 1456 ms. Trace:
W20260709 12:24:14.964900 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 397634us
I20260709 12:24:14.965998 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2541) took 1458 ms. Trace:
W20260709 12:24:14.970645 13723 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.029s	sys 0.039s
I20260709 12:24:14.975725 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:14.975446 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:14.992602 13725 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:24:14.996995 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:14.671092 13801 raft_consensus.cc:2368] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:15.049242 13795 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:15.099100 20428 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:15.116276 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.001s	sys 0.013s
I20260709 12:24:15.119537 20796 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.119378 20694 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:15.128782 13800 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:15.149850 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:15.160732 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.001s	sys 0.013s
W20260709 12:24:15.146533 13616 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.011s	sys 0.067s
W20260709 12:24:15.154677 13723 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.029s	sys 0.039s
I20260709 12:24:15.155416 13720 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:24:15.134958 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.010s	sys 0.009s
I20260709 12:24:15.155354 13719 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:15.139101 20857 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.161887 20694 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.147246 20748 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.184412 20857 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.161226 20796 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.188030 20796 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.159638 20860 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:24:15.190922 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:15.160481 20428 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.140015 20940 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:15.192763 20940 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.157151 13725 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:15.139816 20472 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:15.199872 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 905898us
I20260709 12:24:15.150601 13645 rpcz_store.cc:276] 0709 12:24:13.507395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.719712 (+212317us) service_pool.cc:224] Handling call
0709 12:24:13.720114 (+   402us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:13.745963 (+ 25849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.746051 (+    88us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:13.763621 (+ 17570us) log.cc:844] Serialized 52 byte log entry
0709 12:24:14.203296 (+439675us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:14.499193 (+295897us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.555855 (+ 56662us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.583472 (+ 27617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.583571 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.962856 (+379285us) raft_consensus.cc:1685] finished
0709 12:24:14.962982 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.963680 (+   698us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51487,"prepare.queue_time_us":470152,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":177250,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":24450,"thread_start_us":608436,"threads_started":2,"wal-append.queue_time_us":516466}
I20260709 12:24:15.135926 20734 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.203418 20734 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.206816 20941 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:15.208683 20933 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:15.176486 13726 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.154466 13649 rpcz_store.cc:276] 0709 12:24:13.507933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.773397 (+265464us) service_pool.cc:224] Handling call
0709 12:24:13.834575 (+ 61178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.834986 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.835026 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.869601 (+ 34575us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.277182 (+407581us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:14.277225 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.277307 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.965169 (+687862us) raft_consensus.cc:1685] finished
0709 12:24:14.965236 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.965681 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40153,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":19390,"thread_start_us":358851,"threads_started":1,"wal-append.queue_time_us":463853}
I20260709 12:24:15.216594 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:15.218343 20765 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:15.222429 20941 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:15.223891 20941 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:15.225314 13722 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.659s	user 0.004s	sys 0.022s
I20260709 12:24:15.225837 13722 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:15.227543 20694 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.228958 20857 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.229545 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:15.230654 20765 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:15.232848 20884 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:15.233549 20262 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:15.234081 20262 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:15.235771 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.011s	sys 0.004s
I20260709 12:24:15.235478 20262 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:15.236284 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.011s	sys 0.004s
W20260709 12:24:15.236797 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 694873us
I20260709 12:24:15.239528 13727 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:15.239094 20765 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:15.193972 20428 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.241556 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:15.241878 20740 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:15.277822 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44097.4us
I20260709 12:24:15.130229 20927 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:15.282652 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 194679us
I20260709 12:24:15.298293 20884 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.300612 20884 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.302767 20740 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.303617 20740 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:15.305238 20328 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.001s	user 0.009s	sys 0.018s
I20260709 12:24:15.302652 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:15.307371 14422 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:15.307611 20734 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.306104 20740 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:15.319380 20659 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.134s	user 0.017s	sys 0.000s
I20260709 12:24:15.319922 20659 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.321280 13793 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:15.327262 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:15.327945 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:15.331562 20468 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:15.348905 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3437) took 2108 ms. Trace:
I20260709 12:24:15.305771 20328 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.349627 13718 rpcz_store.cc:276] 0709 12:24:13.240126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.509699 (+269573us) service_pool.cc:224] Handling call
0709 12:24:13.689655 (+179956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.691240 (+  1585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.348369 (+1657129us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54001563d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:15.348640 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413019392}
I20260709 12:24:15.368685 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:15.370303 20860 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:15.370790 20647 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:15.371065 20860 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:15.371531 20740 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.371603 20647 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.372427 20740 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.372584 20860 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.374724 20740 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.375345 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2552) took 1115 ms. Trace:
I20260709 12:24:15.375798 13797 rpcz_store.cc:276] 0709 12:24:14.259914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.260102 (+   188us) service_pool.cc:224] Handling call
0709 12:24:14.655212 (+395110us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.655581 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.655613 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.655658 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:14.655702 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.655763 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.374898 (+719135us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b540006b150. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:15.375093 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5245,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5149,"spinlock_wait_cycles":907864960}
W20260709 12:24:15.369163 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.665s	user 0.026s	sys 0.066s
I20260709 12:24:15.379992 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2537) took 1850 ms. Trace:
I20260709 12:24:15.380702 20647 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:15.388504 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:15.388230 20647 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:15.390879 20950 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:15.389427 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:15.391778 20950 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:15.392155 20781 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.681s	user 0.018s	sys 0.001s
I20260709 12:24:15.392854 20781 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.393730 20740 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:15.394459 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:15.402674 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.751s (SENT) [suppressed 11 similar messages]
I20260709 12:24:15.391047 20647 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:15.393404 20950 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.404934 20647 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:15.405784 20781 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.395439 20947 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.406468 20647 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.408244 20647 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:15.408504 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:15.396205 20853 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:15.410988 20853 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.412855 20328 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:15.414458 13798 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 12:24:15.411022 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83280us
I20260709 12:24:15.403496 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:15.423877 20943 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.724s	user 0.011s	sys 0.003s
I20260709 12:24:15.425212 20853 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.425865 20853 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.426523 20853 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.428320 20813 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:15.430487 20855 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:24:15.431047 20855 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:15.431715 20659 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:15.432507 20855 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.433583 20630 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:15.435864 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:15.437950 20936 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.746s	user 0.002s	sys 0.012s
I20260709 12:24:15.434131 20630 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:15.440855 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.440981 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2561) took 1172 ms. Trace:
I20260709 12:24:15.441792 20813 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:15.396845 20946 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.396960 13726 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:15.441439 13793 rpcz_store.cc:276] 0709 12:24:14.268820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.371140 (+102320us) service_pool.cc:224] Handling call
0709 12:24:14.371594 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.321735 (+950141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.440609 (+118874us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:15.440906 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183160064}
I20260709 12:24:15.445644 13720 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:15.399413 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:15.406741 20947 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.395164 20694 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:15.409880 13721 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:15.380493 13646 rpcz_store.cc:276] 0709 12:24:13.495897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.694005 (+198108us) service_pool.cc:224] Handling call
0709 12:24:13.694429 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.148718 (+1454289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.148779 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.148837 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:15.148866 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.345836 (+196970us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001b6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:15.345912 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.346188 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10001,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":276,"spinlock_wait_cycles":3342874880}
I20260709 12:24:15.453164 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:15.444403 20813 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.460618 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:15.455039 20740 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:15.476277 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77043.2us
I20260709 12:24:15.464370 20884 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.487815 20791 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:15.489209 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3786) took 1797 ms. Trace:
I20260709 12:24:15.495972 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:15.489534 20675 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:15.497375 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:15.497324 20675 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.497722 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:15.498041 14420 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:15.491952 20765 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 27.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
W20260709 12:24:15.489087 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 247758us
I20260709 12:24:15.491698 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:15.494683 13792 rpcz_store.cc:276] 0709 12:24:13.691761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.692950 (+  1189us) service_pool.cc:224] Handling call
0709 12:24:13.693232 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.693438 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:13.745447 (+ 52009us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.236889 (+1491442us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b18003fa840. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:24:15.237355 (+   466us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.487791 (+250436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.488083 (+   292us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:15.488430 (+   347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.488598 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.488992 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47426,"prepare.queue_time_us":96386,"raft.queue_time_us":36206,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":11479,"spinlock_wait_cycles":79530624,"thread_start_us":302966,"threads_started":3,"wal-append.queue_time_us":33532,"wal-append.run_cpu_time_us":1272,"wal-append.run_wall_time_us":1006759}
I20260709 12:24:15.498471 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:15.472421 20813 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:15.491449 20853 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.505935 13718 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.498128 20675 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:15.458554 20946 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:15.538287 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.834s	user 0.027s	sys 0.066s
I20260709 12:24:15.540158 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:15.537823 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:15.543661 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:15.502559 20791 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:15.545881 13722 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:24:15.549476 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64098.8us
I20260709 12:24:15.550426 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:15.547179 13725 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:15.539242 20853 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:24:15.569306 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69862.8us
W20260709 12:24:15.565944 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 163483us
I20260709 12:24:15.560333 20765 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:15.577735 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:15.584057 13727 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:15.564579 20853 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:15.648533 20853 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.648749 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:15.581009 20765 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:15.650071 20765 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.651173 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1173) took 2393 ms. Trace:
I20260709 12:24:15.651556 13796 rpcz_store.cc:276] 0709 12:24:13.257564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.257694 (+   130us) service_pool.cc:224] Handling call
0709 12:24:13.258022 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:13.900648 (+642626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.650657 (+1750009us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54003d4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:15.650957 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476140928}
I20260709 12:24:15.653107 13796 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:15.587525 13722 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:15.590866 20791 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.597433 20840 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.598428 20694 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.598434 20829 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:24:15.599280 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.9074e+06us
I20260709 12:24:15.600502 20675 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.604936 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:15.604756 20659 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:15.605607 20328 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:15.609866 20946 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.610344 20884 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:15.613659 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:15.614727 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:15.615360 13793 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:15.616581 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:15.617892 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:15.622781 20955 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:15.624044 20953 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.626598 20954 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:15.633211 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:15.634465 13646 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:24:15.639369 20898 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:15.689723 20898 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:15.690209 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3795) took 1887 ms (client timeout 1614 ms). Trace:
W20260709 12:24:15.641384 14419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1106) took 1982 ms (client timeout 1735 ms). Trace:
W20260709 12:24:15.690703 13800 rpcz_store.cc:269] 0709 12:24:13.803350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.803506 (+   156us) service_pool.cc:224] Handling call
0709 12:24:15.689930 (+1886424us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:15.690112 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3037248384}
I20260709 12:24:15.642470 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2570) took 1370 ms. Trace:
I20260709 12:24:15.584887 20947 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.644532 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:15.572584 20472 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.647197 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:15.647971 14418 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:15.665207 14420 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:15.670547 20940 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.692148 13795 rpcz_store.cc:276] 0709 12:24:14.271531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.653757 (+382226us) service_pool.cc:224] Handling call
0709 12:24:14.654145 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.159601 (+505456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.642145 (+482544us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54001bbcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:15.642376 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906881280}
I20260709 12:24:15.673375 20694 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.674639 20687 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:15.674561 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:15.675467 20840 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.678743 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2569) took 1407 ms. Trace:
I20260709 12:24:15.683885 20829 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:15.643338 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1343) took 2427 ms. Trace:
I20260709 12:24:15.692785 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:15.646189 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:15.691437 14419 rpcz_store.cc:269] 0709 12:24:13.658845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.658986 (+   141us) service_pool.cc:224] Handling call
0709 12:24:15.580218 (+1921232us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54007a0154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:15.641121 (+ 60903us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280347904}
W20260709 12:24:15.697917 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93111us
I20260709 12:24:15.670611 20675 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:15.706187 13651 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:24:15.709244 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1125) took 2320 ms. Trace:
I20260709 12:24:15.690976 20898 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.710429 20688 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:15.710809 13797 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:15.714300 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1522) took 1409 ms. Trace:
I20260709 12:24:15.715055 20855 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:24:15.726267 14416 rpcz_store.cc:276] 0709 12:24:13.216028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.218413 (+  2385us) service_pool.cc:224] Handling call
0709 12:24:15.642967 (+2424554us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:15.643144 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36515712}
I20260709 12:24:15.715566 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:15.684497 20884 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:15.691957 20262 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:15.722657 13798 rpcz_store.cc:276] 0709 12:24:14.271282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.372336 (+101054us) service_pool.cc:224] Handling call
0709 12:24:14.374709 (+  2373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.415006 (+1040297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.678408 (+263402us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003d54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:15.678681 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2389915520}
W20260709 12:24:15.732682 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.731849 13651 pending_rounds.cc:85] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 12
I20260709 12:24:15.748319 13651 pending_rounds.cc:107] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 21.13
W20260709 12:24:15.745606 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.731943 14418 rpcz_store.cc:276] 0709 12:24:13.388603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.388736 (+   133us) service_pool.cc:224] Handling call
0709 12:24:15.647412 (+2258676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.708102 (+ 60690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.708978 (+   876us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54006c1e50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:15.709147 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5193478400}
I20260709 12:24:15.744994 14424 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:15.752076 13718 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:15.748775 13651 raft_consensus.cc:2889] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.760052 13721 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:15.762147 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:15.766228 20883 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:15.766527 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:15.767663 20883 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:15.768080 20883 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:15.768908 20472 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.769124 20883 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.740484 13723 rpcz_store.cc:276] 0709 12:24:14.305094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.305302 (+   208us) service_pool.cc:224] Handling call
0709 12:24:14.317605 (+ 12303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.202477 (+884872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.202532 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.203513 (+   981us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.462494 (+258981us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:15.462532 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.466078 (+  3546us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.713974 (+247896us) raft_consensus.cc:1685] finished
0709 12:24:15.714001 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.714211 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66855,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":14451,"spinlock_wait_cycles":27520,"thread_start_us":1118149,"threads_started":2,"wal-append.queue_time_us":363181}
I20260709 12:24:15.771463 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3425) took 1349 ms. Trace:
I20260709 12:24:15.750489 20884 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:15.778234 13650 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:15.748852 20688 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:15.782369 20671 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.774997 13721 rpcz_store.cc:276] 0709 12:24:14.411876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.412039 (+   163us) service_pool.cc:224] Handling call
0709 12:24:15.408209 (+996170us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acff8c 00007f5390acfef6 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:24:15.760698 (+352489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14361600}
I20260709 12:24:15.778898 20787 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:15.784852 13650 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:15.789902 20787 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:15.791044 13650 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:15.829728 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 426452us
I20260709 12:24:15.838819 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:15.763377 20954 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.747169 20953 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.825047 20472 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:15.791844 20675 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:15.849186 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:15.848117 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:15.855295 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:15.847057 20898 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:15.867624 13648 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:24:15.871084 20865 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:15.876420 20848 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:15.877014 20848 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.877380 20848 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.883104 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:15.883986 13647 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:15.791221 20787 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.862008 20675 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.787047 20955 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:15.894379 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.894496 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:15.898247 20849 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.898483 20862 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:24:15.899004 20849 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.899565 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:15.901185 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:15.901522 20849 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.901774 13649 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:15.902750 20472 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.854709 13644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:15.889364 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:15.901227 13652 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:15.920995 13693 leader_election.cc:400] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:15.921481 13693 leader_election.cc:403] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:15.897027 13618 connection.cc:256] RPC call timeout handler was delayed by -1.2791795879998062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 12:24:15.924332 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1525) took 1590 ms. Trace:
I20260709 12:24:15.924808 13724 rpcz_store.cc:276] 0709 12:24:14.306449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.306607 (+   158us) service_pool.cc:224] Handling call
0709 12:24:14.324518 (+ 17911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.588875 (+264357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.588924 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.589005 (+    81us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:14.589057 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.859162 (+1270105us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54002f6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:15.863273 (+  4111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.897168 (+ 33895us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607409920}
I20260709 12:24:15.926612 20962 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:15.931788 20961 raft_consensus.cc:2764] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election decision vote started in defunct term 24: won
I20260709 12:24:15.934681 20883 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:15.934810 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:15.938498 20860 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:15.939107 20860 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:15.940100 20860 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:15.941622 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.925599 20884 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.942843 13650 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:15.944104 20884 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:15.944633 20927 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:15.945050 20927 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:15.946136 20927 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.947460 20840 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.947631 20927 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:15.881384 20829 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:15.951287 20630 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.511s	user 0.005s	sys 0.008s
I20260709 12:24:15.951836 20630 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.958555 20940 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:15.959662 20659 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:15.960290 20659 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:15.961422 20659 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:15.962531 20884 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.963112 20884 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.967505 13651 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:15.988611 13646 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:15.988598 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79969.9us
W20260709 12:24:15.991802 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71048.2us
I20260709 12:24:15.909194 13644 raft_consensus.cc:606] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:15.993681 13644 raft_consensus.cc:614] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:15.855077 20671 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:15.896064 20675 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:15.906226 20472 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:16.016922 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101226us
I20260709 12:24:15.954203 20630 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:15.998091 20940 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:16.043524 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:16.047372 13722 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:16.049170 13724 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:16.020757 20965 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.044819 20962 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:16.051537 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:16.059492 20965 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:16.063515 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 290580us
I20260709 12:24:16.067759 20791 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:16.051337 20962 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.075879 20962 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:16.086699 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:16.113507 13801 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:16.087913 13798 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:16.114717 20694 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:16.115612 20688 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.091861 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:16.119545 13797 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:16.093472 20787 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.106446 20965 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:16.108443 13792 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:16.109225 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.111061 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.111477 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.129773 13800 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:24:16.129516 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 231613us
I20260709 12:24:16.129052 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:16.127573 13722 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
W20260709 12:24:16.133834 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86096.1us
I20260709 12:24:16.133440 13796 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:16.146988 13650 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:16.128403 20884 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.168071 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.174526 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3916) took 2496 ms. Trace:
W20260709 12:24:16.170610 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72949.6us
W20260709 12:24:16.178174 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 153182us
I20260709 12:24:16.175170 13650 rpcz_store.cc:276] 0709 12:24:13.677819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.267528 (+589709us) service_pool.cc:224] Handling call
0709 12:24:14.267773 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.774516 (+1506743us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b540031c2a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:24:16.147901 (+373385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.174441 (+ 26540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122766592}
I20260709 12:24:16.174045 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:16.188974 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.181002 13727 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:16.190601 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.179211 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.191561 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:16.204797 13801 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:16.197865 13792 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:16.129317 20765 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.174690 20748 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:16.219023 20963 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.219s	user 0.004s	sys 0.000s
W20260709 12:24:16.220605 20842 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
W20260709 12:24:16.226004 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 195145us
I20260709 12:24:16.175782 20941 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:24:16.242116 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:16.243289 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.223037 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:16.247052 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:24:16.221370 20853 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.005s
I20260709 12:24:16.251698 13643 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:16.252110 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.149660 20961 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:24:16.261320 13725 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:16.186905 20947 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:24:16.213126 20630 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:16.271246 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67325.3us
W20260709 12:24:16.284221 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 105112us
I20260709 12:24:16.262127 20953 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:16.294102 14424 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:24:16.270645 20965 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:24:16.295558 20472 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:16.296315 20855 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.001s
I20260709 12:24:16.296806 20961 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:16.297113 20714 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:16.297329 20961 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:16.297816 20714 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:16.293941 20962 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:16.298983 20714 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:16.313277 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107384us
I20260709 12:24:16.314162 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:16.313987 20953 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:16.315517 20953 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:16.315737 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.315945 20953 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:16.316998 20953 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.318622 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.320895 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.296357 20472 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:16.326632 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.327349 13792 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:16.259238 20853 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.257537 20862 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:24:16.248966 20748 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.332175 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.332284 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:16.275004 20954 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:16.335206 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 120364us
I20260709 12:24:16.334926 20472 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.336241 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:16.336504 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5130502990000423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 12:24:16.385643 13693 connection.cc:256] RPC call timeout handler was delayed by -1.498650299000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260709 12:24:16.386039 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4842502989999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
I20260709 12:24:16.337870 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:16.338838 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.339123 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.339342 20961 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.340229 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.341355 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.338884 14424 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:16.342613 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:16.342898 20813 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.004s
I20260709 12:24:16.343602 20962 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:16.343710 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.344810 20955 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.355636 20671 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.402981 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.404258 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.404839 13798 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:16.405181 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.405928 13648 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:16.406641 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.407085 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.407531 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.408248 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.408489 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.409834 13648 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:24:16.410672 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.411087 20980 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.411818 20980 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.412319 13649 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:16.320856 20941 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:16.267373 20940 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:24:16.414232 20980 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:16.422955 20849 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:16.418543 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2875926130000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
W20260709 12:24:16.431041 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2579125709999062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 12:24:16.431492 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2502502990000721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
W20260709 12:24:16.431773 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2471125709998887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 12:24:16.432045 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2430502990000605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
W20260709 12:24:16.361899 20687 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.633s	user 0.006s	sys 0.007s
I20260709 12:24:16.367185 14416 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:16.367936 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.368746 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.370169 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.371292 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:16.383708 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60176.9us
I20260709 12:24:16.392400 13720 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:16.427301 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:16.436293 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:16.437358 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:16.387219 20898 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 12:24:16.437738 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.396500 20813 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.439755 13799 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:16.440429 20813 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:16.441011 20765 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:16.399639 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:16.400753 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.401844 20472 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.406664 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.359917 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.275489 20946 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.418720 20975 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:16.444407 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50806.9us
W20260709 12:24:16.444679 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130634us
W20260709 12:24:16.392395 20262 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.621s	user 0.013s	sys 0.000s
I20260709 12:24:16.435679 13797 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:16.455924 14423 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:16.462083 13646 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:16.415474 20941 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.478826 13793 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:16.416263 20940 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:16.472105 13650 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:16.466845 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.467224 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:16.466678 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:16.468154 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.441785 20765 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:16.466131 20853 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.480556 20694 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:16.466681 13724 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:16.441309 20731 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.484230 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:16.444169 20953 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.482652 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:16.480862 20941 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:16.488566 13651 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:16.469087 20748 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:16.495565 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 58334.5us
W20260709 12:24:16.496227 20848 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.618s	user 0.014s	sys 0.000s
I20260709 12:24:16.496726 20848 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.497470 20946 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.498580 20961 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:16.430469 20675 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.481451 20940 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:16.525230 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.562624 20862 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:24:16.564656 13648 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:16.564796 20940 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.565310 20675 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.565531 13792 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:16.565703 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.500752 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.566270 20940 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:16.502916 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66938.5us
I20260709 12:24:16.520853 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.493964 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3448) took 1023 ms. Trace:
I20260709 12:24:16.506315 13725 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:16.502373 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.510432 20972 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:16.562544 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65224.1us
I20260709 12:24:16.523793 13793 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:16.504505 13726 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:16.571036 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.538478 13795 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:16.539038 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:16.572976 13651 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:16.567286 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.577246 13650 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:16.567202 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.577826 13801 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:16.559095 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:16.525607 13798 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:16.517338 20787 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:16.583810 13652 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:16.530926 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.530926 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.514428 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.585799 13796 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:16.586229 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:16.587165 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.590973 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:16.591915 13724 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:16.524861 13794 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:24:16.599284 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.518080 20765 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:16.603081 13647 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:24:16.603788 20765 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.604306 20328 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:16.516873 20955 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:16.499406 20731 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.605180 13726 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:24:16.605082 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.501037 20687 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:16.569002 13719 rpcz_store.cc:276] 0709 12:24:15.470574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.475002 (+  4428us) service_pool.cc:224] Handling call
0709 12:24:15.490349 (+ 15347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.681021 (+190672us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:15.896670 (+215649us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.375014 (+478344us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:16.409005 (+ 33991us) log.cc:844] Serialized 26 byte log entry
0709 12:24:16.409229 (+   224us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:16.409271 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.409344 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.493749 (+ 84405us) raft_consensus.cc:1685] finished
0709 12:24:16.493772 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.493904 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25669,"prepare.queue_time_us":274095,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":204609,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":182,"thread_start_us":529059,"threads_started":4,"wal-append.queue_time_us":277957}
I20260709 12:24:16.603127 13648 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:16.606274 20829 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:16.606560 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.505513 20976 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:16.516623 20962 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.607146 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.607851 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:16.608723 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:16.638234 13724 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:24:16.584197 20787 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.585299 20972 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:16.645040 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.604120 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:16.605093 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102827us
I20260709 12:24:16.604206 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.618628 13645 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:16.499224 20694 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.533795 20848 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:16.656126 20262 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:16.620513 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:16.625202 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:16.656715 13652 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:16.499995 20946 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.632445 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:16.639901 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2562) took 2364 ms (client timeout 2198 ms). Trace:
I20260709 12:24:16.642812 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:16.657840 13726 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:16.643045 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.619488 20959 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:16.640558 20687 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:16.658442 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.622578 20977 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:16.662293 13723 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.664163 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.663182 13651 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:24:16.657583 13799 rpcz_store.cc:269] 0709 12:24:14.269349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.323992 (+ 54643us) service_pool.cc:224] Handling call
0709 12:24:16.611759 (+2287767us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b54000d1bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:16.633574 (+ 21815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4821452672}
I20260709 12:24:16.664937 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.641513 20840 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:16.701870 20840 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:16.702356 20842 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:16.703025 20842 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.703081 20840 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.704561 20840 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:16.706027 20671 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:16.643386 20883 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.066s
W20260709 12:24:16.709578 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 89158us
I20260709 12:24:16.667789 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.671353 13797 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:16.661510 20959 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:16.672799 20687 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.674487 20326 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:24:16.723214 13650 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:16.675489 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:16.724285 13801 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:16.675675 20328 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:16.678072 20933 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.687736 13726 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:16.688506 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:16.732481 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44060.8us
I20260709 12:24:16.689702 20898 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:24:16.691900 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.693642 13723 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:16.694713 20865 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:16.695816 20884 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:16.639804 20955 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.700289 20972 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.633246 20849 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:24:16.719460 20262 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:16.748240 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53347.7us
I20260709 12:24:16.656494 20694 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:16.735713 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:16.696780 20983 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.193s	user 0.004s	sys 0.000s
I20260709 12:24:16.730782 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.738116 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:16.730786 20328 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.642424 20962 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.719885 20848 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:16.719488 20959 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:16.750526 20947 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:16.753320 14418 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:16.738302 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:16.746351 20829 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:16.762061 13652 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:16.763782 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.756837 20262 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:16.768148 20927 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.820s	user 0.011s	sys 0.004s
I20260709 12:24:16.765161 20328 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.794755 13650 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
W20260709 12:24:16.792730 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59046.7us
I20260709 12:24:16.766758 20848 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.797398 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:16.800297 13644 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 12:24:16.800931 13644 pending_rounds.cc:85] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 17
I20260709 12:24:16.801157 13644 pending_rounds.cc:107] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:24:16.801370 13644 raft_consensus.cc:2889] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:16.772047 20829 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:16.772163 20428 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:16.803999 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:16.783308 20972 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:16.811848 13652 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:16.813467 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.816617 13724 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:16.802541 20829 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:16.766830 20933 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:16.803339 20848 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:16.828719 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 136925us
I20260709 12:24:16.806401 20428 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:16.811241 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:16.836290 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:16.845353 20326 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:16.847357 20326 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:16.847730 20326 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:16.845575 20999 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:16.859958 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:16.862540 20941 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:16.866890 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55881.8us
I20260709 12:24:16.867875 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:16.838366 20428 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:16.882474 20865 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.883206 20865 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:16.885643 20865 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:16.885309 20326 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:16.897258 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.899540 20997 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:24:16.943518 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100556us
W20260709 12:24:16.903038 20984 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:24:16.869275 20787 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:16.846033 20884 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:24:16.983907 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.889235 20947 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:16.987895 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.988564 13646 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:16.984640 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141341us
I20260709 12:24:16.890643 20865 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:16.990335 13647 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:16.990588 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:16.992100 13648 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:16.993712 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:16.997484 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.001763 20898 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.002549 20898 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.005545 13693 leader_election.cc:400] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:24:16.994524 13725 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:17.018131 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.019043 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:17.019177 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 151395us
I20260709 12:24:17.021200 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:17.020401 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.022410 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.029407 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:17.030108 13725 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:17.031303 14418 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:17.035964 13764 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:17.037993 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.045226 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.046092 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.047524 13646 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:24:17.051425 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65767.1us
W20260709 12:24:17.056457 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.059230 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.034411 20962 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.073230 20848 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.076102 20848 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.061030 20855 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:17.053730 20995 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:17.086604 20855 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:17.087206 20855 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:17.078663 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.090723 13793 raft_consensus.cc:2376] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24. Re-sending same reply.
I20260709 12:24:17.070878 20933 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.091811 20933 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:17.051215 20976 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:24:17.093389 20976 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:17.093870 20976 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:17.094986 20976 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.041662 20898 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.097093 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:17.098008 13644 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:17.081610 20472 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.002s
W20260709 12:24:17.087316 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.093066 20933 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:17.062013 20961 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
I20260709 12:24:17.104492 20855 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.106105 20933 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.098605 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:17.062922 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.080014 20326 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.081796 20959 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.102113 20898 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:24:17.102684 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:17.128961 20985 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.622s	user 0.003s	sys 0.006s
I20260709 12:24:17.130836 20976 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:17.112097 13796 raft_consensus.cc:2368] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19.
W20260709 12:24:17.116464 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54037.1us
I20260709 12:24:17.137607 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.139151 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.139081 20865 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:17.140070 13645 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:17.140163 20865 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:17.141034 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.141268 20865 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.146559 13726 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:17.148351 20959 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.117638 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:17.148541 20829 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.150609 20996 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:17.138881 20328 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.152287 13645 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:24:17.152796 13647 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:17.155128 13648 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:17.158496 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.167196 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:17.168071 14418 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:17.168098 20659 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:17.170439 13799 raft_consensus.cc:2376] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25. Re-sending same reply.
I20260709 12:24:17.172817 13801 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:24:17.152381 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:17.174216 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.175598 13644 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:24:17.112586 20955 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.108067 20855 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:17.271271 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 134387us
I20260709 12:24:17.108143 20731 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:17.189857 13648 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:24:17.274780 13648 pending_rounds.cc:85] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 13
I20260709 12:24:17.275051 13648 pending_rounds.cc:107] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:24:17.275348 13648 raft_consensus.cc:2889] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.182222 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:17.276582 13650 raft_consensus.cc:606] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:17.276947 13650 raft_consensus.cc:614] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:17.277612 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:17.278241 14423 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:17.272323 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3465) took 1134 ms. Trace:
W20260709 12:24:17.280447 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 181936us
I20260709 12:24:17.157531 20996 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:17.283993 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 123166us
I20260709 12:24:17.288857 13644 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:17.272473 20472 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:17.290716 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:17.271838 20659 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.291973 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.275950 20849 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:17.305819 13799 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:17.285286 13722 rpcz_store.cc:276] 0709 12:24:16.137777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.164359 (+ 26582us) service_pool.cc:224] Handling call
0709 12:24:16.164826 (+   467us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:16.317736 (+152910us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:16.318137 (+   401us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.626152 (+308015us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.629710 (+  3558us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.629904 (+   194us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:16.642967 (+ 13063us) log.cc:844] Serialized 78 byte log entry
0709 12:24:16.643121 (+   154us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:16.643321 (+   200us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.643614 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.077257 (+433643us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:24:17.080192 (+  2935us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b18003f9fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:17.090894 (+ 10702us) spinlock_profiling.cc:243] Waited 7.65 ms on lock 0x7b18003f9fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:17.092931 (+  2037us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b18003f9fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9
0709 12:24:17.093630 (+   699us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b18003f9fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:17.271966 (+178336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.272051 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.272098 (+    47us) raft_consensus.cc:1685] finished
0709 12:24:17.272112 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.272262 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20086,"prepare.queue_time_us":1331083,"prepare.run_cpu_time_us":144,"prepare.run_wall_time_us":151,"raft.queue_time_us":78267,"raft.run_cpu_time_us":3991,"raft.run_wall_time_us":481666,"spinlock_wait_cycles":61090176,"thread_start_us":595947,"threads_started":3,"wal-append.queue_time_us":451454}
I20260709 12:24:17.308072 13647 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:17.303184 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:17.310634 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:17.172619 20428 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.310711 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:17.314561 13646 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
W20260709 12:24:17.285758 20975 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.011s	sys 0.053s
I20260709 12:24:17.285641 20996 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:17.320462 13793 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:17.293195 20472 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.322669 13644 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:17.323951 13647 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:17.333362 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.340714 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:17.341723 13651 raft_consensus.cc:2376] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18. Re-sending same reply.
I20260709 12:24:17.324331 20714 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:24:17.335807 20975 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.012s	sys 0.061s
I20260709 12:24:17.335299 20472 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:17.353703 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2579) took 1015 ms. Trace:
I20260709 12:24:17.354221 13792 rpcz_store.cc:276] 0709 12:24:16.338698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.338820 (+   122us) service_pool.cc:224] Handling call
0709 12:24:17.353450 (+1014630us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54003d45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:17.353627 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407877504}
I20260709 12:24:17.312341 20659 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.321928 20849 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:17.358546 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de32481a3be147a5b854251468fc667b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:17.360311 13651 raft_consensus.cc:606] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:17.361459 13651 raft_consensus.cc:614] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:17.348389 20714 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.365911 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:17.367851 13647 raft_consensus.cc:2376] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16. Re-sending same reply.
W20260709 12:24:17.372918 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65893.6us
W20260709 12:24:17.374598 20694 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.005s
I20260709 12:24:17.375118 20694 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.376437 14420 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:17.378069 20659 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:17.379180 14423 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:17.376317 21005 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.384899 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:17.388566 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:17.361155 20997 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:17.391786 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:17.395671 13647 raft_consensus.cc:2376] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24. Re-sending same reply.
I20260709 12:24:17.370930 20714 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:17.405622 14418 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:17.407186 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:17.426731 20849 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.427752 20860 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:17.428169 20860 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:17.429025 20860 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.370960 21001 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.219s
I20260709 12:24:17.437757 14419 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:17.417636 20328 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:17.442960 13647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:17.392263 21005 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.443749 13647 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:17.444329 13647 raft_consensus.cc:614] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:17.422448 20829 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.438654 20731 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:17.440079 20860 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:17.455047 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 136564us
I20260709 12:24:17.456212 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:17.459988 21005 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.461172 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:17.462791 20428 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.464277 20694 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:17.465049 21020 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 12:24:17.464600 20472 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:17.473768 21011 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:24:17.473804 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:17.476922 13724 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:17.478761 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:17.479421 13722 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:17.487869 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:17.490720 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.415832 20997 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.489332 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:17.467386 20999 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:17.455914 20731 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:17.500864 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 191450us
I20260709 12:24:17.469410 20980 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:24:17.508589 21016 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:24:17.508879 20980 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.509366 20980 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:17.481635 20328 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:17.510659 20328 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:17.511674 20328 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.513106 20328 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:17.514467 13719 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:17.518442 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:17.519817 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:17.502408 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:17.525312 13720 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:17.506524 13722 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:24:17.527048 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.528882 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:17.547966 20472 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.548913 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:17.550612 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:17.553395 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:17.549980 21027 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:17.594834 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:17.594925 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:17.526081 20999 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:17.600519 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 215680us
I20260709 12:24:17.563236 20472 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.590847 20849 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:17.591940 21027 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.594372 20972 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.597645 20933 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.664140 20933 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.665014 20980 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.666195 13724 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:17.666519 21000 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:17.667337 21000 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:17.667124 20714 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:17.668690 21026 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:17.668421 21000 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.669477 20714 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:17.651209 20694 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:17.652665 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:17.675268 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 170388us
I20260709 12:24:17.641027 21027 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:17.644164 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:17.676244 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:17.606976 20962 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:24:17.680954 20883 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:17.681699 20883 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:17.682622 20883 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.651315 20974 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.142s
I20260709 12:24:17.683926 20883 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:17.653484 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.685192 20842 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.685416 13797 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:17.685817 20842 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:17.686077 20933 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.657204 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:17.687031 20694 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:17.687471 20694 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:17.688416 20694 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.689702 21022 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:17.622133 20849 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.690493 20849 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:17.690442 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:17.691439 21022 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:17.691809 21022 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.692183 21022 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:17.606712 20630 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:17.694754 20630 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.695360 20630 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:17.631278 20472 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.681715 20962 consensus_queue.cc:1243] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:24:17.700515 20472 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.701280 20472 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:17.701694 20472 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.659165 21030 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:24:17.703639 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.660395 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:17.660058 20999 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.704865 20996 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.661136 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.660881 20731 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.661542 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.661813 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.719396 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:17.662354 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.662417 20972 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.641682 20959 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:24:17.663293 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.670537 20659 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:17.674485 20976 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.004s
I20260709 12:24:17.555100 20953 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:24:17.680833 21014 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:17.686233 20842 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:17.660727 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:17.654538 20955 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.712632 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:17.690068 20694 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:17.702077 20472 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:17.735791 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:17.818171 13801 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:17.846277 13725 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:17.809629 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:17.814009 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1365) took 1091 ms. Trace:
I20260709 12:24:17.799865 20980 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.736501 20714 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:17.726879 20965 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.723721 20428 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:24:17.736570 20962 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.781749 21014 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:17.834723 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102933us
I20260709 12:24:17.842841 13721 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:24:17.849154 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135146us
I20260709 12:24:17.847898 14420 rpcz_store.cc:276] 0709 12:24:16.722386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.722559 (+   173us) service_pool.cc:224] Handling call
0709 12:24:17.813698 (+1091139us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400498c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:17.813921 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397268224}
I20260709 12:24:17.850227 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.855623 13798 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:17.857999 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.867844 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:17.883641 13792 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:17.868055 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.862301 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.843955 20976 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.856447 20941 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.897889 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.859622 20953 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:24:17.901602 20953 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:17.898118 20941 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.903646 13795 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:17.846918 20959 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:17.910537 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:17.916460 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:17.902096 20953 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:17.903522 21009 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:24:17.856590 20955 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.915868 20941 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.864424 20714 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:17.966930 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3477) took 1378 ms. Trace:
I20260709 12:24:17.927484 20996 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:17.975066 13723 rpcz_store.cc:276] 0709 12:24:16.588185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.588339 (+   154us) service_pool.cc:224] Handling call
0709 12:24:17.966669 (+1378330us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b54001a5c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:17.966839 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56147840}
I20260709 12:24:17.945298 20959 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.033988 20941 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.033092 20953 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.988824 20842 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.959481 20840 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:18.134948 20953 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:18.134999 20842 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:18.146876 21034 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.003s
I20260709 12:24:17.904178 20965 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.961573 20980 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.039314 13801 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:17.908999 20962 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.072654 21027 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:17.908012 20976 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.117151 20472 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.118198 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1195) took 1126 ms. Trace:
I20260709 12:24:18.119498 20714 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:18.121271 20955 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.123226 20959 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.124922 13546 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 10 to 4, term changed from 22 to 10, leader changed from <none> to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from VOTER to NON_VOTER, now has a pending config: VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65), VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66), VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67), VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 10 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:18.131552 20933 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.133257 20997 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:17.969373 20940 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.031275 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 163545us
I20260709 12:24:18.165936 20472 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.193912 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.255251 13797 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:24:18.194870 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.237329 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:18.259479 14419 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:18.236495 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:18.224359 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.196931 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.207448 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.200214 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.240078 14418 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:24:18.166208 13796 rpcz_store.cc:276] 0709 12:24:16.991718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.991918 (+   200us) service_pool.cc:224] Handling call
0709 12:24:18.051484 (+1059566us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b54000d2fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:18.118142 (+ 66658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171841408}
W20260709 12:24:18.194381 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.251784 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:18.267016 13723 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:18.219300 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:18.275434 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56243.9us
I20260709 12:24:18.219914 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1567) took 1066 ms. Trace:
I20260709 12:24:18.283560 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:18.286940 14420 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:18.274303 20862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.015s	sys 0.075s
I20260709 12:24:18.292686 14420 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:18.220358 20849 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.001s
W20260709 12:24:18.154170 21029 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.115s	user 0.003s	sys 0.000s
W20260709 12:24:18.155316 21028 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:24:18.277850 13727 rpcz_store.cc:276] 0709 12:24:17.154104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.154241 (+   137us) service_pool.cc:224] Handling call
0709 12:24:17.154784 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.187340 (+ 32556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.187381 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.294531 (+107150us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.586567 (+292036us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:17.586641 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.586771 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.219639 (+632868us) raft_consensus.cc:1685] finished
0709 12:24:18.219670 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.219842 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130079,"raft.queue_time_us":27423,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":92352,"thread_start_us":252883,"threads_started":1,"wal-append.queue_time_us":642054}
W20260709 12:24:18.248339 21022 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.006s
I20260709 12:24:18.211267 20898 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:18.311312 14420 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:18.170742 20974 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:18.163640 21027 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:18.166087 20976 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.166193 20962 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.154881 20972 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.161490 20965 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.164575 20765 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:24:18.319535 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.026s	sys 0.049s
W20260709 12:24:18.294176 20862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.016s	sys 0.075s
I20260709 12:24:18.181934 20630 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:18.323352 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.020s	sys 0.054s
I20260709 12:24:18.325987 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:18.356001 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.020s	sys 0.054s
I20260709 12:24:18.290019 21005 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.330052 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.323418 20898 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:18.291527 20829 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:24:18.173508 20940 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.172511 20933 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.290154 20941 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:18.209689 20955 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.113s
I20260709 12:24:18.241711 20999 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:24:18.158955 20980 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.344740 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:18.352169 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.026s	sys 0.049s
I20260709 12:24:18.353616 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:18.274509 21012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.032s	sys 0.062s
I20260709 12:24:18.397147 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:18.397744 13793 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:18.354171 20765 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:18.360776 14419 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:18.360812 20472 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:18.284075 20428 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:24:18.362605 13799 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:24:18.423758 21046 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:18.424190 21046 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:18.322926 20974 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:18.425160 21046 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:18.426244 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.334996 20883 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.007s
I20260709 12:24:18.425947 20898 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:18.340440 20865 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.025s	sys 0.037s
W20260709 12:24:18.427335 20865 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.025s	sys 0.038s
I20260709 12:24:18.373061 13723 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
W20260709 12:24:18.431269 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86656.4us
I20260709 12:24:18.387374 14418 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:18.305979 20849 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:18.339869 20848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.019s	sys 0.042s
I20260709 12:24:18.383891 20976 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:18.174554 21014 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.324090 21022 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.422422 20472 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.422928 20955 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:18.415679 21012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.033s	sys 0.062s
I20260709 12:24:18.425348 20974 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:18.359917 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.014s	sys 0.049s
I20260709 12:24:18.363619 20965 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:18.434772 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.016s	sys 0.053s
W20260709 12:24:18.481082 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.016s	sys 0.053s
W20260709 12:24:18.481575 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 928219us
I20260709 12:24:18.481173 20965 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.482620 20472 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:18.483510 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.532s (SENT)
I20260709 12:24:18.484076 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:18.331523 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:18.486163 20962 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:18.486831 14420 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:18.491029 13719 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:24:18.491518 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.451076 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 774914us
I20260709 12:24:18.492534 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:18.493412 13724 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:18.493860 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:18.495653 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:18.496734 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:18.393700 20941 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:18.453833 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:18.353996 20997 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.079s
W20260709 12:24:18.476189 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.014s	sys 0.049s
I20260709 12:24:18.490840 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:18.499002 13723 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:24:18.497846 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44165.2us
I20260709 12:24:18.364187 21005 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:18.445168 20848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.020s	sys 0.043s
I20260709 12:24:18.514813 20941 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:18.519296 14416 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:18.519721 20472 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:18.525261 20472 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:18.458210 21046 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:18.520396 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:18.525763 20472 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:18.527652 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:18.532124 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.522500 20840 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:18.536665 13724 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:18.534710 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 985803us
I20260709 12:24:18.541168 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:18.544853 13723 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:18.542586 13793 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:24:18.552920 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:18.553947 13724 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:18.555230 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:18.556401 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:18.523444 20940 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.522716 20959 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:18.455823 20976 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.528460 20975 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.499668 20955 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.568614 13792 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:18.518795 21040 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:18.573422 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:18.575042 20472 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.575393 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:18.576048 13723 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:24:18.546622 21005 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:18.588398 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:18.598405 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:18.557857 20842 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.617372 21047 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:18.618005 20842 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.618578 21047 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:18.619012 21047 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:18.623368 20849 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:18.625281 13648 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:24:18.626931 13718 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:18.558390 20883 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:18.628147 13644 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:18.564260 20840 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:18.563747 20977 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:18.542520 21045 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:18.600857 20940 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.603592 20630 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.605731 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:18.606563 20472 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.607040 20865 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:18.610545 21036 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:18.610919 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:18.611089 20997 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:18.611150 20687 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:18.647902 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:18.648396 21022 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.648558 13646 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:18.649853 20865 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:18.613389 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:18.614217 21037 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:18.611246 21001 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 12:24:18.615011 21042 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:18.611455 21027 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:24:18.611308 20714 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:24:18.616104 20974 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:18.616556 20962 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:18.626739 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:18.604775 21038 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:18.630451 13651 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:24:18.676976 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:18.632083 14420 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:18.630194 20883 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:18.683084 13646 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:18.639297 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:18.641001 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:18.647665 20472 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:18.654707 13723 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:18.655767 20977 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:18.601348 20975 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.656159 20865 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:18.611321 20965 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.631646 20933 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:18.663159 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52314us
I20260709 12:24:18.710216 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:18.679224 20992 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:18.613907 20953 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:24:18.638394 21045 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:18.671011 13650 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:24:18.658066 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:18.677650 13651 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:18.716687 13651 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:24:18.633703 13724 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:18.696208 20472 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.697369 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:18.719573 13646 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:24:18.656118 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 378871us
I20260709 12:24:18.702373 13644 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:18.665751 20731 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.007s	sys 0.036s
I20260709 12:24:18.703205 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:18.698414 13645 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:18.703992 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:18.685791 20694 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.007s
I20260709 12:24:18.645892 20940 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.663022 20687 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:18.723114 13793 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:24:18.704514 20829 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:18.709683 21047 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.631347 20840 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.727735 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:18.683983 20883 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:18.745507 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:18.727353 20731 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.007s	sys 0.038s
I20260709 12:24:18.709988 21040 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.786646 13646 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:24:18.631444 20765 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.740931 20980 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.693282 21038 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:18.742875 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:18.726538 21045 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:18.801972 13794 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:18.778092 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.742657 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:18.778401 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:18.803633 13652 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:18.754078 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:18.739087 13618 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:18.804821 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 65866.2us
I20260709 12:24:18.781245 20472 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:18.747900 20975 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:18.742374 20953 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:18.740938 20933 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:18.742338 21012 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:18.785990 20992 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:18.717145 13651 pending_rounds.cc:85] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 11
I20260709 12:24:18.832150 13651 pending_rounds.cc:107] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:24:18.832505 13651 raft_consensus.cc:2889] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.781098 20862 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:18.742857 20940 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:18.777963 20999 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:24:18.806284 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52262.4us
I20260709 12:24:18.849058 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:18.726583 20965 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:18.796257 21040 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.851192 20965 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.852023 20965 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:18.826347 20953 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:18.853566 21040 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.742201 20976 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.778857 20630 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:18.800030 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46364.8us
I20260709 12:24:18.745916 20694 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:24:18.746613 20829 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.859736 14391 leader_election.cc:400] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19.
I20260709 12:24:18.862908 14391 leader_election.cc:403] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:24:18.743239 20977 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.827348 20933 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:18.832391 20765 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:18.866340 13645 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:18.811403 21037 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:24:18.839025 21022 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:18.844914 20997 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.847807 20898 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:18.743953 20687 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.866919 21005 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:18.854208 20953 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.826985 20865 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.875933 21012 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19.
W20260709 12:24:18.910760 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 414073us
I20260709 12:24:18.867479 20630 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.885919 21051 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:24:18.917555 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 423393us
I20260709 12:24:18.915772 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:18.893198 20955 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:18.907534 21005 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:18.882416 20997 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.947227 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:18.888965 20941 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:18.924139 21051 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:18.912279 21001 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:24:18.947467 20953 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:18.948694 20976 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:18.887115 21038 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:18.950548 20865 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:18.943151 20694 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.961392 21051 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:24:18.951081 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.001765 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 133042us
W20260709 12:24:19.002732 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 129925us
W20260709 12:24:19.002596 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.964821 21014 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:18.966372 20959 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:19.060388 20959 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.004726 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:19.023955 20997 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:19.024936 21050 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:24:19.026418 21022 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:24:19.027859 21048 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:24:19.028378 20941 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:19.029930 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.012596 20694 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:19.032897 21049 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:24:19.034739 20630 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.002216 20714 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.975302 20976 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:19.042586 20965 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.013132 21036 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:19.052733 20829 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.048441 21051 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:19.053519 21043 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.396s	user 0.002s	sys 0.000s
I20260709 12:24:18.934541 20992 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.134285 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:19.057574 21046 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.003s
I20260709 12:24:19.058457 21061 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:19.063056 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58443.1us
I20260709 12:24:19.073686 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:19.096539 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.058537 21039 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:18.948462 20933 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.075702 21001 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:19.121847 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:19.129575 20976 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:19.147526 13723 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:19.148198 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:19.149703 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94145.8us
W20260709 12:24:19.159766 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1627447539999594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 12:24:19.127624 20630 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.131539 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 518190us
I20260709 12:24:19.155568 14420 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
W20260709 12:24:19.160250 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1420447539999259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 12:24:19.163586 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1033447540003181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17320000000000002s for a response
W20260709 12:24:19.163956 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1015447540003152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
I20260709 12:24:19.152257 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:19.164336 13693 connection.cc:256] RPC call timeout handler was delayed by -1.043944754000222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 12:24:19.163997 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:19.167959 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.168632 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.168982 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:18.948706 20955 consensus_queue.cc:1175] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 25 status { last_received { term: 25 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:19.174248 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.182590 13645 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:19.175065 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.172963 20428 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:19.172963 21046 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:19.178452 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.179263 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:19.185205 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:19.128659 20714 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.150588 21039 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:19.183701 13651 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.206225 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.121536 21005 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.207620 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.207046 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.231930 13645 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:19.232344 13647 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:24:19.212602 20842 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.215929 13643 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:24:19.218592 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.206393 13650 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:19.150727 21027 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:24:19.170398 20996 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:19.157407 21061 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:19.258123 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 103728us
I20260709 12:24:19.273146 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.171759 20630 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.279757 13652 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:19.261335 20996 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:19.273694 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.286604 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1369) took 1551 ms. Trace:
I20260709 12:24:19.287325 14416 rpcz_store.cc:276] 0709 12:24:17.728247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.728453 (+   206us) service_pool.cc:224] Handling call
0709 12:24:17.781882 (+ 53429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.528413 (+746531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.278780 (+750367us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:19.278983 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1691594496}
I20260709 12:24:19.287920 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.230834 20955 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.290741 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.291502 20694 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.291389 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.307749 13798 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:19.292215 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.239284 21078 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 12:24:19.239007 21036 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.314677 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.315965 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.316666 13800 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:19.293543 21045 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:19.294760 21060 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 12:24:19.295500 21068 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:24:19.297628 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.295346 20996 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.228976 20883 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.300549 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.301748 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.303323 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:19.303696 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43700.5us
I20260709 12:24:19.330888 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.331084 13645 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:19.323664 13651 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:19.325327 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.298079 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.333978 13724 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.337514 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.346706 13724 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:19.347065 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.349165 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.354005 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.354898 13645 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:19.328027 21046 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.356206 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.357705 13722 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
W20260709 12:24:19.360930 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.328660 21001 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:19.310536 20955 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.367833 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.368474 20714 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:19.369482 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.370182 13645 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
W20260709 12:24:19.370738 20974 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.003s
I20260709 12:24:19.371104 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.365120 21051 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.376500 21045 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.377733 13649 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:24:19.378621 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.379546 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.379619 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:19.339255 20862 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:19.384127 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:19.386179 20765 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.000s
W20260709 12:24:19.386564 13544 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 256) took 19541 ms (client timeout 15000 ms). Trace:
I20260709 12:24:19.387557 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.387773 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.387566 13795 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:19.372014 21061 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.392109 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.392711 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.393677 13645 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:19.394055 13646 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:24:19.394444 13646 pending_rounds.cc:85] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 7
I20260709 12:24:19.394708 13646 pending_rounds.cc:107] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:24:19.394932 13646 raft_consensus.cc:2889] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:19.396948 20977 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.399120 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.400979 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:19.401470 20977 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:19.407485 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.409185 21001 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.412029 20630 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:19.421674 13652 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
W20260709 12:24:19.426337 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.427107 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.389871 21051 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.435447 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:19.436638 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.358958 21046 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.438438 14419 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:19.438601 20977 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:19.421366 20883 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.437001 20630 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.451472 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:19.452514 13651 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:19.452587 13648 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:19.441098 20996 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:19.464282 13648 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:19.468216 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.468354 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:19.469110 13648 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
W20260709 12:24:19.470067 21077 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:24:19.471442 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.472280 13648 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:19.473275 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.474017 13648 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:19.475644 13648 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:24:19.480199 20933 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:19.388088 13544 rpcz_store.cc:269] 0709 12:23:59.845167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.597893 (+752726us) service_pool.cc:224] Handling call
0709 12:24:19.386310 (+18788417us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:09.462790 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:09.463500 (+   710us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:09.463559 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:24:09.463586 (+    27us) tablet.cc:662] Decoding operations
0709 12:24:09.711933 (+248347us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:09.712053 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:09.712094 (+    41us) tablet.cc:685] Acquiring locks for 32 operations
0709 12:24:09.760881 (+ 48787us) tablet.cc:701] Row locks acquired
0709 12:24:09.760950 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:24:09.761138 (+   188us) write_op.cc:270] Start()
0709 12:24:09.761310 (+   172us) write_op.cc:276] Timestamp: P: 1783599849761070 usec, L: 0
0709 12:24:09.761346 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:24:09.762123 (+   777us) log.cc:844] Serialized 12437 byte log entry
0709 12:24:10.318016 (+555893us) op_driver.cc:464] REPLICATION: finished
0709 12:24:11.177343 (+859327us) write_op.cc:301] APPLY: starting
0709 12:24:11.177501 (+   158us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:12.080107 (+902606us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:12.080259 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:13.244290 (+1164031us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:13.317080 (+ 72790us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=0,mrs_lookups=0
0709 12:24:13.317155 (+    75us) write_op.cc:312] APPLY: finished
0709 12:24:13.317823 (+   668us) log.cc:844] Serialized 276 byte log entry
0709 12:24:13.547202 (+229379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:13.578566 (+ 31364us) write_op.cc:454] Released schema lock
0709 12:24:13.600999 (+ 22433us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:13.635164 (+ 34165us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:13.635203 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1571711,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":815377,"apply.run_cpu_time_us":113657,"apply.run_wall_time_us":2480411,"cfile_cache_hit":130,"cfile_cache_hit_bytes":138143,"num_ops":32,"prepare.queue_time_us":376322,"prepare.run_cpu_time_us":42689,"prepare.run_wall_time_us":491733,"raft.queue_time_us":317324,"raft.run_cpu_time_us":20057,"raft.run_wall_time_us":852792,"replication_time_us":550808,"thread_start_us":1486046,"threads_started":5,"wal-append.queue_time_us":463408,"wal-append.run_cpu_time_us":99460,"wal-append.run_wall_time_us":2423358}]]}
I20260709 12:24:19.483470 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:19.443099 21051 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.501166 13792 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:19.508152 20731 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:19.508674 20731 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:19.511523 20976 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.514389 20976 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.516860 13796 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:19.518389 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.519956 20849 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.520109 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.520646 20849 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.521309 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.522127 13724 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:24:19.527115 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.531255 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.509639 20731 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.333801 21084 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.203s
I20260709 12:24:19.533231 20955 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.534370 20849 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:19.491282 21066 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:19.540181 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:19.499210 21036 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:19.458422 20630 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.546952 13645 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:19.552328 13649 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:19.544866 13724 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:19.487805 20883 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:24:19.546177 20731 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.531010 21070 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.565984 13644 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:19.576622 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:19.582701 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.561591 21036 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.586490 13724 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:19.594154 21046 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:19.574492 21066 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:19.596251 21066 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.600684 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:19.601730 20630 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.602073 21037 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.602516 21037 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.602890 21037 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.587910 21005 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:24:19.612560 14416 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:19.615465 14419 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:19.476799 21039 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.619247 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:19.550712 20865 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:24:19.593577 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.613113 21005 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:19.618731 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:19.632359 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.632513 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:19.590969 20953 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.002s
I20260709 12:24:19.625202 13724 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:19.637163 21039 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:19.637349 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.625428 13645 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:19.630578 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.650385 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3504) took 1161 ms. Trace:
I20260709 12:24:19.651708 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:19.652089 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3507) took 1162 ms. Trace:
I20260709 12:24:19.658947 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.659550 13721 rpcz_store.cc:276] 0709 12:24:18.489542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.494609 (+  5067us) service_pool.cc:224] Handling call
0709 12:24:18.495150 (+   541us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:18.495602 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.495627 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:18.496465 (+   838us) log.cc:844] Serialized 52 byte log entry
0709 12:24:18.902454 (+405989us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:19.090183 (+187729us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.651357 (+561174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.651482 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.651546 (+    64us) raft_consensus.cc:1685] finished
0709 12:24:19.651747 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.652003 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":559806,"prepare.queue_time_us":136820,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":23926,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":578098,"threads_started":2,"wal-append.queue_time_us":501866}
I20260709 12:24:19.653496 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3517) took 1017 ms. Trace:
I20260709 12:24:19.643113 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3506) took 1153 ms. Trace:
I20260709 12:24:19.588276 21036 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:19.665957 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.662820 13718 rpcz_store.cc:276] 0709 12:24:18.636045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.636173 (+   128us) service_pool.cc:224] Handling call
0709 12:24:18.636475 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.636711 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:18.929111 (+292400us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.652974 (+723863us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.653056 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:19.653137 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.653229 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.653412 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":543488,"prepare.queue_time_us":401984,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"thread_start_us":471876,"threads_started":2,"wal-append.queue_time_us":410868}
I20260709 12:24:19.668433 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.669180 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.622524 21004 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.624135 20630 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.658016 13727 rpcz_store.cc:276] 0709 12:24:18.488639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.491463 (+  2824us) service_pool.cc:224] Handling call
0709 12:24:18.491800 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.492091 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:18.865457 (+373366us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.649838 (+784381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.649925 (+    87us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:19.649997 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.650098 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.650299 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":582953,"prepare.queue_time_us":546291,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"thread_start_us":563857,"threads_started":2,"wal-append.queue_time_us":286313}
I20260709 12:24:19.670557 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.669978 13718 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:19.670457 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.681259 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.637163 20883 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.697916 20688 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:19.635672 21081 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:19.698396 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:19.682946 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 497795us
I20260709 12:24:19.684737 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.628935 21005 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:19.703960 13651 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:24:19.689703 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:19.702852 13797 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:24:19.687045 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1581) took 1090 ms. Trace:
I20260709 12:24:19.713315 13720 rpcz_store.cc:276] 0709 12:24:18.596365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.598054 (+  1689us) service_pool.cc:224] Handling call
0709 12:24:19.686789 (+1088735us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b5400174654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:19.686972 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274337536}
I20260709 12:24:19.715687 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.716645 13720 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:19.581470 21070 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.709198 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.664147 13725 rpcz_store.cc:276] 0709 12:24:18.489236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.489380 (+   144us) service_pool.cc:224] Handling call
0709 12:24:18.489753 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.489979 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:18.906766 (+416787us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.641011 (+734245us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b1800093d60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:24:19.642221 (+  1210us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.642562 (+   341us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.642633 (+    71us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:19.642708 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.642829 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.643021 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47985,"prepare.queue_time_us":491184,"prepare.run_cpu_time_us":228,"prepare.run_wall_time_us":8198,"spinlock_wait_cycles":45841792,"thread_start_us":519483,"threads_started":2,"wal-append.queue_time_us":380012}
I20260709 12:24:19.705231 13721 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:19.703583 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.702049 13650 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:19.719214 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.701467 20940 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:19.774892 20940 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.777045 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.780838 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.690831 20980 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:19.780957 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.698416 20688 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:19.802476 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99938.6us
I20260709 12:24:19.782335 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.720227 13724 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:19.769345 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:19.743618 20883 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:19.795567 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.822181 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.823177 13725 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:19.802088 13726 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:24:19.699448 20630 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.657760 21001 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:24:19.719646 21081 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.783830 13723 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:19.809564 13727 raft_consensus.cc:2376] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18. Re-sending same reply.
I20260709 12:24:19.829967 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.830755 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.691461 20992 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.098s
I20260709 12:24:19.826671 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.876175 13724 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:19.877254 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:19.770455 21101 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:24:19.824085 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.769313 21051 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:19.835054 13720 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:24:19.699496 20849 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 12:24:19.745483 20865 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
W20260709 12:24:19.841187 21096 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:19.801740 20980 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:19.843647 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58823.7us
I20260709 12:24:19.844633 21086 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:19.846791 21005 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.706343 21004 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.908877 20980 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:19.916182 21004 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:19.919395 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:19.921283 13794 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:19.928918 20965 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:19.856823 13649 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:24:19.861007 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:19.940526 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 79621us
I20260709 12:24:19.821249 21052 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.956696 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:19.822036 20688 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:19.825977 20953 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.034926 20992 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
I20260709 12:24:19.864753 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.821718 20883 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.866684 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.879678 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.884408 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:19.886720 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.890460 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.891235 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:19.893534 20940 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:19.908152 13651 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:19.910887 21088 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.504s	user 0.002s	sys 0.005s
W20260709 12:24:19.912041 21094 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:24:19.912423 20714 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:19.913599 21005 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:19.947639 13652 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:19.839095 21081 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:19.963639 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:19.897432 21000 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:20.016849 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:20.011580 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.997679 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.001852 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.029960 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.029758 21052 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:19.963764 20840 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:20.034934 20996 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:24:20.001681 21084 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:20.018803 20977 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:20.003777 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:20.080778 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48688.6us
W20260709 12:24:20.083214 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49248.2us
W20260709 12:24:20.015722 13693 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34464 dropped due to backpressure: service queue is full with 50 items
I20260709 12:24:20.300100 20953 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.300845 13726 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:20.301548 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.302372 20883 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:20.304148 13721 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:20.101199 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.305627 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.306944 20965 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.308004 20965 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.308815 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.309705 13720 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:20.300658 21073 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:20.108353 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.314765 13724 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:20.315582 13722 raft_consensus.cc:2410] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:24:20.311637 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:20.276911 13793 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:20.077421 20428 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.310137 13726 raft_consensus.cc:2410] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 12:24:20.311589 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.072918 20933 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.001s
I20260709 12:24:20.318488 13725 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:20.312861 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.342079 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3468) took 1133 ms. Trace:
W20260709 12:24:20.124774 21099 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:24:20.306900 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:20.341455 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.121706 13646 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:24:20.344146 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.314787 21073 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:20.341701 20428 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.345469 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:20.342486 13726 rpcz_store.cc:276] 0709 12:24:19.208941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.832518 (+623577us) service_pool.cc:224] Handling call
0709 12:24:20.342001 (+509483us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:20.347131 21102 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
W20260709 12:24:20.347934 21104 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:24:20.381743 13652 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:20.318696 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:20.351845 13723 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:20.357146 13727 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:20.124320 21070 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:20.387626 13643 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:20.389170 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:20.390717 13650 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:20.379263 13646 pending_rounds.cc:85] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 13
I20260709 12:24:20.393640 13646 pending_rounds.cc:107] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 12:24:20.394032 13646 raft_consensus.cc:2889] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:20.290942 20731 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.007s
W20260709 12:24:20.385751 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67169us
I20260709 12:24:20.343448 20840 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:20.399224 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:20.347952 21047 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:20.400277 13721 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:24:20.318413 21084 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.107048 21014 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:20.321267 20992 consensus_queue.cc:1243] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:24:20.387306 20933 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:24:20.342576 21000 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.148175 13692 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34464 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3498)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3499)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3503)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3523)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3513)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3537)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3534)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3525)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3479)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3523)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3522)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3524)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3525)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3528)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3529)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3531)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3533)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3484)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3485)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3535)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1588)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1589)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1590)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1593)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1597)
I20260709 12:24:20.290635 20630 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:20.413584 13650 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:20.423054 20862 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:24:20.410748 21069 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.430557 20848 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:20.456393 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.433713 20975 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:20.438781 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.411455 21109 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.420966 21037 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.818s	user 0.012s	sys 0.004s
I20260709 12:24:20.471863 14419 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:20.442422 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3978) took 1013 ms. Trace:
I20260709 12:24:20.480435 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.480854 13649 rpcz_store.cc:276] 0709 12:24:19.429021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.566229 (+137208us) service_pool.cc:224] Handling call
0709 12:24:19.856439 (+290210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.959797 (+103358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.442179 (+482382us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400112950. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:20.442352 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655925632}
I20260709 12:24:20.435853 21073 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.447537 21069 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.437002 20953 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.449321 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3521) took 1294 ms. Trace:
I20260709 12:24:20.451583 21052 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:20.451927 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.453446 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3979) took 1023 ms. Trace:
W20260709 12:24:20.455267 20980 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.007s
I20260709 12:24:20.432967 20865 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:20.144047 21036 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.307616 20714 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:20.320700 20977 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.476634 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:20.503216 13727 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:24:20.448932 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3984) took 1017 ms. Trace:
I20260709 12:24:20.512025 14418 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:20.512647 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1219) took 1229 ms. Trace:
I20260709 12:24:20.509179 13719 rpcz_store.cc:276] 0709 12:24:19.155012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.177180 (+ 22168us) service_pool.cc:224] Handling call
0709 12:24:19.215363 (+ 38183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.687642 (+472279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.687680 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.688383 (+   703us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.861738 (+173355us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:19.861785 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.861857 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.449024 (+587167us) raft_consensus.cc:1685] finished
0709 12:24:20.449060 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.449243 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":471799,"raft.queue_time_us":475449,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":172895,"threads_started":1,"wal-append.queue_time_us":374189}
I20260709 12:24:20.511817 13724 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:20.519536 14424 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:24:20.522575 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46066.9us
I20260709 12:24:20.511770 13644 rpcz_store.cc:276] 0709 12:24:19.429628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.572840 (+143212us) service_pool.cc:224] Handling call
0709 12:24:19.573351 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.574054 (+   703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.574086 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.574827 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.797158 (+222331us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:19.850539 (+ 53381us) log.cc:844] Serialized 14 byte log entry
0709 12:24:19.850915 (+   376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.851010 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.453190 (+602180us) raft_consensus.cc:1685] finished
0709 12:24:20.453214 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.453380 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22195,"prepare.queue_time_us":53904,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":11385,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":8595,"thread_start_us":191774,"threads_started":1,"wal-append.queue_time_us":300631}
I20260709 12:24:20.470779 20992 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:24:20.532554 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:20.523588 13648 rpcz_store.cc:276] 0709 12:24:19.283093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.478320 (+195227us) service_pool.cc:224] Handling call
0709 12:24:19.479125 (+   805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.479889 (+   764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.479927 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.480911 (+   984us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.494870 (+1013959us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800314da0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1
0709 12:24:20.495181 (+   311us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:20.495228 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.495305 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.495369 (+    64us) raft_consensus.cc:1685] finished
0709 12:24:20.495387 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.512574 (+ 17187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"spinlock_wait_cycles":8092032,"thread_start_us":19496,"threads_started":1,"wal-append.queue_time_us":19555}
I20260709 12:24:20.535800 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:20.537834 13724 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 12:24:20.528918 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110784us
I20260709 12:24:20.523051 13651 rpcz_store.cc:276] 0709 12:24:19.431409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.805481 (+374072us) service_pool.cc:224] Handling call
0709 12:24:19.907938 (+102457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.313172 (+405234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.448629 (+135457us) spinlock_profiling.cc:243] Waited 86.8 ms on lock 0x7b54001b5b50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:20.448848 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199605888}
I20260709 12:24:20.469278 21014 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.539449 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:20.548520 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:20.549213 13723 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:20.541617 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2591) took 1269 ms. Trace:
I20260709 12:24:20.550911 13793 rpcz_store.cc:276] 0709 12:24:19.271717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.271851 (+   134us) service_pool.cc:224] Handling call
0709 12:24:20.541433 (+1269582us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b54000d16d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:20.541534 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1959979904}
I20260709 12:24:20.551559 13644 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:20.554934 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.555676 13726 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:20.558219 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.559190 13721 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:20.562978 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.563840 13719 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:20.506536 21037 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.565898 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.566721 13724 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:20.569149 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3479) took 1054 ms. Trace:
I20260709 12:24:20.569572 13724 rpcz_store.cc:276] 0709 12:24:19.515139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.565536 (+1050397us) service_pool.cc:224] Handling call
0709 12:24:20.569069 (+  3533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.528829 20953 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:20.530059 21073 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:20.563763 13618 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:20.579074 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:20.504261 21109 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.581262 13724 raft_consensus.cc:2410] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:24:20.582472 13720 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:24:20.515632 20980 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.585031 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.588310 13724 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:24:20.564970 20933 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.589354 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.586040 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:20.590719 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.598831 13724 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:20.599328 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:20.602447 13727 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:20.576987 20953 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:20.604763 13645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.015s	sys 0.022s
I20260709 12:24:20.605525 13718 raft_consensus.cc:2368] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21.
I20260709 12:24:20.607321 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:20.609815 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:20.610201 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3522) took 1440 ms. Trace:
I20260709 12:24:20.612741 13726 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:20.613350 13645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.016s	sys 0.022s
I20260709 12:24:20.613655 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3488) took 1095 ms. Trace:
I20260709 12:24:20.598047 20630 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:24:20.614102 13726 rpcz_store.cc:276] 0709 12:24:19.518051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.589434 (+1071383us) service_pool.cc:224] Handling call
0709 12:24:20.613573 (+ 24139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.567668 21038 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:24:20.611440 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3478) took 1096 ms. Trace:
I20260709 12:24:20.619470 13727 rpcz_store.cc:276] 0709 12:24:19.514713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.590357 (+1075644us) service_pool.cc:224] Handling call
0709 12:24:20.611340 (+ 20983us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:20.607990 13766 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34464 dropped due to backpressure: service queue is full with 50 items [suppressed 35 similar messages]
W20260709 12:24:20.603379 20688 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.000s
I20260709 12:24:20.529445 20840 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.616986 13724 rpcz_store.cc:276] 0709 12:24:19.170010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.603767 (+1433757us) service_pool.cc:224] Handling call
0709 12:24:20.604220 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.604458 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.604496 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.604558 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:20.604593 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.604729 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.610115 (+  5386us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:20.614122 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 223199us
I20260709 12:24:20.616034 13723 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:24:20.623788 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3524) took 1419 ms. Trace:
I20260709 12:24:20.622929 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3527) took 1409 ms. Trace:
I20260709 12:24:20.624629 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3528) took 1410 ms. Trace:
I20260709 12:24:20.624584 13727 rpcz_store.cc:276] 0709 12:24:19.213934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.620756 (+1406822us) service_pool.cc:224] Handling call
0709 12:24:20.621078 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.622519 (+  1441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.622599 (+    80us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.622647 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:20.622686 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.622748 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.622872 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.625178 13724 rpcz_store.cc:276] 0709 12:24:19.214635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.623342 (+1408707us) service_pool.cc:224] Handling call
0709 12:24:20.624140 (+   798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.624320 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.624335 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.624396 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:20.624433 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.624490 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.624581 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.608398 20883 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.624439 13723 rpcz_store.cc:276] 0709 12:24:19.204563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.615510 (+1410947us) service_pool.cc:224] Handling call
0709 12:24:20.615796 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.623411 (+  7615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.623718 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.614037 20996 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:20.612054 20953 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:20.614967 20842 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.629951 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3529) took 1406 ms. Trace:
I20260709 12:24:20.631314 13724 rpcz_store.cc:276] 0709 12:24:19.223664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.627514 (+1403850us) service_pool.cc:224] Handling call
0709 12:24:20.629018 (+  1504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.629205 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.629229 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.629269 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:20.629302 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.629727 (+   425us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.629875 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.618608 21052 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:20.634984 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.641201 13727 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:20.647498 21014 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.647740 20883 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:20.630681 20996 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.649503 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.639868 20842 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.651829 21109 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:20.652341 20842 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.606329 20933 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.649266 20996 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:20.659226 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.649405 14416 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:20.582012 20714 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.663971 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:20.624884 20630 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:20.664697 13793 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:20.635308 21051 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:20.639132 20694 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.687379 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:20.662071 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3531) took 1399 ms. Trace:
I20260709 12:24:20.689131 13727 rpcz_store.cc:276] 0709 12:24:19.263009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.640517 (+1377508us) service_pool.cc:224] Handling call
0709 12:24:20.640940 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.661781 (+ 20841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.662003 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.626708 20688 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.692945 13727 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:24:20.699860 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3533) took 1413 ms. Trace:
I20260709 12:24:20.700380 13727 rpcz_store.cc:276] 0709 12:24:19.286779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.691988 (+1405209us) service_pool.cc:224] Handling call
0709 12:24:20.692526 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.699563 (+  7037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.699802 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.702586 13727 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:20.679972 21052 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:20.705412 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3475) took 1271 ms. Trace:
I20260709 12:24:20.706707 13727 rpcz_store.cc:276] 0709 12:24:19.434490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.701709 (+1267219us) service_pool.cc:224] Handling call
0709 12:24:20.701966 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.705080 (+  3114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.705337 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:20.707875 13797 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:20.697472 20842 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:20.688526 21084 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.693926 20977 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:20.702509 20714 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:24:20.684517 20992 consensus_queue.cc:1175] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dafe08a4f404502822a0732fe570de4" responder_term: 28 status { last_received { term: 25 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.18-27.18]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:20.694118 21069 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.732751 21047 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.733296 21047 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.695492 20694 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:20.736795 20694 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:20.731602 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.715366 20842 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.739181 20842 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:20.692912 21051 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.740125 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3988) took 1297 ms. Trace:
I20260709 12:24:20.740553 13645 rpcz_store.cc:276] 0709 12:24:19.443032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.857920 (+414888us) service_pool.cc:224] Handling call
0709 12:24:19.889459 (+ 31539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:19.954901 (+ 65442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.954933 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:19.959669 (+  4736us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.613811 (+654142us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:20.613858 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.613933 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.739887 (+125954us) raft_consensus.cc:1685] finished
0709 12:24:20.739909 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:20.740062 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56014,"raft.queue_time_us":56157,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":43934,"thread_start_us":572429,"threads_started":1,"wal-append.queue_time_us":604239}
I20260709 12:24:20.742419 21051 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:20.744063 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:20.745114 13793 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:20.750639 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.751660 14419 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:20.753010 21047 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.706974 21036 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:20.754947 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:20.754860 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:20.757819 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 149866us
I20260709 12:24:20.711804 20941 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.764354 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:20.765143 13645 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
W20260709 12:24:20.766050 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.714766 21000 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.788641 20980 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:20.311456 13693 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:34464 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3525)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1587)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3518)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3519)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1594)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3479)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3522)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3478)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3488)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3523)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3522)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3524)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3525)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3527)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3528)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3529)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3530)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3531)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3533)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3475)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3476)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3484)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3485)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3490)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3492)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3494)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3535)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3536)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3539)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1588)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1589)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1590)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1591)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1592)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1593)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1597)
I20260709 12:24:20.800441 21128 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:20.863878 21128 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.864444 21128 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:20.908277 20933 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:20.914060 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.910643 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:20.823818 21052 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:20.920029 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.652s (SENT)
I20260709 12:24:20.920925 13793 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:20.785949 20630 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.928435 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.931022 14419 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:24:20.923844 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 160541us
I20260709 12:24:20.933815 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:20.910138 20688 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:20.911113 21131 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.134s	user 0.004s	sys 0.000s
I20260709 12:24:20.929503 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:20.922580 21119 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.946002 13617 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.552s (SENT)
W20260709 12:24:20.932722 21005 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.011s
I20260709 12:24:20.960402 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:20.962993 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.988366 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 220599us
W20260709 12:24:20.947125 21117 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.232s	user 0.000s	sys 0.007s
I20260709 12:24:20.910507 20941 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:20.995843 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.934988 20996 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.004133 20975 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.010684 21120 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:21.012526 20848 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:21.015960 21037 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:21.017109 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:21.018812 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:21.019186 21127 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:21.019044 21081 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:21.019673 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:21.058682 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:21.021158 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1224) took 1161 ms. Trace:
W20260709 12:24:21.023056 21125 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:24:21.024596 13800 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:24:21.026870 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.027298 20965 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:21.029455 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:21.029781 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.030436 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:21.065337 14418 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:21.035465 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3523) took 1852 ms. Trace:
I20260709 12:24:20.978994 20933 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:20.982118 21047 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:21.038985 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:21.039443 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:21.038671 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.049803 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:20.990546 20688 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:24:21.061388 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3525) took 1850 ms. Trace:
I20260709 12:24:21.083810 13726 rpcz_store.cc:276] 0709 12:24:19.211436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.619534 (+1408098us) service_pool.cc:224] Handling call
0709 12:24:20.628854 (+  9320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.629485 (+   631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.629524 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.632946 (+  3422us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.696701 (+ 63755us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:20.698348 (+  1647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.698451 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.061141 (+362690us) raft_consensus.cc:1685] finished
0709 12:24:21.061165 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.061326 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":150,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4139,"thread_start_us":49901,"threads_started":1,"wal-append.queue_time_us":67179}
I20260709 12:24:21.062052 13727 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:21.061702 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.061451 13794 rpcz_store.cc:276] 0709 12:24:19.859849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.926634 (+ 66785us) service_pool.cc:224] Handling call
0709 12:24:19.926892 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.494600 (+567708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.494657 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:20.494702 (+    45us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:20.494731 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.020823 (+526092us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:21.020881 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.021073 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305022208}
W20260709 12:24:21.086182 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.065884 13720 rpcz_store.cc:276] 0709 12:24:19.182795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.456085 (+1273290us) service_pool.cc:224] Handling call
0709 12:24:21.035177 (+579092us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:21.035378 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264167168}
I20260709 12:24:21.090503 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.091612 13720 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:21.092761 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.093777 13720 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:24:21.065368 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:21.068711 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3499) took 1470 ms. Trace:
I20260709 12:24:21.095614 13722 rpcz_store.cc:276] 0709 12:24:19.598194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.026582 (+1428388us) service_pool.cc:224] Handling call
0709 12:24:21.068643 (+ 42061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.068585 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:20.987864 21073 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:24:21.072080 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1123) took 1298 ms. Trace:
I20260709 12:24:21.099232 14421 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:21.099550 14416 rpcz_store.cc:276] 0709 12:24:19.774102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.774308 (+   206us) service_pool.cc:224] Handling call
0709 12:24:19.774890 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.663813 (+888923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.071794 (+407981us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54007125d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:21.072012 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010698496}
I20260709 12:24:21.003535 21027 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:24:21.050160 20975 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:20.985872 21119 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:21.085429 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.104436 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.061733 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:21.088438 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.550s (SENT)
I20260709 12:24:21.109386 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.102548 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1151) took 1159 ms. Trace:
I20260709 12:24:21.114171 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:21.117128 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.616s (SENT)
I20260709 12:24:21.113631 13724 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:21.118371 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.116103 14420 rpcz_store.cc:276] 0709 12:24:19.943607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.944825 (+  1218us) service_pool.cc:224] Handling call
0709 12:24:20.029932 (+ 85107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.036301 (+  6369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.036350 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.798597 (+762247us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b2c000d57a8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:24:20.914074 (+115477us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.086703 (+172629us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:21.086768 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.087003 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.087099 (+    96us) raft_consensus.cc:1685] finished
0709 12:24:21.087120 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.102449 (+ 15329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5932,"raft.queue_time_us":28347,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":7163,"spinlock_wait_cycles":3019520,"thread_start_us":136652,"threads_started":1,"wal-append.queue_time_us":141888}
W20260709 12:24:21.120263 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.679s (SENT)
I20260709 12:24:21.120831 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:21.122275 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.838s (SENT)
I20260709 12:24:21.122778 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:21.100724 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.123905 13720 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 13.
W20260709 12:24:21.128825 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.853s (SENT)
I20260709 12:24:21.130450 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:21.132465 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:21.106580 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67216us
I20260709 12:24:21.135933 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:21.145864 13722 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:21.148464 13617 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.868s (SENT)
I20260709 12:24:21.149020 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.148694 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.150826 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.151840 13719 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 12.
I20260709 12:24:21.098038 20980 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:21.176268 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:21.101869 21014 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.191480 20933 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.194945 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:21.196115 13726 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:21.126121 20688 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.135978 20965 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:21.201444 20688 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:21.202451 21014 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.204208 20842 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.204913 21014 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.207957 13646 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:21.205617 21014 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:21.209654 20953 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.001s
I20260709 12:24:21.210148 20953 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.191660 21005 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:21.211330 21005 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.211851 21005 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:21.211385 13643 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
W20260709 12:24:21.214538 21132 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:24:21.215288 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.216070 13719 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:21.219020 20933 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:21.220611 20996 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:21.222149 20842 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.220695 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:21.207469 21128 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.223484 13794 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:21.223716 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3530) took 1966 ms. Trace:
I20260709 12:24:21.224313 13723 rpcz_store.cc:276] 0709 12:24:19.257503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.640512 (+1383009us) service_pool.cc:224] Handling call
0709 12:24:20.675282 (+ 34770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.675720 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.675765 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.676384 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.765240 (+ 88856us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:20.765281 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.765382 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.223415 (+458033us) raft_consensus.cc:1685] finished
0709 12:24:21.223445 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.223636 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":302,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":11956,"thread_start_us":62658,"threads_started":1,"wal-append.queue_time_us":257382}
I20260709 12:24:21.225781 21128 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.226722 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:21.227522 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79100.6us
I20260709 12:24:21.227687 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:21.227716 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:21.228318 13617 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.885s (SENT)
I20260709 12:24:21.232368 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.137949 21047 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:21.217986 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:21.244980 13724 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:21.224733 13726 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:24:21.237787 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.914s (SENT)
I20260709 12:24:21.251453 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:21.277825 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.127826 21081 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:24:21.281581 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43827.8us
W20260709 12:24:21.285356 13617 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.963s (SENT)
I20260709 12:24:21.285995 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:21.127558 21051 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.242522 20933 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:21.140446 21072 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:21.292541 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.293634 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1154) took 1349 ms. Trace:
I20260709 12:24:21.294068 20842 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:24:21.297302 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.987s (SENT)
I20260709 12:24:21.297945 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:21.296376 13722 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:21.296239 14423 rpcz_store.cc:276] 0709 12:24:19.944495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.945098 (+   603us) service_pool.cc:224] Handling call
0709 12:24:21.293398 (+1348300us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:21.293568 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895522560}
I20260709 12:24:21.301638 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:21.308171 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.302474 13722 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:21.311342 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.312897 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.313596 13722 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:21.299898 20630 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:21.322407 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.322755 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.323769 13721 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:21.324576 13722 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:21.324725 13721 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:21.325217 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3484) took 1808 ms. Trace:
I20260709 12:24:21.325249 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3485) took 1808 ms. Trace:
I20260709 12:24:21.325740 13721 rpcz_store.cc:276] 0709 12:24:19.517002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.324337 (+1807335us) service_pool.cc:224] Handling call
0709 12:24:21.324552 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.325077 (+   525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.325208 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.327129 13721 raft_consensus.cc:1217] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 13.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 12:24:21.327951 13721 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:24:21.328828 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3490) took 1764 ms. Trace:
I20260709 12:24:21.329207 13721 rpcz_store.cc:276] 0709 12:24:19.564571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.326572 (+1762001us) service_pool.cc:224] Handling call
0709 12:24:21.327011 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.328479 (+  1468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.328754 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.281868 21128 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.331560 13721 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:21.332288 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3492) took 1737 ms. Trace:
I20260709 12:24:21.332603 13721 rpcz_store.cc:276] 0709 12:24:19.595213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.330156 (+1734943us) service_pool.cc:224] Handling call
0709 12:24:21.331359 (+  1203us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.332014 (+   655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.332233 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.336282 13794 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:21.339851 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:21.339465 13722 rpcz_store.cc:276] 0709 12:24:19.516720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.324092 (+1807372us) service_pool.cc:224] Handling call
0709 12:24:21.324396 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.324980 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.325165 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.243754 20975 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:21.339643 13617 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.040s (SENT)
I20260709 12:24:21.131403 20862 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.177s
W20260709 12:24:21.351603 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.357905 13724 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:21.361501 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3536) took 1654 ms. Trace:
I20260709 12:24:21.361835 13724 rpcz_store.cc:276] 0709 12:24:19.707552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.360867 (+1653315us) service_pool.cc:224] Handling call
0709 12:24:21.361140 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.361282 (+   142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.361297 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.361332 (+    35us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.361352 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.361385 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.361457 (+    72us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:21.359987 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 211429us
I20260709 12:24:21.363714 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.364259 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3538) took 1643 ms. Trace:
I20260709 12:24:21.342875 21128 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:21.364974 13724 rpcz_store.cc:276] 0709 12:24:19.721612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.363317 (+1641705us) service_pool.cc:224] Handling call
0709 12:24:21.363630 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.363819 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.363845 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.363889 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:21.363970 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.364026 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.364206 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.367830 13724 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:21.368079 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.368501 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3539) took 1631 ms. Trace:
I20260709 12:24:21.369717 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.372581 13722 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:21.477986 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.466142 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3476) took 2021 ms. Trace:
I20260709 12:24:21.467867 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.677364 13720 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:21.677038 13727 rpcz_store.cc:276] 0709 12:24:19.444842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.713950 (+1269108us) service_pool.cc:224] Handling call
0709 12:24:21.061854 (+347904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.098639 (+ 36785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.465883 (+367244us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001f0750. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:21.466065 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776367360}
I20260709 12:24:21.432276 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1588) took 1603 ms. Trace:
I20260709 12:24:21.678912 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:21.410581 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3474) took 2018 ms (client timeout 2040 ms). Trace:
I20260709 12:24:21.680124 21052 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:21.365556 21014 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.425698 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:21.680711 21052 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.681301 21052 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:21.681589 21014 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.546851 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:21.556006 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1589) took 1722 ms. Trace:
I20260709 12:24:21.590764 21119 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.681372 13723 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:21.625331 20326 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.413252 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.687542 13725 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:21.687310 20326 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.480553 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:21.688143 13725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3503) took 2053 ms (client timeout 2167 ms). Trace:
I20260709 12:24:21.371377 21128 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:21.689281 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 208883us
I20260709 12:24:21.689821 20326 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:21.689553 21128 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.374778 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3535) took 1681 ms. Trace:
I20260709 12:24:21.501865 20688 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:21.483461 20953 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.413702 13721 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:21.505672 21142 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.474179 20977 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:21.692112 21142 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.690729 13722 rpcz_store.cc:276] 0709 12:24:19.693234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.356709 (+1663475us) service_pool.cc:224] Handling call
0709 12:24:21.372373 (+ 15664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.374426 (+  2053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.374705 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.694006 13719 rpcz_store.cc:276] 0709 12:24:19.833443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.554955 (+1721512us) service_pool.cc:224] Handling call
0709 12:24:21.555360 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.555535 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.555565 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.555603 (+    38us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:21.555639 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.555688 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.555901 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.568346 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:21.589208 21038 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:24:21.681093 13726 rpcz_store.cc:276] 0709 12:24:19.829207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.431247 (+1602040us) service_pool.cc:224] Handling call
0709 12:24:21.431720 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.431903 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.431933 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.431972 (+    39us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.432027 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.432075 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.432214 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.400609 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:21.370101 13724 rpcz_store.cc:276] 0709 12:24:19.737630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.367342 (+1629712us) service_pool.cc:224] Handling call
0709 12:24:21.367647 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.368279 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.368455 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.699982 13651 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:21.701185 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1595) took 1739 ms. Trace:
I20260709 12:24:21.699486 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1593) took 1806 ms. Trace:
I20260709 12:24:21.562217 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:21.702646 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1596) took 1721 ms. Trace:
I20260709 12:24:21.459374 20428 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:21.427102 21143 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:21.420106 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:21.705152 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:21.706548 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:21.688550 21147 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:21.683015 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 90545.7us
I20260709 12:24:21.704921 13726 rpcz_store.cc:276] 0709 12:24:19.981273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.701751 (+1720478us) service_pool.cc:224] Handling call
0709 12:24:21.702100 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.702313 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.702338 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.702383 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:21.702418 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.702487 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.702590 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.707486 21147 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.708210 21147 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:21.711354 21052 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.690115 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.369154 20980 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.715319 13723 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:21.716121 20694 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:21.701594 13724 rpcz_store.cc:276] 0709 12:24:19.962661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.700337 (+1737676us) service_pool.cc:224] Handling call
0709 12:24:21.700662 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.700827 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.700856 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.700908 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:21.700958 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.701008 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.701121 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.481750 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:21.399350 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1219) took 1708 ms (client timeout 2177 ms). Trace:
I20260709 12:24:21.692412 20977 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:21.693084 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3494) took 2097 ms. Trace:
I20260709 12:24:21.487877 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:21.680392 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1591) took 1795 ms. Trace:
I20260709 12:24:21.732174 13727 rpcz_store.cc:276] 0709 12:24:19.885266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.679608 (+1794342us) service_pool.cc:224] Handling call
0709 12:24:21.679887 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.680054 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.680076 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.680138 (+    62us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:21.680177 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.680233 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.680337 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:21.680410 13718 rpcz_store.cc:269] 0709 12:24:19.392217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.718753 (+326536us) service_pool.cc:224] Handling call
0709 12:24:21.410287 (+1691534us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:21.410506 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917772160}
I20260709 12:24:21.462699 20997 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:21.537705 21135 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:21.695343 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:21.703048 14420 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:21.549669 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4008) took 1005 ms. Trace:
W20260709 12:24:21.397367 21114 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.809s	user 0.000s	sys 0.003s
I20260709 12:24:21.702390 13722 rpcz_store.cc:276] 0709 12:24:19.897453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.697456 (+1800003us) service_pool.cc:224] Handling call
0709 12:24:21.698061 (+   605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.698879 (+   818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.698925 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.699016 (+    91us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.699222 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.699296 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.699424 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.554581 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:21.766530 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 212034us
I20260709 12:24:21.586671 21084 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:21.491680 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.905s (SENT)
I20260709 12:24:21.769598 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.706380 21143 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.407428 21144 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:21.527155 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:21.516723 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:21.778496 13646 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:24:21.696765 13719 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:21.781598 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1592) took 1888 ms. Trace:
I20260709 12:24:21.488538 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:21.733888 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.688549 13725 rpcz_store.cc:269] 0709 12:24:19.634722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.412870 (+1778148us) service_pool.cc:224] Handling call
0709 12:24:21.688080 (+275210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.782205 13794 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:24:21.719623 13797 rpcz_store.cc:269] 0709 12:24:19.691332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.743340 (+ 52008us) service_pool.cc:224] Handling call
0709 12:24:21.399043 (+1655703us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540012b454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:21.399262 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2059311488}
I20260709 12:24:21.720636 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1597) took 1708 ms. Trace:
I20260709 12:24:21.787425 13723 rpcz_store.cc:276] 0709 12:24:20.011910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.719775 (+1707865us) service_pool.cc:224] Handling call
0709 12:24:21.720135 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.720321 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.720346 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.720389 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.720433 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.720483 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.720579 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.789670 13723 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:24:21.790830 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3546) took 1204 ms. Trace:
I20260709 12:24:21.791159 13723 rpcz_store.cc:276] 0709 12:24:20.585538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.788820 (+1203282us) service_pool.cc:224] Handling call
0709 12:24:21.789269 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.790029 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.790208 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.796018 13723 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:21.796554 13651 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:21.720729 13724 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:21.718359 13650 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:21.731637 13721 rpcz_store.cc:276] 0709 12:24:19.596248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.333303 (+1737055us) service_pool.cc:224] Handling call
0709 12:24:21.413472 (+ 80169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.691661 (+278189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.692872 (+  1211us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b54003fd9d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:21.693009 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183531392}
I20260709 12:24:21.731573 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:21.744971 13718 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
W20260709 12:24:21.745589 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 375897us
I20260709 12:24:21.801374 13718 pending_rounds.cc:85] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 9
I20260709 12:24:21.802354 13718 pending_rounds.cc:107] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 12.10
W20260709 12:24:21.726672 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 114938us
I20260709 12:24:21.802731 13718 raft_consensus.cc:2889] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.679286 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1590) took 1817 ms. Trace:
W20260709 12:24:21.752295 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 326724us
I20260709 12:24:21.803714 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3541) took 1298 ms. Trace:
I20260709 12:24:21.803714 13720 rpcz_store.cc:276] 0709 12:24:19.862094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.678354 (+1816260us) service_pool.cc:224] Handling call
0709 12:24:21.678758 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.678963 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.678990 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.679037 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:21.679069 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.679127 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.679231 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.805203 13719 rpcz_store.cc:276] 0709 12:24:19.892747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.696024 (+1803277us) service_pool.cc:224] Handling call
0709 12:24:21.696447 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.781311 (+ 84864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.781541 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:21.812948 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81489us
I20260709 12:24:21.816260 13794 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:21.780773 13646 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
W20260709 12:24:21.817070 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 325422us
I20260709 12:24:21.695214 21142 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.750835 13643 rpcz_store.cc:276] 0709 12:24:20.544565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.544795 (+   230us) service_pool.cc:224] Handling call
0709 12:24:20.568487 (+ 23692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.212605 (+644118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.549120 (+336515us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b54001e5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:21.549560 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477960576}
I20260709 12:24:21.763935 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:21.835821 13726 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:21.770519 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62739us
I20260709 12:24:21.767328 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3544) took 1231 ms. Trace:
I20260709 12:24:21.565532 20941 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:21.429211 20965 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:21.779247 13644 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:21.467368 21005 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:21.506932 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.717967 21052 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:21.820777 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:21.850996 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:21.761546 13649 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:21.826514 13725 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:21.862987 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3545) took 1278 ms. Trace:
I20260709 12:24:21.863742 13725 rpcz_store.cc:276] 0709 12:24:20.584615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.783737 (+1199122us) service_pool.cc:224] Handling call
0709 12:24:21.807040 (+ 23303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.862658 (+ 55618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.862926 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.827626 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3543) took 1296 ms. Trace:
I20260709 12:24:21.865191 13718 rpcz_store.cc:276] 0709 12:24:20.531153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.734729 (+1203576us) service_pool.cc:224] Handling call
0709 12:24:21.735139 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.827247 (+ 92108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.827539 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:21.827785 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.176s (SENT)
I20260709 12:24:21.866935 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.802420 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.837059 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.837116 13722 rpcz_store.cc:276] 0709 12:24:20.536323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.766179 (+1229856us) service_pool.cc:224] Handling call
0709 12:24:21.766664 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.766897 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.766929 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:21.766983 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:21.767027 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.767089 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.767246 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.839172 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:21.859514 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.744549 20997 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:21.878530 13648 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:21.804142 13724 rpcz_store.cc:276] 0709 12:24:20.505472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.720280 (+1214808us) service_pool.cc:224] Handling call
0709 12:24:21.720525 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.803448 (+ 82923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.803654 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:21.855439 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:21.883412 13797 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:21.766979 21119 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:21.834347 21047 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:21.857679 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:21.897812 13794 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:24:21.858497 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.858704 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:21.775380 20630 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:21.860893 14417 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:21.771245 21135 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:21.884495 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81040.9us
W20260709 12:24:21.887887 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.836071 21140 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:21.893010 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 91014.1us
I20260709 12:24:21.907610 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 3995) took 2192 ms. Trace:
W20260709 12:24:21.910277 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82534us
W20260709 12:24:21.922930 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.183s (SENT)
I20260709 12:24:21.923776 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:21.925518 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.857338 14420 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:24:21.906618 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.868768 14424 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:24:21.907109 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65715.5us
I20260709 12:24:21.820153 21084 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:21.930382 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:21.928004 20941 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:21.875811 20428 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:21.924012 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:21.938515 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.943985 13725 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:24:21.939457 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:21.921890 13647 rpcz_store.cc:276] 0709 12:24:19.715433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.935877 (+220444us) service_pool.cc:224] Handling call
0709 12:24:19.936207 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.345487 (+409280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.345533 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.346529 (+   996us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.472908 (+126379us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:20.472993 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.028213 (+555220us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:21.028270 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.396670 (+368400us) spinlock_profiling.cc:243] Waited 9.26 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:21.907383 (+510713us) raft_consensus.cc:1685] finished
0709 12:24:21.907411 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.907553 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":558,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":94,"spinlock_wait_cycles":961592960,"thread_start_us":125853,"threads_started":1,"wal-append.queue_time_us":187011}
I20260709 12:24:21.944655 13722 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:21.948324 13718 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:21.896368 21119 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:21.907081 20965 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:21.948482 13722 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:21.907409 20980 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.896762 20326 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:21.910235 21089 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:21.965855 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 151589us
I20260709 12:24:21.965583 13726 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:21.955664 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:21.960855 13722 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:24:21.944842 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:21.979745 13797 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:24:21.982015 13650 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:21.908159 21052 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 12:24:21.947135 20842 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:21.943197 21142 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:21.991324 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:21.960956 21047 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:21.869148 20977 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:21.947033 21144 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.002987 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:22.013624 13794 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:21.903558 21151 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.090s
I20260709 12:24:22.007184 13797 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:22.018788 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "255f10165e844f2cb50d654996d639cf"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:24:21.959241 21119 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:22.007694 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:21.978986 20326 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:22.024305 13801 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
W20260709 12:24:22.024366 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68813.9us
W20260709 12:24:22.012336 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80968.5us
I20260709 12:24:22.019685 13798 raft_consensus.cc:606] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:22.025230 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:21.979601 20941 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:22.034900 13794 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:24:22.006448 21142 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.035876 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:22.014156 21014 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:22.028558 20326 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:22.034704 20941 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.041424 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:22.028843 21047 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:22.003692 21089 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.040850 21144 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:22.089991 21089 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:22.046818 21073 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:22.043264 21142 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:22.086234 21152 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:24:22.086511 21047 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:22.087209 21159 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:24:22.087855 20941 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:22.087944 21153 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:21.958186 20997 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.038066 20687 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:22.037598 21036 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:22.178815 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 153327us
I20260709 12:24:22.107601 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:22.046870 20975 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:22.115778 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:22.129086 21156 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:24:22.086386 21084 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:22.134418 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.162032 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.172725 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 248823us
I20260709 12:24:22.097421 21144 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:22.051954 20965 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:22.158164 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 213378us
W20260709 12:24:22.197983 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.198105 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 82421.8us
W20260709 12:24:22.217135 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 134017us
I20260709 12:24:22.206097 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:22.207343 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3542) took 1684 ms. Trace:
I20260709 12:24:22.164062 21073 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:22.219031 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:22.230438 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:22.231302 13645 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:22.219305 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:22.219520 13727 rpcz_store.cc:276] 0709 12:24:20.522712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.733668 (+1210956us) service_pool.cc:224] Handling call
0709 12:24:21.734169 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.737293 (+  3124us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:21.958922 (+221629us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.206767 (+247845us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.206867 (+   100us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:22.206967 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.207073 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.207255 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":296457,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2076,"raft.queue_time_us":24220,"thread_start_us":447097,"threads_started":2}
I20260709 12:24:22.173338 21143 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:22.263633 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44687.6us
I20260709 12:24:22.263739 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:22.264437 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:22.162156 21005 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:22.264539 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:22.198608 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:22.186815 20975 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:22.268373 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57286.8us
I20260709 12:24:22.277015 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:22.271015 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:22.278469 13649 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:22.280320 13648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:22.281157 13648 raft_consensus.cc:606] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:22.184036 21140 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:22.273985 13647 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:24:22.281630 13648 raft_consensus.cc:614] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:22.195227 21036 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:22.266686 20630 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:22.289467 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:22.290309 13794 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:24:22.216698 20941 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:22.282836 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:22.267997 21132 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:24:22.300354 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 282287us
I20260709 12:24:22.280370 20975 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.305521 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4019) took 1170 ms. Trace:
W20260709 12:24:22.299048 21147 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.006s
I20260709 12:24:22.228782 21084 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:24:22.300226 21014 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:22.327008 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.301785 21143 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:22.309201 21135 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:22.339913 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57570.5us
I20260709 12:24:22.327905 21037 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:22.311769 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 344032us
I20260709 12:24:22.326079 21170 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.021s
I20260709 12:24:22.363389 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:22.311206 13652 rpcz_store.cc:276] 0709 12:24:21.135364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.535175 (+399811us) service_pool.cc:224] Handling call
0709 12:24:21.535528 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.535883 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.535918 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.536374 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.763146 (+226772us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:21.763212 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.763328 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.305305 (+541977us) raft_consensus.cc:1685] finished
0709 12:24:22.305327 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.305459 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1093,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":35584,"thread_start_us":226326,"threads_started":1,"wal-append.queue_time_us":360084}
I20260709 12:24:22.314554 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:22.229434 20965 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:22.318215 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:22.316052 13645 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:22.373636 13643 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.323885 13649 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:24:22.307736 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:22.298981 20941 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.336519 20714 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:22.310662 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:22.319639 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.386s (SENT)
I20260709 12:24:22.326951 21147 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:22.377699 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 158343us
W20260709 12:24:22.357237 21157 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:24:22.331578 21181 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:24:22.330861 21169 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:24:22.379407 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3554) took 1268 ms. Trace:
I20260709 12:24:22.347711 21037 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:22.392766 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:22.383848 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:22.345933 20862 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:24:22.394302 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83757.3us
W20260709 12:24:22.394919 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.398665 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:22.321764 21144 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.401036 13723 rpcz_store.cc:276] 0709 12:24:21.111508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.797659 (+686151us) service_pool.cc:224] Handling call
0709 12:24:21.798026 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.804987 (+  6961us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:22.032695 (+227708us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.378967 (+346272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.379071 (+   104us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:22.379126 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.379206 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.379343 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18150,"prepare.queue_time_us":290401,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":3680,"thread_start_us":517586,"threads_started":2}
W20260709 12:24:22.394096 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86419us
I20260709 12:24:22.382364 20941 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.349025 21120 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:24:22.331751 21073 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:22.480195 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121640us
I20260709 12:24:22.452246 13651 raft_consensus.cc:606] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:22.481721 13651 raft_consensus.cc:614] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:22.481797 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:22.408504 21161 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.230s	user 0.000s	sys 0.002s
I20260709 12:24:22.344254 21140 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.347539 21036 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:22.396495 21014 raft_consensus.cc:2764] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
W20260709 12:24:22.461673 21167 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:24:22.462348 21144 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:22.400830 20694 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:22.477866 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 92993.4us
W20260709 12:24:22.479368 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 159803us
I20260709 12:24:22.410076 21037 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:22.442201 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:22.459142 13693 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
W20260709 12:24:22.508720 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.457546 21143 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:22.378747 21177 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.123s
I20260709 12:24:22.522956 13693 leader_election.cc:403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:22.572449 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.524789 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82715.5us
I20260709 12:24:22.519007 20694 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.597828 20694 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:22.598554 21171 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 12:24:22.600286 21187 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:24:22.601745 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3557) took 1372 ms. Trace:
I20260709 12:24:22.521170 21120 consensus_queue.cc:1243] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:24:22.602828 21168 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.603440 21168 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.605011 21120 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:22.597198 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 138195us
I20260709 12:24:22.491521 21052 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:24:22.609259 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:22.615885 13643 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
W20260709 12:24:22.630915 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.641708 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51375.9us
I20260709 12:24:22.612211 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:22.572191 21170 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:22.602160 13721 rpcz_store.cc:276] 0709 12:24:21.229762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.800340 (+570578us) service_pool.cc:224] Handling call
0709 12:24:21.800616 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.832632 (+ 32016us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.832680 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.833480 (+   800us) log.cc:844] Serialized 26 byte log entry
0709 12:24:22.300052 (+466572us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:22.300097 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.300177 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.601350 (+301173us) raft_consensus.cc:1685] finished
0709 12:24:22.601379 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.601645 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19858,"raft.queue_time_us":31620,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":15896,"thread_start_us":407651,"threads_started":1,"wal-append.queue_time_us":492715}
I20260709 12:24:22.706101 21151 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:22.706389 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 186909us
I20260709 12:24:22.706671 21151 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.707073 21151 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.708719 21147 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:22.709568 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.680s (SENT)
W20260709 12:24:22.711295 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 189902us
W20260709 12:24:22.607455 21167 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:24:22.714352 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 313704us
W20260709 12:24:22.714957 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.558786 21135 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.147s
I20260709 12:24:22.650038 21014 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17.
I20260709 12:24:22.710026 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:22.728214 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 246487us
I20260709 12:24:22.712842 21170 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.734289 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:22.714736 21174 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:22.735546 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.978s (SENT)
W20260709 12:24:22.725342 21180 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 12:24:22.734058 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 269903us
I20260709 12:24:22.724668 21147 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:24:22.755241 20965 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:22.760489 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.731710 21175 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:24:22.738687 21170 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:22.807101 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.731798 20842 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:22.744486 21174 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:22.815783 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 87821.9us
W20260709 12:24:22.828029 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 96752us
I20260709 12:24:22.813925 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:22.781692 21014 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:22.834509 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:22.832235 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109277us
I20260709 12:24:22.823227 21037 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.836509 21168 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:22.837249 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:22.837874 13723 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:24:22.838402 21037 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:22.839787 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:22.841254 13727 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:22.846150 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:22.854951 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.830195 21174 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.843215 21183 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:24:22.862385 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 219782us
I20260709 12:24:22.866744 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:22.867430 13643 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:22.851164 20326 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.810s	user 0.018s	sys 0.000s
I20260709 12:24:22.873042 21183 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:22.882635 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:22.898283 13723 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:22.898934 13643 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:24:22.882537 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.748s (SENT)
I20260709 12:24:22.905328 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:22.906775 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:22.909075 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 175646us
I20260709 12:24:22.870872 21198 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:24:22.901154 21207 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:24:22.872792 20326 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:22.918238 14419 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:24:22.913018 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:22.924595 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:22.927791 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:22.928715 13643 raft_consensus.cc:2368] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:24:22.931007 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:22.931672 13797 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:22.948740 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:22.958735 14417 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:22.920089 21128 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:22.962898 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51522.5us
I20260709 12:24:22.903352 21206 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:24:22.944510 21014 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:22.971334 20326 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:22.925756 21205 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:22.971935 13546 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 153) took 90464 ms (client timeout 15000 ms). Trace:
I20260709 12:24:22.972223 21205 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.972697 21205 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:22.973589 13547 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:22.974721 21128 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:22.976447 13547 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 12, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 12 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:22.957957 21168 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:22.978199 21168 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:22.966933 21204 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:24:22.981858 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:22.994882 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.513s (SENT)
I20260709 12:24:22.957866 21211 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:22.998190 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:22.986354 21200 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 12:24:23.007341 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83489.6us
I20260709 12:24:23.006337 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
W20260709 12:24:23.010840 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.004333 13797 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
W20260709 12:24:22.991639 21201 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:24:23.014348 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:23.031303 13643 raft_consensus.cc:606] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:23.032552 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:23.040282 13643 raft_consensus.cc:614] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:23.040498 13646 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:22.986665 21128 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:22.986053 21168 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:22.980535 21209 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:24:23.126904 13651 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:23.131547 21144 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.001s
W20260709 12:24:23.087188 21213 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:23.088264 21178 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:23.088166 21182 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:24:23.093432 14419 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:23.003052 21005 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.077s
W20260709 12:24:23.136255 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.148332 13617 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.662s (SENT)
I20260709 12:24:23.149029 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:23.123903 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111320us
W20260709 12:24:22.972538 13546 rpcz_store.cc:269] 0709 12:22:52.554512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:52.558407 (+  3895us) service_pool.cc:224] Handling call
0709 12:24:22.971809 (+90413402us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:19.126871 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:19.178454 (+ 51583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:19.178521 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:24:19.178555 (+    34us) tablet.cc:662] Decoding operations
0709 12:24:19.179123 (+   568us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:19.179229 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:19.179258 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:19.179611 (+   353us) tablet.cc:701] Row locks acquired
0709 12:24:19.179662 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:24:19.179878 (+   216us) write_op.cc:270] Start()
0709 12:24:19.180041 (+   163us) write_op.cc:276] Timestamp: P: 1783599859179797 usec, L: 0
0709 12:24:19.180093 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 12:24:19.228549 (+ 48456us) log.cc:844] Serialized 842 byte log entry
0709 12:24:19.795746 (+567197us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:20.628172 (+832426us) op_driver.cc:464] REPLICATION: finished
0709 12:24:21.004280 (+376108us) write_op.cc:301] APPLY: starting
0709 12:24:21.004702 (+   422us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:21.006664 (+  1962us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:21.006707 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:21.007523 (+   816us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:21.079958 (+ 72435us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:24:21.080038 (+    80us) write_op.cc:312] APPLY: finished
0709 12:24:21.080393 (+   355us) log.cc:844] Serialized 25 byte log entry
0709 12:24:21.080633 (+   240us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:21.080746 (+   113us) write_op.cc:454] Released schema lock
0709 12:24:21.080935 (+   189us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:21.125758 (+ 44823us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:21.129486 (+  3728us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1645917,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":352810,"apply.run_cpu_time_us":5333,"apply.run_wall_time_us":77284,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8315,"num_ops":1,"prepare.queue_time_us":522032,"prepare.run_cpu_time_us":16901,"prepare.run_wall_time_us":285322,"raft.queue_time_us":294324,"raft.run_cpu_time_us":10796,"raft.run_wall_time_us":122047,"replication_time_us":1434555,"spinlock_wait_cycles":32912128,"thread_start_us":728482,"threads_started":4,"wal-append.queue_time_us":177729,"wal-append.run_cpu_time_us":58239,"wal-append.run_wall_time_us":2690524}]]}
W20260709 12:24:23.031553 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.014020 20326 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:23.156750 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.081102 13547 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 11 to 14, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 14 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:23.052589 20997 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:23.166591 13792 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:23.076084 21069 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.017874 21215 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:23.059525 21047 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:23.037774 21211 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:23.056036 21196 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:23.047000 20941 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.004s
I20260709 12:24:23.168208 20965 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:23.276733 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.277441 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.183712 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:22.983829 21183 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:23.197299 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.037853 20428 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:23.151162 21207 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.076345 21168 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:23.074398 21014 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:23.213443 13547 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 4 to 10, term changed from 10 to 19, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to <none>, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65), VOTER b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66), VOTER a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67), VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 19 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:23.134240 21178 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:23.215411 21151 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 12:24:23.166987 21169 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:23.048147 20694 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:23.301708 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.250080 20842 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:23.276038 21119 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.238s	user 0.016s	sys 0.000s
I20260709 12:24:23.214658 20326 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:23.183899 21037 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.123s
I20260709 12:24:23.316227 13547 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:23.293711 20428 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:23.282840 20941 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.162110 20630 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.144s
I20260709 12:24:23.312008 13794 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:24:23.318701 21211 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:23.314142 21120 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.709s	user 0.014s	sys 0.005s
I20260709 12:24:23.293833 20965 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.364100 13648 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:23.365581 20941 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:23.368708 20630 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:23.183657 21069 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:23.377689 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.279044 21047 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.284585 21196 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.386086 14418 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22.
I20260709 12:24:23.309640 21168 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.307032 21207 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:23.430591 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.404435 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5922404269999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:24:23.431416 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:23.452121 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 145053us
W20260709 12:24:23.452453 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5832404269999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 12:24:23.351871 21151 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:23.453994 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5139404269998522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 12:24:23.461597 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:23.388458 20975 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.061s	user 0.014s	sys 0.003s
W20260709 12:24:23.463393 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5040404269998362s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 12:24:23.361590 21205 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.464205 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:23.466498 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:23.467377 13723 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
W20260709 12:24:23.468341 13693 connection.cc:256] RPC call timeout handler was delayed by -2.3051404269999693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
W20260709 12:24:23.468786 13693 connection.cc:256] RPC call timeout handler was delayed by -2.23134042699985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 12:24:23.469244 13693 connection.cc:256] RPC call timeout handler was delayed by -2.2079404269998122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260709 12:24:23.469597 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1989404269997976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 12:24:23.469940 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1899404269997831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 12:24:23.445591 21175 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:24:23.470288 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1611404269997365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
I20260709 12:24:23.481570 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.359623 21119 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.489389 13792 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:24:23.490713 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1431404269997074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 12:24:23.491447 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0549404270000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
W20260709 12:24:23.491968 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9523404269998537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
I20260709 12:24:23.387342 21220 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.359200 20842 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:23.500976 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9442404269998406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
W20260709 12:24:23.501749 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2188404270000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3053s for a response
I20260709 12:24:23.504665 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:23.503356 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0784404269998049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3209s for a response
I20260709 12:24:23.506407 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:23.389127 21120 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:23.515312 13800 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:23.528352 13646 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
W20260709 12:24:23.540489 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.541764 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.542469 13617 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.730s (SENT)
I20260709 12:24:23.542850 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:23.543399 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.544190 13796 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:23.584379 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1244) took 1270 ms. Trace:
I20260709 12:24:23.585063 13648 rpcz_store.cc:276] 0709 12:24:22.313939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.314193 (+   254us) service_pool.cc:224] Handling call
0709 12:24:23.584064 (+1269871us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:23.584243 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294930816}
I20260709 12:24:23.587507 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3560) took 2342 ms. Trace:
I20260709 12:24:23.588043 13720 rpcz_store.cc:276] 0709 12:24:21.245312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.807769 (+562457us) service_pool.cc:224] Handling call
0709 12:24:21.837120 (+ 29351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.837353 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:21.993677 (+156324us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.586876 (+1593199us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18003dc720. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:24:23.587036 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:23.587112 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:23.587180 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.587276 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.587429 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23542,"prepare.queue_time_us":324458,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":20578,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6007,"spinlock_wait_cycles":27688320,"thread_start_us":319326,"threads_started":2}
I20260709 12:24:23.591018 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1162) took 1818 ms. Trace:
I20260709 12:24:23.591521 14423 rpcz_store.cc:276] 0709 12:24:21.772682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.772892 (+   210us) service_pool.cc:224] Handling call
0709 12:24:21.841078 (+ 68186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.956042 (+114964us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:22.347531 (+391489us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.580496 (+232965us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.580710 (+   214us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:22.580825 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.590711 (+1009886us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540028ffd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:23.590778 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.590947 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13105,"prepare.queue_time_us":764787,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":38558,"raft.run_cpu_time_us":484,"raft.run_wall_time_us":64030,"spinlock_wait_cycles":263822720,"thread_start_us":557875,"threads_started":2,"wal-append.queue_time_us":405073}
I20260709 12:24:23.597345 20688 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:23.597931 20688 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:23.598995 20688 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.600368 20688 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:23.609514 21225 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.227s	user 0.001s	sys 0.002s
I20260709 12:24:23.618495 13764 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22.
I20260709 12:24:23.618911 13764 leader_election.cc:403] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:23.619724 21190 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:23.620051 21144 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:23.620325 21209 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:23.620754 21144 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.621510 21084 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:23.621894 21084 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:23.622777 21084 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.630378 21190 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22.
I20260709 12:24:23.633838 20977 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:23.635099 21144 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:23.637763 13643 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:23.645356 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:23.664814 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:23.666074 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:23.666736 13794 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:23.652792 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:23.646142 13800 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:24:23.648074 21084 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:23.653048 13547 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 10 to 11, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 11 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.666908 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:23.654770 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.654597 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:23.490792 20975 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.499310 21220 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.669716 13796 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:23.670288 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.537473 21215 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:23.654989 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:23.678030 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:23.682363 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.690034 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44783.2us
I20260709 12:24:23.691540 13795 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:23.452757 21014 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.461988 20428 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.663859 21144 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:23.495188 20326 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:23.719627 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.664570 21169 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:23.682435 21179 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:23.741969 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64062.1us
I20260709 12:24:23.681829 21047 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:23.657727 20941 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.060s
I20260709 12:24:23.741753 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3559) took 2497 ms. Trace:
I20260709 12:24:23.689640 21128 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:23.879215 13794 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
W20260709 12:24:23.865613 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.912817 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 165305us
W20260709 12:24:23.854408 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 157951us
I20260709 12:24:23.872144 21217 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:23.872355 21230 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:23.744957 21073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.029s	sys 0.032s
I20260709 12:24:23.708630 21066 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:24:23.873330 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.747249 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:23.830242 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.743314 21183 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:23.884394 20630 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:23.885994 20965 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:23.882799 21047 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:23.905797 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.908533 13719 rpcz_store.cc:276] 0709 12:24:21.244803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.806388 (+561585us) service_pool.cc:224] Handling call
0709 12:24:21.869481 (+ 63093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.906770 (+ 37289us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:21.980960 (+ 74190us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.910125 (+929165us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b18000a74a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:24:23.038877 (+128752us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.350233 (+311356us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.350882 (+   649us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.351086 (+   204us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:23.351113 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.351166 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:23.741489 (+390323us) raft_consensus.cc:1685] finished
0709 12:24:23.741520 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.741679 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22181,"prepare.queue_time_us":476424,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":33826,"raft.queue_time_us":119563,"raft.run_cpu_time_us":448,"raft.run_wall_time_us":14072,"spinlock_wait_cycles":17417984,"thread_start_us":741009,"threads_started":4,"wal-append.queue_time_us":403598}
W20260709 12:24:23.928707 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 422493us
I20260709 12:24:23.952896 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:23.754804 21144 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.857468 21169 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.925207 21217 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:23.950505 21120 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.987529 21120 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:23.988462 21238 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:23.953276 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1240) took 1024 ms. Trace:
W20260709 12:24:23.907156 21142 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.750s	user 0.007s	sys 0.011s
I20260709 12:24:23.938966 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:23.926384 20694 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:23.982435 13643 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:24:23.923995 21215 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.020300 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:23.981540 21183 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:23.985816 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.022177 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.023401 13643 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:24.023226 21183 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:23.904811 21239 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:23.992040 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.024564 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.025650 13643 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:23.942485 20714 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:24.026778 20714 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:24.027711 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.027783 20714 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.032068 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:23.909026 13547 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:24:24.042484 21226 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.501s	user 0.001s	sys 0.000s
I20260709 12:24:24.042683 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:24.046437 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 93625.5us
I20260709 12:24:24.047724 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:24.048326 21170 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.208s	user 0.008s	sys 0.006s
W20260709 12:24:24.048812 21174 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.177s	user 0.008s	sys 0.005s
I20260709 12:24:24.048859 21170 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.049464 21174 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.939797 21168 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:23.948031 20630 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:23.939882 21073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.030s	sys 0.033s
I20260709 12:24:23.910244 21179 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:24.129688 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.925832 21128 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:24.054770 21229 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.369s	user 0.000s	sys 0.001s
W20260709 12:24:24.029455 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 486986us
I20260709 12:24:23.957813 21119 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:23.957270 21069 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:24.070825 21196 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:23.908123 21178 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:24.060976 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4033) took 1054 ms. Trace:
I20260709 12:24:23.994508 21217 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:24.076489 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 409671us
I20260709 12:24:24.066865 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.220196 13725 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 12.
I20260709 12:24:24.220775 20694 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:24.221702 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.222172 20694 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:24.222683 20694 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:24.223300 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:24.224037 13725 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:24.050130 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:24.061280 14423 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:24.064724 13547 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:24.009413 21208 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:24.060160 21236 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:23.937104 21066 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.942659 21230 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:23.976289 21047 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.009570 21207 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:24.139317 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.745s (SENT)
I20260709 12:24:24.058835 21245 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.013643 13798 rpcz_store.cc:276] 0709 12:24:22.929206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.930972 (+  1766us) service_pool.cc:224] Handling call
0709 12:24:22.931359 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.931641 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:22.960199 (+ 28558us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.631561 (+671362us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b180046d680. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:23.951830 (+320269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:23.951946 (+   116us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:23.952102 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.952363 (+   261us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.953020 (+   657us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6054,"prepare.queue_time_us":25921,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":12340,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":12631,"spinlock_wait_cycles":3520768,"thread_start_us":312385,"threads_started":1}
I20260709 12:24:23.976104 21231 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.153s
I20260709 12:24:24.093430 21209 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:24.036773 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:24.055833 21144 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.059024 21220 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.100526 21240 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.148041 13649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.013s	sys 0.005s
I20260709 12:24:24.229470 21249 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.052147 21169 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.030900 21142 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.135023 21151 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:24.215370 13644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.002s	sys 0.017s
W20260709 12:24:24.264153 13644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.002s	sys 0.018s
I20260709 12:24:24.092701 21190 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:24:24.093214 20842 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.223623 20694 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.005524 20965 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.227324 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.290439 14420 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:24.149225 21168 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.229221 13645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.012s	sys 0.015s
W20260709 12:24:24.306223 13645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.012s	sys 0.016s
I20260709 12:24:24.306222 13650 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:24.228839 13651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.002s	sys 0.025s
W20260709 12:24:24.307144 13651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.003s	sys 0.026s
I20260709 12:24:24.307912 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4038) took 1155 ms. Trace:
I20260709 12:24:24.308291 13651 rpcz_store.cc:276] 0709 12:24:23.152627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.152789 (+   162us) service_pool.cc:224] Handling call
0709 12:24:23.153571 (+   782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.153797 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:23.525753 (+371956us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.307578 (+781825us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.307635 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:24.307679 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.307744 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.307854 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":335030,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":1030389,"threads_started":2}
I20260709 12:24:24.238831 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:24.236485 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.004s	sys 0.016s
W20260709 12:24:24.311723 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.004s	sys 0.017s
W20260709 12:24:24.276288 21241 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:24:24.243152 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:24.214709 20975 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:24.253634 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 217065us
W20260709 12:24:24.257169 13649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.013s	sys 0.005s
I20260709 12:24:24.321966 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4039) took 1167 ms. Trace:
I20260709 12:24:24.109965 20630 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.237615 21066 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:24.255856 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 318803us
I20260709 12:24:24.175858 21244 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
W20260709 12:24:24.256915 20997 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.068s	user 0.013s	sys 0.002s
I20260709 12:24:24.310990 20965 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.253221 21209 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.326109 21169 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:24.257397 21207 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:24.326363 21246 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:24.337379 13644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:24.338171 13644 raft_consensus.cc:606] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:24.338554 13644 raft_consensus.cc:614] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:24.172478 13652 rpcz_store.cc:276] 0709 12:24:23.006710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.016048 (+  9338us) service_pool.cc:224] Handling call
0709 12:24:23.037672 (+ 21624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.038751 (+  1079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.038782 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.058847 (+ 20065us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.307913 (+249066us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:23.307962 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.308053 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.060690 (+752637us) raft_consensus.cc:1685] finished
0709 12:24:24.060715 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.060881 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12499,"raft.queue_time_us":676,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":33350,"thread_start_us":248392,"threads_started":1,"wal-append.queue_time_us":405683}
I20260709 12:24:24.339514 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4041) took 1147 ms. Trace:
I20260709 12:24:24.335803 13793 raft_consensus.cc:2410] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:24:24.264637 21254 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.314747 14419 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:24.265321 21232 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:24.274677 21220 raft_consensus.cc:2764] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
W20260709 12:24:24.322299 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 182989us
I20260709 12:24:24.322387 13649 rpcz_store.cc:276] 0709 12:24:23.154404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.154580 (+   176us) service_pool.cc:224] Handling call
0709 12:24:23.197674 (+ 43094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.198001 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:23.489024 (+291023us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.321589 (+832565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.321650 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:24.321698 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.321781 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.321904 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163365,"prepare.queue_time_us":341012,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":992412,"threads_started":2}
I20260709 12:24:24.267078 21249 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.253576 21245 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:24.240597 13647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.699s	user 0.006s	sys 0.013s
I20260709 12:24:24.339973 13645 rpcz_store.cc:276] 0709 12:24:23.192272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.463838 (+271566us) service_pool.cc:224] Handling call
0709 12:24:23.464678 (+   840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.465039 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.465066 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.488694 (+ 23628us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.306745 (+818051us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:24.306790 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.306873 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.339264 (+ 32391us) raft_consensus.cc:1685] finished
0709 12:24:24.339292 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.339450 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20662,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4943,"thread_start_us":768358,"threads_started":1,"wal-append.queue_time_us":811364}
I20260709 12:24:24.311923 21170 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.333040 21174 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:24.334307 21242 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:24:24.383402 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.382094 13647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.007s	sys 0.013s
I20260709 12:24:24.272853 21240 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:24.403474 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.754s (SENT)
W20260709 12:24:24.334606 21251 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:24:24.337442 21246 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.314428 21168 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.336066 21207 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.404186 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4040) took 1249 ms. Trace:
I20260709 12:24:24.342384 21198 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.330629 21230 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.335391 21209 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:24.409402 13647 rpcz_store.cc:276] 0709 12:24:23.154926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.182060 (+ 27134us) service_pool.cc:224] Handling call
0709 12:24:23.182538 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.182827 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:23.525041 (+342214us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.403740 (+878699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.403801 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:24.403858 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.403938 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.404117 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":165594,"prepare.queue_time_us":330323,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":3303,"thread_start_us":1097459,"threads_started":2,"wal-append.queue_time_us":815838}
I20260709 12:24:24.354466 21232 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.404167 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:24.410543 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.415841 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 365907us
W20260709 12:24:24.416638 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.437994 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:24.441220 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.760s (SENT)
I20260709 12:24:24.442015 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:24.444065 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:24.451351 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:24.411998 21179 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.413602 21246 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:24.413988 21207 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:24.404723 21247 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.458105 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:24.459882 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3871) took 1308 ms. Trace:
I20260709 12:24:24.456640 13648 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.423808 20941 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.382268 21254 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.428018 21198 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:24.560974 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.463824 13725 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:24:24.565539 21245 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.548502 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:24.492944 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 164977us
I20260709 12:24:24.463155 13797 rpcz_store.cc:276] 0709 12:24:23.151957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.164526 (+ 12569us) service_pool.cc:224] Handling call
0709 12:24:23.164889 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.459194 (+294305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.459228 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.459887 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.538834 (+ 78947us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:23.538882 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.624339 (+ 85457us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000e5bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:23.624394 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.459393 (+834999us) raft_consensus.cc:1685] finished
0709 12:24:24.459428 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.459783 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7881,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":24122,"spinlock_wait_cycles":665291136,"thread_start_us":78471,"threads_started":1,"wal-append.queue_time_us":677692}
I20260709 12:24:24.492552 13647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de32481a3be147a5b854251468fc667b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:24.440186 21232 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:24.448891 21169 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:24.449144 21230 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:24.457033 21174 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:24.571820 13648 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:24.470470 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:24.603430 13647 raft_consensus.cc:606] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:24.605695 13647 raft_consensus.cc:614] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:24.519806 21084 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.000s
W20260709 12:24:24.527642 21268 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:24.458971 21220 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:24.599643 21263 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
W20260709 12:24:24.600560 21264 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 12:24:24.612987 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 142600us
I20260709 12:24:24.460893 21128 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.463429 21205 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:24.572551 21267 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
W20260709 12:24:24.603957 21262 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:24:24.564181 21189 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.004s	sys 0.024s
I20260709 12:24:24.471068 21215 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:24.638069 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:24.508194 21223 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.660727 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:24:24.570432 21261 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:24.519299 20842 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.645704 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:24.642570 21220 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.602152 21230 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.539935 21047 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:24.659389 21189 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.005s	sys 0.025s
I20260709 12:24:24.470983 21247 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.667493 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:24.677657 13693 leader_election.cc:400] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:24:24.677958 13693 leader_election.cc:403] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:24:24.647936 21245 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:24.572515 20941 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.683284 20941 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.671619 21230 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:24.691049 21272 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:24:24.693922 21215 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:24.702409 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:24.674032 21269 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:24:24.668200 21179 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:24.676219 20977 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.004s	user 0.006s	sys 0.009s
I20260709 12:24:24.670480 21220 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:24.673631 21014 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:24.713201 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.706882 21174 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:24:24.676491 21168 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.715891 13647 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:24.756136 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.756875 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:24.756908 13801 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:24:24.757949 20731 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.721630 21247 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.724344 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:24.717480 21014 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.726336 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.727521 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:24.730878 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.762015 13651 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:24.730865 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.731177 21142 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.733203 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:24.763072 13652 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:24.734896 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:24.673230 20842 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.738195 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.776748 13727 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 12: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:24.670782 21066 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.667486 21223 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.679833 21245 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.741047 21120 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.751376 21240 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:24.751398 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.720496 21260 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:24:24.759222 21014 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.708181 21249 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.772559 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:24.736263 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:24.758625 20731 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.790853 14417 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:24.773722 21274 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.792132 21274 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.795295 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.796588 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.797214 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1246) took 1151 ms. Trace:
I20260709 12:24:24.797600 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.782742 21174 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.708798 21215 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:24.798802 21174 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.799532 21215 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.797617 13792 rpcz_store.cc:276] 0709 12:24:23.654025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.654311 (+   286us) service_pool.cc:224] Handling call
0709 12:24:24.797045 (+1142734us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:24.797153 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205342976}
I20260709 12:24:24.804695 21254 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:24.809995 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.815920 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3575) took 1347 ms. Trace:
W20260709 12:24:24.816675 21221 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.461s	user 0.007s	sys 0.000s
I20260709 12:24:24.714859 21271 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:24.818563 21189 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.819276 21189 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.819900 21120 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.820464 21120 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.820734 21223 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.822573 21120 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.824304 13792 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:24.826665 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.827963 21208 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.828467 21274 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.828646 21208 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:24.829219 13723 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:24.828966 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.830493 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.828500 13792 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:24.831281 13723 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:24.832324 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.833545 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.834628 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:24.836798 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:24.837141 13796 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:24.837682 13797 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:24.839695 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3873) took 1684 ms. Trace:
I20260709 12:24:24.827117 21066 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:24.840283 13793 rpcz_store.cc:276] 0709 12:24:23.155363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.156412 (+  1049us) service_pool.cc:224] Handling call
0709 12:24:24.839265 (+1682853us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:24.839590 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701323648}
W20260709 12:24:24.844144 21202 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.293s	user 0.004s	sys 0.005s
I20260709 12:24:24.843776 21209 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.845674 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.847117 21209 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:24.781570 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.850858 13727 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:24.851090 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.851804 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.851868 21198 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.852923 20694 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:24.853291 21284 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:24.855316 21240 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.855875 20630 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 12:24:24.856109 21240 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.856325 20630 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:24.857723 13643 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:24.858384 21198 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.859153 21198 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.861708 21198 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.864192 20428 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:24.783074 21247 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:24.865384 21247 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:24.836239 21217 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.599s	user 0.014s	sys 0.000s
I20260709 12:24:24.866539 21217 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:24.867677 13794 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:24.868160 20842 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.869117 21217 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:24.869757 13795 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:24.870792 13796 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:24.869153 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1247) took 1194 ms. Trace:
I20260709 12:24:24.872097 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.872152 13794 rpcz_store.cc:276] 0709 12:24:23.674739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.929300 (+254561us) service_pool.cc:224] Handling call
0709 12:24:23.931650 (+  2350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.868071 (+936421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.868899 (+   828us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54000e56d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:24.869072 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152158592}
I20260709 12:24:24.872911 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:24.875738 13800 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:24.878109 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:24.816398 13724 rpcz_store.cc:276] 0709 12:24:23.468345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.468790 (+   445us) service_pool.cc:224] Handling call
0709 12:24:23.500736 (+ 31946us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.957393 (+456657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.957428 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.988125 (+ 30697us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.291780 (+303655us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:24.484316 (+192536us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.500962 (+ 16646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.501288 (+   326us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.815658 (+314370us) raft_consensus.cc:1685] finished
0709 12:24:24.815682 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.815851 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95795,"prepare.queue_time_us":232100,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":487027,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":29301,"spinlock_wait_cycles":9419136,"thread_start_us":908691,"threads_started":3,"wal-append.queue_time_us":472191}
W20260709 12:24:24.893344 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.898326 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.902004 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:24.903191 13727 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.911108 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.924355 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:24.927178 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:24.786255 21245 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:24.944736 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.945165 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.945886 14417 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:24.933429 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:24.948118 13646 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:24:24.937413 20842 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:24.938444 21274 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:24.938544 20941 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:24.801867 21174 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:24.951246 21274 consensus_queue.cc:1243] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:24.951977 21174 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:24.945797 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2111887069995646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 12:24:24.954229 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48067.6us
W20260709 12:24:24.956350 13617 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.899s (SENT)
I20260709 12:24:24.932739 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:24.956916 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:24.959502 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.962095 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.964311 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.964540 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:24.965835 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.975140 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:24.923146 21239 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:24.966815 13644 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:24.978142 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:24.978798 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:24.977957 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:24.984469 13646 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:24.988688 13800 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:24:24.988534 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:24.989326 13644 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:24.926438 21240 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:24.989964 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57343.2us
I20260709 12:24:24.997272 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:24.995671 13617 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.926s (SENT)
I20260709 12:24:25.001307 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:24.927034 21189 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:25.008374 14419 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:25.009346 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.010587 13724 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:24:25.015674 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:25.015076 21240 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.017187 21244 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:24.955587 21174 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:24.971071 21173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.407s	user 0.019s	sys 0.018s
W20260709 12:24:25.019300 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.020s (SENT)
I20260709 12:24:25.028437 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:24.975186 20842 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:25.035661 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:25.035773 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 116093us
I20260709 12:24:25.002672 21047 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:25.036458 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.062671 13727 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:25.007699 21128 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:24:25.035918 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 129367us
I20260709 12:24:24.987221 21239 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:24.986860 21246 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:25.005848 21168 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:25.036365 13646 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:24:25.038651 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:25.065717 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:25.073019 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.073834 13796 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:25.037662 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:25.061491 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:25.060518 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.034s (SENT)
I20260709 12:24:25.075675 13645 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:25.075711 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:25.017645 21240 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.057564 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:25.054078 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3582) took 1069 ms. Trace:
I20260709 12:24:25.027643 20941 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:25.035374 21173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.471s	user 0.020s	sys 0.018s
I20260709 12:24:25.081213 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:25.077355 13720 rpcz_store.cc:276] 0709 12:24:23.984571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.984691 (+   120us) service_pool.cc:224] Handling call
0709 12:24:23.985157 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:24.062724 (+ 77567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.062759 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:24.063396 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.410018 (+346622us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:24.775197 (+365179us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.053591 (+278394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.053802 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.053847 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:25.053858 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.054002 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82835,"prepare.queue_time_us":124314,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":63,"raft.queue_time_us":323252,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":12150,"thread_start_us":863947,"threads_started":3,"wal-append.queue_time_us":416496}
W20260709 12:24:25.083267 13617 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.155s (SENT)
I20260709 12:24:25.083909 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.083953 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:25.085044 13720 raft_consensus.cc:2410] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:24:25.090054 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:25.066716 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.090976 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:25.092782 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.093580 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
W20260709 12:24:25.066130 21207 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.003s
I20260709 12:24:25.098472 21289 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:25.098740 21207 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:25.100955 21207 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:25.102169 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.103480 21274 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:25.104880 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:25.106215 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.107005 13727 raft_consensus.cc:2368] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:25.107892 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.057773 21142 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.061447 21169 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:25.109386 13721 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:25.109961 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.053301 21247 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:25.057251 21248 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:24:25.114939 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 149554us
I20260709 12:24:25.070178 21239 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:25.065344 21128 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:25.095819 21047 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:25.127548 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:25.096129 21244 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:25.152841 13645 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:25.128506 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.129352 13646 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:25.104965 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.155344 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.156270 13720 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:24:25.157235 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.134133 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.109392 21215 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:25.163614 13796 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:25.164232 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.139770 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:25.141202 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:25.105971 21144 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:25.164911 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.140172 20941 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:25.051491 21230 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.109526 21120 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:25.180557 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.050490 21232 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.122081 21168 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:25.120028 13724 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:25.165408 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:25.184172 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58874.7us
I20260709 12:24:25.200584 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:25.201457 13645 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
I20260709 12:24:25.202581 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3577) took 1730 ms. Trace:
I20260709 12:24:25.203052 13718 rpcz_store.cc:276] 0709 12:24:23.471821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.472020 (+   199us) service_pool.cc:224] Handling call
0709 12:24:23.472440 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:24.726661 (+1254221us) spinlock_profiling.cc:243] Waited 87.8 ms on lock 0x7b18000458e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393becbfb 00007f5393bf764b
0709 12:24:24.726815 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.726843 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:24.727451 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 12:24:24.988039 (+260588us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:24.988088 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.988167 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.202207 (+214040us) raft_consensus.cc:1685] finished
0709 12:24:25.202234 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.202510 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":244873,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":20547,"spinlock_wait_cycles":201836416,"thread_start_us":538771,"threads_started":2,"wal-append.queue_time_us":472579}
I20260709 12:24:25.205129 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.205369 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.206082 13718 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:24:25.207234 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.197419 21120 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:25.207928 21120 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:25.202579 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:25.208513 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:25.208734 13645 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 12.
I20260709 12:24:25.210309 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:25.210363 21162 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:25.211059 21162 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.212316 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:25.213101 13645 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:25.150527 21247 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:25.214829 21247 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:25.217187 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51869.1us
I20260709 12:24:25.219408 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:25.220351 13645 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:25.141736 20694 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:25.165376 21244 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:25.222494 13645 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:25.219408 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.229429 20630 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:25.229730 21247 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:25.233105 20428 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:25.233741 21179 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.004s
I20260709 12:24:25.197577 20941 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:25.109761 21174 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.233884 20428 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.227763 21215 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:25.122567 21142 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.230012 21144 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:25.150321 21261 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:25.246415 13800 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
W20260709 12:24:25.235803 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60649.3us
I20260709 12:24:25.245538 13722 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:25.245625 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.242902 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:25.234999 14424 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 12:24:25.229748 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.280460 14424 pending_rounds.cc:85] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 17
W20260709 12:24:25.280779 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51177.2us
I20260709 12:24:25.280884 14424 pending_rounds.cc:107] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 12:24:25.281680 14424 raft_consensus.cc:2889] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:25.243297 21215 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:25.258018 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 200523us
W20260709 12:24:25.278445 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 137359us
I20260709 12:24:25.283202 13727 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:25.292378 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:25.285589 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:25.286597 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.261601 13645 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:24:25.243896 21286 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
I20260709 12:24:25.297235 20977 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:25.283638 21215 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:25.298362 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1614) took 1241 ms. Trace:
I20260709 12:24:25.281730 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.298797 13725 rpcz_store.cc:276] 0709 12:24:24.057294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.605317 (+548023us) service_pool.cc:224] Handling call
0709 12:24:24.605748 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:24.612370 (+  6622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.612420 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:24.641921 (+ 29501us) log.cc:844] Serialized 52 byte log entry
0709 12:24:24.772388 (+130467us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:24.772440 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.772520 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.298043 (+525523us) raft_consensus.cc:1685] finished
0709 12:24:25.298067 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.298292 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7612,"raft.queue_time_us":14880,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":2692,"thread_start_us":106167,"threads_started":1,"wal-append.queue_time_us":322911}
I20260709 12:24:25.299947 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.301010 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:25.301501 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.302852 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.227607 21047 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:25.304912 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.314146 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.311096 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.315783 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.316982 13725 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:24:25.318440 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.320387 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.234233 21179 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:25.321766 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.322956 13724 raft_consensus.cc:2368] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:25.237893 20941 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:25.248569 21244 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:25.324175 14417 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:25.278424 21144 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:25.235809 21240 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:25.235265 21162 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:25.337164 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:25.337617 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.337994 13725 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:25.338743 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.339871 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.241930 20694 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:25.333978 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:25.343366 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57900.1us
I20260709 12:24:25.335071 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.334879 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.339995 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.343617 13800 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:25.284833 21147 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.335680 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:25.347030 14419 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:25.348711 21142 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:25.292173 21198 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:25.349895 21198 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.193781 20842 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.350504 21198 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:25.350028 20694 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.292501 21223 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:25.354595 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.355445 13725 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:25.357009 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1618) took 1298 ms. Trace:
I20260709 12:24:25.362349 13725 rpcz_store.cc:276] 0709 12:24:24.067944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.356032 (+1288088us) service_pool.cc:224] Handling call
0709 12:24:25.356357 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.356673 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.356709 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.356753 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:25.356788 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.356842 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.356954 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:25.357761 13799 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:25.358243 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.358987 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.368587 13796 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:25.320770 21254 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:25.313222 21047 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:25.353524 20428 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:25.367584 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.376613 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.380110 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:25.332245 21140 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:24:25.299969 21261 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:25.331956 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.488933 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1244) took 1853 ms. Trace:
I20260709 12:24:25.382815 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.383036 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:25.969376 13800 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
W20260709 12:24:25.886901 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.005s	sys 0.007s
W20260709 12:24:25.972206 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.015s	sys 0.010s
I20260709 12:24:25.705509 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1171) took 1756 ms. Trace:
I20260709 12:24:25.968731 13801 rpcz_store.cc:276] 0709 12:24:23.640692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.640956 (+   264us) service_pool.cc:224] Handling call
0709 12:24:23.641250 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.757759 (+1116509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.488514 (+730755us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003d4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:25.488833 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2564930816}
I20260709 12:24:25.975406 14424 rpcz_store.cc:276] 0709 12:24:23.949090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.949351 (+   261us) service_pool.cc:224] Handling call
0709 12:24:23.949890 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.282623 (+1332733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.705092 (+422469us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54006be4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:25.705395 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2954075520}
I20260709 12:24:25.895840 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:25.495877 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:25.472479 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:25.973272 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 671788us
W20260709 12:24:25.865212 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.006s	sys 0.002s
I20260709 12:24:25.904191 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1616) took 1846 ms. Trace:
I20260709 12:24:25.899724 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:25.902809 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3592) took 1846 ms. Trace:
I20260709 12:24:25.921849 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:25.948720 13792 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.019s	sys 0.018s
I20260709 12:24:25.672330 20731 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:25.998667 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.926875 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:26.001539 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75293.5us
I20260709 12:24:25.891981 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:25.740938 20714 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:26.007381 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:26.010161 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:26.010747 20694 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:26.011675 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:26.012521 13792 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.019s	sys 0.028s
I20260709 12:24:25.992724 13722 rpcz_store.cc:276] 0709 12:24:24.057796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.665411 (+607615us) service_pool.cc:224] Handling call
0709 12:24:24.665779 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.280183 (+614404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.903585 (+623402us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:25.903945 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332834688}
I20260709 12:24:26.016218 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:25.994398 13721 rpcz_store.cc:276] 0709 12:24:24.056137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.310092 (+253955us) service_pool.cc:224] Handling call
0709 12:24:24.341870 (+ 31778us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.116789 (+774919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.902042 (+785253us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400174154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:25.902528 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764642048}
W20260709 12:24:25.996860 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.018s	sys 0.025s
I20260709 12:24:26.018314 13799 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
W20260709 12:24:26.022420 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.503s (SENT) [suppressed 26 similar messages]
W20260709 12:24:25.886373 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1237) took 3656 ms (client timeout 3151 ms). Trace:
I20260709 12:24:25.339089 21144 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:25.754097 21014 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.832s	user 0.013s	sys 0.001s
I20260709 12:24:25.370013 21147 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.673699 21128 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.038725 21014 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.041075 21128 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:25.885587 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4056) took 1899 ms (client timeout 2184 ms). Trace:
W20260709 12:24:26.036478 13650 rpcz_store.cc:269] 0709 12:24:22.229836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.232889 (+  3053us) service_pool.cc:224] Handling call
0709 12:24:25.881951 (+3649062us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54001b5654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:25.886091 (+  4140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4621605632}
W20260709 12:24:25.765236 21289 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.005s
I20260709 12:24:26.115203 21289 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:26.043365 13643 rpcz_store.cc:269] 0709 12:24:23.986032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.027167 (+ 41135us) service_pool.cc:224] Handling call
0709 12:24:25.884717 (+1857550us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001e7b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:25.885274 (+   557us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1905656704}
I20260709 12:24:26.125237 14416 raft_consensus.cc:2410] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:24:26.126147 13718 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:26.129436 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:25.562736 21276 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.020s	user 0.004s	sys 0.004s
I20260709 12:24:25.369055 20977 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:26.132139 21270 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:25.677031 20965 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.134351 21168 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:26.134807 20965 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:26.135197 21168 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:26.137842 21168 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.143618 21271 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:25.714349 21299 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.333s	user 0.003s	sys 0.000s
I20260709 12:24:25.609074 21230 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.147084 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:26.147617 21230 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:26.148121 13722 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:26.148458 21230 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:26.149873 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.06657e+06us
W20260709 12:24:26.150959 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.164s (SENT)
I20260709 12:24:25.769446 21244 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:25.763263 21215 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:26.154497 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 699048us
W20260709 12:24:26.155234 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.378916 21207 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:26.159628 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4062) took 1252 ms. Trace:
I20260709 12:24:26.160899 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:25.384425 21047 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:26.162084 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:25.559226 21305 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:24:25.758096 21120 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.549s	user 0.006s	sys 0.006s
I20260709 12:24:26.165444 21120 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:25.503463 21246 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.151556 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:25.359524 21209 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:26.160161 13648 rpcz_store.cc:276] 0709 12:24:24.907483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.908987 (+  1504us) service_pool.cc:224] Handling call
0709 12:24:24.909412 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:24.909848 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.909877 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:24.929563 (+ 19686us) log.cc:844] Serialized 52 byte log entry
0709 12:24:25.089455 (+159892us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:25.089583 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.089708 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.713446 (+623738us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:26.158848 (+445402us) raft_consensus.cc:1685] finished
0709 12:24:26.158888 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.159352 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":554,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":13829,"spinlock_wait_cycles":16636032,"thread_start_us":79467,"threads_started":1,"wal-append.queue_time_us":336623}
I20260709 12:24:26.173659 21244 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:25.446941 21220 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.000s
I20260709 12:24:25.737566 21174 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:25.761663 21198 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:25.645332 21208 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.001s
I20260709 12:24:26.042781 20941 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.183138 21147 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.184834 21151 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.185738 21248 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:26.187376 13798 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:25.377774 21254 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:25.638873 21249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.010s	sys 0.017s
I20260709 12:24:26.190371 21120 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:26.190872 21274 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:25.369747 21240 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:25.708348 21173 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:26.133543 20977 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:25.361454 21223 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:25.567436 21291 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.350s	user 0.000s	sys 0.001s
I20260709 12:24:26.244403 13724 raft_consensus.cc:2368] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
I20260709 12:24:26.170497 21014 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:25.540925 21245 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.007s
W20260709 12:24:26.255898 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 364506us
I20260709 12:24:25.355888 21294 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:24:25.632205 21232 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.247363 21248 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.284335 13727 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:24:26.283900 21151 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.284494 21230 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.260334 21271 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:26.298962 13649 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.000s	sys 0.008s
I20260709 12:24:26.293937 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3552) took 1579 ms. Trace:
W20260709 12:24:26.291221 21249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.025s	sys 0.036s
I20260709 12:24:26.301532 13727 rpcz_store.cc:276] 0709 12:24:24.710137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.241887 (+1531750us) service_pool.cc:224] Handling call
0709 12:24:26.282071 (+ 40184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.287143 (+  5072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.289541 (+  2398us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.302811 13718 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:24:26.306459 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 188158us
W20260709 12:24:26.307274 13649 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.212s	user 0.001s	sys 0.009s
I20260709 12:24:26.308298 21254 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:26.309077 21274 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.310599 21223 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:26.311190 20326 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.064s	user 0.024s	sys 0.027s
I20260709 12:24:26.311094 21220 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.295629 21271 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.299106 21244 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.319010 21208 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.293864 20941 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.494402 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:26.495191 13652 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:26.495997 13651 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:26.496112 20941 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.497021 21256 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:26.497219 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:26.497984 13644 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:26.497829 21256 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.502039 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:26.489466 21220 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.503679 13799 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:26.505208 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 257063us
I20260709 12:24:26.507606 21014 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.508261 21014 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.289222 21248 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:26.404781 21245 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.520162 21245 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:26.405997 20842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.016s	sys 0.021s
I20260709 12:24:26.298593 21173 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:26.521888 20842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.168s	user 0.017s	sys 0.021s
W20260709 12:24:26.400928 21239 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.038s	user 0.017s	sys 0.000s
I20260709 12:24:26.522817 20842 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.522663 21239 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.523545 20842 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.525578 21239 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.526881 13723 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:26.526743 13727 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:26.527779 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3602) took 1703 ms. Trace:
I20260709 12:24:26.527472 21173 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:26.528920 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 224593us
I20260709 12:24:26.529101 21173 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:26.526662 13647 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:26.532374 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:26.533617 21151 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.534041 21066 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.534220 21151 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.534662 21066 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.534930 21151 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:26.308454 21298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.023s	sys 0.026s
I20260709 12:24:26.307982 21036 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:26.310140 13797 raft_consensus.cc:2368] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28.
I20260709 12:24:26.303210 21168 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:26.541751 13652 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:26.463524 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1621) took 1708 ms. Trace:
W20260709 12:24:26.303210 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 281088us
I20260709 12:24:26.470191 21147 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:24:26.478087 20326 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.251s	user 0.024s	sys 0.027s
I20260709 12:24:26.478638 21240 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:26.559646 21240 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:26.560600 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.468030 21271 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:26.480130 21244 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.568733 21244 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:26.484275 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 1.10405e+06us
I20260709 12:24:26.570114 21223 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.571246 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.571547 13722 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:26.572993 21223 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.573125 21147 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:26.573733 21147 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:26.573913 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1252) took 1238 ms. Trace:
I20260709 12:24:26.574980 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3599) took 1751 ms. Trace:
I20260709 12:24:26.575414 13720 rpcz_store.cc:276] 0709 12:24:24.823278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.256582 (+1433304us) service_pool.cc:224] Handling call
0709 12:24:26.304269 (+ 47687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.574508 (+270239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.574545 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.574630 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:26.574685 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.574756 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.574888 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.488242 21208 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.489444 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.491174 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:26.580624 13648 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:26.491351 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:26.491844 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.492457 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:26.493638 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.516137 21261 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.582397 13650 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:26.516687 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.582646 13643 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:26.528165 13727 rpcz_store.cc:276] 0709 12:24:24.824830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.305850 (+1481020us) service_pool.cc:224] Handling call
0709 12:24:26.306527 (+   677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.527423 (+220896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.527672 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.544127 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.583600 13800 raft_consensus.cc:2410] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:24:26.583854 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:26.517644 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.545398 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.546025 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.547302 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:26.548871 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.526142 20842 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:26.550832 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 399948us
I20260709 12:24:26.554862 21294 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:26.555030 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46124.9us
I20260709 12:24:26.487242 21274 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.574400 13792 rpcz_store.cc:276] 0709 12:24:25.335019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.358791 (+ 23772us) service_pool.cc:224] Handling call
0709 12:24:25.368343 (+  9552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.368595 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:24:26.191164 (+822569us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.536226 (+345062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.573175 (+ 36949us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:26.573323 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.573440 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.573679 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48794,"prepare.queue_time_us":132226,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":27681,"raft.queue_time_us":55671,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":167,"thread_start_us":842817,"threads_started":2,"wal-append.queue_time_us":307337}
W20260709 12:24:26.582966 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 124611us
I20260709 12:24:26.548709 21256 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:26.587237 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.543182 13718 rpcz_store.cc:276] 0709 12:24:24.751909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.192339 (+1440430us) service_pool.cc:224] Handling call
0709 12:24:26.247135 (+ 54796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.305746 (+ 58611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.460389 (+154643us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:26.589299 21298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.026s	sys 0.033s
I20260709 12:24:26.599824 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:26.618419 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.594038 21294 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:26.576284 21219 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:26.596141 21223 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:26.636844 21256 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
W20260709 12:24:26.645448 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.167s (SENT)
W20260709 12:24:26.648478 21323 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:24:26.649376 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 320141us
I20260709 12:24:26.575011 21147 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.652503 21209 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:26.660916 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:26.664021 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1622) took 1827 ms. Trace:
I20260709 12:24:26.665611 21151 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.670078 13792 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:24:26.671176 21147 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:26.673321 21014 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.622213 21248 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.655260 21208 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:26.664377 13727 rpcz_store.cc:276] 0709 12:24:24.836187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.617576 (+1781389us) service_pool.cc:224] Handling call
0709 12:24:26.655216 (+ 37640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.663738 (+  8522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.663760 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.663795 (+    35us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:26.663826 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.663871 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.663971 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.654268 21036 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:26.663257 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.679193 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.669674 21240 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.657317 20842 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.674458 20326 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.685456 13721 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:24:26.685591 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87640.4us
I20260709 12:24:26.680708 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:26.696175 13795 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
W20260709 12:24:26.699397 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 128358us
I20260709 12:24:26.700699 13720 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:26.702668 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3603) took 1877 ms. Trace:
I20260709 12:24:26.703451 13793 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:26.697747 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.703166 13720 rpcz_store.cc:276] 0709 12:24:24.825098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.616979 (+1791881us) service_pool.cc:224] Handling call
0709 12:24:26.675889 (+ 58910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.701033 (+ 25144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.702602 (+  1569us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:26.707677 21320 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.171s	user 0.005s	sys 0.000s
I20260709 12:24:26.707931 20842 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.698055 21168 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:26.711714 21315 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:26.712252 21315 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:26.697631 21252 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:26.717798 13801 raft_consensus.cc:2410] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:24:26.714813 21185 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:26.718667 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.719054 21298 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.719174 21185 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.715281 20326 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.722508 21209 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.723218 21185 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:26.725232 20326 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.702421 21014 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:26.719941 21298 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.726336 21223 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.715497 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.729601 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:26.730413 21144 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.690s	user 0.016s	sys 0.003s
I20260709 12:24:26.730737 21223 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.731884 21248 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.732276 13727 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:26.733076 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1623) took 1609 ms. Trace:
I20260709 12:24:26.734558 21120 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:26.734862 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99621.5us
I20260709 12:24:26.735605 21208 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:26.736248 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:26.730444 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.731092 21144 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.733572 20842 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.739468 21248 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:26.740374 20975 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:26.740821 20975 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:26.740558 21271 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.742489 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.744683 14419 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:26.741798 20975 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.746203 20975 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:26.749416 21252 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:26.730494 21047 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.568s	user 0.017s	sys 0.000s
I20260709 12:24:26.740679 21315 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.750979 21047 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.752660 21315 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:26.747867 21208 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:26.758675 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.739508 21144 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.733414 13727 rpcz_store.cc:276] 0709 12:24:25.123272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.692879 (+1569607us) service_pool.cc:224] Handling call
0709 12:24:26.732020 (+ 39141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.732751 (+   731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.732978 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.755709 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:26.762912 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.733547 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:26.756490 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119876us
I20260709 12:24:26.750466 21252 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:24:26.765391 14417 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:26.767925 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:26.766827 21252 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.768556 13644 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:26.768904 21252 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.770556 21208 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:26.771152 21208 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:26.771626 21208 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:26.773161 13643 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:26.773880 21217 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:26.774745 13648 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:26.777846 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:26.733429 21223 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.781379 21144 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:26.783267 21144 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.785220 21066 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:26.787691 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:26.788090 21298 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:26.788508 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1620) took 2119 ms. Trace:
I20260709 12:24:26.789124 21066 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:26.790426 21128 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.800138 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:24:26.802402 13726 rpcz_store.cc:276] 0709 12:24:24.662426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.374060 (+711634us) service_pool.cc:224] Handling call
0709 12:24:25.895816 (+521756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.353872 (+458056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.457464 (+103592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.461891 (+  4427us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:26.465464 (+  3573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.779572 (+314108us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:26.779612 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.781392 (+  1780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":840011264}
I20260709 12:24:26.764515 21271 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.807688 21248 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:24:26.809052 13727 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:26.822538 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1625) took 1483 ms. Trace:
I20260709 12:24:26.823001 13727 rpcz_store.cc:276] 0709 12:24:25.331837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.808093 (+1476256us) service_pool.cc:224] Handling call
0709 12:24:26.808684 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.814338 (+  5654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.814690 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.803789 21144 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.825302 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:26.829493 13650 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
W20260709 12:24:26.839210 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 101083us
I20260709 12:24:26.809695 21128 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.838629 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:26.844820 13727 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:26.828189 21066 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:26.843225 13644 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:26.843886 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:26.810110 21254 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.854056 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.854893 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3595) took 2032 ms. Trace:
I20260709 12:24:26.859498 13721 rpcz_store.cc:276] 0709 12:24:24.822051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.200188 (+1378137us) service_pool.cc:224] Handling call
0709 12:24:26.277445 (+ 77257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.693671 (+416226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.854671 (+161000us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54002f5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:26.854817 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934885120}
I20260709 12:24:26.855706 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:26.856221 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:26.862833 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75254.5us
I20260709 12:24:26.868137 13726 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:26.872048 13800 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:26.860925 21298 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:26.875780 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3581) took 2829 ms. Trace:
I20260709 12:24:26.876438 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:26.876641 20694 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:26.877763 21066 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.884335 21219 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:26.888535 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.869652 20428 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
W20260709 12:24:26.894577 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.894876 20428 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:26.895395 20428 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:26.896458 20428 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.897862 20428 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:26.872047 21036 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:26.903930 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.904847 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:26.912001 21208 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:26.913781 13796 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:26.914201 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.914891 13720 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:26.920486 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:26.892628 21219 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:26.867677 20842 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:26.914052 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3560) took 1759 ms. Trace:
I20260709 12:24:26.931874 21208 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.936601 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1626) took 1049 ms. Trace:
I20260709 12:24:26.937496 13721 rpcz_store.cc:276] 0709 12:24:25.887674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.866765 (+979091us) service_pool.cc:224] Handling call
0709 12:24:26.874331 (+  7566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.926545 (+ 52214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.926592 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.936304 (+  9712us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:26.936349 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.936416 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.936542 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:26.933727 13724 rpcz_store.cc:276] 0709 12:24:25.154864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.341087 (+186223us) service_pool.cc:224] Handling call
0709 12:24:26.913784 (+1572697us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400174654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:26.913971 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435299328}
I20260709 12:24:26.940841 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:26.952375 13726 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:26.889163 21047 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:26.886588 13719 rpcz_store.cc:276] 0709 12:24:23.984140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.984272 (+   132us) service_pool.cc:224] Handling call
0709 12:24:23.987001 (+  2729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:24.109933 (+122932us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:24.326843 (+216910us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.917641 (+1590798us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b18003dc720. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:26.248504 (+330863us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.406731 (+158227us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.482857 (+ 76126us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:26.658879 (+176022us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.659187 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.674782 (+ 15595us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.790977 (+116195us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b18003dc720. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:26.810039 (+ 19062us) raft_consensus.cc:1685] finished
0709 12:24:26.810060 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.813126 (+  3066us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120327,"prepare.queue_time_us":553729,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":103,"raft.queue_time_us":309179,"raft.run_cpu_time_us":831,"raft.run_wall_time_us":72682,"spinlock_wait_cycles":8505984,"thread_start_us":405315,"threads_started":3,"wal-append.queue_time_us":1949676}
I20260709 12:24:26.938997 21252 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:26.933162 20842 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:26.967639 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 120782us
I20260709 12:24:26.935627 21298 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:26.970624 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:26.973882 21338 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:24:26.977977 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:26.985759 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.980086 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3608) took 1080 ms. Trace:
I20260709 12:24:26.989225 13727 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:26.970340 20975 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:26.991916 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59055.3us
I20260709 12:24:26.975787 20842 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:26.999353 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:27.000021 13719 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:27.000375 13644 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:27.000794 13724 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:27.007503 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3611) took 1073 ms. Trace:
I20260709 12:24:27.007858 13724 rpcz_store.cc:276] 0709 12:24:25.934443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.973334 (+1038891us) service_pool.cc:224] Handling call
0709 12:24:26.998017 (+ 24683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.007239 (+  9222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.007443 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.002307 13727 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:27.009557 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.015193 13648 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:27.015714 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.056473 13643 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:27.018883 21298 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.021130 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:27.023514 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:26.979151 21183 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.192s
I20260709 12:24:27.015247 20965 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.032410 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3613) took 1093 ms. Trace:
I20260709 12:24:27.034415 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3610) took 1069 ms. Trace:
I20260709 12:24:26.998795 21047 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:26.989740 13718 rpcz_store.cc:276] 0709 12:24:25.899289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.910597 (+1011308us) service_pool.cc:224] Handling call
0709 12:24:26.911263 (+   666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.911702 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:26.912995 (+  1293us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.975633 (+ 62638us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.975779 (+   146us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:26.975866 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.976002 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.980019 (+  4017us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":10403,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":7144,"thread_start_us":54754,"threads_started":1}
I20260709 12:24:27.039115 21294 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.043125 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.064116 13644 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:27.065153 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.044996 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.066563 21315 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:27.067195 21144 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:27.067536 21315 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:27.068324 21144 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:27.068754 21144 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:27.044955 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.070358 13651 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26. Re-sending same reply.
I20260709 12:24:27.069315 21315 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.044956 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.050907 21254 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.051105 13793 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:27.051251 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:27.053645 13650 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:27.059163 14423 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:27.059762 13719 rpcz_store.cc:276] 0709 12:24:25.939487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.982516 (+1043029us) service_pool.cc:224] Handling call
0709 12:24:26.997100 (+ 14584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.032128 (+ 35028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.032351 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.059942 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1628) took 1085 ms. Trace:
I20260709 12:24:27.059969 13726 rpcz_store.cc:276] 0709 12:24:25.933996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.972703 (+1038707us) service_pool.cc:224] Handling call
0709 12:24:26.988663 (+ 15960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.003249 (+ 14586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.003280 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:27.003334 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:27.003383 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.003464 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.003580 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.062554 21047 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:27.080711 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57313.5us
I20260709 12:24:27.076274 13727 rpcz_store.cc:276] 0709 12:24:25.974705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.001730 (+1027025us) service_pool.cc:224] Handling call
0709 12:24:27.002107 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.059656 (+ 57549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.059880 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.084847 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:27.060662 21173 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.531s	user 0.016s	sys 0.000s
I20260709 12:24:27.083527 21252 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:27.100807 21147 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:27.101336 21147 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.101740 21147 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:27.102448 21047 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.109315 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:27.110648 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44834us
I20260709 12:24:27.114223 13648 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:27.120472 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.095556 20941 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.130941 13724 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:24:27.131462 13724 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:24:27.069746 20965 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.069808 21144 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:27.138222 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 485941us
I20260709 12:24:27.131716 13724 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:24:27.142460 13724 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:27.143679 13644 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:27.151121 20941 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.153371 21198 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.156322 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3891) took 1953 ms. Trace:
I20260709 12:24:27.139335 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4063) took 2226 ms. Trace:
I20260709 12:24:27.142953 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:27.136111 13617 connection.cc:256] RPC call timeout handler was delayed by -2.0135930399997051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22820000000000001s for a response
I20260709 12:24:27.161360 13643 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:27.156818 13798 rpcz_store.cc:276] 0709 12:24:25.203333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.206648 (+  3315us) service_pool.cc:224] Handling call
0709 12:24:25.206981 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.288737 (+1081756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.155977 (+867240us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54000e5bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:27.156249 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2253379328}
I20260709 12:24:27.165910 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1629) took 1190 ms. Trace:
I20260709 12:24:27.166451 13724 rpcz_store.cc:276] 0709 12:24:25.975149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.082413 (+1107264us) service_pool.cc:224] Handling call
0709 12:24:27.104286 (+ 21873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.165591 (+ 61305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.165840 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:27.159611 21047 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.182834 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51689us
W20260709 12:24:27.194924 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.561s (SENT)
I20260709 12:24:27.157989 13649 rpcz_store.cc:276] 0709 12:24:24.908143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.909197 (+  1054us) service_pool.cc:224] Handling call
0709 12:24:24.934275 (+ 25078us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:24.934718 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.934749 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:24.935409 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 12:24:25.068729 (+133320us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:26.901025 (+1832296us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b180047b5a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:24:26.901233 (+   208us) log.cc:844] Serialized 14 byte log entry
0709 12:24:27.059461 (+158228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.093673 (+ 34212us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.106119 (+ 12446us) raft_consensus.cc:1685] finished
0709 12:24:27.117620 (+ 11501us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.134715 (+ 17095us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5417,"prepare.queue_time_us":138076,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":14219,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":25305,"spinlock_wait_cycles":29903232,"thread_start_us":1410732,"threads_started":3,"wal-append.queue_time_us":259816,"wal-append.run_cpu_time_us":2712,"wal-append.run_wall_time_us":1125965}
I20260709 12:24:27.211828 13799 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:27.219386 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:27.219740 13724 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:24:27.219616 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:27.178524 21185 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:27.229588 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.183528 21256 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.249382 13649 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:24:27.254781 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49285.8us
I20260709 12:24:27.255162 20965 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.256462 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:27.256924 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:27.257325 21185 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.259390 20694 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:27.237397 20326 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.023s
I20260709 12:24:27.260442 21173 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:27.261261 20326 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:27.261688 20326 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:27.262725 20326 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.264554 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3609) took 1365 ms. Trace:
I20260709 12:24:27.255012 20941 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.264405 21215 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.235715 21261 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:27.265062 13721 rpcz_store.cc:276] 0709 12:24:25.899576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.957010 (+1057434us) service_pool.cc:224] Handling call
0709 12:24:26.967757 (+ 10747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.991427 (+ 23670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.991453 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.030619 (+ 39166us) log.cc:844] Serialized 26 byte log entry
0709 12:24:27.176689 (+146070us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:27.176724 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.176786 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:27.264280 (+ 87494us) raft_consensus.cc:1685] finished
0709 12:24:27.264316 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.264486 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":142316,"threads_started":1,"wal-append.queue_time_us":231565}
I20260709 12:24:27.261266 21173 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:27.267854 21173 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:27.268281 21173 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:27.265214 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 89505.9us
I20260709 12:24:27.269917 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:27.273638 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.567s (SENT)
I20260709 12:24:27.274140 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:27.273644 13649 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:27.279848 13724 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:27.209086 21183 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:24:27.231091 21144 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:27.226929 21047 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.300420 21047 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.301613 13799 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:27.283569 21211 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:27.311554 21211 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:27.312932 21211 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.272914 20842 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.317214 20842 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.319360 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:27.320571 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3545) took 2612 ms (client timeout 1754 ms). Trace:
W20260709 12:24:27.321199 13723 rpcz_store.cc:269] 0709 12:24:24.708151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.832045 (+123894us) service_pool.cc:224] Handling call
0709 12:24:27.320180 (+2488135us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54001f0754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:27.320424 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3894643584}
I20260709 12:24:27.264418 20326 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:27.285969 21256 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.326129 21256 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.326529 21211 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:27.327114 13721 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:27.290004 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1253) took 1954 ms (client timeout 2120 ms). Trace:
W20260709 12:24:27.330899 13797 rpcz_store.cc:269] 0709 12:24:25.335292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.358737 (+ 23445us) service_pool.cc:224] Handling call
0709 12:24:27.289707 (+1930970us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:27.289908 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103289600}
I20260709 12:24:27.332247 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.333509 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.333628 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:27.294293 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.959s (SENT)
W20260709 12:24:27.294586 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.294822 21261 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:27.340672 13649 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:27.310341 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.310667 21219 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.320781 13648 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:27.334741 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.317863 21294 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.292605 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:27.335726 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.335698 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.351567 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:27.352340 13643 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:27.329672 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.335072 13799 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:27.340147 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:27.353514 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60992.6us
I20260709 12:24:27.289012 21240 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.354568 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:27.258052 21185 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:24:27.274154 21147 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.361395 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.367125 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.372177 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.376647 13651 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:27.384263 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:27.385771 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91529.3us
I20260709 12:24:27.387163 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.388619 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.390774 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:27.393869 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.395236 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.395789 13646 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:27.395970 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1619) took 3303 ms. Trace:
I20260709 12:24:27.396438 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.396363 13725 rpcz_store.cc:276] 0709 12:24:24.092458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.363644 (+1271186us) service_pool.cc:224] Handling call
0709 12:24:25.372140 (+  8496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.513341 (+1141201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.538354 (+ 25013us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.541632 (+  3278us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:26.541679 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.395541 (+853862us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54001a5254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:27.395594 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:27.395776 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2101153152}
I20260709 12:24:27.397650 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.398432 21176 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:27.400468 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.402293 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.402536 20694 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:27.403520 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.404079 20694 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:27.404716 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.405264 13649 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:27.407014 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3598) took 2584 ms. Trace:
I20260709 12:24:27.407598 14416 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:24:27.407383 13722 rpcz_store.cc:276] 0709 12:24:24.822893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.218318 (+1395425us) service_pool.cc:224] Handling call
0709 12:24:26.290869 (+ 72551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.571990 (+281121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.390740 (+818750us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54003fded4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:27.406937 (+ 16197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644954496}
I20260709 12:24:27.423981 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.425534 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.428054 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.429543 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.344131 20842 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.432940 20842 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.435847 20842 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.436897 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.436887 21173 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.438433 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.439381 21173 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.443693 21219 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:27.444540 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.445245 13718 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:24:27.450383 21066 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:27.450908 21066 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.383579 21215 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.451267 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.452008 21215 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.451283 21066 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:27.454881 21271 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.455294 21294 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.455440 21271 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.455888 21294 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.457731 21271 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.458431 21144 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.383205 21248 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.458994 21144 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.459513 21248 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:27.460626 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 108456us
I20260709 12:24:27.461393 21144 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.464665 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:27.465809 13652 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:27.443774 13721 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:24:27.389874 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.476771 13643 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:27.477492 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:27.478466 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.480759 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.481624 13718 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:27.484313 13793 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:27.490302 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.492782 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1624) took 2174 ms. Trace:
I20260709 12:24:27.493287 13720 rpcz_store.cc:276] 0709 12:24:25.318149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.718089 (+1399940us) service_pool.cc:224] Handling call
0709 12:24:26.914661 (+196572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.915392 (+   731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.492578 (+577186us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400174650. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:27.492717 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274204928}
I20260709 12:24:27.500351 13646 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:24:27.506875 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.507608 13722 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:27.469310 21215 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.503489 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:27.536891 13649 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:27.461689 21248 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.462823 21271 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.541623 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.510207 21326 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:24:27.511353 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4065) took 2602 ms. Trace:
I20260709 12:24:27.506067 14416 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:27.494210 21261 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:24:27.508791 21219 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:27.548285 21219 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:27.517100 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:27.520334 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.552237 13644 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:27.521168 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:27.529207 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 64636.4us
I20260709 12:24:27.534482 13719 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:24:27.458004 21294 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.544749 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:27.509222 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.494474 20842 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:27.562783 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:27.544306 13652 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:27.544930 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:27.543169 13647 rpcz_store.cc:276] 0709 12:24:24.908805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.908929 (+   124us) service_pool.cc:224] Handling call
0709 12:24:24.909243 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.531391 (+1622148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.490653 (+959262us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:27.511253 (+ 20600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3718568704}
I20260709 12:24:27.567938 13648 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:24:27.570909 21271 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.543586 21256 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:24:27.574790 21326 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:27.575343 21326 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:27.526111 21128 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.555864 21185 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:27.576573 21066 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.577378 21185 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.577874 21185 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:27.555532 21147 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:27.581998 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 325175us
I20260709 12:24:27.586601 21177 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:27.587244 13800 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 18. Ops: []
I20260709 12:24:27.563592 14417 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:27.573836 21271 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:27.594594 13651 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:27.596730 13650 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:27.549615 21219 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.603479 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:27.608410 13643 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:27.645901 13646 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:27.649816 13647 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:27.601951 21177 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:27.618695 21128 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:27.601384 21211 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:24:27.698753 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:27.701923 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 315376us
I20260709 12:24:27.576586 21326 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:27.691308 21128 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.705374 13646 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:27.706827 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:27.708712 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.721342 13643 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:27.727193 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:27.731086 14417 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:27.733480 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:27.735442 21361 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:24:27.738703 13646 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:27.741029 13793 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:27.699019 21066 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:27.748561 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.863s (SENT)
I20260709 12:24:27.728936 21128 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:27.757928 13801 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:27.761547 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.763268 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:27.776443 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173826us
I20260709 12:24:27.780160 13724 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:27.787171 13719 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:27.793296 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.801980 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:27.802640 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60109.3us
I20260709 12:24:27.803041 13721 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
W20260709 12:24:27.803714 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.805696 21369 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:24:27.806521 21326 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:27.811398 21183 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:27.811998 21183 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:27.818605 21211 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:27.819332 21211 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:27.820645 21211 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.821064 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:27.822275 13722 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:27.822487 21211 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:27.827389 13647 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:27.813295 21183 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:27.833918 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 371545us
W20260709 12:24:27.837055 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.850353 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.985s (SENT)
I20260709 12:24:27.851274 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:27.852442 21261 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:27.853945 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73150.6us
W20260709 12:24:27.854902 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100005us
W20260709 12:24:27.833791 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44687.3us
I20260709 12:24:27.865654 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:27.867700 21249 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:27.868386 21249 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:27.885846 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:27.888960 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:27.890537 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:27.778681 21066 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 12:24:27.900561 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43683.7us
I20260709 12:24:27.909559 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:27.926697 13617 connection.cc:256] RPC call timeout handler was delayed by -1.6973953010001424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2757s for a response
I20260709 12:24:27.921921 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:27.869385 21249 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:27.935925 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48484.3us
W20260709 12:24:27.945462 14391 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.855s (SENT)
I20260709 12:24:27.950908 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:27.935456 21364 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:27.825989 21177 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.947757 20862 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:24:27.957998 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:27.960035 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:27.964474 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:27.965528 21271 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:27.956506 21364 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:27.981204 21271 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:27.981691 21271 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:27.987926 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:27.991150 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:27.975625 21217 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:27.998276 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:27.968842 20842 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:28.019448 21371 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:28.020126 21371 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:28.020642 21371 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:28.023126 21378 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:27.976509 21381 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
W20260709 12:24:28.041822 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86361.5us
I20260709 12:24:28.007486 21294 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:27.982677 21380 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 12:24:28.062834 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 64694.6us
I20260709 12:24:28.005918 21217 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.014724 21144 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:28.004429 21382 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:28.010116 20862 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:24:28.057971 21374 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:24:28.070494 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.060420 21183 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:28.118709 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 154461us
I20260709 12:24:28.124109 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.078907 21294 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:28.145133 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 177929us
W20260709 12:24:28.140465 21354 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:28.137168 21382 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:28.144109 21047 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:28.135921 20862 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:28.161064 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97249.7us
I20260709 12:24:28.162375 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.144325 21379 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:28.151095 21248 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:28.168711 21248 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.170070 21248 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.170946 21381 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:28.171574 21381 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.167744 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.183337 21294 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:28.187064 21383 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:24:28.186770 21382 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.190891 20862 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.192337 21209 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:28.192801 21209 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.196074 21217 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.199366 21393 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:28.200313 21393 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:28.200667 21393 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.201524 21393 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:28.205360 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 163597us
I20260709 12:24:28.163314 21219 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:24:28.159242 21364 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:28.208525 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.503s (SENT)
I20260709 12:24:28.209200 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:28.210073 21364 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.212205 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:28.212786 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95069.6us
I20260709 12:24:28.213285 14418 raft_consensus.cc:2376] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23. Re-sending same reply.
I20260709 12:24:28.193674 21209 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:28.235028 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86470.5us
I20260709 12:24:28.166630 21380 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:24:28.245040 13649 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:28.245762 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:28.252910 21208 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:28.253599 21208 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.255287 21208 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.260537 21393 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:28.261044 21271 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:28.263785 21363 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.471s	user 0.003s	sys 0.000s
W20260709 12:24:28.265882 21185 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.000s
I20260709 12:24:28.266455 21185 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.272060 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:28.240041 21386 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:24:28.275557 21380 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:28.276932 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:28.277381 21128 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.503s	user 0.013s	sys 0.000s
I20260709 12:24:28.278187 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:28.278865 13644 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.280160 13643 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:28.205035 20975 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:24:28.240383 21398 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:28.167385 21379 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.303817 21271 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:28.304554 21208 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.306422 21047 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.279309 21185 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:28.292858 21248 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:28.308763 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 141858us
W20260709 12:24:28.311823 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62744.3us
I20260709 12:24:28.322690 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.337817 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.277873 21128 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.345376 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.316336 21398 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.352586 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:28.356885 21142 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:28.367635 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90793.2us
I20260709 12:24:28.348186 21271 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:28.380898 13644 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:28.363008 21248 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:28.391007 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45731.9us
I20260709 12:24:28.392102 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.394016 21405 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.395262 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.394862 21405 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.397603 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:28.399374 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.401504 21398 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.406394 21128 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:28.406517 21407 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.407750 21379 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:28.409173 21177 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:28.385830 21294 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:28.419759 21413 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.389731 21271 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.423182 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:28.424715 21407 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.425388 21142 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:28.427062 14418 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:28.432706 21271 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.416301 21248 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.437186 21248 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:28.440062 21407 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:28.441548 21198 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:28.442005 21198 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.437935 21248 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:28.445665 21248 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.446134 21248 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:28.453365 21405 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:28.454820 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66064.6us
I20260709 12:24:28.456130 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.457228 21142 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:28.434347 21413 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.445567 21198 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.474664 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.414125 21393 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:24:28.477241 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.454134 21405 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:28.478550 13719 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:24:28.478989 13719 pending_rounds.cc:85] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:24:28.479269 13719 pending_rounds.cc:107] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:24:28.479525 13719 raft_consensus.cc:2889] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.482923 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.483062 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:28.487350 13726 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:28.478083 21405 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.464263 21371 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.491734 21405 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.495682 21248 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.496277 21409 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:28.498780 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.500581 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.501479 21326 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.502211 21326 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:28.461639 21413 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:28.478578 21183 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:28.502956 21326 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.504594 20326 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:28.507167 20326 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:28.507540 20326 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:28.504580 21185 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:28.493891 21371 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:28.502142 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.541177 13726 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:24:28.536360 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:28.542973 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:28.543368 13723 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:28.549201 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47225.6us
I20260709 12:24:28.550433 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.490358 21294 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.551999 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.552887 21185 consensus_queue.cc:1243] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:24:28.553558 21185 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:28.553901 21185 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:28.551911 21294 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.554934 21185 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.556286 21185 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:28.563759 13727 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:28.564123 13723 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:28.560668 13719 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:28.568408 20326 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.567107 20842 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:28.571940 21128 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:28.572870 21389 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.561223 21249 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:28.573472 21389 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.577402 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:28.579535 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:28.584276 21248 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:28.585479 21367 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.790s	user 0.004s	sys 0.000s
I20260709 12:24:28.591933 20326 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:28.532366 21409 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.576186 21389 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:28.597182 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.542541 21198 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:28.601401 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 206222us
I20260709 12:24:28.602713 14418 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:28.602958 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.604374 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:28.601933 13795 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:28.616334 13724 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:28.625752 13651 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:28.628015 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:28.635414 13646 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:24:28.636660 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.639585 13644 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:28.662640 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:28.663637 13644 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:28.651168 21208 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:28.667279 21208 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:28.668707 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.670501 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.671423 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.672384 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:28.675401 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:28.678807 13645 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:28.612627 21371 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:24:28.681411 21271 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:28.646682 21382 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.684940 13798 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:28.685042 21382 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.688107 21382 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:28.692804 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.698900 14418 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:28.630887 21389 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:24:28.709555 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 157643us
I20260709 12:24:28.710947 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:28.711997 13721 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:28.716318 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:28.721310 13645 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:28.702935 21411 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:28.728644 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.743959 13644 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:28.707652 21271 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.715865 21371 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:28.757810 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.714836 21389 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:28.752104 13652 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:24:28.753450 20975 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:28.707458 21249 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:24:28.609448 21409 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.762387 21411 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:28.758765 21208 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:28.770807 21047 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:28.821869 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93389.8us
I20260709 12:24:28.772223 21371 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:28.824527 14391 leader_election.cc:336] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 3.151s (SENT)
I20260709 12:24:28.827119 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:28.831115 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73429us
I20260709 12:24:28.832389 21128 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.823542 21380 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:24:28.841439 21294 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:28.839722 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:28.842684 21177 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:28.843333 21177 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.847206 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:28.849162 21219 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:28.849591 21219 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.850579 21219 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.850782 21371 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.851758 21219 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:28.854455 20842 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.855202 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.841459 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:28.856649 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.857353 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.826458 21369 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.852833 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:28.852316 21371 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:28.841251 21393 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:28.855906 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.865736 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:28.863818 13722 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:28.868991 14416 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:28.897769 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:28.874047 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:28.862048 21389 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.862105 20428 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.910585 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:28.860504 21128 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.844457 21177 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.859769 21271 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:28.924546 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:28.948614 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.947650 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:28.951275 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 115211us
W20260709 12:24:28.936455 21421 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:24:28.969678 21412 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:28.984777 13644 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:28.863170 20842 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.988546 13652 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:28.899904 21294 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:28.899351 21047 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:28.995947 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:28.912503 21418 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:28.994318 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:29.003602 13646 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:24:28.994158 21373 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:24:28.996321 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.002247 21414 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:28.950163 21298 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:29.022472 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:29.005540 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:24:29.029917 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.005787 13800 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:29.031059 13646 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:29.009136 21209 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.012233 13617 leader_election.cc:400] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
I20260709 12:24:29.035450 13617 leader_election.cc:403] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:29.004920 21248 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:28.949450 21120 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:24:29.014689 21409 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:29.016350 21387 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.855s	user 0.000s	sys 0.004s
I20260709 12:24:29.007515 21414 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:28.995332 21412 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.970355 21393 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:24:29.019012 21176 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:29.024899 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:29.018003 20842 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:29.025715 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.026443 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:29.027138 13650 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:28.868211 21369 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.906113 21413 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.096s
I20260709 12:24:28.914863 20975 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:24:29.030686 13798 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:29.036717 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.037145 13723 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:24:29.038870 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:28.949648 20862 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:24:28.932723 20428 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:28.971076 21142 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.935002 21411 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.125175 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:29.100242 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3913) took 1549 ms. Trace:
I20260709 12:24:29.096783 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:29.125313 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:29.126798 13723 pending_rounds.cc:85] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:24:29.121212 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:29.127216 13723 pending_rounds.cc:107] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:24:29.127629 13723 raft_consensus.cc:2889] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.095520 13719 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:29.116008 13727 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:29.124200 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:29.112903 13726 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:29.131572 13722 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.120482 20326 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.124743 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.132973 20326 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.106778 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.123030 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.121956 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.096711 13721 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:24:29.122656 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.133845 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:29.144527 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.145692 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.153806 13643 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:29.038414 21144 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.012s	sys 0.010s
W20260709 12:24:29.163337 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.098804 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:29.168336 13724 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:29.168567 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.162087 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:29.138334 13721 pending_rounds.cc:85] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 13
I20260709 12:24:29.171620 13721 pending_rounds.cc:107] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:24:29.172035 13721 raft_consensus.cc:2889] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.174091 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:29.175073 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:29.175513 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.180536 20326 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:29.125989 13799 rpcz_store.cc:276] 0709 12:24:27.551355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.551603 (+   248us) service_pool.cc:224] Handling call
0709 12:24:27.552188 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.552693 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.552725 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:27.836431 (+283706us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:24:27.837003 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:24:27.978350 (+141347us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:28.003894 (+ 25544us) log.cc:844] Serialized 14 byte log entry
0709 12:24:28.006066 (+  2172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.006139 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.016032 (+1009893us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b2400227378. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b4609d 00007f539377810e 00007f539378f88c 00007f539378ebb6 00007f539378e7d8 00007f5393798221
0709 12:24:29.099912 (+ 83880us) raft_consensus.cc:1685] finished
0709 12:24:29.099949 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.100163 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1935,"prepare.queue_time_us":38905,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":16582,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"spinlock_wait_cycles":116151808,"thread_start_us":140459,"threads_started":1,"wal-append.queue_time_us":250124}
I20260709 12:24:29.188917 13726 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:29.170595 21144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.013s	sys 0.011s
I20260709 12:24:29.111670 21416 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.090516 21176 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:29.119982 21412 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:29.046746 21248 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.122562 21381 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.099465 21414 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:29.330248 13645 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.000s
I20260709 12:24:29.194684 13795 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:29.249276 13800 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:29.332897 13645 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 13.
W20260709 12:24:29.330091 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 360774us
I20260709 12:24:29.091002 21047 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:29.231863 14417 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:24:29.192720 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 269310us
I20260709 12:24:29.237862 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:29.337507 14416 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:29.231614 13719 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:29.238770 14418 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:29.336773 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:29.316919 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 294525us
W20260709 12:24:29.240288 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 211939us
I20260709 12:24:29.208175 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.303663 20428 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:29.366236 20428 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:29.346446 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:29.349078 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:29.183758 21183 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:29.304776 21428 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:24:29.169668 20842 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
W20260709 12:24:29.370818 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 193813us
I20260709 12:24:29.377055 21249 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:29.378695 21249 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.378397 21176 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.379210 21249 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:29.379801 21176 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:29.370651 21381 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:29.394812 21415 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:24:29.397970 21427 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:29.398541 13794 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:29.401424 13722 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:29.401633 21381 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:24:29.340792 21047 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.422323 21047 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:29.232766 21185 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.676s	user 0.015s	sys 0.000s
I20260709 12:24:29.335526 21144 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:29.427824 21144 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.430406 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3916) took 1283 ms. Trace:
I20260709 12:24:29.176846 21382 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.175s
I20260709 12:24:29.231449 21036 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:29.168773 21142 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.403363 21412 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:29.434954 13648 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:29.371423 21413 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:29.436761 21413 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:29.438166 21413 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.364239 13643 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:29.409771 21436 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.411099 21369 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:29.412050 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 400006us
I20260709 12:24:29.376372 21418 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:29.370471 20975 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:24:29.414332 21395 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:29.431202 13793 rpcz_store.cc:276] 0709 12:24:28.146953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.147096 (+   143us) service_pool.cc:224] Handling call
0709 12:24:28.153853 (+  6757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.322776 (+168923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.322810 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:28.322873 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:28.322903 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.429972 (+1107069us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:29.430056 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.430312 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387983616}
W20260709 12:24:29.446089 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.476959 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:29.377022 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:29.228082 21380 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.148s
W20260709 12:24:29.475633 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134230us
W20260709 12:24:29.401471 21429 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
W20260709 12:24:29.479384 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 281630us
I20260709 12:24:29.495152 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:29.486212 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 109283us
I20260709 12:24:29.473507 21418 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:29.473642 21208 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:29.470714 21128 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:29.351042 21416 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.473919 21381 consensus_queue.cc:1243] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:24:29.497632 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:29.481045 21185 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:29.496868 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 319142us
I20260709 12:24:29.507084 13652 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:24:29.494067 21415 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:24:29.514935 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.507225 13651 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:29.523264 13645 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:29.497174 21420 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:24:29.523046 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:29.535775 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:29.538414 13649 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:29.540092 21409 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:29.540676 21409 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.541117 21409 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:29.543973 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1637) took 1063 ms. Trace:
I20260709 12:24:29.545135 13645 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:29.546447 21128 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:29.547181 21036 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:29.547722 21036 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:29.548138 13650 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:29.551126 21420 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:29.499429 21208 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:29.551985 21420 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:29.552716 21208 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.553831 21208 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:29.553226 21420 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.558684 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.499657 21418 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.559567 13649 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:29.571153 21248 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:29.578001 13801 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:24:29.582897 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106080us
I20260709 12:24:29.584659 13650 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:29.584782 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:29.588517 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:29.594280 13645 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:24:29.600031 13649 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:29.574385 21294 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:24:29.481519 21436 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.544431 13720 rpcz_store.cc:276] 0709 12:24:28.480793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.480957 (+   164us) service_pool.cc:224] Handling call
0709 12:24:28.489152 (+  8195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:28.534072 (+ 44920us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:28.591602 (+ 57530us) log.cc:844] Serialized 14 byte log entry
0709 12:24:28.852164 (+260562us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b18004708c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:29.178180 (+326016us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:29.428343 (+250163us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:24:29.428945 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:24:29.429123 (+   178us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:29.429181 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.429284 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.543743 (+114459us) raft_consensus.cc:1685] finished
0709 12:24:29.543769 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.543905 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181436,"prepare.queue_time_us":57328,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":9273,"raft.queue_time_us":157035,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":15511,"spinlock_wait_cycles":117314688,"thread_start_us":234174,"threads_started":1,"wal-append.queue_time_us":616091}
I20260709 12:24:29.612803 21438 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:29.621384 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:29.585638 21415 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:29.638975 21415 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.641733 21442 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.642633 21442 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:29.643596 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 69441.6us
I20260709 12:24:29.644227 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.645566 21442 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:29.647198 21416 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:29.648547 21426 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:24:29.650238 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.655728 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.656565 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.657572 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:29.596029 21369 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:29.658658 21369 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:29.659574 21369 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.660490 21176 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:29.660900 21176 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:29.660962 21369 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:29.661374 21176 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:29.649757 21416 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:29.649796 20975 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:29.662863 20975 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:29.664079 20975 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.668855 21142 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:29.670704 20975 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:29.591725 21198 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:29.672605 21198 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.674999 21198 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:29.675652 21249 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.676227 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:29.676904 14420 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:24:29.693629 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 72305us
I20260709 12:24:29.694239 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1428) took 1003 ms. Trace:
I20260709 12:24:29.694810 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:29.529471 21185 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.694659 14420 rpcz_store.cc:276] 0709 12:24:28.690552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.691372 (+   820us) service_pool.cc:224] Handling call
0709 12:24:29.676037 (+984665us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:24:29.694169 (+ 18132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16890112}
W20260709 12:24:29.698320 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 293524us
I20260709 12:24:29.634689 21430 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.716866 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:29.727349 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:29.728400 14417 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:29.733736 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.729714 21248 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:29.740422 21371 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.871s	user 0.013s	sys 0.001s
I20260709 12:24:29.694965 21198 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:24:29.707864 21426 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:24:29.760618 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:29.762593 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 228323us
I20260709 12:24:29.752400 21430 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:24:29.759454 21209 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.723s	user 0.007s	sys 0.007s
W20260709 12:24:29.774888 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 256254us
W20260709 12:24:29.779636 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 194957us
I20260709 12:24:29.783198 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:29.784507 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.786355 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:29.789249 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 143234us
I20260709 12:24:29.779954 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:29.787206 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:29.789100 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94404.1us
I20260709 12:24:29.792235 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:29.796094 13649 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:24:29.758049 21412 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:29.798566 14423 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:24:29.796649 20428 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:24:29.844386 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58187us
W20260709 12:24:29.837375 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57501us
W20260709 12:24:29.843858 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 83360.9us
I20260709 12:24:29.848275 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:29.849627 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:29.845500 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:29.850750 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.854589 13645 pending_rounds.cc:85] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 18
I20260709 12:24:29.855545 13645 pending_rounds.cc:107] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 25.19
I20260709 12:24:29.855960 13645 raft_consensus.cc:2889] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.777871 21142 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:29.802943 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.864836 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:29.874414 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:29.875366 13648 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:29.884250 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.885639 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.794800 21183 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:29.907552 13644 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:29.795953 21144 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:29.857107 20842 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:29.845839 21219 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:29.846807 21412 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:29.910200 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112829us
I20260709 12:24:29.845067 21430 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:29.865729 21128 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:29.830540 21371 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:29.908299 13722 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
W20260709 12:24:29.918298 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46001.5us
W20260709 12:24:29.880571 21425 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:29.871105 21177 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:29.878046 21142 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.906985 21437 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:29.875876 21066 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.924562 13725 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:29.924814 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:29.862361 21294 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:29.926469 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:29.790737 21208 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:24:29.926667 21412 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:29.928985 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.931149 13722 pending_rounds.cc:85] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 7
I20260709 12:24:29.971261 13722 pending_rounds.cc:107] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:24:29.971755 13722 raft_consensus.cc:2889] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.888839 21381 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:29.908977 21380 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:29.937392 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 102936us
I20260709 12:24:29.873771 21248 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:24:29.983258 13719 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:24:29.983054 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:29.860092 21418 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:29.880865 21047 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.923306 21392 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:24:29.972630 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:29.973508 21198 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:29.973735 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.927786 21453 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:29.977500 13796 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:29.939294 21177 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:29.929970 21128 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:29.992336 13797 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:24:30.027808 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:29.994887 13720 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:29.923309 21442 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:24:30.035969 21409 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.039583 21440 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.003191 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.040308 21440 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.041900 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.042430 21450 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:30.042905 21450 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:30.043921 21450 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.045270 21450 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:30.003572 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:29.926466 21395 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 12:24:30.048264 21444 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:24:29.929775 21441 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.316s
I20260709 12:24:29.945652 21445 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.119s
I20260709 12:24:30.016155 21144 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:30.050966 21463 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:24:30.058993 21144 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.059381 21142 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.059607 21144 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:30.062359 21436 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.051213 21464 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:30.062909 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.063611 21464 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:30.064085 21464 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.064188 13726 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:30.066437 21459 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.059s
I20260709 12:24:30.067236 21459 consensus_queue.cc:1243] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:24:30.067435 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.031078 21177 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:30.073549 21036 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.525s	user 0.012s	sys 0.000s
I20260709 12:24:30.057849 21128 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:30.053126 21435 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:30.078065 21294 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:30.047299 21451 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:24:29.926041 21458 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.117s
I20260709 12:24:29.991858 21381 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:29.933760 21430 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.083956 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.056681 13724 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:30.088629 21176 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.092484 21393 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:30.095381 21442 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:29.990635 21249 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.083734 13799 raft_consensus.cc:2376] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22. Re-sending same reply.
I20260709 12:24:30.101003 13722 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:30.103719 21209 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:30.104236 20862 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:30.104470 21443 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:30.106244 21047 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.112937 13726 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:24:30.118530 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 176889us
W20260709 12:24:30.154395 14391 leader_election.cc:336] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.120s (SENT)
I20260709 12:24:30.155032 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:29.970079 21371 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.030195 21437 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.143088 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.053215 21441 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:24:29.993526 21066 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.156852 13794 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:24:30.028031 21198 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.074097 21036 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.185994 21441 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:30.008499 21380 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.133422 21393 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:30.151305 20975 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.199744 13795 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:30.202400 20326 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:30.137943 21409 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.135874 21442 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:30.206768 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52410.9us
W20260709 12:24:30.215512 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.644s (SENT)
I20260709 12:24:30.215950 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.025281 21183 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.207479 13793 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:30.218809 21459 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.199242 21418 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.255064 20428 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:24:30.259313 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.216087 21185 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:30.266537 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 401796us
I20260709 12:24:30.219326 21440 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:30.278401 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.287882 21452 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:30.199721 21437 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.297648 20326 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.301579 20326 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.304639 13801 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
W20260709 12:24:30.338873 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 413177us
I20260709 12:24:30.350457 21389 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:30.351718 21389 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.353935 13646 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
I20260709 12:24:30.354480 13644 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:30.356357 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.357245 13648 raft_consensus.cc:2368] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21.
I20260709 12:24:30.357328 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:30.357343 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.358139 13643 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:30.359514 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.362305 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.362848 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.362324 21389 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.363788 13645 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:30.364622 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.365729 13646 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:30.368486 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:30.374794 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 508695us
I20260709 12:24:30.376518 21294 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:24:30.377403 21458 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:30.380744 13645 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:24:30.380831 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:30.382952 13644 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.383538 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.384195 13647 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
W20260709 12:24:30.395067 21413 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.011s	sys 0.035s
W20260709 12:24:30.395963 21413 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.011s	sys 0.035s
I20260709 12:24:30.267251 21445 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:30.402426 20326 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:30.233150 20975 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.408262 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:30.231539 21412 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.419613 13646 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:30.238446 21459 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.429124 13797 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:30.436972 13792 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:30.437721 13644 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:30.418633 21409 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:30.442440 13723 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:30.336932 21393 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.446066 20326 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:30.453874 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.455284 20326 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:30.459810 20326 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:30.450974 21381 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.424347 21442 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.448730 21409 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.448066 21462 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:24:30.464583 21036 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:30.466905 21393 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:30.501320 21471 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 12:24:30.502463 21467 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:24:30.536175 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76587.3us
I20260709 12:24:30.544998 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.549568 20428 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:30.550341 20428 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:30.550638 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.551327 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:30.476526 21430 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.552635 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.512256 21066 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.537902 21298 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:30.539776 21443 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:30.558772 21483 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:30.570725 21443 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.573407 21443 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.559077 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.574205 21437 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.560808 21414 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:30.554729 21442 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:30.561409 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:30.535156 21381 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.580369 21381 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.582409 14423 pending_rounds.cc:85] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 18
I20260709 12:24:30.582798 14423 pending_rounds.cc:107] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 25.19
I20260709 12:24:30.583042 14423 raft_consensus.cc:2889] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:30.583112 21460 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:30.583719 21460 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.584046 21460 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:30.584645 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:30.585461 13644 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:30.586658 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.587515 13651 raft_consensus.cc:2368] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21.
I20260709 12:24:30.588474 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.589165 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:24:30.567394 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:30.535712 21198 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:30.598767 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.599107 13796 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.000s
I20260709 12:24:30.602852 13796 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:30.539324 21380 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:30.639009 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:30.640609 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.605993 13726 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 14: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:30.642946 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.644300 13796 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:30.649855 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.652151 14391 leader_election.cc:400] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21.
I20260709 12:24:30.653317 14391 leader_election.cc:403] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:30.657618 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.654274 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:30.664870 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.665645 13796 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:30.652619 21047 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:30.678190 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.679335 13724 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:30.680153 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.681322 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.683914 21371 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:30.688980 21450 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.002s
I20260709 12:24:30.670135 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.693876 13722 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:30.671411 21462 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.696956 20428 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.671634 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:30.689496 21128 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.002s
I20260709 12:24:30.689918 14419 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:30.667110 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:30.648523 20975 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:30.698921 14419 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:30.701221 13718 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:24:30.691650 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.692163 13724 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:30.693131 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:30.703459 13726 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:30.695422 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.704089 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:30.668386 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.705015 13796 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.696813 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:30.655506 21412 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.699548 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.700569 13722 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:24:30.642676 21298 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.718528 21298 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.721199 21441 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.723567 14422 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:30.724927 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:30.726471 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:30.727684 21494 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:24:30.727795 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:30.728117 21144 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.668s	user 0.007s	sys 0.006s
I20260709 12:24:30.728514 14421 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:30.729221 14419 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.730402 14419 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:30.731014 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:30.732071 14425 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:30.732766 20975 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.733150 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:30.733304 20975 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.733785 14425 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:24:30.691902 21464 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.613s	user 0.014s	sys 0.000s
I20260709 12:24:30.707931 21497 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:30.714210 21430 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:30.734673 21464 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.711489 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.727721 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.740759 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:30.742916 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.749915 13796 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:30.750648 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:30.754433 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.758062 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.751751 21497 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:30.755811 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:30.737627 21177 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.004s
I20260709 12:24:30.760723 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:30.766587 20428 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.753394 21381 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:30.709702 21409 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.746800 21430 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:30.732071 21459 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.774642 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.782229 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.728618 21144 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.788312 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:30.760499 21418 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:30.789214 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.789263 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.757113 21464 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.708951 21128 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.767562 21219 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:30.768146 21441 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:24:30.776774 21120 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:30.777232 21497 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:30.787153 21248 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:30.791165 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.760607 21416 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:24:30.815395 13727 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:24:30.785284 21381 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.784966 21409 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:24:30.820331 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.802199 21482 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:30.790165 21371 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.769199 21177 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.805012 21219 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:30.813791 21120 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.831825 13800 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:30.833499 21177 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.806232 21298 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:30.826303 21219 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:30.835796 21128 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.835589 21298 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.825194 21371 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:30.838510 21371 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.838810 21298 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.823390 21294 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:24:30.821740 21411 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:24:30.840689 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:30.822857 21381 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:30.844875 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.845124 13644 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:30.846213 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.848481 13648 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:30.803292 21420 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:24:30.852737 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:30.855356 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.804287 21459 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:24:30.860806 14425 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:30.824342 21418 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:30.863130 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:30.876365 14421 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:30.831880 21120 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:30.880648 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:30.882131 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.860280 21499 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:30.887538 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 233343us
I20260709 12:24:30.888077 14417 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:24:30.889654 21369 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.228s	user 0.016s	sys 0.001s
I20260709 12:24:30.890583 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:30.891912 21496 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:24:30.898720 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:30.899700 14425 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:30.901640 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:30.898727 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.864820 21381 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:30.902701 13650 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:30.911449 14419 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:24:30.912938 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:30.918018 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:30.918969 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.919725 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.910351 21271 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:30.882467 21298 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:30.910916 21499 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.930367 21499 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.934978 21144 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:30.936937 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.937680 13646 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:30.938720 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.940255 13647 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:30.941398 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.943774 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.886325 21177 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:30.946805 21409 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:30.947033 21144 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.947450 21409 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.947608 21144 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.949784 21409 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.949853 21144 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:30.950796 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.952145 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:30.952736 13652 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
W20260709 12:24:30.919260 21498 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:24:30.954037 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:30.910524 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:30.959628 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49206.4us
I20260709 12:24:30.962231 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.962819 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:30.964366 21298 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:30.965454 21298 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.973157 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:30.976164 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.977380 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:30.978127 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.978789 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:30.979635 13796 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:24:30.962963 13795 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:30.984973 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:30.974196 14425 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:24:30.991210 21219 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:30.993517 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:31.000833 21409 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:31.002164 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:31.003219 13650 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:31.012833 21248 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:31.012964 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:30.913726 21466 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:31.013600 21248 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.020701 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:31.022843 13795 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 11
I20260709 12:24:31.027132 13795 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:24:31.027446 13795 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:31.025029 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:31.028442 14420 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:24:31.029479 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75530.7us
I20260709 12:24:30.957561 21381 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:30.956835 21294 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:30.957422 21177 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:31.083856 21177 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.050493 21381 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:31.056270 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:31.086585 21381 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:31.087217 21382 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:24:31.038479 20975 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.090855 20975 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:31.060166 13798 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:30.952266 21411 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 12:24:31.059440 21482 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:31.093223 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1436) took 1333 ms. Trace:
I20260709 12:24:31.071807 21176 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.029197 21248 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:31.090118 21439 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:31.095781 21439 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:31.096146 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:31.097600 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:30.921185 21271 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.101558 21271 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.104110 21466 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:31.104676 21466 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:31.105321 21219 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.078060 21294 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:31.107307 21294 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:31.110859 21412 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:31.091116 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3662) took 1029 ms. Trace:
I20260709 12:24:31.096400 21439 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:31.072487 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2665) took 1099 ms. Trace:
I20260709 12:24:31.025559 21369 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:31.069593 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2660) took 1098 ms. Trace:
I20260709 12:24:31.093681 14417 rpcz_store.cc:276] 0709 12:24:29.759379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.760218 (+   839us) service_pool.cc:224] Handling call
0709 12:24:29.780619 (+ 20401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.888517 (+1107898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.092917 (+204400us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:31.093132 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2546077440}
I20260709 12:24:31.104840 21462 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21.
I20260709 12:24:31.027998 21409 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:31.113967 13719 rpcz_store.cc:276] 0709 12:24:30.061866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.068748 (+  6882us) service_pool.cc:224] Handling call
0709 12:24:30.069087 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.148568 (+ 79481us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:30.348380 (+199812us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.372274 (+ 23894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.372334 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:30.372381 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.090847 (+718466us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b5400027bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:31.090897 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.091052 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81115,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":1005,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":181858688,"thread_start_us":169205,"threads_started":2,"wal-append.queue_time_us":16803}
I20260709 12:24:31.114552 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3650) took 1227 ms. Trace:
I20260709 12:24:31.121417 13801 rpcz_store.cc:276] 0709 12:24:29.972729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.067100 (+ 94371us) service_pool.cc:224] Handling call
0709 12:24:31.072327 (+1005227us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003d45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:31.072433 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337878912}
I20260709 12:24:31.132241 14418 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
W20260709 12:24:31.132925 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.891s (SENT) [suppressed 14 similar messages]
I20260709 12:24:31.133715 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.122694 13800 rpcz_store.cc:276] 0709 12:24:29.971220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.971362 (+   142us) service_pool.cc:224] Handling call
0709 12:24:30.016618 (+ 45256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.832293 (+815675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.069186 (+236893us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b5400103454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:31.069486 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873959680}
I20260709 12:24:31.137012 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.138936 13720 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:31.139792 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.156935 13722 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:24:31.054988 21120 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.162097 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.144515 13721 rpcz_store.cc:276] 0709 12:24:29.887146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.887352 (+   206us) service_pool.cc:224] Handling call
0709 12:24:29.921858 (+ 34506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.922105 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:30.014349 (+ 92244us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.089886 (+1075537us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b180045c340. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:24:31.090116 (+   230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.090175 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:31.090228 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.114357 (+ 24129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.114483 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":287415,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":67,"raft.queue_time_us":3873,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":10115456,"thread_start_us":280603,"threads_started":2,"wal-append.queue_time_us":329037}
I20260709 12:24:31.120831 21430 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:31.120759 21066 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:31.176121 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.159226 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:31.156328 21415 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.142792 21409 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.157855 21369 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:31.175450 21460 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.002s
I20260709 12:24:31.141389 21176 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:24:31.202253 13719 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 12. Current term is 13. Ops: []
W20260709 12:24:31.210920 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44169us
I20260709 12:24:31.214787 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:31.218112 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:31.181368 21066 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:31.190814 21409 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:31.189531 21271 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:31.190366 21369 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:31.264536 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2662) took 1292 ms. Trace:
I20260709 12:24:31.226472 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.271054 13799 rpcz_store.cc:276] 0709 12:24:29.971869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.971999 (+   130us) service_pool.cc:224] Handling call
0709 12:24:31.264320 (+1292321us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:31.264465 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228771584}
I20260709 12:24:31.265535 14425 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:24:31.272984 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:31.249403 21465 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.982s	user 0.002s	sys 0.000s
I20260709 12:24:31.202068 21379 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:24:31.256511 21120 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.224506 21482 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:24:31.243459 21497 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:31.258235 13719 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:31.266937 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:31.258706 21066 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:31.267650 21381 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.267944 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:31.294953 13725 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:24:31.296594 21381 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.268121 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3665) took 1205 ms. Trace:
I20260709 12:24:31.298370 13727 rpcz_store.cc:276] 0709 12:24:30.063057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.066189 (+  3132us) service_pool.cc:224] Handling call
0709 12:24:30.066656 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.816070 (+749414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.267813 (+451743us) spinlock_profiling.cc:243] Waited 748 ms on lock 0x7b54001563d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:31.268055 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1721221888}
I20260709 12:24:31.300086 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.261468 21442 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:31.301152 21442 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:31.261669 21414 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:31.278849 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:31.302896 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:31.291831 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158780us
I20260709 12:24:31.191603 21415 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.192003 21217 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:24:31.244526 21371 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:31.212797 21460 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:31.317358 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50601us
I20260709 12:24:31.368171 13794 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:24:31.332254 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.356638 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.317376 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:31.290809 21209 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:31.263759 21393 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.105s
I20260709 12:24:31.244308 21144 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
W20260709 12:24:31.392081 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.392620 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.397533 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:31.302533 21414 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:31.399660 13720 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:31.399963 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:31.403878 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.289717 21400 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:31.302021 21198 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:31.393301 21523 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:24:31.418853 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:31.420338 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:31.421496 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:31.275565 21248 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.424875 21248 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.425668 13724 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:31.425621 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:31.374787 20975 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:31.414346 21198 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:31.426434 20975 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:31.426896 20975 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:31.412215 14416 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:31.422115 14417 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:24:31.437022 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:31.403452 21510 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:24:31.478394 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:31.490696 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:31.395649 21371 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:31.310846 21185 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:31.428529 21198 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:31.480358 21443 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.513242 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:31.404418 21209 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.487929 21066 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:31.493031 21426 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:31.493001 21380 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:31.519816 14417 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:31.515180 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.536736 13725 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:31.561380 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:31.568724 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.492729 21219 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:24:31.514492 21434 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:31.428960 21442 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.543035 21380 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.603753 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:31.606487 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:24:31.599206 13719 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:31.567893 21185 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:31.598869 14420 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.563s	user 0.014s	sys 0.000s
I20260709 12:24:31.643482 14420 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:31.599294 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.549211 21443 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.610318 21380 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:31.631945 21466 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:24:31.627373 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.653949 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:31.655791 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:31.658629 14420 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:31.656586 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:31.660969 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3656) took 1733 ms. Trace:
I20260709 12:24:31.661816 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:31.563911 21414 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:31.612207 21442 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:31.661427 13718 rpcz_store.cc:276] 0709 12:24:29.927942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.934181 (+  6239us) service_pool.cc:224] Handling call
0709 12:24:29.934536 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.708567 (+774031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.660681 (+952114us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b540021fcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:31.660896 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760834816}
I20260709 12:24:31.666476 21219 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:24:31.667047 21525 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:24:31.668588 13794 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:24:31.669024 13794 pending_rounds.cc:85] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 10
I20260709 12:24:31.669281 13794 pending_rounds.cc:107] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:24:31.666890 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:31.669605 13794 raft_consensus.cc:2889] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:31.669653 14420 pending_rounds.cc:85] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 11
I20260709 12:24:31.670063 14420 pending_rounds.cc:107] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:24:31.665587 21447 raft_consensus.cc:2764] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:24:31.604115 21434 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.692055 21443 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.668159 21298 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:24:31.693483 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:31.663580 21381 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:31.676474 21514 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:24:31.679450 21415 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:31.708406 21460 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:31.719094 21371 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:31.720083 21464 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:31.720580 21464 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.720955 21464 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:31.698199 21047 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:31.728837 21047 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.730626 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:31.741520 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:31.742197 13720 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:31.677800 14420 raft_consensus.cc:2889] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:31.667084 21219 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:31.684725 21439 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.005s
W20260709 12:24:31.698450 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.03134e+06us
I20260709 12:24:31.744985 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:31.747924 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:31.737851 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:31.779073 13795 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
W20260709 12:24:31.780622 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.784114 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46417.5us
I20260709 12:24:31.784945 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:31.786736 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:31.796497 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:31.800086 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:31.817557 13793 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:31.822011 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:31.662102 21466 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:31.785259 21414 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:31.798487 21443 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:31.846174 21128 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:31.846767 21128 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:31.849938 21481 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.818671 21066 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:31.862869 21219 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:31.865269 21369 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.001s
I20260709 12:24:31.869464 21443 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:31.838424 21298 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:31.785004 21439 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:31.875535 21409 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.000s
W20260709 12:24:32.066339 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 404716us
I20260709 12:24:31.882594 13725 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:31.884775 14417 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:31.808020 21389 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:31.837103 21437 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.837602 21472 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:31.893957 21128 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:32.127122 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1161) took 1100 ms. Trace:
I20260709 12:24:32.141513 14425 rpcz_store.cc:276] 0709 12:24:31.026149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.026334 (+   185us) service_pool.cc:224] Handling call
0709 12:24:31.026671 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.273158 (+246487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.125757 (+852599us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400555fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:32.126753 (+   996us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548416896}
I20260709 12:24:32.144063 13795 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:32.106925 21434 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:32.116350 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:32.189944 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 121603us
I20260709 12:24:32.191979 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2680) took 1175 ms. Trace:
W20260709 12:24:32.196302 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 324367us
W20260709 12:24:32.198489 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 870232us
W20260709 12:24:32.198843 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 865006us
I20260709 12:24:32.200043 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1657) took 1026 ms. Trace:
I20260709 12:24:31.897959 21066 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:24:32.206359 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.192484 13796 rpcz_store.cc:276] 0709 12:24:31.016870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.017204 (+   334us) service_pool.cc:224] Handling call
0709 12:24:31.264485 (+247281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.264885 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.264915 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.265413 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.360127 (+ 94714us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:31.391859 (+ 31732us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.392137 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.392218 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.674933 (+282715us) raft_consensus.cc:1685] finished
0709 12:24:31.674953 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.191742 (+516789us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54003c0850. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:32.191899 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13687,"prepare.queue_time_us":34352,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":8060,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":7442,"spinlock_wait_cycles":567394048,"thread_start_us":94196,"threads_started":1,"wal-append.queue_time_us":338730}
I20260709 12:24:32.084571 21298 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.084796 21217 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.319049 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 119272us
I20260709 12:24:32.087770 21533 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.324646 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1158) took 1742 ms. Trace:
I20260709 12:24:32.325215 14416 rpcz_store.cc:276] 0709 12:24:30.582222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.634554 (+ 52332us) service_pool.cc:224] Handling call
0709 12:24:30.651146 (+ 16592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.429013 (+777867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.324316 (+895303us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:32.324566 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749772672}
I20260709 12:24:32.197306 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:32.200523 13721 rpcz_store.cc:276] 0709 12:24:31.173737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.173977 (+   240us) service_pool.cc:224] Handling call
0709 12:24:31.188603 (+ 14626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.188878 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:31.306438 (+117560us) log.cc:844] Serialized 14 byte log entry
0709 12:24:32.199683 (+893245us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.199739 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:32.199788 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.199869 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.199982 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":815761,"prepare.queue_time_us":339168,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":192633,"threads_started":2,"wal-append.queue_time_us":308471}
W20260709 12:24:32.204151 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 369243us
W20260709 12:24:32.322922 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86141.1us
I20260709 12:24:32.328161 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:32.215677 20975 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.005s
I20260709 12:24:32.204459 21497 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:31.899188 21481 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.367009 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:32.362867 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1201) took 1477 ms. Trace:
I20260709 12:24:32.365240 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:32.071662 21369 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.088443 21409 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.216306 21066 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:32.378063 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 163053us
W20260709 12:24:32.376055 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.372499 14421 rpcz_store.cc:276] 0709 12:24:30.885307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.885472 (+   165us) service_pool.cc:224] Handling call
0709 12:24:30.915264 (+ 29792us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.953788 (+ 38524us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.953821 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.953866 (+    45us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:30.953896 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.362530 (+1408634us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:32.362613 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.362782 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88118656}
I20260709 12:24:32.383524 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:32.320742 21380 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.000s
I20260709 12:24:32.190645 21389 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.322546 21382 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:32.380745 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60612.4us
I20260709 12:24:32.329993 21549 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:32.318329 21298 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:32.373680 21177 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:32.393481 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:32.407687 13727 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:32.230912 21447 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.170s
I20260709 12:24:32.335561 21395 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.205068 21437 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.317935 21472 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.428244 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:32.379997 21393 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:32.382462 21544 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:32.339447 21533 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.334765 21217 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.448216 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:32.450145 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:32.465432 13648 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:32.379734 21534 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:24:32.468425 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:32.468813 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:32.409030 21548 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:32.470194 14425 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:24:32.377125 21545 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:24:32.380827 20975 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.477934 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:32.477919 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 280679us
I20260709 12:24:32.481737 14416 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:32.398275 21185 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:32.479563 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:32.498703 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:32.335403 21443 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.456513 21393 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.485971 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:32.551813 13798 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:32.396142 21380 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.456954 21371 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:24:32.485700 21546 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:32.402899 21382 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.447764 21395 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.589607 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:32.593078 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3689) took 1010 ms. Trace:
I20260709 12:24:32.593794 13721 rpcz_store.cc:276] 0709 12:24:31.583477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.429773 (+846296us) service_pool.cc:224] Handling call
0709 12:24:32.592322 (+162549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.592594 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.592639 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.592704 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:32.592751 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.592827 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.592995 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:32.596558 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3690) took 1012 ms. Trace:
I20260709 12:24:32.433914 21549 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.646113 13721 rpcz_store.cc:276] 0709 12:24:31.583889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.595447 (+1011558us) service_pool.cc:224] Handling call
0709 12:24:32.595890 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.596127 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.596165 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:32.596225 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:32.596271 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.596341 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.596484 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:32.564719 21466 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:32.566640 21393 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:32.624600 20326 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.729709 20326 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.469815 21544 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.498068 21548 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:32.800582 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 345208us
I20260709 12:24:32.792908 20326 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:32.806535 13650 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:32.771139 21439 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:32.737720 21553 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:32.591698 21546 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.498528 21497 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:32.816390 13649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.003s	sys 0.010s
W20260709 12:24:32.835918 13649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.003s	sys 0.010s
W20260709 12:24:32.839074 21538 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:24:32.840958 21537 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 12:24:32.842219 21536 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:32.842876 21536 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.847185 21217 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:32.848706 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4151) took 2005 ms. Trace:
I20260709 12:24:32.849097 13643 rpcz_store.cc:276] 0709 12:24:30.843221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.843355 (+   134us) service_pool.cc:224] Handling call
0709 12:24:30.843702 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.725258 (+881556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.725298 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.725849 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.876654 (+150805us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:31.876708 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.876788 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.848470 (+971682us) raft_consensus.cc:1685] finished
0709 12:24:32.848491 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:32.848640 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":881164,"raft.queue_time_us":882801,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":65,"thread_start_us":150231,"threads_started":1,"wal-append.queue_time_us":807281}
I20260709 12:24:32.851568 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:32.833199 21371 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:32.848145 21389 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:32.848145 21533 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:32.853463 21409 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:32.845412 21536 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:32.868522 21481 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:32.881414 21414 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.039s	user 0.012s	sys 0.000s
W20260709 12:24:32.887343 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 308262us
I20260709 12:24:32.885753 21185 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:24:32.879621 21535 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
W20260709 12:24:32.887969 21464 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.164s	user 0.015s	sys 0.000s
I20260709 12:24:32.940608 20975 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:32.902374 21472 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:32.954852 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 465649us
W20260709 12:24:32.955957 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 152199us
I20260709 12:24:32.885262 21371 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.904117 21248 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:32.954177 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.008s	sys 0.013s
I20260709 12:24:32.960075 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:32.940474 21389 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:24:32.962677 14418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1163) took 1899 ms (client timeout 2367 ms). Trace:
I20260709 12:24:32.938824 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:32.968310 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.008s	sys 0.013s
W20260709 12:24:32.955493 21442 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.278s	user 0.013s	sys 0.000s
I20260709 12:24:32.949514 21549 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:32.955897 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:32.973847 14418 rpcz_store.cc:269] 0709 12:24:31.055966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.097334 (+ 41368us) service_pool.cc:224] Handling call
0709 12:24:32.954732 (+1857398us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b54007a0154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:32.955392 (+   660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77939072}
I20260709 12:24:33.026885 21561 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.027730 21561 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:32.940526 21464 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:32.960956 21380 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:32.956487 21553 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:33.036749 13792 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:24:32.936609 21185 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:33.068104 13765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.003s	sys 0.018s
W20260709 12:24:33.041404 13722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.005s	sys 0.009s
I20260709 12:24:33.040014 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4175) took 1310 ms. Trace:
W20260709 12:24:33.069387 13722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.005s	sys 0.009s
W20260709 12:24:33.070802 13765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.003s	sys 0.018s
I20260709 12:24:33.069460 13649 rpcz_store.cc:276] 0709 12:24:31.729265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.733967 (+  4702us) service_pool.cc:224] Handling call
0709 12:24:31.734300 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.734540 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:24:32.136640 (+402100us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.039594 (+902954us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.039651 (+    57us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:33.039710 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.039792 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.039950 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126028,"prepare.queue_time_us":692458,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":121501,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":4560,"thread_start_us":1269320,"threads_started":3,"wal-append.queue_time_us":780213}
I20260709 12:24:32.971906 21248 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:33.034680 21550 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:24:33.040171 13798 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:24:32.973965 21389 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:32.971133 21395 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:32.971359 21563 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:33.081378 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 199491us
I20260709 12:24:32.978688 21409 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:33.070011 21551 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:24:33.071229 21420 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.037933 21566 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:33.067564 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 111772us
I20260709 12:24:33.097506 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:33.077912 21472 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:33.100699 21560 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.097057 21420 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.101284 21371 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.082324 21544 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.081926 21437 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.101462 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.145198 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:33.138679 21481 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:33.147100 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 73581.6us
I20260709 12:24:33.150220 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:33.151706 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56777.7us
W20260709 12:24:33.154431 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57040.1us
I20260709 12:24:33.156447 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:33.112300 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 173562us
I20260709 12:24:33.158717 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:33.161792 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:33.163576 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:33.174893 21539 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
I20260709 12:24:33.141086 21548 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.176786 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4174) took 1447 ms. Trace:
I20260709 12:24:33.177026 21546 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.092394 21409 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.153364 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:33.177287 13652 rpcz_store.cc:276] 0709 12:24:31.728955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.731942 (+  2987us) service_pool.cc:224] Handling call
0709 12:24:31.732342 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:31.732775 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.732803 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.733429 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.136106 (+402677us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:32.136155 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.136240 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.176495 (+1040255us) raft_consensus.cc:1685] finished
0709 12:24:33.176519 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.176713 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43916,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":18717,"thread_start_us":402234,"threads_started":1,"wal-append.queue_time_us":1008464}
I20260709 12:24:33.105399 21566 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.179502 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:33.179811 13652 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:24:33.181144 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:33.103992 21472 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.182565 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.183452 13719 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:33.184561 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.103732 21430 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:33.185285 13719 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:33.136552 20975 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.133490 21561 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.101346 21560 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.136281 21249 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:33.215266 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.217036 13719 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
W20260709 12:24:33.239457 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86187us
I20260709 12:24:33.149253 21371 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:33.244176 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.097365 21563 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.182799 21573 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.202131 21472 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:33.273546 13767 connection.cc:256] RPC call timeout handler was delayed by -1.1443989690001217s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:33.202554 21536 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.147446 21481 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.281766 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.283919 13647 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:24:33.284409 13797 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:33.281900 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.285356 13798 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:33.222731 21571 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.282701 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:33.293926 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 135288us
I20260709 12:24:33.294984 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:33.239065 21548 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:33.221972 21587 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:33.300541 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.209012 21544 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.067s
W20260709 12:24:33.277154 21128 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.072s	user 0.010s	sys 0.007s
I20260709 12:24:33.242357 21545 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:33.213953 21437 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:24:33.231062 21249 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.292800 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:33.234154 20975 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.299731 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:33.320456 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1660) took 2119 ms (client timeout 1603 ms). Trace:
I20260709 12:24:33.311324 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.321615 13718 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:24:33.270915 21430 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.297386 21482 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:24:33.330318 13720 rpcz_store.cc:269] 0709 12:24:31.203709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.217764 (+ 14055us) service_pool.cc:224] Handling call
0709 12:24:33.320232 (+2102468us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54000285d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.320375 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415108480}
I20260709 12:24:33.149647 21389 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.314870 21546 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:33.323694 21439 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:33.328784 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:33.345503 21464 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:33.324993 21562 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
W20260709 12:24:33.345654 21565 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 12:24:33.347580 21560 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.348416 21572 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.312495 21571 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.348929 21589 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.222594 21553 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.318535 21566 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.353991 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3693) took 1709 ms. Trace:
I20260709 12:24:33.324199 21459 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.329246 21558 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:33.360145 21545 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.272251 21371 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.333498 21559 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.284953 21573 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.344965 21561 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:33.343451 21430 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.392434 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2646) took 1054 ms. Trace:
I20260709 12:24:33.332298 21128 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.298148 21536 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.387842 21389 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:33.485097 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3671) took 2226 ms. Trace:
I20260709 12:24:33.486853 21577 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:33.487774 21577 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:33.443810 21543 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.911s	user 0.005s	sys 0.000s
I20260709 12:24:33.491848 21434 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:33.492311 21434 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:33.492684 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1167) took 2165 ms. Trace:
I20260709 12:24:33.493227 21434 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.493067 14420 rpcz_store.cc:276] 0709 12:24:31.327289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.669023 (+341734us) service_pool.cc:224] Handling call
0709 12:24:31.669461 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.800723 (+131262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.800777 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.812597 (+ 11820us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.203829 (+391232us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:32.203878 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.203955 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.353495 (+1149540us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:33.492323 (+138828us) raft_consensus.cc:1685] finished
0709 12:24:33.492350 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.492612 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34331520,"thread_start_us":390762,"threads_started":1,"wal-append.queue_time_us":635223}
I20260709 12:24:33.489926 21420 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:33.498616 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4188) took 1384 ms (client timeout 1555 ms). Trace:
I20260709 12:24:33.499383 21571 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.503726 21599 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.504662 21599 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:33.505404 21066 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.103s	user 0.011s	sys 0.003s
W20260709 12:24:33.507944 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3676) took 2247 ms (client timeout 2148 ms). Trace:
W20260709 12:24:33.508373 13726 rpcz_store.cc:269] 0709 12:24:31.260387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.402902 (+142515us) service_pool.cc:224] Handling call
0709 12:24:33.507722 (+2104820us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b54003fe3d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.507864 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4443618048}
I20260709 12:24:33.485558 13718 rpcz_store.cc:276] 0709 12:24:31.258935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.672586 (+413651us) service_pool.cc:224] Handling call
0709 12:24:33.321360 (+1648774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.330941 (+  9581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.484862 (+153921us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54000285d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.485018 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3790948608}
I20260709 12:24:33.515035 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:33.327111 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:33.368224 21546 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:33.519181 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:33.520474 14422 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:24:33.390316 13721 rpcz_store.cc:276] 0709 12:24:31.644220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.647829 (+1003609us) service_pool.cc:224] Handling call
0709 12:24:32.773735 (+125906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.979518 (+205783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.979574 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.035178 (+ 55604us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.261355 (+226177us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:33.261418 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.261534 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.353714 (+ 92180us) raft_consensus.cc:1685] finished
0709 12:24:33.353739 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.353924 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12533,"raft.queue_time_us":197451,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":62939,"thread_start_us":379226,"threads_started":2,"wal-append.queue_time_us":284089}
I20260709 12:24:33.392978 21589 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.393293 21248 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.387079 21464 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:33.443965 21581 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.448468 13646 rpcz_store.cc:276] 0709 12:24:32.338536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.338720 (+   184us) service_pool.cc:224] Handling call
0709 12:24:32.339136 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:32.867310 (+528174us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:32.867350 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:32.876676 (+  9326us) log.cc:844] Serialized 52 byte log entry
0709 12:24:33.086924 (+210248us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:33.087064 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.087194 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.392177 (+304983us) raft_consensus.cc:1685] finished
0709 12:24:33.392206 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.392369 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39701,"raft.queue_time_us":517629,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":15456,"thread_start_us":633780,"threads_started":2,"wal-append.queue_time_us":455520}
I20260709 12:24:33.367899 21439 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.464624 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:33.464830 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 165142us
I20260709 12:24:33.446758 21185 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:33.467505 21568 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:33.408633 21558 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.469470 21563 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.397231 21459 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.385553 21572 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.472651 21128 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.473217 21462 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:33.477105 21430 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.480640 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3682) took 2085 ms. Trace:
W20260709 12:24:33.480954 21578 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:24:33.481513 21584 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:33.499082 13651 rpcz_store.cc:269] 0709 12:24:32.115245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.115903 (+   658us) service_pool.cc:224] Handling call
0709 12:24:33.498401 (+1382498us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400096854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.498545 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686812544}
I20260709 12:24:33.531286 13726 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:24:33.533631 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 354210us
I20260709 12:24:33.534788 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:33.649883 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:33.536082 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 219499us
I20260709 12:24:33.505901 21066 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.545077 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:33.642205 13722 rpcz_store.cc:276] 0709 12:24:31.395039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.198876 (+803837us) service_pool.cc:224] Handling call
0709 12:24:32.199348 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.199824 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:32.364712 (+164888us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.078439 (+713727us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.079039 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.079194 (+   155us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:33.079418 (+   224us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.079600 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.079695 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.480366 (+400671us) raft_consensus.cc:1685] finished
0709 12:24:33.480392 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.480567 (+   175us) inbound_call.cc:177] Queueing success response
0709 12:24:33.588097 (+107530us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b18003a9cc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
Metrics: {"mutex_wait_us":48994,"prepare.queue_time_us":154710,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":2775,"raft.queue_time_us":291771,"raft.run_cpu_time_us":540,"raft.run_wall_time_us":60150,"spinlock_wait_cycles":63822208,"thread_start_us":661752,"threads_started":1,"wal-append.queue_time_us":994775}
I20260709 12:24:33.561825 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:33.562680 21380 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.564174 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:33.651842 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.564503 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:33.652745 14425 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:33.571077 21441 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:33.578647 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1649) took 2445 ms (client timeout 1625 ms). Trace:
W20260709 12:24:33.660287 13724 rpcz_store.cc:269] 0709 12:24:31.133629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.134198 (+   569us) service_pool.cc:224] Handling call
0709 12:24:33.578234 (+2444036us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.578517 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604023168}
I20260709 12:24:33.585515 21209 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.592177 21382 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.602052 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.681226 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:33.604898 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2645) took 3203 ms. Trace:
W20260709 12:24:33.613153 21530 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.968s	user 0.022s	sys 0.029s
W20260709 12:24:33.532119 21565 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.119s	user 0.000s	sys 0.004s
W20260709 12:24:33.533135 21579 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.330s	user 0.000s	sys 0.006s
I20260709 12:24:33.615516 13721 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:33.502240 21369 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.718940 21369 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:33.719571 21369 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:33.720067 21369 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:33.620635 21581 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:33.692358 21575 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:24:33.536176 21409 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.537904 21464 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.460253 21545 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.637123 21371 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:33.771049 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 121311us
I20260709 12:24:33.772308 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:33.730540 21439 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:33.731986 21558 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:33.645345 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 318280us
I20260709 12:24:33.749799 21573 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.413136 21559 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.785509 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3668) took 2563 ms. Trace:
I20260709 12:24:33.558276 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1292) took 1972 ms. Trace:
I20260709 12:24:33.691855 21441 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.623417 21185 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.618847 21568 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.641793 21462 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.683205 13645 rpcz_store.cc:276] 0709 12:24:30.401798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.402020 (+   222us) service_pool.cc:224] Handling call
0709 12:24:30.842363 (+440343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.723877 (+881514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.723920 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.724535 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.840835 (+116300us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:31.840879 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.840955 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:32.853918 (+1012963us) raft_consensus.cc:1685] finished
0709 12:24:32.853952 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.604367 (+750415us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400095e50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.604799 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":881032,"raft.queue_time_us":883789,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":149,"spinlock_wait_cycles":1011696896,"thread_start_us":115857,"threads_started":1,"wal-append.queue_time_us":647790}
I20260709 12:24:33.801012 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.801194 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2687) took 1333 ms. Trace:
I20260709 12:24:33.712882 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1293) took 2126 ms. Trace:
I20260709 12:24:33.801642 13792 rpcz_store.cc:276] 0709 12:24:32.467911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.468073 (+   162us) service_pool.cc:224] Handling call
0709 12:24:33.800906 (+1332833us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54002cbfd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.801119 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1288823552}
I20260709 12:24:33.713996 21599 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.801853 13799 rpcz_store.cc:276] 0709 12:24:31.586525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.586639 (+   114us) service_pool.cc:224] Handling call
0709 12:24:31.595190 (+  8551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.685724 (+ 90534us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:31.685765 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:31.685816 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:31.685852 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.712528 (+2026676us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:33.712589 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.712763 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207635328}
I20260709 12:24:33.713874 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:33.715437 21298 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.275s	user 0.013s	sys 0.006s
I20260709 12:24:33.806865 21298 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.715867 21589 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.680907 21380 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.813012 21380 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.814468 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:33.816010 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:33.747429 13726 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:33.633679 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.819201 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.654296 21602 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.658804 21430 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:33.636106 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:33.761844 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:33.762715 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 105687us
I20260709 12:24:33.680505 21584 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.761875 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:33.641872 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 313175us
I20260709 12:24:33.776404 14418 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:24:33.778788 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 233847us
I20260709 12:24:33.735060 21066 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:33.616420 21434 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:33.726899 21577 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.748301 21481 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.765882 21464 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:33.783291 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:33.547834 21548 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.654960 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.786165 13723 rpcz_store.cc:276] 0709 12:24:31.222488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.222707 (+   219us) service_pool.cc:224] Handling call
0709 12:24:31.223077 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.223438 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.223467 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.224120 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.493803 (+269683us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:31.493860 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.493958 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.612091 (+2118133us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:33.780346 (+168255us) raft_consensus.cc:1685] finished
0709 12:24:33.780390 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.785400 (+  5010us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1834,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":25828,"spinlock_wait_cycles":6742912,"thread_start_us":223425,"threads_started":1,"wal-append.queue_time_us":342283}
I20260709 12:24:33.788851 13794 rpcz_store.cc:276] 0709 12:24:31.586199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.586334 (+   135us) service_pool.cc:224] Handling call
0709 12:24:31.605692 (+ 19358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.670128 (+ 64436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.557906 (+1887778us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:33.558196 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143665536}
I20260709 12:24:33.854068 13722 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:33.773859 20975 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:33.858217 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75039.8us
I20260709 12:24:33.800045 21462 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:33.783775 21128 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:33.830780 14416 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:33.809990 21298 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:33.698665 21546 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:33.855971 21472 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 12:24:33.791076 21603 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:33.818817 21568 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:33.856649 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 299718us
W20260709 12:24:33.857043 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84859.5us
I20260709 12:24:33.860262 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2656) took 1482 ms. Trace:
I20260709 12:24:33.861195 21548 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.866595 21558 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:33.823909 21536 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.869361 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:33.869361 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:33.870009 13725 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:33.867062 21589 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:24:33.867084 21459 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.826807 21430 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:33.878609 21393 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.098s	user 0.017s	sys 0.000s
I20260709 12:24:33.906144 21393 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.907462 13801 raft_consensus.cc:2368] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21.
I20260709 12:24:33.908874 21393 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:33.887184 13726 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:33.888110 21548 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.885178 13652 rpcz_store.cc:276] 0709 12:24:32.377505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.180735 (+803230us) service_pool.cc:224] Handling call
0709 12:24:33.181085 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.181434 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.181467 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.221165 (+ 39698us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.517469 (+296304us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:33.517512 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.517577 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.860029 (+342452us) raft_consensus.cc:1685] finished
0709 12:24:33.860055 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.860203 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15484,"raft.queue_time_us":57883,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":22366,"thread_start_us":217622,"threads_started":1,"wal-append.queue_time_us":431287}
I20260709 12:24:33.911954 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:33.834092 21602 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.887434 13724 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:33.915925 21602 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.916707 13646 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:33.918586 21472 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.919384 13726 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:24:33.891875 21545 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.866602 21441 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:33.877197 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2373283729994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
I20260709 12:24:33.921854 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:33.923492 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:33.923782 21559 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.924981 13651 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:33.926640 21589 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:33.927768 21589 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:33.928395 21589 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:33.929436 21589 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.841578 21598 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:33.841693 21407 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:33.931046 21598 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:33.931272 21407 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.933499 21598 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.844606 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:33.920791 21472 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.819132 21185 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:33.913264 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.876772 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:33.895147 13727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3681) took 2602 ms (client timeout 1729 ms). Trace:
I20260709 12:24:33.896180 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2686) took 1428 ms. Trace:
W20260709 12:24:33.946302 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69596.5us
W20260709 12:24:33.946084 13727 rpcz_store.cc:269] 0709 12:24:31.292995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.299697 (+  6702us) service_pool.cc:224] Handling call
0709 12:24:33.894908 (+2595211us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002f5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.895064 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2537254784}
I20260709 12:24:33.905033 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.948130 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:33.905541 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:33.904100 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2654) took 1534 ms (client timeout 2010 ms). Trace:
W20260709 12:24:33.950513 13650 rpcz_store.cc:269] 0709 12:24:32.369836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.447882 (+ 78046us) service_pool.cc:224] Handling call
0709 12:24:33.903915 (+1456033us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54002b3ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:33.904029 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786433920}
I20260709 12:24:33.952085 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.952837 13650 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:33.953756 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:33.884708 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.902021 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:33.935523 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 91054us
I20260709 12:24:33.886390 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:33.901253 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:33.938126 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:33.957427 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.958148 13801 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:33.961129 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:33.962159 13794 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:33.962479 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.962659 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.963342 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.963600 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.963960 13644 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:33.964903 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.965533 13644 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
W20260709 12:24:33.968284 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82003.5us
I20260709 12:24:33.969072 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:33.970896 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:33.972286 13792 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 12:24:33.967479 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66300.1us
I20260709 12:24:33.955562 14422 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:24:33.976866 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:33.977468 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:33.979475 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:33.979674 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:33.980569 13799 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:33.946512 13795 rpcz_store.cc:276] 0709 12:24:32.467529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.467702 (+   173us) service_pool.cc:224] Handling call
0709 12:24:32.468126 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.539353 (+ 71227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.539405 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.548177 (+  8772us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.041151 (+492974us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:33.213743 (+172592us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.215030 (+  1287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.351277 (+136247us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b1800323200. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:24:33.895847 (+544570us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b5400103454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:33.895906 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.895965 (+    59us) raft_consensus.cc:1685] finished
0709 12:24:33.895980 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.896118 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19790,"prepare.queue_time_us":371441,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":53250,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":36225,"spinlock_wait_cycles":61614592,"thread_start_us":581189,"threads_started":2,"wal-append.queue_time_us":827629}
I20260709 12:24:33.983677 13651 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:33.899230 21430 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:33.954437 13650 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:33.989604 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:33.992838 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:33.934231 21407 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:34.002980 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.003857 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.897876 21249 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.986780 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:33.892091 21584 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.892804 21572 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:33.903234 21566 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:33.994895 14420 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:33.980834 21545 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:24:33.995187 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:33.984133 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:33.987346 13644 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:33.996235 13618 leader_election.cc:400] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21.
I20260709 12:24:33.995474 21602 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:34.027195 21573 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:34.027554 21602 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.027912 21573 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.028527 21573 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:33.974463 21379 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:34.030483 21379 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:33.998407 20975 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:34.030920 21379 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:33.994654 21548 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.031785 21548 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.034958 21560 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:34.035389 21560 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:34.036743 21389 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 12:24:33.900473 21581 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.018570 21572 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.037756 21572 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:34.038239 21572 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.037540 21548 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:33.998576 21577 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:33.999562 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.007793 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:33.985574 21393 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:33.943253 21481 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:33.978380 21472 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:34.010905 21414 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:24:34.017349 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:34.048424 13795 raft_consensus.cc:2368] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:34.020906 21395 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:34.009778 21566 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:34.023942 21185 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:34.026405 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2649) took 1686 ms. Trace:
I20260709 12:24:34.026995 21545 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:34.028126 21602 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:34.004506 13792 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:34.015588 21407 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:24:34.017298 13618 leader_election.cc:403] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:34.036248 21560 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.005079 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.016881 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.014622 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:34.062952 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48430.8us
I20260709 12:24:33.996737 13646 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:34.059217 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:34.003511 21603 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.007629 21128 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.051908 13649 rpcz_store.cc:276] 0709 12:24:32.339642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.072969 (+733327us) service_pool.cc:224] Handling call
0709 12:24:33.073491 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.364805 (+291314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.364860 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.365932 (+  1072us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.648602 (+282670us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:33.648652 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.735799 (+ 87147us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:33.735855 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.026013 (+290158us) raft_consensus.cc:1685] finished
0709 12:24:34.026040 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.026328 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64214,"raft.queue_time_us":157778,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":17595,"spinlock_wait_cycles":253896448,"thread_start_us":342807,"threads_started":2,"wal-append.queue_time_us":588069}
I20260709 12:24:34.067277 14425 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:34.067535 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.053470 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
W20260709 12:24:34.055416 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65954.1us
I20260709 12:24:34.050436 21393 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.066241 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:34.069381 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61708.9us
I20260709 12:24:34.069768 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.063766 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:34.067374 13798 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:34.061331 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 65255us
I20260709 12:24:34.070843 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:34.016211 21249 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.074817 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:34.075259 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:34.079185 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:34.091702 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.056763 21407 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.073808 21393 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.076568 21481 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:34.076792 21249 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.050962 21566 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.074833 21128 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.121860 21558 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.049738 21430 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.093020 21584 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:24:34.077939 21414 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:34.217010 21407 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:34.172945 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.174275 21371 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.068825 21472 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:24:34.174969 21369 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.052615 21545 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.182646 21501 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:34.182782 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.183476 13618 leader_election.cc:400] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:34.175019 21573 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.089044 13649 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 15: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:34.070101 21524 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.190168 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.191291 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.263473 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.264689 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.265326 13724 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:34.265065 21572 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.191706 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.266736 13718 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:34.236848 21568 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.267663 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.268930 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.197784 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.271298 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.271960 13726 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:34.272782 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.199847 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.274619 13725 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:34.200773 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.201727 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:33.105346 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:34.188797 21208 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:34.206395 14422 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:24:34.211808 14419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1203) took 3239 ms (client timeout 2712 ms). Trace:
I20260709 12:24:34.213055 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1665) took 1170 ms. Trace:
I20260709 12:24:34.210639 13644 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
W20260709 12:24:34.278688 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 222982us
I20260709 12:24:34.216156 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.279541 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.270216 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:34.279577 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.080587 21271 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:34.172103 21560 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:34.225764 21249 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.294147 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.201228 21462 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.227830 21558 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.206005 21464 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.255357 13618 leader_election.cc:403] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:24:34.256898 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.298480 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:34.257086 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.260552 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.275925 13727 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:24:34.087815 21603 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.277086 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.278005 14418 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:34.225805 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 154029us
I20260709 12:24:34.281680 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:34.262032 13723 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 13.
I20260709 12:24:34.278887 13720 rpcz_store.cc:276] 0709 12:24:33.042828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.356076 (+313248us) service_pool.cc:224] Handling call
0709 12:24:33.356438 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.414157 (+ 57719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.466706 (+ 52549us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.536428 (+ 69722us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.830817 (+294389us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:33.830956 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.201724 (+370768us) spinlock_profiling.cc:243] Waited 52.6 ms on lock 0x7b54000294d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:34.201779 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.201831 (+    52us) raft_consensus.cc:1685] finished
0709 12:24:34.201845 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.212974 (+ 11129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67467,"raft.queue_time_us":1024,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":57725,"spinlock_wait_cycles":121093376,"thread_start_us":278118,"threads_started":1,"wal-append.queue_time_us":283223}
I20260709 12:24:34.285869 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.286697 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:34.278645 14419 rpcz_store.cc:269] 0709 12:24:30.962423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.962552 (+   129us) service_pool.cc:224] Handling call
0709 12:24:34.201741 (+3239189us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b5400651dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:34.201874 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5771912576}
I20260709 12:24:34.289325 13719 raft_consensus.cc:2368] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:34.302335 13722 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:34.289765 21271 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.304093 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:34.311285 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 129121us
I20260709 12:24:34.315024 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:34.316128 13649 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.261287 21524 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.312490 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:34.291301 21481 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:34.273710 21568 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.292778 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.274360 21430 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:34.299973 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.256352 21501 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:34.278306 21208 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:34.304785 21558 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.317817 21459 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:34.255493 21371 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.323042 21549 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:34.284884 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:34.329298 13644 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:34.317507 21271 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:34.329172 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:34.329784 13645 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:34.323800 21524 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.334532 21600 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:34.338515 21414 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:34.393401 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2685) took 1926 ms. Trace:
I20260709 12:24:34.393895 13796 rpcz_store.cc:276] 0709 12:24:32.467203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.469037 (+  1834us) service_pool.cc:224] Handling call
0709 12:24:32.469396 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.547804 (+ 78408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.547846 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.548434 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.496736 (+948302us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b18000417a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:24:33.784175 (+287439us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:33.784224 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.393024 (+608800us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:34.393094 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.393147 (+    53us) raft_consensus.cc:1685] finished
0709 12:24:34.393163 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.393333 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36417,"raft.queue_time_us":15313,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":16169,"spinlock_wait_cycles":152358144,"thread_start_us":287208,"threads_started":1,"wal-append.queue_time_us":1318492}
I20260709 12:24:34.371007 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:34.386943 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 157579us
I20260709 12:24:34.396660 13644 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.340250 21533 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:34.404011 13649 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
W20260709 12:24:34.404953 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 120948us
I20260709 12:24:34.407373 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.408473 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:34.409684 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:34.374238 21553 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:34.411028 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.343746 21459 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.385157 21572 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:34.343087 21442 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:24:34.398712 21547 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.392171 21414 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.396134 21600 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:34.436318 21600 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:34.440275 21619 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:24:34.441852 21618 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 12:24:34.418553 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 96930.2us
I20260709 12:24:34.418790 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:34.421672 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:34.445175 21581 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:24:34.422070 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112648us
W20260709 12:24:34.446866 21621 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:34.407918 21481 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:34.450884 21481 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.453238 21481 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:34.435387 21464 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.435623 21414 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:34.457104 21617 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:24:34.413810 21533 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:34.413126 21568 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:34.459036 21533 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.459699 21568 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.461360 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.461887 21568 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.443861 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:34.462731 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.463451 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.463692 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:34.463913 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.467459 21584 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:34.467911 21584 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.468117 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.468756 21584 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:34.468888 13718 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:34.471839 21614 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:24:34.479508 21128 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.409829 21549 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:34.414664 21605 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:34.492554 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.519702 13801 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:34.497449 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:34.470520 21581 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:34.502710 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.520960 14418 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:34.385903 21208 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.505472 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:34.506108 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 63386.9us
I20260709 12:24:34.506975 13719 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:34.523919 13795 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:34.507149 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.525040 13725 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:34.507913 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.509794 13644 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:34.510753 13722 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:34.528322 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.415591 21459 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:34.502969 13643 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:24:34.535494 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2660) took 2140 ms. Trace:
I20260709 12:24:34.423411 21545 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:34.535853 13643 rpcz_store.cc:276] 0709 12:24:32.400807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.487701 (+2086894us) service_pool.cc:224] Handling call
0709 12:24:34.502636 (+ 14935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.535197 (+ 32561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.535435 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.522436 13801 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:34.513828 21120 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:24:34.531903 21369 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.525962 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:34.532788 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:34.534375 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:34.536034 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63624.8us
I20260709 12:24:34.530308 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:34.537194 13796 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:34.537977 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.540028 13794 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:24:34.540283 13643 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:34.561434 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2662) took 2130 ms. Trace:
I20260709 12:24:34.561973 13643 rpcz_store.cc:276] 0709 12:24:32.431589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.539685 (+2108096us) service_pool.cc:224] Handling call
0709 12:24:34.539968 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.561104 (+ 21136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.561358 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.533723 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.564705 13727 raft_consensus.cc:2376] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23. Re-sending same reply.
I20260709 12:24:34.514406 21549 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.578039 13799 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:34.579689 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.533322 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.584010 13722 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:34.584547 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58684.9us
I20260709 12:24:34.584767 21128 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:34.586943 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:34.582808 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:34.608594 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100763us
I20260709 12:24:34.524847 21066 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.614629 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:34.609189 13725 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:34.615382 14418 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:24:34.619562 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 89448.1us
I20260709 12:24:34.622357 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:34.622946 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:34.623250 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:34.623821 14417 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:34.623904 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:34.555107 21389 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:34.611421 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.612473 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:34.613207 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.628365 14425 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:34.642807 14418 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:24:34.429675 21379 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.591204 21549 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.647998 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:34.609810 21128 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.619885 21208 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:34.620499 21462 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.486822 21547 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.654835 14419 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:34.655581 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:34.660079 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48753.9us
I20260709 12:24:34.679368 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:34.660362 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:34.660609 14423 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:34.610702 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:34.614370 21628 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:34.684860 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.685696 13794 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:34.609534 13643 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:24:34.430523 21501 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.684360 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:34.687551 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2665) took 1884 ms. Trace:
I20260709 12:24:34.620098 21573 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:34.676312 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.681816 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.682152 21545 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:24:34.683179 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.020s	sys 0.039s
W20260709 12:24:34.691128 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.020s	sys 0.039s
I20260709 12:24:34.644878 21581 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:34.655014 21369 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:34.661425 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:34.696079 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 85509.4us
I20260709 12:24:34.691430 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.697486 13796 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:34.691612 13799 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:34.691457 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.12969e+06us
I20260709 12:24:34.689954 13643 rpcz_store.cc:276] 0709 12:24:32.803194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.585842 (+1782648us) service_pool.cc:224] Handling call
0709 12:24:34.609189 (+ 23347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.687186 (+ 77997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.687479 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.666087 21128 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:34.673245 21481 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:24:34.683849 21628 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:34.678654 21437 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:34.674213 21442 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:34.627938 21066 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.707484 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.688304 21568 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:24:34.711761 14424 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:34.715950 13718 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:34.706844 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.717543 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4191) took 1213 ms. Trace:
I20260709 12:24:34.723232 13643 rpcz_store.cc:276] 0709 12:24:33.503977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.707249 (+1203272us) service_pool.cc:224] Handling call
0709 12:24:34.707545 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.707738 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.707763 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.707809 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:34.707842 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.707897 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.717456 (+  9559us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.726441 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4192) took 1222 ms. Trace:
I20260709 12:24:34.727370 13643 rpcz_store.cc:276] 0709 12:24:33.504307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.725098 (+1220791us) service_pool.cc:224] Handling call
0709 12:24:34.725465 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.725820 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.725873 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.725936 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:34.725977 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.726049 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.726363 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.703393 21185 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:34.700690 21553 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:34.690711 21626 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:34.705786 21120 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:34.703380 21209 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:34.703120 21603 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:34.740662 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.776341 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.777407 13796 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:34.722417 21481 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:34.783842 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4200) took 1276 ms. Trace:
I20260709 12:24:34.786248 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.703886 21545 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.790676 13646 rpcz_store.cc:276] 0709 12:24:33.507030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.620738 (+113708us) service_pool.cc:224] Handling call
0709 12:24:34.783546 (+1162808us) spinlock_profiling.cc:243] Waited 89.4 ms on lock 0x7b54001e5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:34.783746 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205706496}
I20260709 12:24:34.790693 13794 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:34.717718 21628 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:34.703677 21581 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.731909 21472 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21.
I20260709 12:24:34.741483 21626 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.772043 21120 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.739112 21185 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.805573 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:34.811620 21393 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.003s
I20260709 12:24:34.737779 21371 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:34.820601 21407 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
W20260709 12:24:34.829336 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 126037us
I20260709 12:24:34.838150 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.782647 21209 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:34.835836 21120 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:34.836658 21185 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:34.831284 21626 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:34.905613 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1216) took 1371 ms. Trace:
I20260709 12:24:34.919979 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2648) took 2580 ms. Trace:
W20260709 12:24:34.932643 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 247493us
I20260709 12:24:34.934203 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1217) took 1033 ms. Trace:
I20260709 12:24:34.907176 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:34.906530 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4199) took 1400 ms. Trace:
I20260709 12:24:34.963064 14420 rpcz_store.cc:276] 0709 12:24:33.900870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.905306 (+  4436us) service_pool.cc:224] Handling call
0709 12:24:34.933995 (+1028689us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b5400555fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:34.934130 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104660864}
I20260709 12:24:34.963738 13645 rpcz_store.cc:276] 0709 12:24:33.506687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.800683 (+293996us) service_pool.cc:224] Handling call
0709 12:24:34.906305 (+1105622us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54002b48d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:34.906454 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214334720}
I20260709 12:24:34.937263 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4201) took 1429 ms. Trace:
I20260709 12:24:34.970942 13651 rpcz_store.cc:276] 0709 12:24:33.507442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.815698 (+308256us) service_pool.cc:224] Handling call
0709 12:24:34.937072 (+1121374us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:34.937194 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249164160}
I20260709 12:24:34.961724 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:34.962809 14416 rpcz_store.cc:276] 0709 12:24:33.533864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.535410 (+  1546us) service_pool.cc:224] Handling call
0709 12:24:33.617926 (+ 82516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.871481 (+253555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.905080 (+1033599us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540028ffd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:34.905300 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488591616}
I20260709 12:24:34.973145 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:34.977552 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.977651 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:34.978232 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.866876 21249 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.562s	user 0.016s	sys 0.003s
I20260709 12:24:34.960903 13648 rpcz_store.cc:276] 0709 12:24:32.339327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.601539 (+262212us) service_pool.cc:224] Handling call
0709 12:24:32.709253 (+107714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:32.794890 (+ 85637us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:32.817526 (+ 22636us) log.cc:844] Serialized 14 byte log entry
0709 12:24:34.186590 (+1369064us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b1800417e20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:34.668468 (+481878us) log.cc:844] Serialized 14 byte log entry
0709 12:24:34.669172 (+   704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.669228 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:34.669292 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.919714 (+250422us) spinlock_profiling.cc:243] Waited 85.2 ms on lock 0x7b54001e7b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:34.919769 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.919912 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1484,"prepare.queue_time_us":524695,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":119,"raft.queue_time_us":7024,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":21283,"spinlock_wait_cycles":203868800,"thread_start_us":479734,"threads_started":2,"wal-append.queue_time_us":1580475}
I20260709 12:24:34.855294 21407 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.913347 21559 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:34.984759 13652 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:34.940492 21120 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.987356 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2652) took 2646 ms. Trace:
I20260709 12:24:34.990738 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:35.012568 13647 rpcz_store.cc:276] 0709 12:24:32.340598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.146342 (+805744us) service_pool.cc:224] Handling call
0709 12:24:33.177943 (+ 31601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.284463 (+106520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.987024 (+1702561us) spinlock_profiling.cc:243] Waited 95.9 ms on lock 0x7b5400221fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:34.987281 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220475648}
I20260709 12:24:34.916517 21437 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:34.927214 21298 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.995857 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4196) took 1490 ms. Trace:
W20260709 12:24:34.946045 21558 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.005s
W20260709 12:24:34.889815 21602 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.837s	user 0.010s	sys 0.014s
I20260709 12:24:35.070073 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4203) took 1562 ms. Trace:
I20260709 12:24:35.070750 13648 rpcz_store.cc:276] 0709 12:24:33.507974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.005060 (+1497086us) service_pool.cc:224] Handling call
0709 12:24:35.069464 (+ 64404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.069699 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.069729 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.069774 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:35.069824 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.069880 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.070009 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:34.903481 21271 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.003s
I20260709 12:24:35.021437 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:35.017521 13645 rpcz_store.cc:276] 0709 12:24:33.505570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.994531 (+1488961us) service_pool.cc:224] Handling call
0709 12:24:34.995061 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.995328 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.995367 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.995443 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:34.995528 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.995606 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.995768 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.850904 21393 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:34.876959 21209 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:34.901886 21443 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:35.090819 13650 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:35.095974 13793 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:24:35.077268 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4206) took 1222 ms. Trace:
I20260709 12:24:35.108965 13795 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:35.109107 13645 rpcz_store.cc:276] 0709 12:24:33.854534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.076080 (+1221546us) service_pool.cc:224] Handling call
0709 12:24:35.076536 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.076768 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.076809 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.076883 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:35.076934 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.077010 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.077181 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:35.100382 13719 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:35.100349 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2659) took 2712 ms. Trace:
I20260709 12:24:35.097033 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4204) took 1588 ms. Trace:
I20260709 12:24:35.115115 13647 rpcz_store.cc:276] 0709 12:24:33.508273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.013847 (+1505574us) service_pool.cc:224] Handling call
0709 12:24:35.096316 (+ 82469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.096549 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.096587 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.096650 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:35.096699 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.096772 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.096935 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:34.930730 21548 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 12:24:35.012645 21380 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:35.012521 21369 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:35.012053 21559 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:35.017068 21437 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.074129 21439 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:35.101733 20975 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.176383 13800 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:35.016278 21249 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:35.186713 13726 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.006s	sys 0.013s
I20260709 12:24:35.133016 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:35.207829 13726 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.006s	sys 0.013s
W20260709 12:24:35.208274 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75324.5us
I20260709 12:24:34.899739 21414 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.180846 13647 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:24:35.209407 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:35.212365 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:35.212927 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 222326us
I20260709 12:24:35.137177 21576 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.074743 21298 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.078866 21558 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.218752 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:35.216648 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2674) took 1332 ms. Trace:
I20260709 12:24:35.233575 13646 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:35.098639 21602 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.106796 21271 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.188230 21217 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:34.999604 21481 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.233935 13647 rpcz_store.cc:276] 0709 12:24:33.883878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.116446 (+1232568us) service_pool.cc:224] Handling call
0709 12:24:35.140460 (+ 24014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.210965 (+ 70505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.216554 (+  5589us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:35.236835 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:35.237340 13720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.033s	sys 0.055s
I20260709 12:24:35.190044 21547 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:35.670070 21547 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:35.670321 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 433658us
I20260709 12:24:35.528789 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 3962) took 1024 ms. Trace:
W20260709 12:24:35.528921 21635 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 12:24:35.233136 21576 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.673929 21576 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:35.674088 21566 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:35.676477 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3716) took 1214 ms. Trace:
I20260709 12:24:35.676901 13721 rpcz_store.cc:276] 0709 12:24:34.462408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.462555 (+   147us) service_pool.cc:224] Handling call
0709 12:24:34.462941 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:34.516078 (+ 53137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.516113 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:34.524281 (+  8168us) log.cc:844] Serialized 52 byte log entry
0709 12:24:34.845367 (+321086us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:35.199089 (+353722us) log.cc:844] Serialized 14 byte log entry
0709 12:24:35.199392 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.199476 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.676202 (+476726us) raft_consensus.cc:1685] finished
0709 12:24:35.676226 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.676407 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53095,"prepare.queue_time_us":292736,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":57766,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":28618,"spinlock_wait_cycles":15243520,"thread_start_us":449476,"threads_started":2,"wal-append.queue_time_us":555096}
I20260709 12:24:35.678882 21439 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:35.679603 21439 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:35.681124 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3725) took 1172 ms. Trace:
I20260709 12:24:35.390106 21545 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:35.346474 21369 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.529902 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:35.686172 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 156354us
I20260709 12:24:35.687394 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:35.237696 21572 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:35.221711 21557 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:35.530676 13723 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:35.531250 21581 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:35.508409 13724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.007s	sys 0.013s
W20260709 12:24:35.694638 13724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.109s	user 0.007s	sys 0.014s
I20260709 12:24:35.695461 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3717) took 1232 ms. Trace:
I20260709 12:24:35.695804 13724 rpcz_store.cc:276] 0709 12:24:34.462702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.470365 (+  7663us) service_pool.cc:224] Handling call
0709 12:24:34.470760 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:34.535072 (+ 64312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.535106 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:34.535662 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:24:35.695112 (+1159450us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:35.695155 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.695221 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.695264 (+    43us) raft_consensus.cc:1685] finished
0709 12:24:35.695275 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.695400 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16410,"raft.queue_time_us":36861,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":30957,"thread_start_us":1109724,"threads_started":1,"wal-append.queue_time_us":953648}
W20260709 12:24:35.371035 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158749us
I20260709 12:24:35.331090 21629 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.206s
I20260709 12:24:35.536566 14416 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:35.537645 21393 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:35.230942 21600 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.788s	user 0.013s	sys 0.000s
I20260709 12:24:35.217551 21371 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:35.547684 21407 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:35.548951 21615 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:24:35.525211 21120 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:35.555550 21602 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:35.524730 21640 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
W20260709 12:24:35.527029 21633 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:24:35.557428 21464 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:35.742233 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2661) took 3336 ms. Trace:
I20260709 12:24:35.558889 21298 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:35.526029 21414 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:35.567123 21249 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:35.567723 21524 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.183s	user 0.009s	sys 0.013s
W20260709 12:24:35.568660 21549 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.893s	user 0.006s	sys 0.006s
I20260709 12:24:35.556495 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:35.586644 21208 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:24:35.587754 21217 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:35.588689 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 347910us
I20260709 12:24:35.759145 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:35.589468 21442 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:24:35.593524 21584 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.124s	user 0.013s	sys 0.004s
I20260709 12:24:35.606664 21648 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:35.644294 21185 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.000s
W20260709 12:24:35.323789 21560 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.030s	user 0.009s	sys 0.004s
I20260709 12:24:35.245442 21546 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:35.241968 21379 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:35.669123 13720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.034s	sys 0.055s
I20260709 12:24:35.240694 13792 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20.
I20260709 12:24:35.670898 13798 rpcz_store.cc:276] 0709 12:24:34.504520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.505000 (+   480us) service_pool.cc:224] Handling call
0709 12:24:34.505535 (+   535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:34.505746 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.505771 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:34.818797 (+313026us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:24:34.819439 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:24:35.049748 (+230309us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:35.049812 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.049937 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.528467 (+478530us) raft_consensus.cc:1685] finished
0709 12:24:35.528486 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.528725 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12057728,"thread_start_us":229638,"threads_started":1,"wal-append.queue_time_us":706991}
I20260709 12:24:35.218046 21066 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:35.682497 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.005s	sys 0.005s
W20260709 12:24:35.788769 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.005s	sys 0.005s
I20260709 12:24:35.682546 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4194) took 2177 ms. Trace:
I20260709 12:24:35.681661 13726 rpcz_store.cc:276] 0709 12:24:34.508344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.511807 (+  3463us) service_pool.cc:224] Handling call
0709 12:24:34.512214 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.512521 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:35.208413 (+695892us) log.cc:844] Serialized 14 byte log entry
0709 12:24:35.680626 (+472213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.680705 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:35.680771 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.680870 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.681039 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13765,"prepare.queue_time_us":500499,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":116,"thread_start_us":1102693,"threads_started":2}
W20260709 12:24:35.789215 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 675755us
I20260709 12:24:35.689379 21660 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:35.689689 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:35.792346 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102741us
I20260709 12:24:35.793087 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:35.691336 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.002s	sys 0.010s
W20260709 12:24:35.794435 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.002s	sys 0.010s
I20260709 12:24:35.794765 21555 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:24:35.795054 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 773700us
I20260709 12:24:35.795461 21555 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:35.796114 21555 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:35.799821 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2666) took 2996 ms. Trace:
I20260709 12:24:35.799589 21217 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.800328 13652 rpcz_store.cc:276] 0709 12:24:32.803479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.654448 (+1850969us) service_pool.cc:224] Handling call
0709 12:24:34.683042 (+ 28594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.005313 (+322271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.799490 (+794177us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54002e45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:35.799738 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693279104}
I20260709 12:24:35.789491 13646 rpcz_store.cc:276] 0709 12:24:33.505016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.795352 (+1290336us) service_pool.cc:224] Handling call
0709 12:24:34.824906 (+ 29554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.234047 (+409141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.682068 (+448021us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b5400095954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:35.682476 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939183872}
I20260709 12:24:35.718755 21658 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
W20260709 12:24:35.705030 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.022s	sys 0.068s
W20260709 12:24:35.833127 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124770us
W20260709 12:24:35.701169 21628 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.875s	user 0.012s	sys 0.000s
I20260709 12:24:35.719214 13651 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
W20260709 12:24:35.735064 21437 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.002s
I20260709 12:24:35.697414 21572 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:35.746380 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 215941us
I20260709 12:24:35.742898 13644 rpcz_store.cc:276] 0709 12:24:32.406613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.538859 (+2132246us) service_pool.cc:224] Handling call
0709 12:24:34.539195 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:34.554586 (+ 15391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.554643 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:34.555306 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.719002 (+163696us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:34.719042 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.719111 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.741890 (+1022779us) raft_consensus.cc:1685] finished
0709 12:24:35.741927 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.742138 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7540,"raft.queue_time_us":15320,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":96,"thread_start_us":135432,"threads_started":1,"wal-append.queue_time_us":424175}
W20260709 12:24:35.720592 21443 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.600s	user 0.000s	sys 0.011s
W20260709 12:24:35.726785 21626 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.007s
I20260709 12:24:35.711968 21371 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:35.762154 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 205759us
I20260709 12:24:35.772524 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:35.743140 21557 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:35.703740 21209 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:24:35.706910 21600 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.851450 21371 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:35.858227 21546 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:35.860152 21600 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:35.804736 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:35.861660 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67483.4us
I20260709 12:24:35.863092 13617 leader_election.cc:400] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:35.697825 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:35.865437 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 168597us
I20260709 12:24:35.866545 21564 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:35.845732 21604 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.868469 21604 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.850724 21626 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.869658 21439 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.872586 21626 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:35.868652 21371 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.728742 21576 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:35.802130 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:35.878793 13652 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:35.784590 21185 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.811713 21644 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.812951 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:35.820413 21559 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.792335 21605 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:35.824818 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:35.830611 21298 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:35.833555 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.022s	sys 0.069s
I20260709 12:24:35.692930 21395 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:24:35.771060 21602 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:24:35.778755 21414 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:35.838999 21628 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.841046 21437 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.752403 21524 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.843988 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:35.901697 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:35.764796 21549 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:35.852808 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80414.8us
I20260709 12:24:35.779384 21584 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.864295 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:35.838037 21271 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:35.873871 13794 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:35.913383 14420 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
W20260709 12:24:35.914584 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50417.7us
I20260709 12:24:35.913456 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:35.915599 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:35.916633 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:35.918155 13644 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:35.893347 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:35.778628 21128 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.055s	user 0.010s	sys 0.004s
W20260709 12:24:35.894724 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.896386 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.04481e+06us
I20260709 12:24:35.926627 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:35.781098 21648 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:35.902446 13652 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:35.874123 21443 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:35.873529 21626 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:35.872645 21439 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:35.919299 13646 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:35.920442 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4215) took 1952 ms. Trace:
I20260709 12:24:35.961858 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:35.962638 13644 rpcz_store.cc:276] 0709 12:24:33.968596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.917635 (+1949039us) service_pool.cc:224] Handling call
0709 12:24:35.917965 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.918765 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.920374 (+  1609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:35.920795 13798 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:35.921114 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:35.926476 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:35.896965 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72258us
I20260709 12:24:35.901237 21602 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.929952 14416 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:24:35.969302 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4216) took 1996 ms. Trace:
W20260709 12:24:35.881434 21459 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.276s	user 0.016s	sys 0.001s
W20260709 12:24:35.933400 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.973544 13645 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:35.855789 21558 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:35.896175 21605 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:35.969684 13644 rpcz_store.cc:276] 0709 12:24:33.972607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.963720 (+1991113us) service_pool.cc:224] Handling call
0709 12:24:35.968754 (+  5034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.968965 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.968990 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.969051 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:35.969087 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.969143 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.969247 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:35.937644 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.975013 13722 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:35.895712 21560 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:35.901679 21209 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:35.978317 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.005185 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:35.982959 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:35.979370 13648 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:35.928159 21600 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:35.992625 14419 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:35.993863 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:35.994724 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.011996 14423 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:36.001936 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:35.893776 21546 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 12:24:35.992877 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.007597 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.007524 13798 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:35.949239 21553 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.178s	user 0.011s	sys 0.005s
I20260709 12:24:36.011373 21558 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:35.892465 21644 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.902165 21298 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.002405 21605 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.022544 13795 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:35.980722 21658 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:35.956699 21464 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:36.026577 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:35.974623 21602 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:36.025134 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.001518 21641 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.024137 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.040207 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:36.033834 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:35.916895 21656 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
W20260709 12:24:36.023339 20975 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.820s	user 0.010s	sys 0.003s
W20260709 12:24:36.033041 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 198257us
I20260709 12:24:35.957108 21660 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:24:35.973062 21249 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:24:35.973654 21271 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:24:35.935854 21128 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.073769 14423 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:24:36.009981 21560 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.043884 13796 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:36.045238 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4214) took 2100 ms. Trace:
W20260709 12:24:36.046445 13690 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.757s (SENT)
I20260709 12:24:35.996215 21501 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.034s
I20260709 12:24:36.047137 13793 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:36.040205 21576 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.977811 21459 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.086378 13718 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:36.087821 21459 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.088829 21459 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:36.020782 21626 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:35.117096 13649 rpcz_store.cc:276] 0709 12:24:32.387539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.419204 (+2031665us) service_pool.cc:224] Handling call
0709 12:24:34.420080 (+   876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.420332 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:34.742219 (+321887us) log.cc:844] Serialized 14 byte log entry
0709 12:24:35.099958 (+357739us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.100018 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:35.100070 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.100141 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.100285 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":426950,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":8556,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":11839,"thread_start_us":578426,"threads_started":3,"wal-append.queue_time_us":360783}
W20260709 12:24:36.095163 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.096426 13799 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:36.002951 21572 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:35.977993 21637 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:24:36.028975 21553 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:36.061285 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:36.031711 21369 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:36.071241 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3718) took 1608 ms. Trace:
I20260709 12:24:36.072364 13652 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:36.043103 13646 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:36.040503 21209 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.096995 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:36.097512 14391 connection.cc:256] RPC call timeout handler was delayed by -3.4445670790000804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
I20260709 12:24:36.029105 21558 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:36.071822 21657 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.287s	user 0.001s	sys 0.001s
I20260709 12:24:36.057092 21371 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.057935 20975 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:36.104394 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.105144 13643 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:36.017563 21557 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.149631 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.117687 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.102326 13720 rpcz_store.cc:276] 0709 12:24:34.463004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.470856 (+  7852us) service_pool.cc:224] Handling call
0709 12:24:34.504539 (+ 33683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.504772 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:34.720450 (+215678us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.070854 (+1350404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:36.070922 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:36.070972 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.071046 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.071175 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130313,"prepare.queue_time_us":338566,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":5971,"raft.queue_time_us":118073,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":23323,"thread_start_us":1539551,"threads_started":3,"wal-append.queue_time_us":924434}
W20260709 12:24:36.100577 21662 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 12:24:36.036284 21658 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.120508 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:36.116323 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.081188 13647 rpcz_store.cc:276] 0709 12:24:33.945017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.371673 (+1426656us) service_pool.cc:224] Handling call
0709 12:24:35.681466 (+309793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.682104 (+   638us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:24:35.836461 (+154357us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.044227 (+207766us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.044543 (+   316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:36.044612 (+    69us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:36.044684 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.044790 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.045093 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2574,"prepare.queue_time_us":345818,"prepare.run_wall_time_us":49,"raft.queue_time_us":2855,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":9490,"spinlock_wait_cycles":6706048,"thread_start_us":352214,"threads_started":2}
I20260709 12:24:36.122611 14422 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:24:36.122838 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.171663 14423 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:36.128322 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.129673 13646 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:36.128530 14417 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:36.099881 21439 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.111071 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49920us
W20260709 12:24:36.123922 21661 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
I20260709 12:24:36.171056 14416 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:36.049536 21641 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:36.175105 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54667us
W20260709 12:24:36.188796 13767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.886s (SENT) [suppressed 12 similar messages]
I20260709 12:24:36.180435 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.180827 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.181757 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.184543 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.190634 13798 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:36.124953 21655 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:36.195351 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.098999 21576 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.198662 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.202857 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.204280 14416 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:24:36.208606 13646 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:24:36.209978 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.102727 21217 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 12:24:36.211948 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.203533 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:36.211584 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 127802us
I20260709 12:24:36.188654 21393 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:36.221664 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.129160 21629 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:24:36.118391 21369 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.234577 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.235512 13726 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:36.237248 13798 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:36.238341 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.239327 13726 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:36.239559 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50699us
I20260709 12:24:36.126370 21669 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 12:24:36.205618 21185 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.243383 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.242195 13719 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:36.210625 21371 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:36.212657 21524 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.216336 21628 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.202800 21665 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:36.268433 14416 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:36.268792 21665 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.269595 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.270433 14421 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:36.270601 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4193) took 2766 ms. Trace:
I20260709 12:24:36.271111 13643 rpcz_store.cc:276] 0709 12:24:33.504725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.728966 (+1224241us) service_pool.cc:224] Handling call
0709 12:24:34.729335 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.129622 (+1400287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.270245 (+140623us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54001478d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:36.270506 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3163527552}
I20260709 12:24:36.273043 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.237942 21648 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.193981 21671 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:24:36.279791 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.280560 14417 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:36.281241 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3727) took 1392 ms. Trace:
I20260709 12:24:36.248220 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.285512 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:36.252317 21298 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.205870 21472 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.043s
I20260709 12:24:36.256603 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:36.256880 21566 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.004s
I20260709 12:24:36.193001 21439 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.229674 21549 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.245865 21371 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:36.258159 21442 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:24:36.258798 21668 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:24:36.259617 21670 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:36.259992 21655 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:36.261104 21576 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.262647 21381 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:36.263734 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.262672 21641 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.265981 21369 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.266990 21560 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.267390 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.231602 21209 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.285909 13727 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.293241 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:36.410594 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:36.413157 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.413656 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.281813 13718 rpcz_store.cc:276] 0709 12:24:34.888732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.888925 (+   193us) service_pool.cc:224] Handling call
0709 12:24:34.889351 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.086891 (+1197540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.280910 (+194019us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400155ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:36.281156 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2752391936}
I20260709 12:24:36.311334 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:36.415109 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 113303us
I20260709 12:24:36.321345 13725 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:36.416282 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.274132 13643 raft_consensus.cc:2410] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:24:36.420111 13721 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:36.305594 13652 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
W20260709 12:24:36.299639 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:36.427459 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.295899 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2677) took 2321 ms. Trace:
I20260709 12:24:36.296635 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:36.437232 21615 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:24:36.441251 21555 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.004s
I20260709 12:24:36.248152 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:36.403409 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 157276us
I20260709 12:24:36.267220 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3728) took 1374 ms. Trace:
I20260709 12:24:36.406520 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.406450 13723 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:36.420856 13721 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:36.328280 21524 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:24:36.450802 13722 rpcz_store.cc:276] 0709 12:24:34.892476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.892694 (+   218us) service_pool.cc:224] Handling call
0709 12:24:34.893117 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.000410 (+1107293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.266892 (+266482us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001a4d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:36.267141 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487465728}
I20260709 12:24:36.336486 21128 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.346591 21558 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.264250 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4211) took 2394 ms. Trace:
I20260709 12:24:36.461057 13645 rpcz_store.cc:276] 0709 12:24:33.870231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.110507 (+1240276us) service_pool.cc:224] Handling call
0709 12:24:35.110929 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.977841 (+866912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.263938 (+286097us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b54001e6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:36.264170 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1983211264}
I20260709 12:24:36.446853 21573 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.463467 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.463735 21573 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.215860 21581 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
W20260709 12:24:36.466239 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141730us
I20260709 12:24:36.468197 21581 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:24:36.468874 21581 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:36.352798 21600 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:36.477943 21572 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:36.478348 21600 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:36.479341 21572 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.211541 21626 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.442231 21555 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.483273 20975 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.484228 20975 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.351120 21644 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.345628 21658 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:36.353706 21646 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.826s	user 0.005s	sys 0.000s
I20260709 12:24:36.271064 21665 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.487653 21628 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:36.249497 21667 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:36.488682 21628 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.489318 21667 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.314800 21655 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.492259 21655 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:36.335808 21666 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:24:36.496666 21644 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.312034 21442 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.499796 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.500978 14423 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:36.243218 21393 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.321074 21381 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.308478 21371 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:36.351985 21379 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:36.347795 21407 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:36.569290 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.569757 21407 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.420447 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.572388 13643 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:36.310657 21564 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:36.297730 21674 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:24:36.328732 21679 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:36.346344 21683 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:24:36.575979 21679 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.577718 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2709) took 1692 ms. Trace:
I20260709 12:24:36.578037 21581 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.578248 13794 rpcz_store.cc:276] 0709 12:24:34.885457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.885650 (+   193us) service_pool.cc:224] Handling call
0709 12:24:34.906740 (+ 21090us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.934714 (+1027974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.576809 (+642095us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54000d16d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:36.577322 (+   513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223559552}
I20260709 12:24:36.578985 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.580283 21581 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:36.580839 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.465629 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.340826 21576 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:36.429294 13644 rpcz_store.cc:276] 0709 12:24:33.974810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.998393 (+2023583us) service_pool.cc:224] Handling call
0709 12:24:35.998823 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.999115 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:24:36.189901 (+190786us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.295549 (+105648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:36.295609 (+    60us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:36.295658 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.295725 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.295841 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1165,"prepare.queue_time_us":310355,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":4010,"raft.queue_time_us":1287,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7793,"thread_start_us":279281,"threads_started":2,"wal-append.queue_time_us":136758}
I20260709 12:24:36.485724 21555 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.446941 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.510003 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.497840 21644 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.468894 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:36.481242 21600 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.534130 14417 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:24:36.543269 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:36.546517 21573 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.562340 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4195) took 3056 ms. Trace:
I20260709 12:24:36.569557 21379 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.571801 21407 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.415282 13726 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:36.302596 21678 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.582214 21655 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:36.601301 13651 rpcz_store.cc:276] 0709 12:24:33.505313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.983646 (+1478333us) service_pool.cc:224] Handling call
0709 12:24:34.983947 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.836900 (+852953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.561525 (+724625us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:36.561996 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690314496}
W20260709 12:24:36.611727 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58558.9us
I20260709 12:24:36.611181 21678 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.612150 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.482138 21602 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.615922 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.616895 21602 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.617360 21683 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:36.510747 21393 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.617817 21564 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:36.618747 21564 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.548599 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:36.620189 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 114551us
I20260709 12:24:36.623077 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:36.624114 14424 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:36.624162 21555 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:36.624915 21555 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.594990 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.598950 13645 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:36.607532 13718 raft_consensus.cc:2368] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:36.608204 21674 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:36.608731 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:36.608984 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 140488us
I20260709 12:24:36.609090 21693 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 17, Time since last communication: 0.087s
I20260709 12:24:36.625378 21555 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:36.629580 13651 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:36.446703 21605 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:36.652808 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109897us
I20260709 12:24:36.653573 21600 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:36.654162 13721 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.656316 21559 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:36.656517 21628 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.643635 20975 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.628640 21576 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:36.611167 21407 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:36.662963 21381 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.673661 13721 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:36.676414 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.678444 21217 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.679385 21217 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.676632 21678 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.681005 13720 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:36.682443 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:36.688611 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.690167 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.692157 13727 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:36.693176 21209 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:36.693683 21209 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.693913 13727 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:36.694109 21209 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:36.695297 13690 leader_election.cc:336] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.663s (SENT)
I20260709 12:24:36.695874 13727 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:36.696823 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.697782 13649 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:36.698871 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.700300 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:36.701073 13649 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:36.681581 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.690840 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:36.715996 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:36.714433 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:36.717422 13727 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [20.15-22.16]
I20260709 12:24:36.728516 21564 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.730500 21659 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:36.731043 21659 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:36.732292 21659 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.743863 21380 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:36.744547 21380 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.744792 21568 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:24:36.746244 21568 consensus_queue.cc:1243] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:24:36.747340 21568 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:36.747730 21568 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:36.755692 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:36.758453 21665 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:36.721482 21217 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.755559 21557 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:24:36.764261 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74580.6us
I20260709 12:24:36.767972 14424 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:36.771494 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:36.780136 21644 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.781775 21655 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:36.782724 21655 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:36.784070 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4217) took 2790 ms (client timeout 1886 ms). Trace:
W20260709 12:24:36.784544 13650 rpcz_store.cc:269] 0709 12:24:33.993360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.058939 (+ 65579us) service_pool.cc:224] Handling call
0709 12:24:36.783506 (+2724567us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b54001e6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:36.783784 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1760895232}
I20260709 12:24:36.784298 21655 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.706914 20975 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
W20260709 12:24:36.790236 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3960) took 2353 ms (client timeout 2155 ms). Trace:
W20260709 12:24:36.790812 13792 rpcz_store.cc:269] 0709 12:24:34.436796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.463442 (+ 26646us) service_pool.cc:224] Handling call
0709 12:24:36.789766 (+2326324us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54000d20d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:36.789990 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774434176}
I20260709 12:24:36.791664 21217 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:36.792351 21217 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:36.793017 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:36.793830 21369 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:36.793673 21217 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.794739 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:36.795051 21369 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:36.795572 21369 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:36.795727 13794 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:36.795639 21217 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:36.797174 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:36.798076 21604 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.798347 21656 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.772192 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.776103 21658 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:36.774721 21667 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.778564 21677 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:36.791416 20975 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:36.777443 21371 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.804229 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.806388 21381 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:36.806834 21679 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.809994 21636 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:36.734093 21659 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:36.817003 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:36.852522 13651 raft_consensus.cc:606] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:36.852907 13651 raft_consensus.cc:614] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:36.785995 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.746102 21380 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.853425 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:36.818662 21658 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:36.820348 21605 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.791422 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:36.834636 13645 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:36.798122 13725 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:36.831946 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:36.837707 13649 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:36.846458 13721 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:24:36.760788 21572 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.852907 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:36.797165 21369 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.822240 21439 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:36.858671 13721 pending_rounds.cc:85] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:24:36.870565 13650 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:36.871315 13799 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:36.851684 21636 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:36.859741 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.861339 13721 pending_rounds.cc:107] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:24:36.877231 13721 raft_consensus.cc:2889] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:36.832860 21683 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:24:36.881250 13650 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:24:36.871351 21380 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:36.884346 21679 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:36.886394 21553 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.886521 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.886986 21553 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.887383 13799 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:36.887624 13651 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:36.888425 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1289) took 2809 ms. Trace:
I20260709 12:24:36.888736 13651 rpcz_store.cc:276] 0709 12:24:34.079283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.865482 (+2786199us) service_pool.cc:224] Handling call
0709 12:24:36.887431 (+ 21949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.888084 (+   653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.888331 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.886214 21636 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:36.890636 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1290) took 2802 ms. Trace:
I20260709 12:24:36.878121 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.869415 21644 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:36.895503 13650 rpcz_store.cc:276] 0709 12:24:34.079588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.871575 (+2791987us) service_pool.cc:224] Handling call
0709 12:24:36.872271 (+   696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.881754 (+  9483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.882001 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.898434 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.899272 14425 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:36.901628 13801 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:36.880745 21677 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.904506 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:36.905421 21677 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:36.908295 13651 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:36.911075 21636 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:36.914422 13799 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:36.923118 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:36.923889 14424 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:24:36.925493 13652 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:36.926472 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1291) took 2846 ms. Trace:
I20260709 12:24:36.927028 13651 rpcz_store.cc:276] 0709 12:24:34.079864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.889564 (+2809700us) service_pool.cc:224] Handling call
0709 12:24:36.900595 (+ 11031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.918676 (+ 18081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.926391 (+  7715us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.928817 13651 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:36.929406 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2681) took 2748 ms. Trace:
I20260709 12:24:36.929718 13651 rpcz_store.cc:276] 0709 12:24:34.180901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.928122 (+2747221us) service_pool.cc:224] Handling call
0709 12:24:36.928615 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.929168 (+   553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.929349 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.856776 21656 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.885385 21658 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.926519 13719 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:36.932204 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1294) took 2746 ms. Trace:
I20260709 12:24:36.935111 13645 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:36.939534 21605 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.939818 13650 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: [22.17-23.18]
W20260709 12:24:36.940008 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148740us
I20260709 12:24:36.940577 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1292) took 2860 ms. Trace:
I20260709 12:24:36.940934 13650 rpcz_store.cc:276] 0709 12:24:34.080116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.913464 (+2833348us) service_pool.cc:224] Handling call
0709 12:24:36.939430 (+ 25966us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:36.940292 (+   862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.940511 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:36.913049 21644 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:36.998399 13644 raft_consensus.cc:2410] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
W20260709 12:24:37.021485 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78030.3us
I20260709 12:24:37.022054 21644 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.023291 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:37.024453 13643 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:36.947216 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 179526us
I20260709 12:24:36.947937 21669 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:36.947227 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.959316 21637 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:36.959442 13651 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:36.912103 21641 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:24:36.963449 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:36.963482 21605 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:36.969537 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:37.071055 13797 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:37.020541 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:36.967494 21371 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:36.964776 21658 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:37.080941 13618 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20.
I20260709 12:24:36.936928 13652 rpcz_store.cc:276] 0709 12:24:34.179902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.924783 (+2744881us) service_pool.cc:224] Handling call
0709 12:24:36.925266 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.925993 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.926194 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.015960 13792 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:24:37.084841 13618 leader_election.cc:403] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:24:36.963964 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:37.067619 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.059368 21669 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:37.068585 21637 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:37.077750 14424 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:37.078492 21605 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.012936 21442 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.081908 21573 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.082602 21546 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:37.110698 21546 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.113648 21546 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:37.115393 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.083261 21128 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.119145 21128 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.120582 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:37.084895 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4218) took 2856 ms. Trace:
I20260709 12:24:36.945355 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2678) took 2963 ms. Trace:
I20260709 12:24:37.004644 21656 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:37.096396 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.096513 13797 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:37.097084 21637 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.098189 21533 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:24:37.098541 13795 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:37.098767 21442 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:37.099308 21573 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.099658 21669 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.083777 21602 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:24:37.102510 13652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:37.143235 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.143831 13652 raft_consensus.cc:606] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:37.071205 21209 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.108947 21665 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:37.109640 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:37.095196 21641 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:37.124933 13645 rpcz_store.cc:276] 0709 12:24:33.975101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.862523 (+2887422us) service_pool.cc:224] Handling call
0709 12:24:36.917296 (+ 54773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.938678 (+ 21382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.938895 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.141621 13796 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:24:37.092644 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 186943us
I20260709 12:24:37.123739 13651 rpcz_store.cc:276] 0709 12:24:34.214398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.934925 (+2720527us) service_pool.cc:224] Handling call
0709 12:24:36.959219 (+ 24294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.070812 (+111593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.071078 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.121915 21128 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.136335 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:37.150552 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:37.144192 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:37.144260 13652 raft_consensus.cc:614] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:37.148936 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.149575 13801 raft_consensus.cc:1217] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Deduplicated request from leader. Original: 22.17->[23.18-23.18]   Dedup: 23.18->[]
W20260709 12:24:37.139412 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58846.7us
I20260709 12:24:37.151350 13721 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:37.151818 13801 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 12:24:37.152311 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:37.155207 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60393.5us
W20260709 12:24:37.158193 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 194379us
I20260709 12:24:37.158834 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:37.159487 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:37.095306 21555 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.172437 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2682) took 2905 ms. Trace:
I20260709 12:24:37.178715 13793 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:37.179940 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:37.182000 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:37.183314 21580 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:37.183795 21580 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:37.184185 21580 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:37.160812 21546 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20.
I20260709 12:24:37.187634 14424 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:24:37.187888 13647 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27.
I20260709 12:24:37.188094 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4219) took 2860 ms. Trace:
I20260709 12:24:37.188774 13651 rpcz_store.cc:276] 0709 12:24:34.328169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.147239 (+2819070us) service_pool.cc:224] Handling call
0709 12:24:37.178249 (+ 31010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.187717 (+  9468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.187763 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.187811 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:37.187846 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.187906 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.188036 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.191751 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:37.192740 13792 raft_consensus.cc:2368] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:37.192617 13726 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:37.194752 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1682) took 1388 ms. Trace:
I20260709 12:24:37.176371 21442 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:37.195312 13723 rpcz_store.cc:276] 0709 12:24:35.806725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.806959 (+   234us) service_pool.cc:224] Handling call
0709 12:24:35.807430 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.482654 (+675224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.194394 (+711740us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b54003bea54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:37.194649 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1375779072}
I20260709 12:24:37.198802 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:37.201043 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.203022 13652 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:37.203760 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2685) took 1648 ms. Trace:
I20260709 12:24:37.204088 13652 rpcz_store.cc:276] 0709 12:24:35.555320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.202348 (+1647028us) service_pool.cc:224] Handling call
0709 12:24:37.202775 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.203502 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.203702 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.204694 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:37.206046 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:37.206372 13646 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:37.207607 13651 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:37.208433 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4221) took 1405 ms. Trace:
I20260709 12:24:37.208503 21443 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:37.208885 13651 rpcz_store.cc:276] 0709 12:24:35.802758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.189751 (+1386993us) service_pool.cc:224] Handling call
0709 12:24:37.207275 (+ 17524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.208172 (+   897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.208376 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.209841 21680 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:37.210327 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.211189 21546 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:37.211560 13721 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:37.211748 21546 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:37.212903 21546 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.215081 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:37.216532 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.221705 21555 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:37.174633 13650 rpcz_store.cc:276] 0709 12:24:34.266880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.941758 (+2674878us) service_pool.cc:224] Handling call
0709 12:24:36.942170 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.029156 (+ 86986us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:37.031076 (+  1920us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.172136 (+141060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.172185 (+    49us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:37.172228 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.172279 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.172389 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9258,"prepare.queue_time_us":941,"prepare.run_cpu_time_us":211,"prepare.run_wall_time_us":212,"thread_start_us":113660,"threads_started":1}
I20260709 12:24:37.187635 13645 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:24:37.226207 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4220) took 1427 ms. Trace:
I20260709 12:24:37.226641 13645 rpcz_store.cc:276] 0709 12:24:35.802405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.186782 (+1384377us) service_pool.cc:224] Handling call
0709 12:24:37.187230 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.225792 (+ 38562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.226129 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:37.233731 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 111483us
W20260709 12:24:37.242818 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.252614 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82350.2us
W20260709 12:24:37.253070 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.254742 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:24:37.256016 13796 raft_consensus.cc:606] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:37.256418 13796 raft_consensus.cc:614] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.258787 13721 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:37.265515 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:37.287905 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:37.265545 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3741) took 1016 ms. Trace:
I20260709 12:24:37.288213 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1299) took 1273 ms. Trace:
I20260709 12:24:37.254783 21559 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:37.298693 21559 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:37.288110 21209 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:37.301160 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:37.283329 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:37.303802 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:37.296049 21414 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:37.292482 13647 rpcz_store.cc:276] 0709 12:24:36.014635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.608156 (+593521us) service_pool.cc:224] Handling call
0709 12:24:37.287953 (+679797us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:37.288127 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267983232}
I20260709 12:24:37.291601 13719 rpcz_store.cc:276] 0709 12:24:36.249495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.266222 (+ 16727us) service_pool.cc:224] Handling call
0709 12:24:36.266516 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.962558 (+696042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.265190 (+302632us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b5400175054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:37.265426 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369316608}
I20260709 12:24:37.302562 21679 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:37.253621 21573 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.287551 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:37.315197 14422 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:24:37.288733 21671 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:37.313421 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:37.352813 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:37.359088 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68875.3us
I20260709 12:24:37.359925 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:37.351586 21702 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.548s	user 0.002s	sys 0.007s
W20260709 12:24:37.365960 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.309500 21414 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:37.283744 21128 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
W20260709 12:24:37.372690 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.374116 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90914.8us
I20260709 12:24:37.299944 21559 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.271616 21690 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:37.356029 21209 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:24:37.400255 13647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:37.402324 21690 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:37.400408 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:37.407727 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:37.415047 21437 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.408201 13647 raft_consensus.cc:606] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:37.415824 21660 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:37.416344 21660 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:37.416105 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:37.419713 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:37.426163 14416 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:24:37.434270 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 146510us
I20260709 12:24:37.435232 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:37.437412 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2701) took 2730 ms (client timeout 3284 ms). Trace:
W20260709 12:24:37.437942 13800 rpcz_store.cc:269] 0709 12:24:34.707075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.707228 (+   153us) service_pool.cc:224] Handling call
0709 12:24:37.436850 (+2729622us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:37.437191 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971634432}
I20260709 12:24:37.439477 13794 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 12:24:37.442699 21217 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.646s	user 0.016s	sys 0.004s
I20260709 12:24:37.418954 21584 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.415603 21414 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.445125 21414 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:37.444550 21379 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.446519 13797 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:37.451072 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3621) took 1028 ms. Trace:
I20260709 12:24:37.451542 13727 rpcz_store.cc:276] 0709 12:24:36.423377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.696821 (+273444us) service_pool.cc:224] Handling call
0709 12:24:36.716883 (+ 20062us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:36.717871 (+   988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.450820 (+732949us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b54003fd9d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:37.450998 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45053824}
I20260709 12:24:37.452203 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1236) took 1004 ms. Trace:
I20260709 12:24:37.454630 14425 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:37.455778 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:37.446184 21379 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:37.416462 21706 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:37.458125 21706 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.458671 21706 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:24:37.459777 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 169898us
I20260709 12:24:37.453173 21568 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.461669 21568 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:37.395651 21555 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:37.465040 21680 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:37.465677 21680 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:37.469000 14424 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:24:37.470801 21718 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:24:37.471572 21716 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:24:37.472903 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4205) took 3618 ms. Trace:
W20260709 12:24:37.473142 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72820.8us
I20260709 12:24:37.474206 21120 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:37.474677 21120 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.475030 21120 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:37.416112 13647 raft_consensus.cc:614] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.462170 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:37.419138 21660 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.473349 13648 rpcz_store.cc:276] 0709 12:24:33.854200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.072064 (+1217864us) service_pool.cc:224] Handling call
0709 12:24:35.133941 (+ 61877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.008103 (+874162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.472332 (+1464229us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b5400096d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:37.472671 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943850368}
W20260709 12:24:37.465059 21659 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.003s
W20260709 12:24:37.481050 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84068.6us
I20260709 12:24:37.458755 21553 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.482879 21553 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:37.483325 21553 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:37.483727 21553 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:37.483615 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:37.483036 13618 leader_election.cc:400] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:37.485438 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:37.485572 21555 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:37.486974 21555 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.488655 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:37.489506 14416 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:37.490381 21580 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.490717 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:37.492515 14418 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:24:37.493239 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.484820 13618 leader_election.cc:403] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:37.473652 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2679) took 3498 ms. Trace:
W20260709 12:24:37.495905 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60765.8us
I20260709 12:24:37.494065 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:37.496414 13649 rpcz_store.cc:276] 0709 12:24:33.975366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.863332 (+2887966us) service_pool.cc:224] Handling call
0709 12:24:36.920212 (+ 56880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.125746 (+205534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.125784 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.146864 (+ 21080us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.249421 (+102557us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:37.272660 (+ 23239us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.318787 (+ 46127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.318912 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.473363 (+154451us) raft_consensus.cc:1685] finished
0709 12:24:37.473386 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.473552 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66078,"prepare.queue_time_us":4564,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":63,"raft.queue_time_us":51988,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":15383,"spinlock_wait_cycles":2761088,"thread_start_us":38502,"threads_started":1,"wal-append.queue_time_us":112713}
I20260709 12:24:37.493912 14419 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:37.393635 21671 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:37.521981 13643 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:37.523432 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2688) took 1337 ms. Trace:
I20260709 12:24:37.525358 21217 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:37.500140 21437 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:37.530042 21660 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:37.547559 21576 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.548811 21576 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:37.542743 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:37.544685 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:37.528162 13648 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:37.553761 21217 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:37.535116 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.556389 21209 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:37.530516 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68483us
I20260709 12:24:37.506671 21693 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:37.559105 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:37.531118 14418 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:24:37.537887 21669 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:37.547596 21437 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.545320 13727 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:37.536332 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1304) took 1451 ms (client timeout 1745 ms). Trace:
I20260709 12:24:37.544277 21671 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:37.531845 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46586.3us
I20260709 12:24:37.536942 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1306) took 1274 ms. Trace:
I20260709 12:24:37.568341 13644 rpcz_store.cc:276] 0709 12:24:36.262104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.615362 (+353258us) service_pool.cc:224] Handling call
0709 12:24:37.536745 (+921383us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400147dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:37.536883 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875681536}
I20260709 12:24:37.558162 13724 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:37.554461 21217 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:37.452661 14420 rpcz_store.cc:276] 0709 12:24:36.447858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.448232 (+   374us) service_pool.cc:224] Handling call
0709 12:24:36.449047 (+   815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.450153 (+  1106us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:24:36.589055 (+138902us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.668560 (+ 79505us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.669418 (+   858us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.669656 (+   238us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:36.669704 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.669808 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.269006 (+599198us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:37.451686 (+182680us) raft_consensus.cc:1685] finished
0709 12:24:37.451816 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.451978 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":151977,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":384,"raft.run_cpu_time_us":539,"raft.run_wall_time_us":543,"spinlock_wait_cycles":5362304,"thread_start_us":154194,"threads_started":2,"wal-append.queue_time_us":44267}
I20260709 12:24:37.610460 14422 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:24:37.618688 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 117301us
I20260709 12:24:37.414368 21690 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:37.566772 13646 rpcz_store.cc:269] 0709 12:24:36.085494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.209633 (+124139us) service_pool.cc:224] Handling call
0709 12:24:37.536076 (+1326443us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:37.536246 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654146304}
I20260709 12:24:37.510366 21546 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:37.621842 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99601.4us
I20260709 12:24:37.506495 21584 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.523743 13643 rpcz_store.cc:276] 0709 12:24:36.185940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.022862 (+836922us) service_pool.cc:224] Handling call
0709 12:24:37.523357 (+500495us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.619585 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:37.534878 21369 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:37.626228 13727 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:37.556425 21659 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:24:37.629562 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.532275 21573 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:37.493135 21580 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:37.641431 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:37.614621 21437 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:37.654825 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:37.622205 21693 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:37.681423 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.643081 21671 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:37.528456 21371 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.129s
W20260709 12:24:37.697383 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 138415us
W20260709 12:24:37.700621 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 81211us
W20260709 12:24:37.701076 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.701675 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.701992 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.705798 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51824.3us
I20260709 12:24:37.656109 21442 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.656904 21659 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:37.693529 21437 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:37.626624 21443 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:24:37.659794 21693 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:37.734611 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.737645 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1681) took 1946 ms. Trace:
I20260709 12:24:37.742388 13724 rpcz_store.cc:276] 0709 12:24:35.791015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.791195 (+   180us) service_pool.cc:224] Handling call
0709 12:24:35.791564 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.557815 (+1766251us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400311da0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:24:37.622487 (+ 64672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.698207 (+ 75720us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:37.737565 (+ 39358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1992265216}
I20260709 12:24:37.734947 21271 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
W20260709 12:24:37.699177 21644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.002s	sys 0.018s
W20260709 12:24:37.764405 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 137089us
W20260709 12:24:37.736357 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.729306 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:37.686672 21546 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.727938 21690 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:37.728426 21655 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:24:37.789067 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 87359.8us
I20260709 12:24:37.685921 21584 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.792902 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:37.793649 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:37.738338 21437 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:37.810209 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:37.772979 21644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.002s	sys 0.019s
I20260709 12:24:37.658723 21209 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.745375 21693 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:37.815464 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.758152 21271 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:37.763864 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:37.799999 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65372.2us
W20260709 12:24:37.832084 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.840009 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46507.1us
W20260709 12:24:37.828835 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.842597 21128 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:37.848032 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:37.843362 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.816465 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.858604 13652 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:24:37.784291 21393 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:37.859974 13645 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:37.865310 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:37.806173 21655 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.753108 21442 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.752941 21637 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 12:24:37.837327 21379 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:37.866123 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:37.877460 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.819931 21437 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:37.794890 21683 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:37.875975 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.841012 21271 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:37.852358 21581 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:37.857292 21128 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.868355 21580 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:37.875123 21655 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:37.896471 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:37.902369 14425 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 12:24:37.904068 21722 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:24:37.911108 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2691) took 1724 ms. Trace:
I20260709 12:24:37.912294 13794 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:37.876804 21379 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.910346 21644 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:37.924759 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:37.939168 13798 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
W20260709 12:24:37.940938 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.913156 21128 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:37.919034 13651 rpcz_store.cc:276] 0709 12:24:36.186764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.209725 (+1022961us) service_pool.cc:224] Handling call
0709 12:24:37.210205 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.270588 (+ 60383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.270620 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.271337 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.478773 (+207436us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:37.478836 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.478939 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.910733 (+431794us) raft_consensus.cc:1685] finished
0709 12:24:37.910758 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.911031 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17788,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":169,"thread_start_us":206930,"threads_started":1,"wal-append.queue_time_us":482395}
W20260709 12:24:37.965670 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.951830 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.972327 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124439us
I20260709 12:24:37.983053 13799 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:37.925349 21379 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:37.926175 21645 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:24:37.976332 21706 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.001s
I20260709 12:24:37.995039 21706 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:37.997695 21706 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:37.988685 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 225012us
W20260709 12:24:37.990558 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.890612 21393 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:37.993324 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:37.993533 21553 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.000s
I20260709 12:24:37.993770 21208 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.178s
I20260709 12:24:37.994287 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:37.992574 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:38.065635 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2690) took 1879 ms. Trace:
I20260709 12:24:37.921784 21414 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:24:38.066423 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:37.941298 21581 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.069667 13645 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:24:38.079587 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.081360 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.085592 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.088119 13652 rpcz_store.cc:276] 0709 12:24:36.186491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.205259 (+1018768us) service_pool.cc:224] Handling call
0709 12:24:37.205686 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.867949 (+662263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.015329 (+147380us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54002e54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:38.065564 (+ 50235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500817280}
I20260709 12:24:38.091159 13646 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21. Re-sending same reply.
I20260709 12:24:38.066444 13801 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
W20260709 12:24:38.099212 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.099993 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.943060 21683 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.106475 13648 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:37.917738 21659 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.066062 21645 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:24:38.086680 21371 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.081748 21658 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:37.943984 21644 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:38.118656 13724 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:24:38.098912 21724 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.156s	user 0.002s	sys 0.003s
I20260709 12:24:38.128237 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:38.133829 13718 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:37.981572 21271 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.141271 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.141324 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 148160us
I20260709 12:24:38.135869 13643 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:24:38.091645 21553 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.089619 21706 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 12:24:38.139385 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 498057us
I20260709 12:24:38.153363 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:38.155670 21393 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.156136 21443 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:38.158167 21656 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.159235 21726 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.102s	user 0.002s	sys 0.003s
I20260709 12:24:38.159724 21434 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:38.144981 21208 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.163722 21645 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:38.164906 21584 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.164987 21381 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:24:38.166517 21725 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:24:38.174885 21644 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.178282 13647 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:38.121439 21568 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.002s
W20260709 12:24:38.180084 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.229012 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:38.199033 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45786.8us
I20260709 12:24:38.160199 21414 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.190580 21680 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.720s	user 0.010s	sys 0.005s
I20260709 12:24:38.219249 21208 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.135103 21371 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.207525 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.232662 13722 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:24:38.239955 14417 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:24:38.241134 21568 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:38.241262 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.242066 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.242715 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.243407 13644 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:38.234756 21584 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.252038 21729 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:38.241856 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.204702 21442 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.252616 21660 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:24:38.259774 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.214622 21602 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.215196 21645 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:38.287771 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:38.289031 21217 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.001s
I20260709 12:24:38.280931 13647 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:38.252038 21553 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.285573 21660 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:38.290429 21584 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.290963 21581 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.291741 21660 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.293190 21660 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:38.260600 21414 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.293551 21584 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.285724 13651 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:38.295428 21558 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.295639 21371 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.287966 21645 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:38.246425 21524 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:38.289644 21217 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.290625 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97660.3us
I20260709 12:24:38.295753 21414 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.306082 13718 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
W20260709 12:24:38.306381 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.286917 21602 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.298892 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1691) took 1008 ms. Trace:
I20260709 12:24:38.208508 21656 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.255505 21680 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.304832 13726 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:38.277964 21568 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:24:38.300815 21369 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.640s	user 0.013s	sys 0.001s
I20260709 12:24:38.253165 21693 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.314023 13643 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:24:38.313719 13721 rpcz_store.cc:276] 0709 12:24:37.290956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.353470 (+ 62514us) service_pool.cc:224] Handling call
0709 12:24:37.365116 (+ 11646us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:37.394816 (+ 29700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.394848 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:37.480108 (+ 85260us) log.cc:844] Serialized 52 byte log entry
0709 12:24:38.079336 (+599228us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:38.079388 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.081000 (+  1612us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.298621 (+217621us) raft_consensus.cc:1685] finished
0709 12:24:38.298648 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.298824 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20844,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":12769,"thread_start_us":457775,"threads_started":1,"wal-append.queue_time_us":659798}
W20260709 12:24:38.318677 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.312647 21669 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.002s
W20260709 12:24:38.334277 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 468182us
I20260709 12:24:38.236200 21434 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.332257 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:38.336915 13649 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:38.315410 21683 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.333372 21524 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:38.356333 21524 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.320724 21128 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.359874 21680 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.360205 21656 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.361076 21128 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:38.362406 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.363863 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:38.364069 21298 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:38.342414 14421 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:24:38.336354 21637 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.345206 21208 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.346566 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:38.348654 21434 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.300652 21690 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:24:38.322042 21568 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.370047 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:38.378876 14422 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:38.376633 13727 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:38.380066 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:38.312469 21437 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:38.381096 13722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3624) took 1943 ms (client timeout 1801 ms). Trace:
I20260709 12:24:38.385170 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:38.385072 13722 rpcz_store.cc:269] 0709 12:24:36.425135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.458451 (+ 33316us) service_pool.cc:224] Handling call
0709 12:24:37.407548 (+949097us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:24:38.367895 (+960347us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b5400174654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:38.368061 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911604992}
W20260709 12:24:38.369462 13617 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.005s	sys 0.012s
I20260709 12:24:38.380842 13645 raft_consensus.cc:606] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:38.386690 13617 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.005s	sys 0.012s
I20260709 12:24:38.386858 13645 raft_consensus.cc:614] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:38.387183 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 542610us
I20260709 12:24:38.368201 21298 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:38.392985 13650 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:24:38.370182 21369 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.413586 13646 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:38.414572 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:38.392903 21671 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.004s
W20260709 12:24:38.424552 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.438949 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:38.439826 14419 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:38.441535 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.395267 21690 consensus_queue.cc:1243] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:38.392474 21546 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.367543 21655 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.440871 21555 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.003s	sys 0.017s
W20260709 12:24:38.459298 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.459677 13650 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:24:38.460636 21298 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.462076 21655 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:38.462486 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.453830 21732 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:38.468364 21602 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.503180 13643 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:24:38.503516 13801 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:38.388757 21637 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.465440 21271 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:38.505306 13792 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:24:38.506214 21637 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.507700 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:38.507372 13541 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 15 to 16, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 16 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:38.508697 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:38.466049 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.466771 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.511606 13794 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:24:38.511679 13651 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:38.425261 21369 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.468057 21120 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.993s	user 0.006s	sys 0.009s
I20260709 12:24:38.512992 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:38.513808 13797 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:38.468428 21576 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.919s	user 0.016s	sys 0.000s
I20260709 12:24:38.514626 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1478) took 1645 ms. Trace:
I20260709 12:24:38.515496 21298 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:38.515125 14417 rpcz_store.cc:276] 0709 12:24:36.868945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.869097 (+   152us) service_pool.cc:224] Handling call
0709 12:24:36.880677 (+ 11580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.266681 (+1386004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.514276 (+247595us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:38.514507 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3109175168}
I20260709 12:24:38.471158 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:38.517956 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46862.2us
I20260709 12:24:38.472100 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:38.519204 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.474289 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.475314 21693 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.476117 21739 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:38.476727 14416 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:24:38.477099 13796 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:38.477721 21584 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:24:38.445147 21553 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:38.479528 21437 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.456686 21217 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:38.476766 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158915us
I20260709 12:24:38.460850 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:38.543449 14420 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:38.511032 21669 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:24:38.520323 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 280290us
I20260709 12:24:38.518850 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:38.526227 21546 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.525955 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:38.526816 21128 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.526142 21655 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:38.545622 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:38.526482 21740 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:24:38.528663 21380 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:24:38.532047 21730 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.328s	user 0.007s	sys 0.002s
I20260709 12:24:38.479826 21379 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.502009 21732 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:38.539119 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 374559us
I20260709 12:24:38.513474 21641 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:38.513538 21120 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.560321 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:38.556027 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.548270 13643 raft_consensus.cc:606] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:38.591645 13727 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:24:38.579440 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:38.538589 21738 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:24:38.520244 21555 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.004s	sys 0.017s
I20260709 12:24:38.548511 14417 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:38.592922 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.594803 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:38.596766 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 307258us
I20260709 12:24:38.603675 13650 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:38.603920 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:38.561223 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:38.605043 21437 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:38.605305 13801 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:38.605629 21437 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.606034 21437 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.606654 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:38.607735 21271 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:38.610052 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:38.611862 21271 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:38.605089 21693 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:38.612541 21271 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:38.603675 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:38.613931 21271 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.629721 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:38.630846 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:38.590621 21641 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:38.546186 21669 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:38.546475 21553 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.629973 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.642936 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:38.509631 13541 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 13, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 13 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:38.646659 13649 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:38.655521 13541 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 12 to 15, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 15 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:38.661118 13793 raft_consensus.cc:2368] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
I20260709 12:24:38.569401 21655 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.601357 21738 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.663533 13647 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:38.647735 21553 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:38.570020 21128 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.597780 21564 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:24:38.633023 21558 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:24:38.635174 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56203.5us
I20260709 12:24:38.634521 21644 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.686769 13767 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.886s (SENT)
I20260709 12:24:38.658912 21706 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:38.598165 21645 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.682585 21658 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.686116 21738 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:38.660658 21659 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.659829 21743 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:38.727052 13722 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:38.642392 21524 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:24:38.728325 21379 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.630163 21546 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.721428 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 130877us
I20260709 12:24:38.724264 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:38.720482 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 201708us
W20260709 12:24:38.726579 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.714473 21706 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:38.732630 21602 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:38.727092 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:38.741710 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:38.747437 21369 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.759542 21369 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.760130 21369 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.761655 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:24:38.750741 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72025.8us
I20260709 12:24:38.750957 21568 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.763641 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:38.730660 21658 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.741371 21380 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:24:38.758438 21602 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:38.785208 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43651.5us
I20260709 12:24:38.785781 21602 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.743630 21581 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:24:38.786300 21759 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:24:38.785485 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:38.762538 21369 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.728839 21524 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:24:38.763123 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.762488 21560 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.792183 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.780454 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 162314us
W20260709 12:24:38.798789 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 88995.3us
W20260709 12:24:38.791546 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.799325 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.789556 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:38.724521 21669 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.729318 21644 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.736937 21641 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.788749 21568 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.814775 21560 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.787600 21581 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:38.836702 21581 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:38.837638 21644 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:38.790731 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 233) took 61939 ms (client timeout 15000 ms). Trace:
W20260709 12:24:38.862612 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.555s (SENT)
I20260709 12:24:38.864284 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:38.866063 21369 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:24:38.866807 14421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1186) took 1550 ms (client timeout 1683 ms). Trace:
I20260709 12:24:38.866943 21756 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:38.867260 14421 rpcz_store.cc:269] 0709 12:24:37.316128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.316282 (+   154us) service_pool.cc:224] Handling call
0709 12:24:38.866570 (+1550288us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b5400644bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:38.866712 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181841792}
W20260709 12:24:38.787034 21741 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:24:38.737845 21659 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.870813 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81334.9us
I20260709 12:24:38.875335 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.791918 21442 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:38.835175 13690 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.683s (SENT)
I20260709 12:24:38.879601 13718 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:24:38.887625 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:38.819293 21753 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:38.796631 21564 raft_consensus.cc:993] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: : Instructing follower a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:24:38.787137 21602 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.792136 21656 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.883523 14423 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:24:38.788133 21734 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:38.791801 21660 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.792025 21434 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.792272 21443 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.889237 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:38.889534 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.838805 21558 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:38.911310 13726 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:38.912981 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.604s (SENT)
I20260709 12:24:38.890411 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.891244 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.914588 13721 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:38.915030 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.893173 21442 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.883394 21584 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:38.879791 21576 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:38.820621 21546 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.825970 21732 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.893640 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:38.894546 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:38.878007 14419 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:38.825073 21128 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.940490 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:38.941195 14422 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:24:38.927089 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:38.833086 21658 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.814378 21645 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:38.943135 13794 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:24:38.898389 21669 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:38.901489 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:38.946452 21762 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.188s	user 0.000s	sys 0.004s
I20260709 12:24:38.946915 14424 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:38.948168 21298 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.948859 21298 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.838151 21560 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.906543 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:38.950788 13718 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:38.952730 21298 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:38.952595 21706 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.907466 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:38.953751 21706 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:38.899106 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.917646 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:38.915638 13722 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 13.
I20260709 12:24:38.843005 14420 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:38.919376 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 3992) took 1132 ms. Trace:
I20260709 12:24:38.795197 21757 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.104s
I20260709 12:24:38.960009 13796 rpcz_store.cc:276] 0709 12:24:37.778904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.787650 (+  8746us) service_pool.cc:224] Handling call
0709 12:24:38.911032 (+1123382us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:38.911197 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530994048}
W20260709 12:24:38.969957 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43492us
I20260709 12:24:38.788291 21209 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 12:24:38.923789 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:38.925217 14421 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:24:38.898650 21380 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:24:38.910774 21437 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.927035 21690 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:38.916471 21369 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:24:38.928457 21442 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:38.922868 21558 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:38.929342 21576 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:38.936280 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 194881us
W20260709 12:24:38.936708 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101587us
I20260709 12:24:38.988128 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:38.988515 13690 leader_election.cc:336] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.711s (SENT)
I20260709 12:24:38.917717 21660 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.919909 21734 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:38.912293 21656 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.917987 21443 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.837707 21581 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.941248 13792 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:38.981194 21732 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:39.090071 21732 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:39.091404 21732 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.905784 21559 raft_consensus.cc:1081] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Signalling peer a4188193a10c4d50a555d76ea85c60b7 to start an election
I20260709 12:24:38.797508 21524 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:39.094640 21524 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.095494 21739 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:39.096107 21739 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:39.096706 21524 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:38.957557 21393 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:39.097350 21739 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:38.967514 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:39.098996 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 131592us
W20260709 12:24:38.968070 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.102908 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 114858us
I20260709 12:24:38.790449 21759 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.119s
I20260709 12:24:38.971375 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:38.976070 21738 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:38.921470 21584 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.985520 13796 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:24:38.993270 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.200917 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 212440us
W20260709 12:24:39.201695 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.917069 21434 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.085372 21546 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.031394 13797 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:39.000833 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1486) took 1121 ms. Trace:
I20260709 12:24:38.889343 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:38.999161 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:39.087790 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59989.5us
W20260709 12:24:39.088992 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.839591 13547 rpcz_store.cc:269] 0709 12:23:36.850721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:36.850968 (+   247us) service_pool.cc:224] Handling call
0709 12:24:38.790427 (+61939459us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:24.675441 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:24.747531 (+ 72090us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:24.747574 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:24:24.747593 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:24.748574 (+   981us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:24.748641 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:24.748670 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:24:24.749348 (+   678us) tablet.cc:701] Row locks acquired
0709 12:24:24.749364 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:24:24.749664 (+   300us) write_op.cc:270] Start()
0709 12:24:24.749801 (+   137us) write_op.cc:276] Timestamp: P: 1783599864749593 usec, L: 0
0709 12:24:24.749820 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:24:24.789538 (+ 39718us) log.cc:844] Serialized 3228 byte log entry
0709 12:24:26.549383 (+1759845us) op_driver.cc:464] REPLICATION: finished
0709 12:24:26.750098 (+200715us) write_op.cc:301] APPLY: starting
0709 12:24:26.750339 (+   241us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:27.027073 (+276734us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:27.027318 (+   245us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:27.103543 (+ 76225us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:27.154339 (+ 50796us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:24:27.154424 (+    85us) write_op.cc:312] APPLY: finished
0709 12:24:27.154972 (+   548us) log.cc:844] Serialized 81 byte log entry
0709 12:24:27.540267 (+385295us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:27.551530 (+ 11263us) write_op.cc:454] Released schema lock
0709 12:24:27.551622 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:27.552300 (+   678us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:27.552337 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1106640,"reported_tablets":50,"spinlock_wait_cycles":1920,"child_traces":[["op",{"apply.queue_time_us":164336,"apply.run_cpu_time_us":73852,"apply.run_wall_time_us":825480,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37631,"num_ops":8,"prepare.queue_time_us":326433,"prepare.run_cpu_time_us":27275,"prepare.run_wall_time_us":396790,"raft.queue_time_us":963385,"raft.run_cpu_time_us":25813,"raft.run_wall_time_us":369134,"replication_time_us":1563248,"thread_start_us":1333435,"threads_started":5,"wal-append.queue_time_us":928513,"wal-append.run_cpu_time_us":115626,"wal-append.run_wall_time_us":2672981}]]}
W20260709 12:24:39.210003 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.801s (SENT)
I20260709 12:24:39.099946 21559 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:39.101056 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:39.209617 13724 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:24:39.219808 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 118903us
I20260709 12:24:38.789752 13541 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:39.006405 21442 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.204s
I20260709 12:24:39.093178 21732 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.194309 13718 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:39.105300 13801 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:24:39.210481 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:39.196316 21763 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:39.199978 21680 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.212601 21208 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.212934 21576 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.213779 21659 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:39.214010 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 214948us
I20260709 12:24:39.214514 21414 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:39.243671 21414 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:39.244014 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:39.245653 21414 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:39.105854 21637 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:39.207864 21581 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:39.246464 21637 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:39.247326 21637 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:39.248797 21739 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.250636 21217 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:39.252079 21443 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.252225 21680 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.252782 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:39.253579 13801 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:39.254308 21656 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.254472 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:39.254719 21680 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.255501 21763 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.209012 14423 rpcz_store.cc:276] 0709 12:24:37.879514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.879714 (+   200us) service_pool.cc:224] Handling call
0709 12:24:38.868900 (+989186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.900480 (+ 31580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.996635 (+ 96155us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400644bd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:39.000739 (+  4104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2272134528}
W20260709 12:24:39.133486 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44265.2us
I20260709 12:24:39.232951 21558 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.233703 21434 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:39.222972 21271 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:39.234472 21546 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.237176 14421 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:24:39.208324 21747 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.241168 21756 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:39.241729 21769 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:24:39.207453 21584 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.269176 13718 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:39.273135 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.275357 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58368us
I20260709 12:24:39.258898 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:39.193671 21553 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.233626 21655 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:39.288457 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.277446 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:39.237329 21559 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:39.295930 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:39.296680 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:39.301882 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.302796 13649 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 14: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:39.299798 13796 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 14.
I20260709 12:24:39.274621 21560 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:39.297791 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:39.307219 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 86755us
I20260709 12:24:39.307972 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.270220 21208 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.321867 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:39.294124 21769 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.312100 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:39.328274 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.273968 21271 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.298897 21573 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:24:39.336712 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.323436 13647 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
W20260709 12:24:39.310968 21768 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:39.342329 13796 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:39.310580 21434 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:39.345583 13718 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:39.335022 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:39.348922 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.351493 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:39.334975 21564 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:39.330982 21680 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:39.340948 21656 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:39.374924 21656 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.375401 21656 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.375710 21553 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.387292 21738 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.387579 21414 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:39.388458 21414 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.360674 21564 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:39.390725 21564 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:39.391119 21559 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.395771 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.396521 13721 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:39.397684 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.398572 13719 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:39.398635 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.400192 21660 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.291252 21747 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.404119 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:39.404765 21747 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.405961 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:39.337216 21769 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:24:39.360049 21680 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:39.407828 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67082.1us
I20260709 12:24:39.363441 13652 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:24:39.363818 21443 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:24:39.233017 13541 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:39.336273 21379 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:24:39.350136 21434 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.419289 21434 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.420179 21208 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.420825 21208 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:39.339097 21759 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:39.422801 21759 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:39.360882 21655 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:39.423933 21759 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.424918 21442 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:39.425585 21442 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.425978 21442 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:24:39.362088 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152112us
W20260709 12:24:39.427013 13617 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.853s (SENT)
I20260709 12:24:39.427461 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:39.428332 21641 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:39.429384 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.918s (SENT)
I20260709 12:24:39.429863 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:39.439479 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 131038us
I20260709 12:24:39.440428 21128 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:39.441375 21763 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.441651 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:39.407760 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.390070 21271 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:39.408618 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.409312 21680 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:39.418502 21437 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.363154 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.430636 21573 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.330722 21744 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:24:39.431173 21738 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:39.435956 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.439482 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.440347 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:39.467463 13718 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:39.293761 21734 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:39.433539 21758 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.675s	user 0.013s	sys 0.000s
W20260709 12:24:39.445088 13690 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 2.035s (SENT)
W20260709 12:24:39.444558 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.443933 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:39.442773 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.443231 13646 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:39.478538 13650 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:39.480146 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.436108 21564 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.446631 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:39.486207 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:39.492753 13792 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
W20260709 12:24:39.493893 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.497010 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:39.502880 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:39.503727 14423 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.423884 21655 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.542676 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:39.513341 21602 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 12:24:39.485698 21208 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:39.515707 21437 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:24:39.545776 21120 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:39.519641 21555 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:39.546891 21555 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:39.546943 21380 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:39.547920 13652 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 16. Ops: [14.11-14.11]
I20260709 12:24:39.516197 14417 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:24:39.480086 21763 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:39.530951 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 90702.4us
I20260709 12:24:39.481935 21369 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:39.553447 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46674.4us
I20260709 12:24:39.455968 21660 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.078s
I20260709 12:24:39.522086 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.562219 14424 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:39.475809 13796 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:60492
I20260709 12:24:39.563416 13796 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:39.563835 13796 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:39.522298 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4003) took 1018 ms. Trace:
I20260709 12:24:39.540061 13724 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:39.556612 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.556908 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.558300 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.559764 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.564343 13793 rpcz_store.cc:276] 0709 12:24:38.503905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.507393 (+  3488us) service_pool.cc:224] Handling call
0709 12:24:39.521997 (+1014604us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:39.522174 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322098944}
I20260709 12:24:39.566857 14417 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:39.547788 21380 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:39.544054 21564 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.481220 21573 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.548020 21555 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.551295 21434 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.606294 21414 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.609335 21434 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:39.519243 21298 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:39.578449 21437 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:39.610131 21637 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.582145 13652 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:24:39.610678 13796 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.612493 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.613445 13796 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.584141 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.614563 13644 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:39.614760 21271 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.615368 21271 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:39.616479 13644 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:39.616676 13649 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:39.617599 21271 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.587957 13650 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 19. Ops: []
I20260709 12:24:39.564021 21369 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:39.618746 21706 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.010s
I20260709 12:24:39.588654 14425 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:39.603986 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:39.607319 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4255) took 1044 ms. Trace:
I20260709 12:24:39.620365 13793 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:39.620282 13648 rpcz_store.cc:276] 0709 12:24:38.562830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.563034 (+   204us) service_pool.cc:224] Handling call
0709 12:24:38.593910 (+ 30876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.610544 (+ 16634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.610583 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.611271 (+   688us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.693497 (+ 82226us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:38.718674 (+ 25177us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.721952 (+  3278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.722046 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.100954 (+378908us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b1800136300. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:39.432832 (+331878us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:39.607018 (+174186us) raft_consensus.cc:1685] finished
0709 12:24:39.607058 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.607248 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22577,"prepare.queue_time_us":28746,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":105201,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":10111,"spinlock_wait_cycles":27938048,"thread_start_us":81846,"threads_started":1,"wal-append.queue_time_us":146723}
I20260709 12:24:39.610388 21437 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.623281 21380 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.624712 21380 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.625547 13651 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:24:39.549736 21738 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:24:39.628597 21573 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:39.586653 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.624806 21437 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.593665 21208 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:24:39.630537 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:39.515097 21217 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:39.600358 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.642604 14419 raft_consensus.cc:2410] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:39.598348 14391 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27.
I20260709 12:24:39.643966 14391 leader_election.cc:403] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:39.576088 21669 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.001s
W20260709 12:24:39.634241 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 194039us
I20260709 12:24:39.630210 13647 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:39.637113 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:39.512288 21655 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:39.632287 13618 leader_election.cc:400] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
I20260709 12:24:39.674716 13618 leader_election.cc:403] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:24:39.613364 13652 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:39.675282 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:39.641196 21437 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.006s
I20260709 12:24:39.646692 21393 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:24:39.646936 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:39.649201 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:39.696079 14425 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.610069 21298 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:39.642969 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:39.600927 21546 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.678946 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:39.706529 14424 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:39.577984 21769 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:39.679896 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.679733 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:39.682075 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.712517 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 306584us
W20260709 12:24:39.713235 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81090.4us
I20260709 12:24:39.676435 13796 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:39.687224 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 326212us
I20260709 12:24:39.693836 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:39.620185 21369 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:39.636806 13693 connection.cc:256] RPC call timeout handler was delayed by -2.6637639109999327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:24:39.704770 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61892.8us
I20260709 12:24:39.724639 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.726011 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:39.682456 13793 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
W20260709 12:24:39.737408 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.721073 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:39.746037 13800 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:39.737010 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:39.721935 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:39.753911 13718 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:39.766335 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.775238 13646 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:39.776098 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.757910 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:39.784670 21744 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:39.785256 21744 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.599942 21584 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.680181 21217 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:39.786782 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:39.786315 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.787608 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:39.805303 13727 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:24:39.806378 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.806980 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:39.788406 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.790673 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.681545 21645 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:24:39.783233 21739 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.534s	user 0.014s	sys 0.000s
W20260709 12:24:39.794211 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.796746 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82229.5us
I20260709 12:24:39.789143 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.777886 13796 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:39.796861 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:39.786587 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49733us
W20260709 12:24:39.831526 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.701462 21271 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:39.810437 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:39.655126 21738 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.117s
I20260709 12:24:39.705392 21763 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27.
I20260709 12:24:39.790010 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:39.832757 21271 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.833395 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:39.834002 13726 raft_consensus.cc:2410] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:24:39.835055 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.835407 21271 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:39.756428 21573 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:39.833088 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:39.826933 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.846648 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:39.847565 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:39.836383 13719 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:39.832775 13652 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:39.785739 21744 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:39.857276 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.848603 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:39.847483 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.864811 13649 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:39.865211 13723 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:39.792179 21414 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:39.852329 14422 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:39.782368 21434 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:39.854233 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.869899 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.870455 13725 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:24:39.871536 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 273398us
I20260709 12:24:39.864727 13650 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23. Re-sending same reply.
I20260709 12:24:39.876721 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.877709 13721 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:39.877534 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:39.885213 13648 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:24:39.830613 21546 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:39.830930 21120 raft_consensus.cc:2764] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:24:39.903823 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.897148 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:39.899334 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.155s (SENT)
I20260709 12:24:39.890322 21298 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.896091 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.820871 21645 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:39.786315 21442 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:39.874013 21581 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:24:39.883728 21524 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.786s	user 0.017s	sys 0.000s
I20260709 12:24:39.923506 13725 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:39.950960 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:39.925290 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.206s (SENT)
I20260709 12:24:39.953267 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:39.914942 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.933169 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.878659 21414 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.962173 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:39.963361 13650 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:39.952258 13726 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:39.939564 14424 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:39.941510 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.879033 21434 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.975771 21706 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:39.976475 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:39.976707 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51459.3us
I20260709 12:24:39.978241 21759 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:39.979947 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:39.981562 13722 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:39.987414 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:39.988521 13721 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:39.969897 21434 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:39.990619 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.970057 21414 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:39.837967 13541 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 19 to 20, leader changed from <none> to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 20 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:24:40.003687 21732 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.004s
I20260709 12:24:40.005560 21637 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:39.968501 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:40.013008 13721 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:39.929754 21546 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:40.030021 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:40.036629 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.039059 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.045470 13726 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:40.044979 13722 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:40.048518 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.054211 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:40.059407 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 226088us
I20260709 12:24:40.065511 13725 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:24:40.063575 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.006954 21393 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:40.076092 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:40.077162 13650 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:40.080632 13650 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:40.084707 13727 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
W20260709 12:24:40.078470 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 141764us
I20260709 12:24:40.070334 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:40.086948 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.037964 21660 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:24:40.059376 21767 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:24:40.082046 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:40.107981 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:40.061251 21298 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:40.119239 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49140.1us
I20260709 12:24:40.104735 13650 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.133762 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:40.137907 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55956.5us
I20260709 12:24:40.126499 13723 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:40.149566 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.151587 13650 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:40.140969 13723 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:40.160847 13725 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:40.164494 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 331496us
I20260709 12:24:40.014725 21217 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.147081 21706 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:24:40.148042 21656 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.772s	user 0.016s	sys 0.000s
I20260709 12:24:40.128386 21660 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:40.118637 21393 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:40.186362 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.151325 21743 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:24:40.147353 21680 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.001s
I20260709 12:24:40.073415 13541 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:40.230991 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.163374 21584 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:40.204002 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 96159.1us
I20260709 12:24:40.288832 13724 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
W20260709 12:24:40.292339 21787 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:40.223976 21546 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.299245 13727 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:24:40.299759 13727 pending_rounds.cc:85] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 11
I20260709 12:24:40.299997 13727 pending_rounds.cc:107] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:24:40.300238 13727 raft_consensus.cc:2889] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:40.301510 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.302680 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.303987 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.305063 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.305758 13726 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:40.306622 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.307245 13722 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:40.308020 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.309033 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.309505 13722 raft_consensus.cc:2376] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14. Re-sending same reply.
I20260709 12:24:40.209200 13723 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.201339 21637 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:40.237167 13725 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:24:40.105522 21645 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.234824 21706 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:40.275420 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.232581 21782 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:40.224380 21769 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:40.280817 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45580.7us
I20260709 12:24:40.274046 21602 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.312742 14425 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.002s
I20260709 12:24:40.327338 14425 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:40.313416 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.329195 13722 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:40.329367 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.319203 13723 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.274039 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.330454 13727 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:40.330456 13720 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.289341 21743 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.243659 13794 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:24:40.327267 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:40.313692 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 271189us
I20260709 12:24:40.319294 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:40.324501 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:40.322618 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.315564 21442 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.346493 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:40.321139 14423 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.003s
I20260709 12:24:40.350232 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.350675 14423 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:40.351967 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.342519 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.344261 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:40.344835 13726 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.326156 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:40.324994 21706 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:40.349969 21524 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.347903 21645 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:40.364280 21524 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.347541 21743 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.354455 21763 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
W20260709 12:24:40.373890 21573 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.002s
I20260709 12:24:40.356050 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:40.357399 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:40.378903 13795 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:40.379832 21779 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:40.383864 21573 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
I20260709 12:24:40.365473 21208 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:40.376009 21524 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.376807 21744 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.383378 21660 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.349713 13800 raft_consensus.cc:2368] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
W20260709 12:24:40.359777 21781 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:24:40.354163 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:40.408012 13650 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:40.408840 13793 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:40.409027 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.352165 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:40.388527 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:40.394644 13652 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 12:24:40.389015 21208 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:40.327008 21637 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:40.290758 21656 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.406061 13722 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:40.406014 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.343796 21602 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:40.387027 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.313638 21680 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.361886 21414 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.415524 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.437894 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:40.441586 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.440718 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.446010 13722 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:24:40.448212 13719 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.011s
I20260709 12:24:40.448655 13719 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:24:40.440904 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 88838.3us
I20260709 12:24:40.450412 13652 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:40.455536 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.456770 13650 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
W20260709 12:24:40.457573 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.458092 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.460681 13652 raft_consensus.cc:2410] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:40.472775 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:40.474272 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.474754 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:40.475404 14417 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:40.475608 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:40.477058 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:40.478096 13650 raft_consensus.cc:2368] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
I20260709 12:24:40.436167 21208 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.484844 13797 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:40.479378 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:40.479848 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:40.480800 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.465952 13726 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:40.486760 13723 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:40.483175 13647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.010s	sys 0.009s
W20260709 12:24:40.487828 13647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.010s	sys 0.009s
I20260709 12:24:40.488865 14425 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:24:40.487957 13724 raft_consensus.cc:2410] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:24:40.490240 13724 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:40.491894 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:40.492681 14419 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:40.493139 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:40.491003 13724 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:40.490182 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 141470us
I20260709 12:24:40.404284 21393 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.499318 13727 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:40.472841 21778 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.500344 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84925.4us
I20260709 12:24:40.500411 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:40.500866 14422 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:40.502091 13722 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:40.452374 21799 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:40.502120 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.502794 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.506546 13726 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.503450 13724 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:40.498929 21655 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.798s	user 0.007s	sys 0.007s
I20260709 12:24:40.511855 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:40.514930 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:40.504840 21791 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:24:40.515267 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2695) took 1243 ms. Trace:
I20260709 12:24:40.513163 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:40.494522 21524 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:40.499557 21546 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:40.515653 13650 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:40.515910 13645 rpcz_store.cc:276] 0709 12:24:39.271904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.272127 (+   223us) service_pool.cc:224] Handling call
0709 12:24:39.289795 (+ 17668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.297303 (+  7508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.297336 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.523845 (+226509us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b1800105c40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a
0709 12:24:39.524616 (+   771us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:24:39.525303 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.838683 (+313380us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:39.838725 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.838801 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.515021 (+676220us) raft_consensus.cc:1685] finished
0709 12:24:40.515043 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.515194 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12686,"raft.queue_time_us":330,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4580,"spinlock_wait_cycles":28396672,"thread_start_us":312889,"threads_started":1,"wal-append.queue_time_us":497842}
W20260709 12:24:40.517745 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.470976 21434 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.520326 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.523443 14420 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
W20260709 12:24:40.528393 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.529171 13726 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:40.534371 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:40.543581 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.513355 21573 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:40.514729 21564 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:40.549796 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:40.384517 13541 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 11 to 13, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 13 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:40.549993 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.556468 13724 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.550697 21434 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:40.550889 13799 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:40.551965 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51957.2us
I20260709 12:24:40.516551 21799 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:40.546765 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.562557 13721 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:24:40.534559 21769 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:40.534879 21763 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:40.509029 21778 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.567795 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.518934 21655 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:40.553789 21780 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
W20260709 12:24:40.578043 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.595408 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.596292 13651 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:24:40.596767 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1491) took 1119 ms. Trace:
I20260709 12:24:40.597291 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.556089 21802 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:24:40.531706 21794 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:24:40.535444 21524 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.544049 21584 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:24:40.544929 21637 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.581637 13650 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:40.591207 21799 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:40.567123 21380 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.942s	user 0.014s	sys 0.000s
I20260709 12:24:40.607372 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:40.677424 14425 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.678705 21558 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.676257 21743 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.655476 21803 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.607703 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:40.600342 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4018) took 1136 ms. Trace:
I20260709 12:24:40.627105 13645 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:40.599211 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:40.701071 13718 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:40.597198 14421 rpcz_store.cc:276] 0709 12:24:39.477397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.477646 (+   249us) service_pool.cc:224] Handling call
0709 12:24:39.492246 (+ 14600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.492730 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.492760 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.498220 (+  5460us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.704698 (+206478us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:39.704744 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.704823 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.596501 (+891678us) raft_consensus.cc:1685] finished
0709 12:24:40.596531 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.596703 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":180,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":15306,"thread_start_us":195115,"threads_started":1,"wal-append.queue_time_us":673270}
I20260709 12:24:40.607973 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:40.702487 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94627us
I20260709 12:24:40.704020 14391 leader_election.cc:400] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
I20260709 12:24:40.704290 14391 leader_election.cc:403] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:40.679301 21558 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.710839 21558 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.694687 21680 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.711414 21558 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:40.712054 21524 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.711993 21558 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.700069 13797 rpcz_store.cc:276] 0709 12:24:39.464054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.464242 (+   188us) service_pool.cc:224] Handling call
0709 12:24:40.600148 (+1135906us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b54001bbcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:40.600284 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294110592}
I20260709 12:24:40.565361 21744 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:40.716250 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2709) took 1135 ms. Trace:
I20260709 12:24:40.716706 13646 rpcz_store.cc:276] 0709 12:24:39.581234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.582580 (+  1346us) service_pool.cc:224] Handling call
0709 12:24:39.582917 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.775770 (+192853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.715968 (+940198us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54002b3ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:40.716180 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441800960}
I20260709 12:24:40.598068 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:24:40.719765 13651 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:40.689370 14425 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:40.586033 21573 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:40.698218 13725 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 14.
I20260709 12:24:40.606894 21602 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:40.674934 21298 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
W20260709 12:24:40.606232 21796 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.306s	user 0.001s	sys 0.001s
I20260709 12:24:40.600317 21655 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.707465 21743 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.708197 21208 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.728822 13720 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:40.694334 13647 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:40.723884 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:40.651214 21656 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.605019 21769 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.740159 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 205876us
I20260709 12:24:40.733332 13724 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:24:40.743405 13724 pending_rounds.cc:85] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:24:40.744093 13724 pending_rounds.cc:107] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:24:40.729597 21805 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:40.584681 21763 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.744421 13724 raft_consensus.cc:2889] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:40.752159 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 186571us
I20260709 12:24:40.754833 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:40.754949 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:40.757437 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:40.761794 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.764539 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.729597 21803 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.747781 21655 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:40.768724 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:40.744365 21769 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.762398 21602 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.599131 13541 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:40.767562 21735 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:40.763065 21808 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:40.778362 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4017) took 1341 ms. Trace:
I20260709 12:24:40.772408 21584 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:40.783082 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.774421 21546 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:40.771849 21558 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.768415 21802 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 12:24:40.785147 13801 rpcz_store.cc:276] 0709 12:24:39.436791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.475358 (+ 38567us) service_pool.cc:224] Handling call
0709 12:24:39.504916 (+ 29558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.797966 (+293050us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.798018 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.844786 (+ 46768us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.230036 (+385250us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:40.231941 (+  1905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.232059 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.605584 (+373525us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:40.773290 (+167706us) raft_consensus.cc:1685] finished
0709 12:24:40.773325 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.778156 (+  4831us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30790,"raft.queue_time_us":63945,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":72999,"spinlock_wait_cycles":37501824,"thread_start_us":612116,"threads_started":2,"wal-append.queue_time_us":415433}
W20260709 12:24:40.778946 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75046.8us
I20260709 12:24:40.779657 21655 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:40.782857 21735 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.784159 21810 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.798424 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.801999 21814 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.816725 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.815351 21814 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.785113 21808 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.819489 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:40.809136 21809 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:24:40.823804 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.782488 21602 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.792393 21794 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:40.831808 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:40.832175 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.839294 13647 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:40.813352 21776 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.521s	user 0.001s	sys 0.001s
I20260709 12:24:40.833032 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:40.833804 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:40.840198 13650 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.822873 21816 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:40.840684 13650 raft_consensus.cc:614] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:40.840911 21816 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.835084 13649 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:40.843255 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.843990 13649 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:40.823717 21706 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.845142 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.846684 21208 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.846916 21706 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:40.847851 21817 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
W20260709 12:24:40.812454 21798 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
W20260709 12:24:40.811187 21795 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.367s	user 0.003s	sys 0.000s
I20260709 12:24:40.843559 21816 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.852121 21763 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.833122 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:40.854300 21814 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.802067 21735 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:40.795889 21558 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.847929 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:40.788878 21807 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:24:40.841966 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:40.846078 13649 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:40.862764 21794 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:40.864393 21814 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:40.865017 21814 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:40.812899 21802 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:40.867278 14391 leader_election.cc:400] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
I20260709 12:24:40.801926 21584 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:24:40.870782 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.865424 21814 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:40.830706 21808 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:40.874054 21524 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:24:40.875844 13541 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 13 to 20. New cstate: current_term: 20 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:40.879798 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.868564 21802 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:40.884766 21803 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:40.884495 21802 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.885864 21208 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.886474 21813 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:24:40.887795 21769 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.887990 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4274) took 1067 ms. Trace:
I20260709 12:24:40.897347 13648 rpcz_store.cc:276] 0709 12:24:39.820459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.616301 (+795842us) service_pool.cc:224] Handling call
0709 12:24:40.616752 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.617149 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.617186 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.617701 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.744654 (+126953us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:40.744739 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.744818 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.887769 (+142951us) raft_consensus.cc:1685] finished
0709 12:24:40.887792 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.887923 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1229,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"thread_start_us":126521,"threads_started":1,"wal-append.queue_time_us":166875}
I20260709 12:24:40.820771 21810 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.899001 21602 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:40.900300 13541 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:40.901306 21810 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:40.901860 21810 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:40.868675 21558 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.886075 21802 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:40.904697 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4285) took 1072 ms. Trace:
I20260709 12:24:40.912233 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.934065 13720 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:40.813668 21655 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:40.901989 13541 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:40.893597 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:40.932497 13651 rpcz_store.cc:276] 0709 12:24:39.832408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.721539 (+889131us) service_pool.cc:224] Handling call
0709 12:24:40.721866 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.722480 (+   614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.722528 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.739541 (+ 17013us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.791319 (+ 51778us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:40.791377 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.791466 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.904445 (+112979us) raft_consensus.cc:1685] finished
0709 12:24:40.904467 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.904626 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7636,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3329,"thread_start_us":43678,"threads_started":1,"wal-append.queue_time_us":114488}
I20260709 12:24:40.911720 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:40.888616 21800 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.354s	user 0.002s	sys 0.001s
W20260709 12:24:40.944098 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50590.5us
I20260709 12:24:40.944303 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:40.937702 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.945266 13722 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:40.945356 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:40.939993 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.929934 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.947160 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:40.937979 13724 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:40.951289 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.951903 13724 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:40.958279 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:40.958638 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 102707us
I20260709 12:24:40.964910 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:40.965260 13722 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:40.967746 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.968647 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.895958 21803 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:40.968132 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:40.905617 21555 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:40.969880 13798 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:40.970098 13721 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:40.971966 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:40.972891 13799 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:40.975145 14416 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:40.979753 13724 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:40.982067 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:41.003827 14416 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:41.004223 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.982662 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:41.005167 13724 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:41.005879 13727 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:24:40.934907 21443 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.984575 14425 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:40.987350 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.009061 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.010076 13798 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:41.010217 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:41.010833 14420 raft_consensus.cc:2376] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19. Re-sending same reply.
I20260709 12:24:40.957720 21706 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.962064 21763 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:41.023717 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:41.021202 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:40.910740 21208 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:40.957353 21794 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:41.025442 21794 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.014837 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.027420 13800 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:41.023866 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:40.958352 21807 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:41.031471 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:41.035022 13720 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:41.035533 13727 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:24:41.031563 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158650us
I20260709 12:24:41.007019 21764 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:41.041380 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:41.038527 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.935707 21810 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:41.040016 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.041725 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:40.982564 21381 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:41.043674 13724 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:41.049679 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:41.053335 13727 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:41.051286 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:41.055154 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:41.030716 21803 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:41.058924 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.031282 21208 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.018544 21443 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.035961 21822 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:41.091320 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.030035 21706 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.101990 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:41.103785 13720 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:41.104564 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.112336 13798 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:41.112376 13797 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:41.050673 21819 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:24:41.090468 21573 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:41.057418 21381 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.101873 21822 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:41.132097 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 108476us
I20260709 12:24:41.134984 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:41.135799 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.135123 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.142050 13798 raft_consensus.cc:2368] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:24:41.114513 21637 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:41.108430 21764 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:41.143441 14416 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:41.074621 21832 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:24:41.139993 21759 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:24:41.158836 13795 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:41.159412 13792 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:41.158232 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:41.163178 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:41.160951 13799 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 20. Ops: []
I20260709 12:24:41.155254 21826 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:41.163908 13651 raft_consensus.cc:606] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.160053 13796 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 14. Current term is 16. Ops: [14.11-14.11]
I20260709 12:24:41.141718 21822 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:41.171279 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:41.172322 13799 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:41.150231 21584 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.170127 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.143397 21829 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:41.182126 13793 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:41.178448 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.179092 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:41.183118 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:41.183280 21838 raft_consensus.cc:2764] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:24:41.174870 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.133585 21735 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.200253 13799 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:41.190965 13792 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:41.192564 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.210140 13793 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:41.210024 13792 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:24:41.144757 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 89719.6us
W20260709 12:24:41.211336 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66591.9us
W20260709 12:24:41.263233 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51676.5us
I20260709 12:24:41.184528 21837 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:41.232887 21778 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:41.241241 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.685s (SENT) [suppressed 37 similar messages]
I20260709 12:24:41.255349 21443 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.207060 21209 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:41.232399 13690 leader_election.cc:336] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.566s (SENT)
I20260709 12:24:41.256908 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.231603 21381 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.175786 21833 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:41.185519 21829 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:41.302628 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.303578 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.226817 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.255959 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:41.227206 21836 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:41.232086 21810 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.233207 21706 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.230531 21764 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.278833 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:41.313990 13720 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:24:41.271454 13767 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.507s (SENT)
W20260709 12:24:41.280818 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.283635 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.689s (SENT)
I20260709 12:24:41.315819 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.130842 21807 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.282642 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:41.254313 21839 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:41.318727 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.276789 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.291220 21847 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:41.216696 21584 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:41.295171 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62812.9us
I20260709 12:24:41.305326 13792 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:41.320297 13799 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:41.338508 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.329871 13795 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:41.340127 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:24:41.338640 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:41.340490 13793 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:41.340842 13792 raft_consensus.cc:606] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.301796 21833 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:41.344164 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.346761 13800 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:41.347694 13720 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:41.311357 21829 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:41.353621 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70011.3us
I20260709 12:24:41.337548 21810 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:41.355444 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:41.355717 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 84312.9us
I20260709 12:24:41.292197 21837 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.297747 21209 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.328549 21558 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:41.388515 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67418.9us
I20260709 12:24:41.364224 21656 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:41.337020 21794 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:24:41.348932 21769 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:41.353868 21833 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:41.388487 21810 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:41.558599 21852 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:24:41.555656 21803 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:41.522300 21837 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.444550 21437 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:41.644297 13644 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:24:41.521049 21808 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.000s
I20260709 12:24:41.450423 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:41.447681 21846 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:24:41.527701 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:41.646099 13690 leader_election.cc:336] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.984s (SENT)
I20260709 12:24:41.525527 21584 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:41.464228 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.502231 21558 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:41.459791 13648 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:24:41.659584 13648 pending_rounds.cc:85] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 14
I20260709 12:24:41.659868 13648 pending_rounds.cc:107] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:24:41.660112 13648 raft_consensus.cc:2889] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:41.445621 21784 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.153s	user 0.003s	sys 0.003s
I20260709 12:24:41.475684 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:41.661582 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2696) took 2389 ms. Trace:
I20260709 12:24:41.661998 13643 rpcz_store.cc:276] 0709 12:24:39.272261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.272549 (+   288us) service_pool.cc:224] Handling call
0709 12:24:39.273056 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.288340 (+ 15284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.288374 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.294571 (+  6197us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.580104 (+285533us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b18004ba8a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:24:39.822348 (+242244us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:39.822397 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.822470 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.297154 (+1474684us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:41.661350 (+364196us) raft_consensus.cc:1685] finished
0709 12:24:41.661377 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.661517 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8490,"raft.queue_time_us":15392,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5765,"spinlock_wait_cycles":60730368,"thread_start_us":242002,"threads_started":1,"wal-append.queue_time_us":990311}
I20260709 12:24:41.664049 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:41.664639 13796 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:41.661936 13645 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:41.423873 21834 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.241s	user 0.002s	sys 0.000s
I20260709 12:24:41.493556 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:41.667685 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.507342 21830 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.481s	user 0.001s	sys 0.000s
I20260709 12:24:41.564253 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:41.639976 21763 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.002s
I20260709 12:24:41.421504 21443 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.510738 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:41.459252 21735 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:41.447992 21802 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.001s
I20260709 12:24:41.529922 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:41.520118 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:41.565709 21848 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.621071 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.688730 13718 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:24:41.688920 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67995.6us
I20260709 12:24:41.588810 21847 consensus_queue.cc:1243] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:24:41.491452 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45270.3us
W20260709 12:24:41.625289 21851 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:24:41.341290 21839 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.624282 21209 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:41.643725 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 255870us
I20260709 12:24:41.645610 13651 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:41.645118 21759 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.642772 21810 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:41.655462 21645 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:41.655910 21437 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.646649 21803 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.657414 13650 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:41.672700 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:41.661604 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:41.561076 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:41.664677 13643 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:41.675460 13645 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:41.678419 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:41.687144 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4013) took 2711 ms. Trace:
I20260709 12:24:41.726275 13794 rpcz_store.cc:276] 0709 12:24:38.969478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.970795 (+  1317us) service_pool.cc:224] Handling call
0709 12:24:38.983053 (+ 12258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.344737 (+1361684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.680325 (+1335588us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003d4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:41.680561 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2889483008}
W20260709 12:24:41.688000 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3634) took 1965 ms (client timeout 1721 ms). Trace:
I20260709 12:24:41.704489 21837 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:41.670871 21546 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.709791 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 371240us
I20260709 12:24:41.751143 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:41.709786 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:41.680611 21763 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:24:41.718185 13767 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.666s (SENT)
W20260709 12:24:41.727579 13719 rpcz_store.cc:269] 0709 12:24:39.722950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.723139 (+   189us) service_pool.cc:224] Handling call
0709 12:24:41.687792 (+1964653us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b54000f6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:41.687924 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73558272}
I20260709 12:24:41.769510 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:41.705161 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.937s (SENT)
W20260709 12:24:41.770780 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65783.2us
W20260709 12:24:41.772445 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.055s (SENT)
I20260709 12:24:41.671342 21822 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.729142 14425 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:41.671635 21808 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.762019 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:41.687870 21829 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.749593 21209 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.681156 21443 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.724364 21645 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:41.780984 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62843.4us
I20260709 12:24:41.786351 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:41.785636 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:41.787155 13650 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:41.789625 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:41.791482 13722 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:41.794046 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:41.796281 21822 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.798456 21808 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:41.803824 21782 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:41.805107 21864 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:24:41.806968 21645 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.808153 21645 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:41.812345 21443 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.719365 21759 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.710088 21848 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.815608 21848 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.817085 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:41.818310 14425 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:41.792027 21826 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 12:24:41.820892 21826 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:41.823166 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 72162.8us
I20260709 12:24:41.823997 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:41.825690 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:41.825801 21808 raft_consensus.cc:2764] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:24:41.827754 21298 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:24:41.828526 21298 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:24:41.829006 21298 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:24:41.829330 21298 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:41.829918 13647 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:41.785059 21743 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:41.837159 21803 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:41.842437 21836 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:41.843259 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:41.844151 13649 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:41.845232 13649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:41.845854 13649 raft_consensus.cc:606] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:41.833200 14421 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:24:41.846450 13649 raft_consensus.cc:614] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:41.791582 21814 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.894s	user 0.010s	sys 0.005s
I20260709 12:24:41.848956 21814 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.850441 13651 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:41.851719 21814 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.854149 21839 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.768567 21546 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.786548 21209 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:41.857856 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.783730 21861 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:41.863732 21414 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:41.832804 21829 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.835086 21437 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.867085 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:41.867779 14421 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:41.871495 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.873531 13649 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:41.875631 14416 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:24:41.888232 14420 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:24:41.888267 14425 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:41.894562 13650 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
W20260709 12:24:41.899505 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73890us
I20260709 12:24:41.900380 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:41.911875 14421 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:41.913327 13797 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:24:41.915498 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:41.857985 21209 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:41.897739 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:41.916034 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:41.922784 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.177s (SENT)
I20260709 12:24:41.927381 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:41.928788 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.926841 13649 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:41.930377 13690 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:41.931680 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:41.937971 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:41.901886 21806 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:41.902859 21814 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.955930 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.949579 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:41.955811 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.228s (SENT)
I20260709 12:24:41.957526 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:41.939574 21782 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:41.926093 21858 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.997720 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:41.998791 13801 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:41.993506 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.993865 13794 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:41.977519 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:41.991849 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:41.994529 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.005146 13643 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:41.995214 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:42.006017 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50240.4us
W20260709 12:24:41.950770 21832 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.004s	sys 0.022s
I20260709 12:24:42.005813 13797 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:24:41.999399 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:42.017161 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 116865us
I20260709 12:24:41.953374 21826 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:24:41.895272 21298 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.026760 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:42.047959 13643 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.048377 13643 raft_consensus.cc:614] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:42.027956 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:42.049788 13648 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.054374 21414 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:42.054900 21414 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.055254 21414 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.046597 21782 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:42.059293 21782 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:41.967247 21808 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:41.993918 21802 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:24:42.062638 21808 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.063196 21802 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:42.063660 21802 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:41.967644 21209 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:41.967572 21861 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:42.045197 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:42.072645 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:41.990568 21814 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.901567 21655 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.121s
I20260709 12:24:42.013336 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:41.958810 21847 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:24:42.087119 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73878.2us
I20260709 12:24:41.994437 21443 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 12:24:42.095781 13649 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
W20260709 12:24:42.068674 13541 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.009s	sys 0.044s
W20260709 12:24:42.047544 21832 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.007s	sys 0.028s
I20260709 12:24:41.953653 21859 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.156s
I20260709 12:24:42.074141 21524 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:42.016157 21858 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:41.961516 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:42.047775 13650 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:24:42.089661 21847 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.088522 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.094048 13648 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:42.036860 21824 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:24:42.095036 21829 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:42.131467 21829 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:42.131956 21829 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:42.133203 21865 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:24:42.122833 21847 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:42.102695 13541 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.009s	sys 0.045s
I20260709 12:24:42.137923 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4279) took 2316 ms. Trace:
I20260709 12:24:42.138468 21414 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.138437 13646 rpcz_store.cc:276] 0709 12:24:39.821679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.717930 (+896251us) service_pool.cc:224] Handling call
0709 12:24:40.885932 (+168002us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:40.901128 (+ 15196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.901170 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:40.901991 (+   821us) log.cc:844] Serialized 78 byte log entry
0709 12:24:41.020170 (+118179us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:41.212839 (+192669us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.253284 (+ 40445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.253386 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.506658 (+253272us) spinlock_profiling.cc:243] Waited 9.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:41.670092 (+163434us) raft_consensus.cc:1685] finished
0709 12:24:41.670212 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.137734 (+467522us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001e5d50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:42.137857 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14692,"prepare.queue_time_us":189503,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":56624,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":6156,"spinlock_wait_cycles":406669952,"thread_start_us":299022,"threads_started":2,"wal-append.queue_time_us":123695}
I20260709 12:24:42.140897 21764 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.141554 21414 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:42.141830 21764 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.144848 21764 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:42.146595 21827 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.120s	user 0.002s	sys 0.007s
I20260709 12:24:42.146884 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:42.147840 14424 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:42.150134 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.150805 13643 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:42.151551 21824 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:42.152134 21824 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.152487 21824 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:42.152602 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 191157us
I20260709 12:24:42.153805 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.155408 21806 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:42.155958 21637 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:42.156499 21832 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:42.157075 21832 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:42.157480 21832 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:42.160948 21835 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.009s	user 0.006s	sys 0.000s
I20260709 12:24:42.162628 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3805) took 1236 ms. Trace:
I20260709 12:24:42.151939 21858 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.163019 13726 rpcz_store.cc:276] 0709 12:24:40.926438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.926564 (+   126us) service_pool.cc:224] Handling call
0709 12:24:40.926964 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.024886 (+ 97922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.024923 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.025458 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.049605 (+ 24147us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:41.049645 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.506139 (+456494us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:41.506197 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.160247 (+654050us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:42.162351 (+  2104us) raft_consensus.cc:1685] finished
0709 12:24:42.162374 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.162546 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58715,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":58,"spinlock_wait_cycles":107982336,"thread_start_us":81819,"threads_started":2,"wal-append.queue_time_us":67384}
I20260709 12:24:42.165676 21546 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:41.949571 21860 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:42.125123 13649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:42.175479 13649 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.175864 13649 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:42.176115 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.121598 21816 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.191792 21824 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.125681 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.099963 21298 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:24:42.129710 21839 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:42.173717 21799 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.194947 21824 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.195444 21799 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.073118 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:42.192055 21816 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:42.178779 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.201347 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.205957 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.208427 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:42.199637 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77273.4us
I20260709 12:24:42.213105 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:42.219534 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 146570us
I20260709 12:24:42.223474 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.224790 13727 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:24:42.063670 21808 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.236150 13725 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:42.238298 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.239216 13723 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:42.241765 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:42.177819 21861 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:42.156494 21637 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:42.244635 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:42.244764 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:42.245781 13617 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.010s (SENT)
I20260709 12:24:42.194912 21814 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.246373 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.200845 21839 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:42.248801 13721 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:24:42.224296 21803 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:42.253640 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.232290 21524 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:42.254526 13693 leader_election.cc:400] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:24:42.157087 21806 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.094s
I20260709 12:24:42.259147 13693 leader_election.cc:403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:42.256100 21814 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:24:42.262898 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:42.254704 21826 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:42.254715 21805 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:24:42.267807 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:42.267975 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3803) took 1356 ms. Trace:
I20260709 12:24:42.268024 21805 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:42.268445 14422 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:42.238478 21735 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:24:42.247938 21861 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.268416 13718 rpcz_store.cc:276] 0709 12:24:40.911098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.912973 (+  1875us) service_pool.cc:224] Handling call
0709 12:24:41.688543 (+775570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.689151 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.267744 (+578593us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54000f6c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:42.267912 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782588672}
I20260709 12:24:42.270064 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.281965 13726 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:24:42.281745 21861 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.240176 21859 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:24:42.283202 21861 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:42.282119 21832 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.285857 21832 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.286604 21832 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:42.287011 21832 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.287874 21832 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.290036 21832 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.179440 21802 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.291707 21814 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:42.292032 21802 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:42.292699 21814 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.293493 21838 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:42.293915 21838 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.250221 21848 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 12:24:42.201444 21860 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.300474 21860 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.266098 21803 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:42.273612 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.269711 21805 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:42.254402 21739 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:24:42.311154 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:42.321538 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.326363 21810 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.576s	user 0.005s	sys 0.008s
I20260709 12:24:42.327716 21816 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.327401 21805 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.328284 21645 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:42.328482 21847 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.329759 21803 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.266505 21867 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19.
I20260709 12:24:42.333139 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:42.350354 14423 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.360276 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.343079 21816 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.404948 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.405894 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45721.7us
I20260709 12:24:42.293960 21814 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:42.413221 13644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4268) took 2608 ms (client timeout 1896 ms). Trace:
I20260709 12:24:42.339478 21810 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:42.419324 13644 rpcz_store.cc:269] 0709 12:24:39.805103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.805236 (+   133us) service_pool.cc:224] Handling call
0709 12:24:42.412948 (+2607712us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54001b5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:42.413135 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3672060288}
I20260709 12:24:42.331697 21838 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.362212 21805 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:42.407271 21833 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.733s	user 0.010s	sys 0.003s
I20260709 12:24:42.361357 21209 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:42.438694 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.450230 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
W20260709 12:24:42.439019 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.440672 21825 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.339s	user 0.006s	sys 0.000s
I20260709 12:24:42.437566 13796 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:42.449774 14423 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:42.337544 21839 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:42.458380 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 289592us
I20260709 12:24:42.437731 14422 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:42.461149 13796 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:24:42.451071 21848 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:42.469121 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.464099 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.453037 21838 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:42.450104 21802 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.451807 21564 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:42.470613 13539 maintenance_manager.cc:419] P d5e54c6d2c954fbf93d5136276b7b78c: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033369
I20260709 12:24:42.421449 21816 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.482579 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.450269 21833 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:42.470909 21872 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 12:24:42.485327 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 527862us
I20260709 12:24:42.489404 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.473390 21840 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:42.471354 21848 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:42.496930 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:42.503018 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 240151us
I20260709 12:24:42.498605 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:42.506969 14423 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:42.507334 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:42.499687 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:42.509481 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:42.511598 14422 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:24:42.512040 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:42.516108 14423 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:42.479187 21802 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.530052 21802 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.504105 21209 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:42.509510 21873 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:24:42.502483 21848 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:42.536438 21799 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:42.537393 21810 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.537598 21807 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:24:42.538580 21807 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:42.538998 21584 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:42.539397 21584 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:42.540088 21876 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:42.540607 21876 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.540328 21584 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.540985 21876 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:42.541673 21584 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.481069 21564 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.543653 21816 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.544126 21564 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.544275 21816 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:42.519511 21840 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:42.545897 21840 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.510912 21743 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
W20260709 12:24:42.547014 21875 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:24:42.525508 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3812) took 1594 ms. Trace:
I20260709 12:24:42.549198 13722 rpcz_store.cc:276] 0709 12:24:40.931163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.968475 (+ 37312us) service_pool.cc:224] Handling call
0709 12:24:42.525272 (+1556797us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b540033fed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:42.525429 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890759040}
I20260709 12:24:42.551725 21437 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:24:42.552596 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.682s (SENT)
I20260709 12:24:42.552399 21437 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.554072 21829 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.556455 21829 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:42.557130 13800 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:24:42.525434 21847 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.568944 21847 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.569509 21847 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.519651 21867 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:42.571890 21867 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:42.572335 21867 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:42.573897 21819 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:42.574345 21819 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.575091 21847 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.536662 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4294) took 1052 ms. Trace:
I20260709 12:24:42.527524 21833 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.537424 21874 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:24:42.577111 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61264.1us
I20260709 12:24:42.577525 21874 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:42.563267 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:42.567664 21759 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:42.569904 21878 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.072s	user 0.001s	sys 0.002s
I20260709 12:24:42.575951 21819 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.568104 21437 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.580649 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:42.575776 13651 rpcz_store.cc:276] 0709 12:24:41.483873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.851657 (+367784us) service_pool.cc:224] Handling call
0709 12:24:41.851995 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:41.852718 (+   723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.852750 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:41.853342 (+   592us) log.cc:844] Serialized 52 byte log entry
0709 12:24:42.111817 (+258475us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:42.111863 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.111957 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.536371 (+424414us) raft_consensus.cc:1685] finished
0709 12:24:42.536398 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.536589 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"thread_start_us":258024,"threads_started":1,"wal-append.queue_time_us":405316}
I20260709 12:24:42.586727 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.588171 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.619671 13795 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:24:42.596248 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.598178 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.517392 21739 consensus_queue.cc:1243] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:42.621982 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:42.622458 14416 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:42.622884 13798 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:42.623775 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.624619 13644 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:42.624648 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:42.628662 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48118.7us
W20260709 12:24:42.629437 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.760s (SENT)
I20260709 12:24:42.630671 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.623575 14418 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:24:42.637835 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:42.643002 21573 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:42.643476 21573 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.643827 21573 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:42.637185 21437 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:42.645464 21437 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:42.646445 21437 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.626677 21759 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.649669 21759 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.651028 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.651903 21759 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.653662 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.654353 13800 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:42.654402 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.655258 21885 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:42.655791 21885 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.655920 14418 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:42.656134 21887 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:24:42.656591 21887 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:42.656996 21887 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:42.656704 21885 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.658481 21889 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:42.658624 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:42.659006 21889 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.659392 21889 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:42.660984 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.637679 21637 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.663302 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:42.666164 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:42.666776 13651 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:42.668956 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:42.669618 14418 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:42.672693 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.691995 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:42.693840 21807 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:42.694460 21807 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.695216 21807 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:42.700071 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:42.701633 21764 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:24:42.713593 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 133413us
W20260709 12:24:42.715740 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 404698us
I20260709 12:24:42.716818 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:42.717743 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:42.719501 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:42.721688 13651 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:42.578557 21874 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:42.724565 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:42.745774 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.725819 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.640563 21656 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:42.642184 21819 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:42.685611 21564 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.685993 21880 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:24:42.754691 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 256204us
I20260709 12:24:42.675853 21782 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.689966 21524 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:42.631466 21822 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 12:24:42.761833 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:42.764622 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:42.667416 21867 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.732482 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:42.739017 21769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.006s	sys 0.014s
I20260709 12:24:42.717875 21807 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:42.759205 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 236301us
I20260709 12:24:42.761437 13801 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:42.781141 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.763363 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:42.630862 21882 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.131s
I20260709 12:24:42.686885 21876 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.683178 21833 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.072s
I20260709 12:24:42.768669 21656 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:42.808837 21894 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:24:42.788808 21437 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:24:42.769889 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.774168 13643 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:24:42.778940 13650 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:24:42.778816 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.782310 13651 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:42.816115 21645 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:42.816615 21645 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:42.817456 21645 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.781700 21414 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
W20260709 12:24:42.818845 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.820722 21833 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:42.758019 21524 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.777722 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:42.839390 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61777.6us
W20260709 12:24:42.766462 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49983.7us
I20260709 12:24:42.769460 21839 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:24:42.746907 13652 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
W20260709 12:24:42.776081 21769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.006s	sys 0.015s
I20260709 12:24:42.790088 13645 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:42.793290 21867 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.779875 21824 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
W20260709 12:24:42.805341 21893 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 12:24:42.806764 21886 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.240s	user 0.000s	sys 0.002s
W20260709 12:24:42.807294 21897 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:24:42.807265 13801 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:42.809149 21656 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.793294 21876 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:42.768962 21799 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:24:42.824421 21584 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:42.824532 21743 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:42.826649 21645 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:42.824772 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60349.1us
I20260709 12:24:42.834141 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:42.782203 14418 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:42.770910 21888 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:42.786075 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.841614 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:42.765522 21739 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.026s
I20260709 12:24:42.755818 21564 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:42.857767 21546 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.829530 21833 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:42.868335 21829 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:24:42.875399 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83861.5us
I20260709 12:24:42.787652 21209 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 12:24:42.876497 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:42.887239 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45753.8us
I20260709 12:24:42.872926 21882 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:42.764962 21782 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.851168 21839 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:42.885097 13690 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:42.882177 21890 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 12:24:42.885274 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:42.902701 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0608470619999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
I20260709 12:24:42.890756 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.891497 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.892207 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.873354 21779 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:42.878075 21891 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:24:42.887261 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.880229 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:42.872323 21637 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.905025 13724 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:42.911973 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:42.914819 13649 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:42.916575 13649 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:24:42.917598 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.918689 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.919440 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:42.919895 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:42.920442 13646 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:42.920579 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:42.920863 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.921715 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:42.922317 13645 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:42.923156 13724 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:24:42.918869 13651 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:42.927843 13651 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:42.932503 13645 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:42.933076 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.934190 13724 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:24:42.935909 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.936203 13646 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:42.937040 13724 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:24:42.886910 21829 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.896353 21816 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:42.944190 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.941006 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.883069 21584 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.897701 21839 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:24:42.942610 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.958115 13718 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:42.945400 13719 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:42.945564 13724 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:24:42.894119 21882 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:42.975736 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.976027 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.881428 21555 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:24:42.900990 21867 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:42.980507 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:42.980782 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.122s (SENT)
I20260709 12:24:42.980430 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.955843 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.983299 13725 raft_consensus.cc:2376] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19. Re-sending same reply.
I20260709 12:24:42.984508 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:42.974081 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.976833 13722 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:42.976864 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:42.986414 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.986516 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.987025 13722 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:24:42.987681 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.987936 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.988551 13724 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:24:42.989339 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:42.990099 13724 raft_consensus.cc:2410] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:24:42.976204 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.990911 13719 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:42.995364 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.997094 13727 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:24:42.997943 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.140s (SENT)
I20260709 12:24:42.998553 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.002823 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.003772 13727 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:42.945689 21829 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:42.898000 21546 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.005662 13720 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:43.007776 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.019109 13722 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:24:43.019610 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.020210 13720 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:43.020375 13722 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:24:43.020761 13722 pending_rounds.cc:85] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:24:43.020980 13722 pending_rounds.cc:107] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:24:43.021220 13722 raft_consensus.cc:2889] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.008455 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:42.939782 21524 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.022603 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:43.023121 21524 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:43.017803 21867 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:43.025107 21867 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.981999 21833 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.026806 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:42.988230 21637 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:43.027689 21637 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:43.028895 21808 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:43.028548 21637 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.029328 21808 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.029712 21808 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:43.030138 21807 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:42.912588 21779 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.006737 13727 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:24:43.009600 21782 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:42.980057 21437 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:43.012313 21904 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:24:43.015070 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:43.046764 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.016688 13719 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:42.900655 21573 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:42.947173 21816 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:43.007956 21882 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.028234 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:43.043748 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.685s (SENT)
I20260709 12:24:43.079485 13723 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:43.046295 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:43.032534 21848 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.018373 21739 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:43.100623 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56937.2us
W20260709 12:24:43.115645 13617 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.194s (SENT)
I20260709 12:24:43.116101 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:43.107759 21546 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.108951 21889 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.113286 21832 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:43.036446 21838 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:43.123932 14423 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:43.089141 21802 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:43.127612 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.212s (SENT)
W20260709 12:24:43.126113 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.099697 21816 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:43.128144 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:43.033097 21887 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.123004 21888 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:43.036276 21861 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 12:24:43.136142 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 250967us
W20260709 12:24:43.146422 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.094842 21437 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.125826 21832 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:43.129642 21546 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:43.136243 21882 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.122298 21739 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.186640 14418 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:24:43.132369 21735 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:43.180125 13538 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 12:24:43.199193 21909 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:24:43.133448 21802 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:43.228427 21912 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:43.228813 21298 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:43.229132 21829 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:43.231034 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.231313 21759 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.221282 21808 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:43.240937 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.242422 21895 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.525s	user 0.002s	sys 0.002s
I20260709 12:24:43.245020 21880 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:24:43.245544 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 358350us
I20260709 12:24:43.246285 21414 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:43.246769 21906 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:24:43.246438 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 334541us
I20260709 12:24:43.300084 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:43.182310 21832 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:43.268245 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.409s (SENT)
W20260709 12:24:43.272945 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.302922 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:43.277504 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 249300us
W20260709 12:24:43.277783 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.281152 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.308306 13801 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.286028 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:43.262380 21878 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:24:43.290972 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.322821 13797 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:43.309751 14424 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:43.282271 21298 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:43.299893 21819 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.002s
I20260709 12:24:43.330892 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:43.332907 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:43.335798 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:43.297461 21888 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.310359 21882 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:24:43.321069 21807 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.298025 21759 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.325774 21414 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:43.347301 21907 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:24:43.379117 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48379us
I20260709 12:24:43.380594 14421 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:24:43.351678 13725 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:24:43.423424 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.410247 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:43.363824 21913 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:24:43.433122 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.396667 13801 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:43.433380 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.437095 13719 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:43.374341 21546 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:43.388711 13538 maintenance_manager.cc:643] P d5e54c6d2c954fbf93d5136276b7b78c: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.863s	user 0.042s	sys 0.006s Metrics: {"bytes_written":5363,"delete_count":0,"lbm_write_time_us":5014,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":5,"reinsert_count":0,"update_count":199}
I20260709 12:24:43.361238 21808 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.364405 21584 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:43.348803 21838 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.158s
W20260709 12:24:43.421418 21645 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.003s
I20260709 12:24:43.370805 21779 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:43.359988 21805 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.917s	user 0.006s	sys 0.008s
I20260709 12:24:43.374758 21876 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:43.374486 21883 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:43.528847 13722 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:24:43.348636 21743 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:43.386989 21739 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:43.358533 21887 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.371542 21573 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:43.385092 21889 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:43.500994 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 198884us
I20260709 12:24:43.542490 13539 maintenance_manager.cc:419] P d5e54c6d2c954fbf93d5136276b7b78c: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.235726
I20260709 12:24:43.371002 21832 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.500321 13726 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:43.498229 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.583894 13721 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:43.584223 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.503608 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:43.516106 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:43.584956 13726 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:43.517288 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:43.518127 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:43.518386 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:43.518916 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:43.586660 13646 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:24:43.519480 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.519656 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:43.519944 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:43.587677 13723 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.588126 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.522034 13724 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 12:24:43.590456 13724 pending_rounds.cc:85] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 19
I20260709 12:24:43.590787 13724 pending_rounds.cc:107] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 12:24:43.591089 13724 raft_consensus.cc:2889] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:43.520298 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 87870.6us
W20260709 12:24:43.513370 21929 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:43.531059 13722 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:43.607470 13722 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:43.480222 21799 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:24:43.430665 21807 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:43.543223 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.476s (SENT)
I20260709 12:24:43.624313 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:43.371183 21298 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.379315 21900 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:43.519889 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.590184 13719 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:24:43.625891 13720 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 14.
I20260709 12:24:43.523509 21546 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.598729 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:43.627440 13647 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:43.599592 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:43.600679 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 97157.3us
I20260709 12:24:43.601282 21573 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:43.601352 21437 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:43.598794 21919 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.382s	user 0.000s	sys 0.003s
I20260709 12:24:43.531620 21880 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.611722 13723 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:43.635998 13650 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:43.615182 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:43.594657 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:43.583602 21802 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.629580 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:43.645336 21876 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:43.653467 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:43.665350 13650 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:24:43.631589 21573 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.669875 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.637351 21829 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.534204 21883 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.675876 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:43.676931 21824 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:43.650549 21298 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:43.688966 13801 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:43.691972 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.658622 21437 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:43.673353 21573 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:43.701643 13647 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:43.703433 13647 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:24:43.704828 21934 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.705760 13650 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:43.705462 21934 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.707300 21874 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.707854 21874 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.709262 21759 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:43.709436 13648 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:43.709969 21759 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.710173 21874 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:43.710633 21759 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:43.713234 21936 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:24:43.716588 21876 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.615964 21414 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:43.648648 21880 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:43.703222 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 63863.2us
W20260709 12:24:43.730859 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 187647us
W20260709 12:24:43.732781 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.871s (SENT)
I20260709 12:24:43.733280 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:43.739548 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.785032 21824 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:43.786579 14416 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:43.702183 21816 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.004s
W20260709 12:24:43.727038 21839 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.005s
I20260709 12:24:43.797271 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.790719 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.797763 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.800196 13800 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
W20260709 12:24:43.799233 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.801739 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.789052 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.802966 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.802757 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.788671 13801 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:43.807253 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.810752 13797 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:43.813436 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.815120 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.810469 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:43.823215 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:43.823545 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.797s (SENT)
I20260709 12:24:43.823915 13795 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:43.824025 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:43.817625 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.826093 13797 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:43.800747 21743 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:43.838117 21735 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:43.838454 21941 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:24:43.840161 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:43.842923 13797 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:43.846529 13795 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:43.851084 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:43.852845 13799 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:43.856442 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:43.858186 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:43.858999 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.802462 21816 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.838850 21735 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.804924 21839 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:43.808920 21874 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.043s
I20260709 12:24:43.850970 21880 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:43.649637 21655 consensus_queue.cc:1175] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 24 status { last_received { term: 22 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: [23.16-23.16]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:43.860183 21950 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:43.879542 14421 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:43.859789 21759 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.892855 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:43.869899 21832 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.911327 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:43.962699 21874 consensus_queue.cc:1243] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 25.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
W20260709 12:24:44.001791 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.034s (SENT)
I20260709 12:24:43.869174 21900 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:43.873523 21934 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:43.964089 21874 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:24:44.007411 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.062s (SENT)
I20260709 12:24:43.893508 21955 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:43.896138 21546 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:43.895589 21807 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:44.022119 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 129347us
I20260709 12:24:44.023128 21953 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:43.873241 21824 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:44.036465 21943 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 12:24:44.037127 21951 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:24:44.043794 21298 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:44.044363 21298 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:44.045346 21298 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.049275 21735 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.002662 21743 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.051291 21816 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.052543 21900 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.060309 21839 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.028108 21779 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:44.022223 21960 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:24:44.063032 21883 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.069569 21573 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.072427 21573 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:44.073536 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49234.6us
W20260709 12:24:44.075853 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.574s (SENT)
I20260709 12:24:44.076093 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:44.076606 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.077548 21958 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:44.078018 21958 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:44.078434 21958 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.078495 21298 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:44.072499 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1289) took 1396 ms. Trace:
I20260709 12:24:44.074241 21779 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:44.084404 14423 rpcz_store.cc:276] 0709 12:24:42.676287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.677005 (+   718us) service_pool.cc:224] Handling call
0709 12:24:42.677382 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.124452 (+447070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.072212 (+947760us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54004c3e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:44.072413 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1026382080}
I20260709 12:24:44.083964 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:44.091190 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.094414 13792 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:44.114445 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 240759us
I20260709 12:24:44.094414 21951 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:44.098685 21876 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:44.120193 21769 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.132687 21769 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.118716 21950 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.124583 21951 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:44.137648 21950 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.146772 21956 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:44.144096 21814 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:44.147626 21814 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:44.148630 21814 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.169715 21769 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.142026 21951 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:44.175338 13644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:44.176082 13644 raft_consensus.cc:606] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.146195 21950 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.178272 13643 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:44.178716 21950 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.179323 21950 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:44.181262 21914 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.987s	user 0.004s	sys 0.010s
I20260709 12:24:44.186209 13645 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24.
I20260709 12:24:44.187813 13643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:44.188300 21814 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.188526 13643 raft_consensus.cc:606] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.188922 13643 raft_consensus.cc:614] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:44.190187 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2733) took 1338 ms. Trace:
I20260709 12:24:44.190884 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1292) took 1153 ms. Trace:
I20260709 12:24:44.190604 13652 rpcz_store.cc:276] 0709 12:24:42.851899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.852189 (+   290us) service_pool.cc:224] Handling call
0709 12:24:42.852706 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.853022 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.853061 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:42.853680 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.055104 (+201424us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:43.055231 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.055354 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.166246 (+1110892us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:44.189876 (+ 23630us) raft_consensus.cc:1685] finished
0709 12:24:44.189899 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.190125 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62627584,"thread_start_us":160992,"threads_started":1,"wal-append.queue_time_us":299748}
I20260709 12:24:44.195978 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.196789 13644 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:44.199791 14417 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:24:44.201066 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.201920 14417 raft_consensus.cc:2410] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:24:44.202701 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.203202 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.203761 14423 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:44.204494 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.206439 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.206766 14417 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:44.207433 14425 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:24:44.210461 21645 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.211242 21645 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.212129 13651 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:44.215476 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.216554 13651 raft_consensus.cc:2410] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:24:44.220374 13651 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:24:44.221160 21958 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.227314 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:44.229202 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.230023 13798 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:24:44.204726 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.232153 21888 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.232336 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:44.205615 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:44.234889 14416 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:44.191648 14420 rpcz_store.cc:276] 0709 12:24:43.037039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.037520 (+   481us) service_pool.cc:224] Handling call
0709 12:24:43.037915 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.038207 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:43.136992 (+ 98785us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.146864 (+  9872us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.190154 (+1043290us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800454ba0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:24:44.190394 (+   240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.190465 (+    71us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:44.190534 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.190631 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.190805 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1031551,"prepare.queue_time_us":341481,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":60,"raft.queue_time_us":1058174,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":13659,"spinlock_wait_cycles":39693568,"thread_start_us":98269,"threads_started":1}
I20260709 12:24:44.239105 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3718) took 1261 ms. Trace:
I20260709 12:24:44.239578 13725 rpcz_store.cc:276] 0709 12:24:42.978167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.005449 (+ 27282us) service_pool.cc:224] Handling call
0709 12:24:43.005880 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.431280 (+425400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.238884 (+807604us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54003fded4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:44.239049 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768106368}
I20260709 12:24:44.242733 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1291) took 1341 ms. Trace:
I20260709 12:24:44.243315 14418 rpcz_store.cc:276] 0709 12:24:42.900567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.900745 (+   178us) service_pool.cc:224] Handling call
0709 12:24:42.901083 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.193676 (+292593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.241951 (+1048275us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:44.242214 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596884864}
I20260709 12:24:44.252051 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.233122 21888 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.258821 21917 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.259506 21917 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.260244 21564 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.250798 21769 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:24:44.261823 21917 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.264140 21888 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.270112 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:44.270810 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 935042us
I20260709 12:24:44.232232 21840 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:24:44.272018 21928 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:44.254140 21555 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:44.274135 21555 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.274544 21555 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:44.276939 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.277737 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.278087 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.278533 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.279158 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:44.280467 13792 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:44.280685 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:44.286193 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.286933 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.202s (SENT)
I20260709 12:24:44.287508 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.288486 21961 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:44.234529 21950 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.292028 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:44.293754 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:24:44.236806 21645 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:44.298569 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.297866 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:44.299585 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.302222 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.303496 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.306421 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:44.288599 21769 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:44.308075 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.300822 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:44.314575 13795 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.296622 21847 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:44.321481 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.322608 13643 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:44.324680 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.325928 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:44.328653 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.344704 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.302336 21955 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.345853 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.355556 13725 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:44.346613 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:44.313361 21861 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:44.348300 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:44.313182 21735 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:24:44.352607 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.361892 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:44.307287 21832 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.307528 21524 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.353590 13720 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:24:44.353873 13724 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
W20260709 12:24:44.326386 21867 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.173s	user 0.023s	sys 0.052s
I20260709 12:24:44.328091 21961 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:44.347419 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:44.408027 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.408648 13726 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.327927 21840 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:44.386032 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:44.359249 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.360221 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:44.360751 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:44.410562 13797 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:44.411002 13646 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:44.411864 13797 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:44.362376 13651 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:24:44.323942 21802 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:44.364970 13795 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:44.358948 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.431656 13648 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:44.432315 13648 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.432690 13648 raft_consensus.cc:614] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:44.433204 13645 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:44.329219 21810 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:44.418968 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.389503 13722 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:24:44.310678 21769 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:44.379949 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:44.444471 21656 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:44.447324 21847 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.463963 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.407699 21867 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.023s	sys 0.053s
W20260709 12:24:44.462338 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103498us
I20260709 12:24:44.355584 21822 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:44.469184 13643 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21.
I20260709 12:24:44.422829 13646 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:44.425559 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:44.424126 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:44.311246 21950 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.053s
I20260709 12:24:44.329103 21928 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:44.418623 13720 pending_rounds.cc:85] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:24:44.471202 13720 pending_rounds.cc:107] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:24:44.420218 21645 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.407106 21655 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:24:44.360410 21824 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:44.423837 21861 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:44.495213 21861 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.495723 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.496536 13721 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:44.441464 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.497756 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.444739 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.447016 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.454734 13797 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21.
I20260709 12:24:44.442639 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.451565 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.442669 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.460505 13723 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:44.356701 21414 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.112s
I20260709 12:24:44.374734 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.471675 13720 raft_consensus.cc:2889] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:44.473125 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.481035 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56979.9us
I20260709 12:24:44.343065 21298 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 12:24:44.424407 21917 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.511988 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
W20260709 12:24:44.525557 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103904us
I20260709 12:24:44.525911 13651 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.526396 13651 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:44.514552 13800 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:44.514710 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.515756 13796 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:44.500633 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:44.515827 13644 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:44.512025 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:44.517102 13726 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:44.514710 13794 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.454172 21779 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:44.447670 21848 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:44.447170 21819 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:44.463882 21437 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.534855 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.461726 21802 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:44.465376 21564 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:44.465126 21840 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:24:44.539547 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.537518 14420 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:24:44.545588 13799 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:24:44.505142 21883 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:24:44.487321 21824 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:44.464972 21847 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.438704 21524 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.454831 21876 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:24:44.438458 21832 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.547554 21950 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:44.532191 21915 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:24:44.552508 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.519860 21962 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.207s
I20260709 12:24:44.563344 13726 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:24:44.547199 21975 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:44.465427 21810 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.508351 21807 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:24:44.563009 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:44.500939 21645 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.606985 21978 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.607591 21978 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.608536 21645 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:44.535370 21927 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.583817 21950 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.609732 21978 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.609477 21927 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:44.611116 21967 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 12:24:44.612392 21927 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.555397 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:44.504885 21822 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:44.555295 21935 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:44.504462 21951 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.583640 13795 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:44.547860 21779 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:44.615694 21951 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:44.615442 21779 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.616374 13798 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17. Ops: []
I20260709 12:24:44.533241 21934 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:24:44.617345 21555 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.632334 21555 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.594965 13726 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:44.570001 13796 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:44.536851 21546 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:24:44.597273 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.599486 13721 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:24:44.635571 13721 pending_rounds.cc:85] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 8
I20260709 12:24:44.635841 13721 pending_rounds.cc:107] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:24:44.636075 13721 raft_consensus.cc:2889] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.535805 21917 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.638249 21917 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.556509 21883 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:44.639401 21883 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.640429 21883 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:44.571925 21824 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.642522 21840 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.647039 13796 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:44.605620 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:44.552996 21437 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.561781 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.666879 13799 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
W20260709 12:24:44.667603 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62089.4us
I20260709 12:24:44.648692 13796 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:44.617933 21955 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:44.618412 21945 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.624207 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68913.4us
I20260709 12:24:44.678555 13796 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:44.624423 13723 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:24:44.585595 21958 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:44.624043 21848 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:44.610191 21950 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:44.622669 21973 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.004s
I20260709 12:24:44.502141 21928 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.668708 13799 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:44.669349 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.750223 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:44.751266 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4054) took 1487 ms. Trace:
I20260709 12:24:44.671377 14423 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:24:44.752140 14423 pending_rounds.cc:85] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 15
I20260709 12:24:44.752573 14423 pending_rounds.cc:107] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:24:44.752919 14423 raft_consensus.cc:2889] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:44.752946 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 215569us
I20260709 12:24:44.751721 13793 rpcz_store.cc:276] 0709 12:24:43.263836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.264180 (+   344us) service_pool.cc:224] Handling call
0709 12:24:43.273894 (+  9714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.702050 (+1428156us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54003780a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:24:44.702480 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:24:44.702517 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.702571 (+    54us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:44.702604 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.750926 (+ 48322us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:44.751010 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.751189 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11839,"spinlock_wait_cycles":1393405056}
I20260709 12:24:44.502903 21769 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.625018 21915 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:24:44.681200 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.666342 21888 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:44.609805 21847 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.550069 21819 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:44.767277 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86181.9us
I20260709 12:24:44.768965 14419 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:24:44.709796 21917 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:44.680488 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:44.684917 14420 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:44.685501 21940 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
W20260709 12:24:44.687291 21931 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.141s	user 0.000s	sys 0.003s
I20260709 12:24:44.689822 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4324) took 1020 ms. Trace:
I20260709 12:24:44.697989 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3728) took 1687 ms. Trace:
I20260709 12:24:44.712183 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:44.597579 21975 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.647794 21655 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:44.791656 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:44.792825 13648 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:44.781795 13718 rpcz_store.cc:276] 0709 12:24:43.010141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.024902 (+ 14761us) service_pool.cc:224] Handling call
0709 12:24:43.100325 (+ 75423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.116930 (+ 16605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.116968 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.117511 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:24:43.306450 (+188939us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:43.390703 (+ 84253us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.275355 (+884652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.275453 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.275509 (+    56us) raft_consensus.cc:1685] finished
0709 12:24:44.275525 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.697737 (+422212us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b54003bea50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:44.697905 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":900478,"prepare.queue_time_us":230176,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":909025,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":4865,"spinlock_wait_cycles":173264000,"thread_start_us":232830,"threads_started":2,"wal-append.queue_time_us":248367,"wal-append.run_cpu_time_us":4074,"wal-append.run_wall_time_us":1262287}
I20260709 12:24:44.652765 21978 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.751854 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:44.748719 21814 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.760710 21555 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:24:44.773780 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:44.775422 13650 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:24:44.776270 13645 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:24:44.778290 21847 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:44.788379 21867 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.782203 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:44.782696 13647 rpcz_store.cc:276] 0709 12:24:43.669844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.704309 (+ 34465us) service_pool.cc:224] Handling call
0709 12:24:43.704611 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.728869 (+ 24258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.728918 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.728983 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:43.729031 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.689518 (+960487us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:44.689578 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.689749 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55189888}
I20260709 12:24:44.825519 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.826184 13647 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:24:44.758692 21848 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.823416 21814 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.672262 21802 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:44.788882 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:44.839056 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.789960 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:24:44.840015 13796 raft_consensus.cc:606] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.796906 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.782769 21799 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:24:44.771906 21645 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:44.770879 21888 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.781301 21769 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.821058 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.683771 21945 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.791795 21655 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:44.816965 21958 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:24:44.819360 21978 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.837159 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:44.841289 13644 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:44.767246 21927 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.815065 21546 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.871346 13801 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
W20260709 12:24:44.843459 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85744.5us
I20260709 12:24:44.847404 13645 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
W20260709 12:24:44.872817 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.873732 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:44.805841 21861 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:24:44.876689 21861 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:44.877671 21861 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:44.878360 21861 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:44.878742 21861 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:44.879591 21861 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.880872 21861 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:44.792658 21975 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:44.682314 13800 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:44.868491 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:44.846830 21645 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.883551 21645 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:44.848344 21983 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.848589 21888 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.830982 21847 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.891947 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:44.865177 21524 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.834550 21867 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.919898 21867 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.895236 13644 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:24:44.921234 13644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:44.921837 13644 raft_consensus.cc:606] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:44.922161 13644 raft_consensus.cc:614] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:44.901748 21847 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:44.923360 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.925243 21848 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:44.891211 21876 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:44.926589 21832 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.927014 21876 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:44.928090 21814 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.893208 21983 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.928390 21876 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:44.930953 21980 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:44.931321 21983 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.932153 21989 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:44.903112 21819 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.888306 21546 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.845417 21824 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.911019 21928 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.910876 21973 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:44.914431 21978 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.915308 21810 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:44.916376 21945 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:44.916575 21979 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:24:44.917524 21982 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:24:44.962772 13647 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:24:44.961153 13794 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
W20260709 12:24:44.958995 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 207227us
W20260709 12:24:44.946091 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48382us
I20260709 12:24:44.925913 21848 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:24:44.951618 21973 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.888896 21927 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:44.945891 21437 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:44.977367 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.985143 21990 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:24:44.977344 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:44.951551 21876 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:44.979825 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:44.899992 21915 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:44.914453 21655 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:44.980822 21973 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:44.991634 13718 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
W20260709 12:24:44.992822 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 117408us
I20260709 12:24:44.997244 21927 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:44.994318 21802 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:44.995519 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:44.987447 21991 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.009034 21991 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.012022 21991 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.014369 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.014984 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:45.008458 21953 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:45.015077 13724 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:45.025822 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.034891 21546 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:45.037115 21971 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.441s	user 0.001s	sys 0.000s
I20260709 12:24:45.038388 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1716) took 2002 ms. Trace:
I20260709 12:24:45.038781 13727 rpcz_store.cc:276] 0709 12:24:43.035679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.036417 (+   738us) service_pool.cc:224] Handling call
0709 12:24:43.036934 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.037384 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.037420 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:43.037978 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.159608 (+1121630us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b18003caf60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:24:44.257918 (+ 98310us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:44.257967 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.258059 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.036492 (+778433us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:45.038061 (+  1569us) raft_consensus.cc:1685] finished
0709 12:24:45.038089 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.038316 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":48494,"spinlock_wait_cycles":111736960,"thread_start_us":97759,"threads_started":1,"wal-append.queue_time_us":1508182}
W20260709 12:24:45.046712 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.033586 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.023012 21802 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:45.054380 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.030829 21927 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:24:45.025313 21655 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:45.056968 21915 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.058924 21645 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.062774 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.067819 21645 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.023569 21953 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:45.072789 21927 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:24:45.073022 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48891.1us
I20260709 12:24:45.031684 21824 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.081233 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:45.043721 21991 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:45.093981 13693 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21.
I20260709 12:24:45.094496 13693 leader_election.cc:403] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:24:45.097895 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.098619 13727 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.099040 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.099669 13719 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:45.102108 21927 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.106513 21953 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.107851 13651 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:24:45.108238 21953 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.109266 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.109054 21973 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.111389 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.112826 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.114667 13726 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:45.118712 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.086714 21991 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.119591 13726 raft_consensus.cc:2376] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24. Re-sending same reply.
I20260709 12:24:45.123963 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.096815 21802 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:45.121011 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.131422 13722 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:45.132539 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.133698 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.124807 21991 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:45.137522 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.121186 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.138247 13722 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:45.138746 13719 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:45.139591 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.139724 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.079027 21975 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:45.126689 21972 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.516s	user 0.003s	sys 0.000s
I20260709 12:24:45.125069 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.150661 13726 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:45.151151 13727 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:45.124604 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.147531 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.152344 13719 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:45.140506 13722 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:45.165136 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.168176 14424 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:45.114023 21888 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.172459 13693 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21.
I20260709 12:24:45.175202 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:45.170624 21978 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:45.191768 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:45.170244 21810 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:24:45.201087 21838 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:45.204936 21833 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:45.201215 21759 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.206071 21973 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:45.197088 21978 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:45.208222 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:24:45.212357 13719 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:45.214564 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.219853 13722 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:45.199585 21917 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:45.215775 21838 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.220458 21978 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:45.236737 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.228614 21838 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:45.220917 21759 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.240315 13722 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:45.242234 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.248381 21950 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.241241 21949 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:45.246341 21917 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.248553 21927 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21.
I20260709 12:24:45.254652 21759 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.263301 21978 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.256398 21949 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:45.265460 21975 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:45.256593 22000 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:45.269500 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1528) took 2020 ms. Trace:
I20260709 12:24:45.276971 21808 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:45.279897 13724 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:45.280691 14422 rpcz_store.cc:276] 0709 12:24:43.249447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.249608 (+   161us) service_pool.cc:224] Handling call
0709 12:24:43.250100 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:44.193877 (+943777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.193912 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:44.194565 (+   653us) log.cc:844] Serialized 52 byte log entry
0709 12:24:44.317760 (+123195us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:44.352843 (+ 35083us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.353130 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.353215 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.247286 (+894071us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b18004837c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:45.267753 (+ 20467us) raft_consensus.cc:1685] finished
0709 12:24:45.269220 (+  1467us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.269408 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":943314,"prepare.queue_time_us":38119,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":944322,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":5570,"spinlock_wait_cycles":15872896,"thread_start_us":122743,"threads_started":1,"wal-append.queue_time_us":259359}
I20260709 12:24:45.263355 21950 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.288407 21743 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:45.288949 21743 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:45.289978 21743 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.291390 21743 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:45.293363 21949 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.294782 21949 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:45.293393 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.299175 14420 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:45.301527 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.302747 21991 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.303999 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.305087 21991 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.306959 14417 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
W20260709 12:24:45.308683 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.315850 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.317516 13690 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.320729 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.319522 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.321471 13722 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:45.322219 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.320693 14425 raft_consensus.cc:2410] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:24:45.325974 21950 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.330996 13725 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:45.336087 13722 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:24:45.336411 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.326603 21950 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.348311 21950 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.335772 21917 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.349647 21917 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.338339 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.337164 14422 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:45.282914 21808 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:45.359272 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.361227 21917 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.361764 21808 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.362952 21808 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:45.366408 21986 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:45.367023 21986 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.367442 21986 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:45.370909 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.371624 13723 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:24:45.372776 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:45.373442 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.374929 13794 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:24:45.378189 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.383253 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.384131 13793 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:45.384497 13799 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 21. (term mismatch)
I20260709 12:24:45.384871 13799 pending_rounds.cc:85] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 20
I20260709 12:24:45.286547 22000 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:45.385336 13799 pending_rounds.cc:107] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 27.21
I20260709 12:24:45.387353 13799 raft_consensus.cc:2889] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.357302 21991 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:45.379464 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.392269 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.396272 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.396694 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:45.375454 21928 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:45.400768 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.384872 21975 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:24:45.406997 21824 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:24:45.413337 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.414673 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.415699 13799 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:45.418042 13721 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:24:45.406890 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.399842 21759 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:45.400136 21555 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:24:45.391026 21991 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.421325 21991 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:45.419900 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.422629 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:45.423410 21555 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:45.423830 21555 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.424175 21555 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:45.406946 21928 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.427174 21928 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:45.420483 21824 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:45.428355 21833 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.428431 21986 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.429550 21833 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.420606 21975 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:45.431385 21975 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.432132 21824 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:45.432351 21838 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.432766 21833 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.370381 21810 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:24:45.434804 21833 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:45.435382 21833 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:45.434733 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:45.437163 22000 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.436617 21833 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.439807 21656 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:45.440375 21656 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:45.440629 21555 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.443194 21927 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.443660 21573 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:45.444092 21573 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.443819 21759 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:24:45.444449 21573 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:45.446025 21934 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:24:45.446360 22002 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:45.432837 21824 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:45.450700 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:45.453125 14425 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:45.419920 13723 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:45.454794 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.440640 21833 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.456342 21917 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:45.449479 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:45.444406 21986 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:45.407090 21888 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.473774 14425 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:45.458755 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:45.479539 21759 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:45.479542 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.480276 21759 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.480830 21759 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:45.481479 13800 pending_rounds.cc:85] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 15
I20260709 12:24:45.481827 13800 pending_rounds.cc:107] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:24:45.482092 13800 raft_consensus.cc:2889] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:45.474207 21840 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:45.443701 21927 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:45.520679 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.482316 21917 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:45.523938 21838 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.526214 21927 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:45.539675 21824 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.544590 21861 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:45.545137 21861 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:45.547122 21927 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.546083 21861 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.548332 21861 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:45.547904 21927 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.492440 21810 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:45.550913 21810 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:45.551987 21810 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.553035 21822 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.555511 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.555431 21917 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.556262 13723 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:45.559558 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.562644 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:45.563740 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:24:45.564353 21973 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:45.564910 21973 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:45.565389 21973 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.570730 21743 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.571411 21743 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.573695 21743 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:45.575853 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:45.578299 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.582055 13652 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:24:45.584928 13643 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:45.586313 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:45.589172 13646 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:24:45.591144 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4064) took 1362 ms. Trace:
I20260709 12:24:45.591797 13792 rpcz_store.cc:276] 0709 12:24:44.229164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.230102 (+   938us) service_pool.cc:224] Handling call
0709 12:24:44.236060 (+  5958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.284678 (+ 48618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.590890 (+1306212us) spinlock_profiling.cc:243] Waited 44.2 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:45.591077 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101635968}
I20260709 12:24:45.593947 21819 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:45.593880 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:45.597339 13723 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:24:45.597450 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.602326 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:45.613786 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.517702 21840 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.616225 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:45.618362 13646 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:45.619982 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:45.614773 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.618731 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:45.619418 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:45.615445 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:45.624282 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:45.626592 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.626863 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 48896.8us
I20260709 12:24:45.639063 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.639824 13725 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:45.651423 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:45.652597 13797 raft_consensus.cc:2368] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:45.636794 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.653643 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.654240 13800 raft_consensus.cc:2376] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20. Re-sending same reply.
I20260709 12:24:45.655503 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.656348 13650 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:24:45.657550 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.659277 13650 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:45.659297 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:45.661224 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.662007 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.662158 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.662997 13650 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:24:45.671659 21949 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:45.672508 21949 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:45.673298 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.674005 13652 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:45.673595 21949 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.663239 13801 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24. Re-sending same reply.
I20260709 12:24:45.677619 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.636122 21840 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:45.679786 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.643198 21743 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:45.682040 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.683054 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.674806 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:45.690011 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.691015 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.692303 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.694976 21940 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:45.695611 21940 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:45.695899 22011 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:45.696038 21940 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:45.698596 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.641163 21810 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:24:45.641901 21573 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.706012 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.707068 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.709304 13800 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:24:45.710002 21656 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.723577 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.715214 21743 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:45.716058 14423 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:45.716892 13797 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:45.721171 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.714409 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1536) took 1223 ms. Trace:
I20260709 12:24:45.731504 14416 rpcz_store.cc:276] 0709 12:24:44.491137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.491751 (+   614us) service_pool.cc:224] Handling call
0709 12:24:44.510236 (+ 18485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.510562 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:44.534879 (+ 24317us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.713780 (+1178901us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b180021c700. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:24:45.713988 (+   208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:45.714053 (+    65us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:45.714111 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.714178 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.714328 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45326,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":9437,"spinlock_wait_cycles":5211904,"thread_start_us":64659,"threads_started":1,"wal-append.queue_time_us":245311}
I20260709 12:24:45.676738 21779 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:24:45.730440 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.740010 21802 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:45.738396 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.740968 21802 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:45.724520 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:45.741343 21802 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:45.742799 21802 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.744390 21802 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:45.745561 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:45.759461 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.704412 21555 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:45.770669 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.773934 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3855) took 1209 ms. Trace:
I20260709 12:24:45.779373 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:45.782528 21915 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:45.783102 21915 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.780596 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.787336 13645 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:45.800076 13795 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:45.800436 21656 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:45.807528 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.809369 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.810624 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.812631 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:45.814514 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.815207 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.725464 21900 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:45.782063 21975 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:24:45.785737 21915 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:45.838099 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48647.9us
W20260709 12:24:45.840198 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.844170 13719 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:24:45.849830 13649 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:45.823876 21573 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.774537 13720 rpcz_store.cc:276] 0709 12:24:44.564132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.564299 (+   167us) service_pool.cc:224] Handling call
0709 12:24:44.564739 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:44.652954 (+ 88215us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:44.677606 (+ 24652us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.684369 (+  6763us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b18002c7fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:45.298810 (+614441us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b18002c7fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:45.486767 (+187957us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:45.493421 (+  6654us) log.cc:844] Serialized 52 byte log entry
0709 12:24:45.493688 (+   267us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:45.513301 (+ 19613us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.520616 (+  7315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.772095 (+251479us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b18002c7fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:45.773549 (+  1454us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b54001563d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:45.773599 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.773648 (+    49us) raft_consensus.cc:1685] finished
0709 12:24:45.773663 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.773861 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65581,"prepare.queue_time_us":18833,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":97,"raft.queue_time_us":80391,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":13015,"spinlock_wait_cycles":121594496,"thread_start_us":164221,"threads_started":1,"wal-append.queue_time_us":834613}
I20260709 12:24:45.859285 13797 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:24:45.859520 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:45.850883 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.735726 21927 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:45.856062 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.862991 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.865291 13720 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:45.842559 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.866140 21900 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:24:45.868052 21991 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:45.734313 21876 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.129s
I20260709 12:24:45.870656 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:45.871410 21991 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:45.871744 13795 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 15: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:45.854836 21975 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.020s
I20260709 12:24:45.873524 21759 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.849718 21769 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.884907 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:45.887598 13720 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:24:45.962517 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77702us
I20260709 12:24:45.963703 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:45.909458 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:45.929617 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.923211 21555 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:24:45.984306 14422 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:45.984336 14418 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:24:45.908828 21907 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:24:45.938056 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:45.893127 21900 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:24:45.949005 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:46.051357 13643 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
W20260709 12:24:45.950860 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 314223us
I20260709 12:24:45.877494 21656 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:24:45.874331 21940 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:45.887307 21573 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:45.889055 21876 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:24:45.966233 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:45.968830 13720 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:45.970140 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:45.971105 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:45.971701 21824 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:46.066318 13650 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:45.979444 21957 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:46.050477 21882 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.053678 21833 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:46.109522 13650 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:45.963542 21973 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:46.120695 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 240940us
W20260709 12:24:46.077613 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 351454us
I20260709 12:24:46.132882 13767 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24.
I20260709 12:24:45.966137 21769 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.137497 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4068) took 1863 ms. Trace:
I20260709 12:24:46.138702 13798 rpcz_store.cc:276] 0709 12:24:44.273943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.281988 (+  8045us) service_pool.cc:224] Handling call
0709 12:24:44.285624 (+  3636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.616782 (+331158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.137167 (+1520385us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54003e17d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:46.137412 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760197376}
I20260709 12:24:46.140781 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:46.076913 21975 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:24:46.110323 21759 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.116183 21822 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:46.149688 21907 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.166338 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3731) took 1067 ms. Trace:
I20260709 12:24:46.198207 13724 rpcz_store.cc:276] 0709 12:24:45.099074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.108934 (+  9860us) service_pool.cc:224] Handling call
0709 12:24:46.166061 (+1057127us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54000f7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:46.166243 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390911744}
I20260709 12:24:46.200291 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:46.202524 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:46.183312 13650 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:46.203315 13651 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:46.150063 22020 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:24:46.205192 13724 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:24:46.165338 22009 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.697s	user 0.005s	sys 0.000s
I20260709 12:24:46.117321 21882 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.210223 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1728) took 1233 ms. Trace:
I20260709 12:24:46.188345 21883 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:46.153277 21940 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:46.216579 13721 rpcz_store.cc:276] 0709 12:24:44.977146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.977791 (+   645us) service_pool.cc:224] Handling call
0709 12:24:44.978185 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.418642 (+440457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.209973 (+791331us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:46.210158 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010441600}
I20260709 12:24:46.159364 21973 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.157420 21769 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.227864 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:46.239604 13648 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:46.126318 21833 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.230312 21917 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:46.141856 22023 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.088s
W20260709 12:24:46.250474 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 397292us
I20260709 12:24:46.136920 21840 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.275218 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:46.275646 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:46.202584 21907 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:46.273118 13726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.008s	sys 0.024s
W20260709 12:24:46.282891 13726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.008s	sys 0.024s
I20260709 12:24:46.205739 21822 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:46.295719 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.307282 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1308) took 1600 ms. Trace:
I20260709 12:24:46.311200 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:46.320251 14424 rpcz_store.cc:276] 0709 12:24:44.706745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.707087 (+   342us) service_pool.cc:224] Handling call
0709 12:24:46.306885 (+1599798us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54005965d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:46.307176 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886454144}
W20260709 12:24:46.323266 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 255007us
W20260709 12:24:46.324101 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.277269 22020 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:24:46.222795 21994 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:24:46.330344 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1729) took 1352 ms. Trace:
W20260709 12:24:46.339802 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.273751 21939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.657s	user 0.020s	sys 0.021s
I20260709 12:24:46.332108 13718 rpcz_store.cc:276] 0709 12:24:44.977617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.978358 (+   741us) service_pool.cc:224] Handling call
0709 12:24:46.330118 (+1351760us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b54002f5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:46.330273 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9705344}
I20260709 12:24:46.346570 13648 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:24:46.282119 22000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.008s	sys 0.024s
I20260709 12:24:46.354960 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:46.355666 13793 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:46.357362 13648 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:46.322238 21973 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:46.331197 21883 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:46.393110 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:46.393926 13794 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:46.353746 21939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.737s	user 0.021s	sys 0.021s
I20260709 12:24:46.388070 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:46.388283 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 266689us
W20260709 12:24:46.388154 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 418098us
I20260709 12:24:46.383492 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:46.299043 21975 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:46.323720 21927 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:46.351770 22000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.008s	sys 0.024s
I20260709 12:24:46.382537 21824 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:46.326829 21953 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 12:24:46.399281 21928 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.972s	user 0.013s	sys 0.002s
W20260709 12:24:46.401408 21861 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.853s	user 0.014s	sys 0.000s
I20260709 12:24:46.400070 21973 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.402817 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:46.332013 21885 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:46.407974 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:46.408224 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:46.415694 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:46.466707 13538 tablet_replica.cc:1406] Time spent applying in-flights took a long time: real 0.523s	user 0.000s	sys 0.000s
I20260709 12:24:46.330317 21900 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
W20260709 12:24:46.468012 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61158.1us
I20260709 12:24:46.399698 21907 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:46.467599 22019 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:24:46.392625 21883 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:46.354276 21656 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 12:24:46.503507 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95834.7us
I20260709 12:24:46.505918 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:46.467381 22012 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:46.466246 22023 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.507592 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:46.511355 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:46.517277 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:46.513370 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:46.521137 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:46.493468 13794 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:46.520648 22023 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.518533 22025 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.520058 21656 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.052s
I20260709 12:24:46.599910 22044 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:46.600411 22044 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:46.600762 22044 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:46.601392 22043 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:46.601934 22043 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:46.603121 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.604811 22033 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:24:46.605082 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:46.605995 13722 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 12:24:46.606806 22036 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:24:46.520530 21907 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:24:46.627413 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 316297us
I20260709 12:24:46.502957 21917 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:24:46.520751 21782 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:46.501235 22000 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.507412 21861 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:46.479038 21927 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.584168 13795 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:46.545779 22046 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:46.373991 21915 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.543612 13793 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:46.505596 21769 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:24:46.495791 21928 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.598649 22025 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.599092 21934 raft_consensus.cc:2764] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:24:46.707326 21840 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:46.611438 22041 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:46.502683 21885 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.491303 21975 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.608943 21735 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:24:46.614733 21802 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.870s	user 0.013s	sys 0.004s
W20260709 12:24:46.587831 22024 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
I20260709 12:24:46.628433 21888 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.557924 21994 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:46.596199 22022 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:46.735663 22029 raft_consensus.cc:2764] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:24:46.596721 21824 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:46.688733 22030 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.442s	user 0.001s	sys 0.001s
I20260709 12:24:46.661809 21833 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:46.684814 22021 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.405s	user 0.001s	sys 0.000s
I20260709 12:24:46.607949 21883 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.696158 21917 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:46.706704 21861 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.775180 22022 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:46.758301 22038 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 12:24:46.759279 22039 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:24:46.735785 22031 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 12:24:46.723762 22012 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.722939 22046 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:46.713287 22000 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:46.767908 22032 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.265s	user 0.001s	sys 0.001s
I20260709 12:24:46.772042 21994 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:46.500924 22037 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:46.777524 21973 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:46.764941 22049 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.027s
W20260709 12:24:46.785224 22028 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:24:46.768587 22041 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:46.800560 22045 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:24:46.792775 22012 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.802380 22041 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:46.802783 22048 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:24:46.803830 22012 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.802438 22046 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.805071 22046 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:46.806129 21975 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.807847 21928 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:46.786095 22022 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:46.809175 22025 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.809865 22037 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:46.810990 22037 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.820283 22000 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.797322 21994 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:46.833964 22022 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.834962 22044 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.953804 22044 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:46.888540 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.966387 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:46.890995 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4339) took 1272 ms. Trace:
W20260709 12:24:46.891903 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 370680us
I20260709 12:24:46.968776 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:46.970779 21555 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.893159 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 477398us
I20260709 12:24:46.972738 21927 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.970788 22025 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.895156 13719 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:46.896492 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:46.687912 13723 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:24:46.974308 22054 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:46.974133 22025 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.977185 21915 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.977957 22025 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.686347 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:46.980758 13724 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:46.973428 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.681210 13725 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 12:24:46.898695 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 215615us
I20260709 12:24:46.661134 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:46.995601 13722 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:46.902319 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3861) took 1265 ms. Trace:
I20260709 12:24:46.998541 13726 rpcz_store.cc:276] 0709 12:24:45.637427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.637709 (+   282us) service_pool.cc:224] Handling call
0709 12:24:45.638063 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:45.661060 (+ 22997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.661098 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:45.661714 (+   616us) log.cc:844] Serialized 52 byte log entry
0709 12:24:46.283381 (+621667us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:46.283427 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.283511 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.901717 (+618206us) raft_consensus.cc:1685] finished
0709 12:24:46.901760 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.902098 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4670,"raft.queue_time_us":4858,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":24279,"thread_start_us":575751,"threads_started":1,"wal-append.queue_time_us":835158}
I20260709 12:24:46.690352 13797 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:47.001715 13797 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.003640 13797 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.005455 13797 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:46.685894 13798 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:46.677908 21810 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.008991 21810 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.783951 21861 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:46.926446 21883 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:47.011358 22035 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:24:46.685892 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 302463us
I20260709 12:24:46.930540 21885 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.002035 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:46.932045 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1306) took 2647 ms. Trace:
I20260709 12:24:46.934762 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:47.019034 21743 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:46.769081 21782 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.020668 21743 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.018952 14421 rpcz_store.cc:276] 0709 12:24:44.284174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.284376 (+   202us) service_pool.cc:224] Handling call
0709 12:24:44.284745 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:44.285130 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.285164 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:44.288399 (+  3235us) log.cc:844] Serialized 26 byte log entry
0709 12:24:45.752246 (+1463847us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b1800454ba0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:24:46.111327 (+359081us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:46.111581 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.111850 (+   269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.931028 (+819178us) raft_consensus.cc:1685] finished
0709 12:24:46.931076 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.931684 (+   608us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1497,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4236,"spinlock_wait_cycles":9534208,"thread_start_us":358299,"threads_started":1,"wal-append.queue_time_us":2093138}
I20260709 12:24:47.021693 21782 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.023185 21743 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.774145 21888 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:46.952855 21917 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.025034 21882 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:47.027727 21888 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.028617 21888 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:46.870993 13538 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.372s	user 0.000s	sys 0.000s
I20260709 12:24:46.963316 22022 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:46.964505 21928 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.971927 21555 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:46.973655 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:47.035795 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62370.4us
I20260709 12:24:47.036967 21555 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.037595 21885 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:46.755394 14423 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:24:47.038159 21885 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:47.038914 21885 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:46.967669 13644 rpcz_store.cc:276] 0709 12:24:45.618060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.618967 (+   907us) service_pool.cc:224] Handling call
0709 12:24:45.636107 (+ 17140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.052700 (+416593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.052734 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.053332 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.508964 (+455632us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:46.588189 (+ 79225us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.597417 (+  9228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.597504 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.889918 (+292414us) raft_consensus.cc:1685] finished
0709 12:24:46.890014 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.890590 (+   576us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17492,"prepare.queue_time_us":78197,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":4506,"raft.queue_time_us":97955,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":8344,"spinlock_wait_cycles":27667328,"thread_start_us":790728,"threads_started":2,"wal-append.queue_time_us":591740}
I20260709 12:24:47.044092 21810 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.048347 13793 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:47.050633 21573 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.984s	user 0.007s	sys 0.015s
I20260709 12:24:47.051275 21573 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:24:47.051612 21822 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:47.007761 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.024919 21743 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:47.020458 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94237.2us
I20260709 12:24:47.029224 21888 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:47.032351 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4317) took 3485 ms. Trace:
I20260709 12:24:47.025137 21782 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:47.032614 21928 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.037931 21555 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:47.058877 13649 rpcz_store.cc:276] 0709 12:24:43.560433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.612785 (+ 52352us) service_pool.cc:224] Handling call
0709 12:24:44.266395 (+653610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.849626 (+1583231us) spinlock_profiling.cc:243] Waited 7.08 ms on lock 0x7b54003168a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf61c0 00007f5393bf7231 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c
0709 12:24:45.856147 (+  6521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.029094 (+1172947us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54001b5650. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:47.029316 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1467942528}
I20260709 12:24:47.070271 13724 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:47.072005 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:47.083170 13723 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:24:47.083814 13723 pending_rounds.cc:85] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:24:47.084112 13723 pending_rounds.cc:107] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:24:47.084465 13723 raft_consensus.cc:2889] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.085852 21810 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:47.087644 22044 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.088411 22044 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:47.089300 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.089654 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.090284 21949 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.090760 21810 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:47.091185 21949 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.091658 21810 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.093990 13726 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:47.091209 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:47.073882 21928 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.068184 21555 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:47.126588 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52305.8us
I20260709 12:24:47.120095 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:47.126169 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:47.132325 13799 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:47.134295 21822 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:47.140956 21822 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.141112 21888 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.143589 21822 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:47.144369 13615 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 2.274s (SENT) [suppressed 23 similar messages]
I20260709 12:24:47.131263 21555 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:47.148926 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.152630 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98758.8us
I20260709 12:24:47.137028 21573 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:47.168457 21928 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.173589 13722 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
W20260709 12:24:47.172492 22034 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.811s	user 0.005s	sys 0.005s
W20260709 12:24:47.185581 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89811.2us
I20260709 12:24:47.183110 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.174064 21991 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.215636 21810 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.215924 21991 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.216557 21900 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.217434 21900 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.218868 22057 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.219630 22057 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.220122 21934 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:24:47.220795 21934 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.199198 21882 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.203938 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.222842 21882 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.223439 13720 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:24:47.224699 21883 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:47.225571 21883 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.223445 22044 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.226876 21883 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.227392 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.230543 21949 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.231115 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.231917 13800 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:47.234447 21907 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
W20260709 12:24:47.235544 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.235716 21907 consensus_queue.cc:1243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 29.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:24:47.236485 21907 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:47.237347 21994 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.215078 22054 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:47.214910 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:47.220193 21900 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.227491 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4341) took 1608 ms. Trace:
I20260709 12:24:47.235365 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:47.239645 14416 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:24:47.248847 13645 rpcz_store.cc:276] 0709 12:24:45.619421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.619757 (+   336us) service_pool.cc:224] Handling call
0709 12:24:47.227064 (+1607307us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54002224d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:47.227284 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":334435200}
I20260709 12:24:47.196853 22022 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:47.240855 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.241909 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:47.174410 21819 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:24:47.187332 21573 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.208868 21928 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.243453 21940 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:47.262219 21940 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.261987 21573 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.263561 21940 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.265213 21940 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:47.222020 21934 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.199858 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:47.206489 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:47.268280 21928 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:47.268846 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62622us
I20260709 12:24:47.269941 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:47.270583 21900 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:47.271108 21983 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:47.271247 21900 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.271941 21983 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:47.272473 21900 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.273682 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:47.276656 22056 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:47.277243 22056 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.279081 22023 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.278860 22056 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.280927 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.281800 13722 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:24:47.283157 22043 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.213250 21861 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.243969 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.252091 22022 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.251999 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.253002 14418 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:47.253944 14422 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 18: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:47.255121 21991 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.250509 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:47.258836 21882 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:47.193840 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49555.3us
I20260709 12:24:47.327003 13800 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:24:47.247912 22057 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:47.310802 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 58532.1us
I20260709 12:24:47.334692 13721 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:24:47.318356 21986 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.342234 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:47.309890 21810 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.345904 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.347453 14424 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:47.349359 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:47.355718 21994 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:47.357046 22022 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.358314 21861 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.361581 21782 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.362495 21782 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.364427 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:47.366856 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.358724 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 159050us
I20260709 12:24:47.366209 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.369040 13796 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:24:47.366788 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:47.386960 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 193463us
W20260709 12:24:47.367681 22048 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:24:47.401988 13797 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:24:47.367513 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.369832 21782 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.404644 13797 pending_rounds.cc:85] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 14
I20260709 12:24:47.409356 13797 pending_rounds.cc:107] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:24:47.409699 13797 raft_consensus.cc:2889] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:47.411201 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:47.395251 21991 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:24:47.409420 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 64454.4us
I20260709 12:24:47.417573 13793 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:47.378422 21555 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:47.425791 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56180.6us
I20260709 12:24:47.416918 21991 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.437987 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.438948 13794 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:47.441169 21927 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:24:47.441533 13799 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:47.417476 21928 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.423689 21819 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.442024 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.446193 14418 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:24:47.440032 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.418303 21810 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.394687 21986 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.443962 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1542) took 1460 ms. Trace:
I20260709 12:24:47.443125 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1741) took 1557 ms. Trace:
I20260709 12:24:47.423313 21883 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:47.438082 21991 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:47.455168 21833 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:47.458096 13795 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:47.453294 14423 rpcz_store.cc:276] 0709 12:24:45.983180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.984441 (+  1261us) service_pool.cc:224] Handling call
0709 12:24:45.984744 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.038756 (+1054012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.443514 (+404758us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:47.443762 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771122944}
I20260709 12:24:47.469738 21928 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.472434 21928 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.472970 13798 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [21.15-22.16]
I20260709 12:24:47.473562 21900 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.474385 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.474218 21900 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.475788 21819 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.476827 21900 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.477423 21900 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.479473 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.480244 22053 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.481141 22053 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.481576 13796 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:24:47.483726 22053 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.484151 21810 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.454231 13719 rpcz_store.cc:276] 0709 12:24:45.885776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.937743 (+ 51967us) service_pool.cc:224] Handling call
0709 12:24:47.442855 (+1505112us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b5400156dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:47.443046 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084867968}
I20260709 12:24:47.475832 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.488080 14424 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:24:47.492790 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.496153 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.458066 21986 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.498792 21900 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.501004 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:47.501778 21900 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.502168 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:47.506621 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.507066 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.508039 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:47.508986 22076 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:24:47.510242 14425 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:47.510509 21927 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:47.494510 21934 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:47.511627 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.506664 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.513779 14421 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:47.514084 21555 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:47.514420 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 200166us
I20260709 12:24:47.514689 21986 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:47.515352 21986 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:47.515497 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:47.516584 21861 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:47.516506 21986 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.460253 21802 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.518469 21802 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.521011 21802 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.521589 21802 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.461632 21824 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:24:47.522240 22071 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
W20260709 12:24:47.534483 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 83572.7us
I20260709 12:24:47.536628 21555 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:47.538620 21555 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:47.539073 21555 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.540082 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:47.540704 22072 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
I20260709 12:24:47.540823 22044 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:24:47.542719 21759 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:47.517055 21861 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.546013 13793 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
W20260709 12:24:47.562570 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.576066 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:47.527594 22040 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.032s	user 0.002s	sys 0.000s
I20260709 12:24:47.576474 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.580456 14423 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:47.580258 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.524835 21819 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.587787 13801 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:47.589080 21573 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:47.594091 21759 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:47.594861 21759 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.596267 21759 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:47.599630 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.601490 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.602285 13645 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
W20260709 12:24:47.602877 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62923.5us
I20260709 12:24:47.603709 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.605069 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:47.605669 13644 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:47.607589 22054 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:47.608151 22054 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.609236 22054 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.610913 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.611825 13650 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:47.611380 21555 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.612979 21555 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:47.615072 21743 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.000s
I20260709 12:24:47.617285 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.618716 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.619395 13798 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:47.620896 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.622373 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.553884 21824 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:47.623920 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.625298 21824 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.626613 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.627935 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.573945 21861 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:47.632606 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 117261us
W20260709 12:24:47.633741 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52645.3us
I20260709 12:24:47.522202 21802 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.625401 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.524370 21885 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.646463 22043 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.656208 13793 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:47.598130 21759 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:47.512902 21927 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 12:24:47.666903 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:47.672724 22075 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:24:47.678660 22043 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.680552 21885 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.658355 21573 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.681730 21573 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.683034 13723 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:47.684010 21573 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.684617 21953 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:24:47.686314 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:47.688118 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:47.694758 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.695992 13650 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:24:47.696485 21743 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.697115 21743 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.697752 13645 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:24:47.670893 21779 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:24:47.581453 22053 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.602111 22047 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.716452 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.718677 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.719185 13652 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:24:47.719456 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:47.720407 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.722707 21779 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:47.722517 22047 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.723183 21779 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.725425 22047 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.724607 21779 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.726660 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.726477 22053 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.727381 14425 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:47.729259 22053 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.740629 13644 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
W20260709 12:24:47.743173 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80337.1us
I20260709 12:24:47.742797 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.755915 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.757025 13724 raft_consensus.cc:2410] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:24:47.757687 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:47.758488 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1002663479998773s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 12:24:47.758816 21779 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:47.759014 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0615663479998148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:24:47.759415 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0435663479997856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 12:24:47.759713 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0255663479997565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
I20260709 12:24:47.759743 13651 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 12:24:47.760669 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.761893 21885 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:47.762562 21885 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.763833 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.764534 13719 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:47.764871 21885 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.765470 21885 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.766422 21810 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.767095 21810 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.768522 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.769778 21810 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.758525 13647 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:47.708370 21573 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.765782 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.757933 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.775010 13724 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:24:47.703554 22057 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:47.756099 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.771894 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:47.772984 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.710078 22043 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:47.771442 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71033.6us
I20260709 12:24:47.716405 22022 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:47.773857 14418 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
W20260709 12:24:47.785411 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.781296 13645 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:47.713510 21983 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.786898 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.802868 13722 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:47.810388 13721 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:24:47.819502 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.765926 21885 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.815371 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:47.824767 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:47.786386 22044 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:47.779776 22057 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:47.837498 14417 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:47.787036 21810 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.787563 22022 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.835188 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.848631 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:47.850198 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:47.839568 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:47.825202 22023 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.854887 13652 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:24:47.786880 21573 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.839612 22043 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:47.843755 22044 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.854753 22022 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:47.858623 21927 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:47.880295 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:47.876973 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.868369 21743 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:47.891505 22053 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.891966 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:47.890910 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 129046us
I20260709 12:24:47.874816 21810 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.891606 13651 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:24:47.892432 21927 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.897627 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:47.920637 22043 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.921666 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.839691 21983 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.882524 22044 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:47.906809 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:47.924940 13645 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
W20260709 12:24:47.931761 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 230308us
I20260709 12:24:47.874564 21802 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:47.938459 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.941922 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:47.942148 13721 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:47.933051 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.946659 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.947793 13649 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:47.959183 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:47.949993 13652 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:47.969931 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.971072 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:47.972806 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 146012us
I20260709 12:24:47.940203 21927 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:47.980564 13651 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:47.981472 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:47.981003 22057 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:47.982303 13646 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:47.982929 22057 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:47.983332 21810 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:47.984630 21883 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:47.985690 13645 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:47.954206 22053 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.986003 13721 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:47.986932 13645 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:24:47.987377 13721 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:24:47.988207 22053 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:47.988662 21743 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.929035 22056 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:24:47.989262 21743 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.991611 21743 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:47.992709 22063 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:24:47.994508 22063 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:47.995013 22063 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:48.000411 21882 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:48.001235 21882 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.003576 13651 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:48.007905 21882 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:48.011664 22088 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
I20260709 12:24:48.015854 21983 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:48.018328 21957 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.451s	user 0.013s	sys 0.013s
W20260709 12:24:48.018795 21957 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.631s	user 0.013s	sys 0.013s
W20260709 12:24:48.019964 22027 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.492s	user 0.004s	sys 0.028s
W20260709 12:24:48.020341 22027 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.638s	user 0.004s	sys 0.028s
I20260709 12:24:48.021087 21833 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:48.021616 21833 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:47.916728 22023 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.025811 22023 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.027526 21573 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.028702 21885 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:48.029878 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.032423 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:48.033481 21940 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.006s
I20260709 12:24:47.981063 21950 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:47.974337 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.039800 13793 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
W20260709 12:24:48.038772 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62393.4us
I20260709 12:24:47.956023 22043 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.047132 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:48.049705 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.051435 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:48.066720 13799 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:48.072992 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.076329 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:48.053592 21991 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.561s	user 0.012s	sys 0.002s
I20260709 12:24:48.083529 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:48.075047 13647 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:48.052610 21885 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:48.085278 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.047173 21950 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:47.996174 22063 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.087222 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.099968 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.103035 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:48.105386 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 132550us
I20260709 12:24:48.107028 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.102885 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:48.091560 21743 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:48.091392 21882 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:48.128793 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 194503us
I20260709 12:24:48.112694 13647 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:48.121366 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.122895 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:48.057479 21861 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.147533 13793 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:48.090669 21991 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.131633 21882 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.151888 21882 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.152251 22063 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:48.154376 21991 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.155267 14422 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:48.156209 13798 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:48.157477 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:48.149999 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.118758 22056 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:48.150899 13800 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:24:48.118532 21953 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:48.168092 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.168953 21950 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.169624 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:48.173173 21759 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.002s
I20260709 12:24:48.174827 13693 leader_election.cc:400] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:48.191586 13693 leader_election.cc:403] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:48.132457 21940 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:48.173285 21882 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.188437 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:48.161796 21986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.013s	sys 0.032s
I20260709 12:24:48.195456 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.196195 13647 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:24:48.198522 13798 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:48.175791 21953 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.188957 21991 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:24:48.199247 21986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.014s	sys 0.032s
I20260709 12:24:48.193811 21940 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.199589 21900 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:48.232144 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.200731 21885 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:48.236729 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.216714 21953 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:48.242061 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 125618us
I20260709 12:24:48.231084 21940 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:48.257213 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68362.9us
W20260709 12:24:48.259455 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84920.8us
I20260709 12:24:48.260447 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.266445 22108 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.267342 22108 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.270627 22108 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:48.271047 22105 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:24:48.273478 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.274539 21882 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.274847 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.232782 21900 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:48.282471 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.113106 13538 maintenance_manager.cc:643] P d5e54c6d2c954fbf93d5136276b7b78c: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 4.569s	user 0.131s	sys 0.044s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":25556,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":475567,"dirs.run_cpu_time_us":752,"dirs.run_wall_time_us":17335,"lbm_read_time_us":23178,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":22,"lbm_write_time_us":16770,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":10,"peak_mem_usage":269820,"reinsert_count":0,"spinlock_wait_cycles":258493824,"thread_start_us":420925,"threads_started":2,"update_count":199}
I20260709 12:24:48.287417 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.288267 13643 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:48.289599 13643 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:48.288123 22103 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:24:48.293846 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 104141us
I20260709 12:24:48.293606 21915 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.003s
I20260709 12:24:48.298877 22103 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:48.299324 22103 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:48.300232 22103 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.301115 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.301961 22103 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:48.302578 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.254664 22046 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:48.294488 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:48.303892 13801 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:48.309207 13648 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 15.
I20260709 12:24:48.312915 13643 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:24:48.279135 21986 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:48.280159 21900 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:48.317579 13799 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:48.294070 21991 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
W20260709 12:24:48.332967 22100 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:24:48.333925 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.307696 22043 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.335250 13801 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:24:48.335361 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50956.5us
I20260709 12:24:48.313836 21915 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:48.284066 22044 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.347289 13651 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:48.350455 21861 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.353320 21888 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:48.356423 21900 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:48.357016 22096 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.342s	user 0.001s	sys 0.001s
I20260709 12:24:48.339723 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:48.357930 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3749) took 3221 ms. Trace:
I20260709 12:24:48.358337 22057 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:48.359081 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:48.359385 22090 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
W20260709 12:24:48.360615 22094 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.368s	user 0.002s	sys 0.000s
W20260709 12:24:48.361933 22093 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 12:24:48.361536 21953 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.362722 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.365706 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.376699 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.373447 13727 rpcz_store.cc:276] 0709 12:24:45.136832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.154557 (+ 17725us) service_pool.cc:224] Handling call
0709 12:24:46.330429 (+1175872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.332174 (+  1745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.348775 (+ 16601us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:46.349330 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.710665 (+361335us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:46.737183 (+ 26518us) log.cc:844] Serialized 14 byte log entry
0709 12:24:46.737426 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.737518 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:47.223962 (+486444us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b180034c900. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:47.281500 (+ 57538us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b180034c900. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:47.284593 (+  3093us) raft_consensus.cc:1685] finished
0709 12:24:47.284617 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.357549 (+1072932us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002f5150. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:48.357710 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1250,"prepare.queue_time_us":26811,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":36031,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":15823,"spinlock_wait_cycles":2715598080,"thread_start_us":360745,"threads_started":1,"wal-append.queue_time_us":423705,"wal-append.run_cpu_time_us":2835,"wal-append.run_wall_time_us":1005051}
I20260709 12:24:48.386548 13649 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
W20260709 12:24:48.387013 22092 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.365s	user 0.002s	sys 0.000s
I20260709 12:24:48.387749 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:48.389531 22083 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.736s	user 0.001s	sys 0.004s
I20260709 12:24:48.377468 13647 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:48.389531 22044 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.391095 14420 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
W20260709 12:24:48.391557 22078 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.816s	user 0.002s	sys 0.000s
I20260709 12:24:48.385094 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.384632 22056 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:48.393541 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1746) took 1301 ms. Trace:
I20260709 12:24:48.393962 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.373427 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:48.399377 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63774.5us
I20260709 12:24:48.393911 13720 rpcz_store.cc:276] 0709 12:24:47.092549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.224557 (+132008us) service_pool.cc:224] Handling call
0709 12:24:47.225184 (+   627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.225742 (+   558us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.225775 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.226389 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.341449 (+115060us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:24:47.341488 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.341566 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.390900 (+1049334us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:48.393238 (+  2338us) raft_consensus.cc:1685] finished
0709 12:24:48.393258 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.393470 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17557,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":137,"spinlock_wait_cycles":10874624,"thread_start_us":114574,"threads_started":1,"wal-append.queue_time_us":206517}
W20260709 12:24:48.404028 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86430.3us
I20260709 12:24:48.403757 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.405165 13650 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:48.409664 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:48.410393 13651 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.419750 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.424161 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.425046 13797 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:48.413585 22042 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:48.425941 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:48.430126 13794 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:48.421747 22029 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:24:48.431404 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.432322 13797 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:24:48.403141 22057 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.435266 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:48.439404 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.427228 22042 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.442842 22042 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:48.443769 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:48.445360 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.446816 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1342) took 1221 ms. Trace:
I20260709 12:24:48.447275 13795 rpcz_store.cc:276] 0709 12:24:47.225358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.226506 (+  1148us) service_pool.cc:224] Handling call
0709 12:24:47.445441 (+218935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.487482 (+ 42041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.446570 (+959088us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54003d40d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:48.446726 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502515968}
I20260709 12:24:48.449299 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1752) took 1217 ms. Trace:
I20260709 12:24:48.449719 13723 rpcz_store.cc:276] 0709 12:24:47.231897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.280250 (+ 48353us) service_pool.cc:224] Handling call
0709 12:24:47.280631 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:47.683469 (+402838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.448972 (+765503us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:48.449227 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924952832}
I20260709 12:24:48.450904 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.452247 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:48.453078 13794 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
W20260709 12:24:48.457070 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.459247 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1346) took 1114 ms. Trace:
I20260709 12:24:48.459743 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:48.488909 13648 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:48.460745 13792 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:24:48.461836 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.464377 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:48.464871 13652 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:48.466185 13724 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:24:48.470353 22108 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:48.470570 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:48.472261 13647 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:24:48.473594 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.493888 13718 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.494037 13799 raft_consensus.cc:2376] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29. Re-sending same reply.
I20260709 12:24:48.487969 13800 rpcz_store.cc:276] 0709 12:24:47.345351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.345602 (+   251us) service_pool.cc:224] Handling call
0709 12:24:48.458988 (+1113386us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b540012a054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:48.459153 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597519616}
I20260709 12:24:48.476547 13796 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:48.490774 13719 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:48.497792 13649 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:48.477257 21927 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.480270 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.480974 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:48.500017 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:48.506429 13794 raft_consensus.cc:2410] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 12:24:48.440563 22022 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.006s
I20260709 12:24:48.509464 13722 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:48.510344 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:48.510996 13726 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:24:48.511169 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:48.513952 13723 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:48.510547 13618 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:48.516426 22022 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.517819 22057 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.518445 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:48.519711 21927 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.520601 21940 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.521761 21900 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.527456 13799 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:48.527499 13618 leader_election.cc:403] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:48.531226 13645 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:48.531477 13647 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:24:48.532732 22056 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:48.534811 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 112741us
I20260709 12:24:48.496646 22108 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.540511 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:48.545070 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:48.569521 13797 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:24:48.546015 21656 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:48.547051 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.548017 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:48.549068 13801 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:24:48.553516 13643 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:24:48.505352 21986 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:48.560202 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:48.566486 13645 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:48.568198 13644 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
W20260709 12:24:48.578039 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74641.6us
I20260709 12:24:48.585525 13723 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:48.587245 13647 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:24:48.588501 13645 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:48.589665 21833 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.567s	user 0.002s	sys 0.012s
I20260709 12:24:48.577129 22056 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.590443 13798 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:48.571945 22108 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:48.541707 21779 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:24:48.598212 21986 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:48.598927 21986 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.599376 22029 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:48.599833 22029 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.600164 22029 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:48.601382 21986 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.604123 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.605751 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:48.606943 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67394.7us
I20260709 12:24:48.607741 13794 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:24:48.608800 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.610440 21900 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:48.611438 21900 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:24:48.612151 21900 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:48.612460 21900 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:48.610499 21986 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:48.613389 21900 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.581439 21883 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.589665 13645 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:48.610126 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.620858 13723 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:48.624611 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.642066 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1546) took 1131 ms. Trace:
W20260709 12:24:48.581169 22054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.015s	sys 0.023s
I20260709 12:24:48.627861 21986 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:48.642557 14422 rpcz_store.cc:276] 0709 12:24:47.504445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.506743 (+  2298us) service_pool.cc:224] Handling call
0709 12:24:48.635965 (+1129222us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54006a72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:48.636122 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488855424}
I20260709 12:24:48.652593 13718 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:48.581385 21656 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.635915 21882 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.626025 21779 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.658783 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53149.9us
I20260709 12:24:48.630467 22115 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:48.656418 13726 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:48.663549 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:48.657195 22054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.016s	sys 0.023s
W20260709 12:24:48.671540 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.676116 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.676515 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:48.677121 21656 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.679395 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:48.681062 21927 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:48.681543 21927 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.682088 21927 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:48.683400 21900 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:48.683513 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.685104 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.685392 21861 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:48.686239 21861 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:48.680388 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.681478 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.688305 21824 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:48.688380 22103 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:48.688925 21824 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:48.689393 21824 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:48.690136 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.692589 21861 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:48.692440 22042 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.687745 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.696115 22103 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:48.697652 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:48.698465 22103 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:48.647377 22023 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:24:48.702355 21882 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.703140 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:48.703737 13727 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:48.704640 21882 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.704929 22108 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.705940 22022 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.590193 21833 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.706346 21927 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.706517 21779 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.708684 21833 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.708792 21779 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.701936 21940 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.713536 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.658993 22046 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:24:48.714890 22046 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:48.714990 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.715339 22046 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:24:48.716462 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.717026 14417 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:24:48.717396 14417 pending_rounds.cc:85] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 15
I20260709 12:24:48.717605 14417 pending_rounds.cc:107] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:24:48.717829 14417 raft_consensus.cc:2889] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:48.663769 22056 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.718331 21824 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.665010 21983 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:24:48.720683 21927 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.673938 21957 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:24:48.722728 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:48.723512 13720 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:48.723533 13793 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:48.725098 13651 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
W20260709 12:24:48.721222 22118 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:48.726339 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:48.726649 14417 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:48.726933 13793 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:48.700551 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.746134 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:48.746438 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45967.7us
I20260709 12:24:48.743610 13727 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:48.774312 14417 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:24:48.778971 13650 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:24:48.783180 13645 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 12:24:48.733786 22103 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:48.791577 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.741984 21927 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.745226 21940 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.832785 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86537.2us
I20260709 12:24:48.838534 21833 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:48.780076 21819 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:48.789832 13645 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 16
I20260709 12:24:48.846815 13645 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:24:48.847591 13645 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:48.849807 14417 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:48.850451 14417 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:48.851526 14417 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.852862 14417 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:48.853150 13793 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:48.858120 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:48.859172 14422 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:48.861135 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:48.863624 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:48.865187 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.866181 22044 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:24:48.868692 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.869837 22044 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:24:48.873173 22044 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:48.883369 22044 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:48.846323 21824 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:48.898414 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124110us
I20260709 12:24:48.843936 21833 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:48.916528 22121 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:24:48.839305 21940 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.916532 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:48.916997 22054 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:48.940395 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:48.944905 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:48.928386 22042 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.950541 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:48.841157 21927 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.928853 21819 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:48.961521 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:48.948237 21810 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:48.952560 22103 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:24:48.969278 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52911.2us
I20260709 12:24:48.954691 22108 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:48.959102 22023 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:48.981483 22023 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:48.982741 22023 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:48.992029 21940 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:48.994135 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:48.994644 21819 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.995155 13721 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:48.998677 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:48.999044 21833 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:48.999904 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:49.000447 13801 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:49.003818 22029 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.008998 13801 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:49.010780 21833 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:49.012781 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:49.013409 13797 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:48.975814 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1351) took 1623 ms. Trace:
W20260709 12:24:49.020567 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 396080us
I20260709 12:24:49.021615 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:49.018939 13792 rpcz_store.cc:276] 0709 12:24:47.352472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.352679 (+   207us) service_pool.cc:224] Handling call
0709 12:24:48.460263 (+1107584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.489824 (+ 29561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.974195 (+484371us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540012a050. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:48.975735 (+  1540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2546266496}
I20260709 12:24:48.971451 22042 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:49.018196 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:49.042630 13797 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
W20260709 12:24:49.047785 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.060122 21810 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:24:49.072417 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 121965us
W20260709 12:24:49.077376 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59297.8us
I20260709 12:24:49.082034 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.062764 21883 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
W20260709 12:24:49.089054 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 130751us
I20260709 12:24:49.067778 21779 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:49.072358 22029 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:49.029208 21819 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:24:49.092263 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:49.100535 13723 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.094731 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:49.122715 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2810) took 1238 ms. Trace:
I20260709 12:24:49.123279 13798 rpcz_store.cc:276] 0709 12:24:47.884428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.204874 (+320446us) service_pool.cc:224] Handling call
0709 12:24:49.122489 (+917615us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:49.122644 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":801444224}
I20260709 12:24:49.126088 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:49.127717 21986 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:49.127961 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:49.107964 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:49.093765 22103 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:49.102690 21819 consensus_queue.cc:1243] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:49.136842 13723 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:24:49.100446 21940 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.144424 13721 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:24:49.145556 13721 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 13
I20260709 12:24:49.145802 13721 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:24:49.146083 13721 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:49.144340 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63624.1us
W20260709 12:24:49.140547 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45937.3us
I20260709 12:24:49.148857 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:49.149856 22132 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:49.156243 21819 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:49.156705 21819 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.153309 22137 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:49.157179 21819 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.139345 21900 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:24:49.167114 22131 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:49.169251 22131 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:49.131330 21885 raft_consensus.cc:2764] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:24:49.129189 22046 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.171186 21927 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:49.137420 21861 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:24:49.172894 22046 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.153515 22056 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:49.131618 22126 raft_consensus.cc:2764] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:24:49.174568 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:49.150588 21940 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.177637 21940 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.179107 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:49.179883 14425 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:49.182788 22103 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.191054 22131 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:24:49.238708 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.254832 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 109757us
W20260709 12:24:49.270712 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 121967us
I20260709 12:24:49.276451 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.277586 13723 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:49.275591 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:24:49.289099 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 161189us
I20260709 12:24:49.295410 13798 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:49.293210 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:49.274565 22043 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:49.279211 22042 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:49.301728 21940 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.308230 21940 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.310546 21940 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.303541 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:49.312268 22022 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:49.312738 22022 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:49.313690 22022 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.314304 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:49.314742 22046 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:49.326081 14421 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
W20260709 12:24:49.300213 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 208041us
I20260709 12:24:49.316275 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:49.316598 21986 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:49.289106 22117 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:49.318235 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.336109 13723 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:49.319478 14417 pending_rounds.cc:85] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 19
I20260709 12:24:49.337343 14417 pending_rounds.cc:107] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 25.20
I20260709 12:24:49.337678 14417 raft_consensus.cc:2889] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:49.320039 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43910.8us
I20260709 12:24:49.321240 21743 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:49.321698 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2780) took 1534 ms. Trace:
I20260709 12:24:49.335721 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:49.340775 13644 rpcz_store.cc:276] 0709 12:24:47.786827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.787277 (+   450us) service_pool.cc:224] Handling call
0709 12:24:47.787599 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.578654 (+791055us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.321435 (+742781us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54001e6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:49.321632 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794405120}
I20260709 12:24:49.341837 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:49.323894 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 169950us
I20260709 12:24:49.335116 21986 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.326122 22042 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:49.342845 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.310249 21819 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.346962 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.331590 22046 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.352984 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.354660 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.355408 13799 raft_consensus.cc:2368] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:24:49.329164 22043 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.362303 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.347127 21986 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:49.340965 21861 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:49.372723 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.343518 22117 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:49.353984 22022 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:49.359521 22126 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:49.387954 21953 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.387442 22042 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.386463 21888 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:24:49.406524 21888 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:24:49.356752 22046 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.408885 22043 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:49.370819 21900 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:24:49.411212 21819 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:49.383385 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.414755 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.378028 22141 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:24:49.397140 13651 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:49.415619 13799 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:49.398995 13796 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:49.399664 22126 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.416467 22126 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:49.419042 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.419701 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.422897 22046 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.425858 13646 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:49.426770 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:49.427416 14422 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.427979 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:49.436594 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.439246 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.440272 13651 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:49.392812 21861 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.449244 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.450315 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:49.453554 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.459928 13796 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:24:49.460495 13796 pending_rounds.cc:85] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 15
I20260709 12:24:49.460740 13796 pending_rounds.cc:107] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:24:49.460966 13796 raft_consensus.cc:2889] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.456810 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.452320 13646 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:49.438860 22046 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.466578 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.467514 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:49.468182 13796 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:49.468420 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.448975 21779 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.471599 21779 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:49.474463 22138 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.319s	user 0.011s	sys 0.015s
I20260709 12:24:49.440402 22141 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:24:49.478636 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.479035 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:49.479578 14419 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:49.479868 22146 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:49.480463 22141 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:49.480633 22146 consensus_queue.cc:1243] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 22.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:24:49.480890 22141 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:49.482698 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:49.439215 21888 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.028s
I20260709 12:24:49.459074 21861 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.478653 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:49.480008 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.492890 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.492725 21986 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.493608 13643 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:49.490505 22117 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.500964 22057 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:49.501457 22057 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:49.503796 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.509538 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.510550 13646 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:24:49.515727 14422 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:49.517338 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:49.523787 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1350) took 1169 ms. Trace:
I20260709 12:24:49.526170 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:49.527300 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:49.524463 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4372) took 1406 ms. Trace:
I20260709 12:24:49.524245 13650 rpcz_store.cc:276] 0709 12:24:48.354814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.535332 (+180518us) service_pool.cc:224] Handling call
0709 12:24:48.535911 (+   579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.788087 (+252176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.523473 (+735386us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54001e5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:49.523700 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558059392}
I20260709 12:24:49.530856 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.531668 13719 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:49.527987 13652 rpcz_store.cc:276] 0709 12:24:48.117711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.344351 (+226640us) service_pool.cc:224] Handling call
0709 12:24:48.464487 (+120136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.492118 (+ 27631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.524263 (+1032145us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400113d50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:49.524394 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275539200}
I20260709 12:24:49.539491 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:49.565500 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.570242 13795 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: [22.13-24.14]
I20260709 12:24:49.572120 13799 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [14.11-17.12]
I20260709 12:24:49.516469 22139 raft_consensus.cc:2764] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:24:49.527714 21900 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:49.582109 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:49.591109 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.597045 13722 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:49.481837 22141 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.540279 21743 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:49.611832 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:49.621032 13650 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:49.540531 21888 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.076s
I20260709 12:24:49.640841 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.581766 22146 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:24:49.509852 22057 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.575150 22137 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:24:49.644178 13722 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:49.659384 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.660593 22057 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:49.666440 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 91920.2us
I20260709 12:24:49.616797 21743 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:49.540889 22126 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.691226 13719 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:49.699173 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.700222 14419 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:49.701751 13719 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:49.703083 13721 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:49.690999 21743 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.637269 21861 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.701217 21986 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:49.750520 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.759178 13722 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:49.755431 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226794us
I20260709 12:24:49.673395 22046 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:24:49.692497 22146 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.056s
I20260709 12:24:49.746896 22141 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:49.795471 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:49.714293 22134 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:24:49.804262 22133 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.826840 22126 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.797483 21819 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:49.831905 21861 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.832511 21940 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.833671 22046 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:49.834088 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.838785 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.888530 13726 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:49.839645 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.840364 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:49.841313 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.890470 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.891865 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.892619 13727 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:49.842645 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.894101 13723 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:49.849920 13647 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:49.856238 21900 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:49.859783 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.860328 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:49.901832 14421 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:49.904099 13646 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:24:49.861884 13796 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:49.918222 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.919687 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:49.884708 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 291763us
I20260709 12:24:49.920495 13650 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:49.885524 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90164us
I20260709 12:24:49.919791 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:49.926813 13652 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:49.908555 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.867877 13721 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:24:49.925448 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60231.5us
I20260709 12:24:49.923523 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.939532 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:49.933504 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:49.906103 22029 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:49.941259 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 272873us
W20260709 12:24:49.942988 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.945199 13720 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:49.951020 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:49.953676 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.954675 13718 raft_consensus.cc:2368] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:49.938040 13723 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:49.919303 21779 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:49.904583 21819 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.884610 22133 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.964747 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:49.927983 22126 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.977720 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:49.978102 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:49.968487 13721 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:24:49.927601 22022 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.964810 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.906400 21940 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:49.979604 13650 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:49.940730 22106 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:49.983691 13720 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:49.991732 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:49.996932 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1346) took 1644 ms. Trace:
I20260709 12:24:49.999547 13649 rpcz_store.cc:276] 0709 12:24:48.352683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.505573 (+152890us) service_pool.cc:224] Handling call
0709 12:24:48.505912 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.514492 (+  8580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.514525 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.567688 (+ 53163us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.647204 (+ 79516us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:48.765967 (+118763us) log.cc:844] Serialized 14 byte log entry
0709 12:24:48.766291 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.766402 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.830538 (+1064136us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b18001d08c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:24:49.969027 (+138489us) raft_consensus.cc:1685] finished
0709 12:24:49.996725 (+ 27698us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.996865 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":118272,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":12861,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":16443,"spinlock_wait_cycles":3004800,"thread_start_us":51890,"threads_started":1,"wal-append.queue_time_us":243626}
I20260709 12:24:50.008062 13727 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:50.018787 13718 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:50.010892 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:50.020380 13721 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:24:50.023398 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3914) took 1383 ms. Trace:
I20260709 12:24:49.997733 21824 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.023931 13724 rpcz_store.cc:276] 0709 12:24:48.640164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.640347 (+   183us) service_pool.cc:224] Handling call
0709 12:24:48.660073 (+ 19726us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:48.710621 (+ 50548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.710656 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:48.710717 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:48.710761 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.022691 (+1311930us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b54001a4d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:50.023159 (+   468us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.023330 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115658496}
I20260709 12:24:50.028641 13726 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:50.029369 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.020749 21953 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:50.030537 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:50.033295 13720 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
W20260709 12:24:50.034371 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.984519 22160 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:49.998103 22022 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:49.986989 22126 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.041980 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:50.006285 22046 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:24:50.025188 22158 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:50.021713 22164 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:50.050401 13721 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:24:50.061812 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122015us
I20260709 12:24:49.984692 21900 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:50.080617 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 132279us
I20260709 12:24:50.076344 22161 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:24:50.032469 21779 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:24:50.034698 21824 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.074712 22158 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.077291 22164 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.038040 22140 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:24:50.076864 22160 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:50.094939 22161 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:50.104661 22158 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:50.015439 22106 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:50.137169 22174 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:24:50.139072 22106 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:50.139968 22157 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:24:50.103410 21743 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.147706 21743 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:50.148803 21743 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:24:50.150015 22126 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:50.152141 22022 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:50.112710 22164 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:24:50.136323 22173 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:24:50.138944 22155 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:24:50.125445 21819 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:50.178130 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 225225us
I20260709 12:24:50.179361 21819 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.179966 21819 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:50.180796 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 151521us
I20260709 12:24:50.181699 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.187875 21833 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:50.199895 21833 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.200348 21833 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:50.192881 21953 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:50.202979 21953 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:50.203951 21953 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.205497 21953 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:50.185937 21861 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:50.207739 21861 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.208683 21861 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:50.196650 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:50.211602 13795 raft_consensus.cc:2368] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
I20260709 12:24:50.210448 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:50.213629 13796 raft_consensus.cc:2376] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25. Re-sending same reply.
I20260709 12:24:50.182250 21819 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:50.154407 22171 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:24:50.229724 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:50.173573 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91494.4us
I20260709 12:24:50.246528 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:50.232378 22177 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.104s	user 0.000s	sys 0.005s
I20260709 12:24:50.253126 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:50.244264 22179 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.119s	user 0.003s	sys 0.000s
W20260709 12:24:50.233047 22169 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.063s	user 0.001s	sys 0.003s
W20260709 12:24:50.293936 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47496.8us
I20260709 12:24:50.257095 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:50.300949 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:24:50.259183 22167 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 12:24:50.305284 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 48307.2us
I20260709 12:24:50.315186 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:50.230154 22131 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.069s
I20260709 12:24:50.320524 13652 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:50.330822 14421 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:50.343433 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:50.299563 22133 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.337515 21986 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:50.352638 22056 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:50.316694 22182 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:50.327152 21940 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:50.341969 22162 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:24:50.381170 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 199576us
I20260709 12:24:50.381779 21986 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:50.412117 22201 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:50.412317 21986 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.412947 22201 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.382448 22162 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:50.414829 22162 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:50.415553 22201 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:50.418991 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.419483 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4138) took 1694 ms. Trace:
I20260709 12:24:50.421110 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:50.419922 13793 rpcz_store.cc:276] 0709 12:24:48.725003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.856919 (+131916us) service_pool.cc:224] Handling call
0709 12:24:48.857359 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.953399 (+ 96040us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:49.385740 (+432341us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b180035b7e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:24:49.565534 (+179794us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.969935 (+404401us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.970492 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.970647 (+   155us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:49.970680 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.190203 (+219523us) spinlock_profiling.cc:243] Waited 94.7 ms on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:50.192621 (+  2418us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.419217 (+226596us) raft_consensus.cc:1685] finished
0709 12:24:50.419241 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.419408 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45846,"prepare.queue_time_us":467982,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":36672,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":15700,"spinlock_wait_cycles":228707840,"thread_start_us":424727,"threads_started":2,"wal-append.queue_time_us":468007}
W20260709 12:24:50.440287 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96933.1us
W20260709 12:24:50.443356 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 342500us
I20260709 12:24:50.511842 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:50.515422 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.322463 22160 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.542943 13722 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:50.525493 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125987us
I20260709 12:24:50.545871 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:50.545987 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:50.570919 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.344929 22161 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.543428 22188 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:50.519021 22194 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.621117 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:50.541538 22056 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:50.631054 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85293.5us
I20260709 12:24:50.638402 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.639649 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.616530 22103 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:50.619287 22182 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:50.620376 22140 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:50.624822 21819 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:50.627323 22188 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:50.685190 22196 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 12:24:50.686416 22192 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:24:50.669364 22158 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.005s
W20260709 12:24:50.703369 22191 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 12:24:50.706965 21940 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:50.668917 22188 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:50.691272 22056 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.667068 22140 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.663717 22210 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:50.630779 22183 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.059s
I20260709 12:24:50.671633 22022 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.631939 22194 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.661542 22103 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.698693 22182 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:50.720402 21833 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.519s	user 0.014s	sys 0.004s
I20260709 12:24:50.715731 21888 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:50.749388 21882 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:50.752487 22193 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:24:50.757076 22164 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.527s	user 0.008s	sys 0.004s
W20260709 12:24:50.764227 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 218349us
W20260709 12:24:50.768425 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 128857us
I20260709 12:24:50.726444 22158 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.749588 22210 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:50.777336 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:50.778545 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:50.749492 22022 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.788661 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4378) took 1357 ms. Trace:
I20260709 12:24:50.767594 21882 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:50.760281 21833 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:50.804071 22204 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.003s
I20260709 12:24:50.804528 22181 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.818158 22140 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.818655 22181 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.819516 22210 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.770334 22193 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:50.823683 22193 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.825291 22193 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:50.794677 22206 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.817938 22158 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:50.776584 22164 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:50.835314 21861 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.626s	user 0.008s	sys 0.009s
I20260709 12:24:50.836305 21861 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.763363 21888 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.840972 21888 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.822167 22181 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:50.844110 22022 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:50.846208 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56950.4us
I20260709 12:24:50.847092 21882 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.848737 21882 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:50.855304 22158 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:50.856272 22158 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.856555 21833 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.816509 22194 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.824379 22103 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.860020 22158 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:50.866714 22188 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.797962 13647 rpcz_store.cc:276] 0709 12:24:49.430905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.439439 (+  8534us) service_pool.cc:224] Handling call
0709 12:24:50.788371 (+1348932us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b5400199454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:50.788562 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":798713216}
I20260709 12:24:50.877377 22164 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:50.878388 22195 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:24:50.881124 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3784) took 1043 ms. Trace:
I20260709 12:24:50.881515 13723 rpcz_store.cc:276] 0709 12:24:49.837371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.965603 (+128232us) service_pool.cc:224] Handling call
0709 12:24:49.969487 (+  3884us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.179153 (+209666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.179193 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.179829 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.307059 (+127230us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:50.616359 (+309300us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.619615 (+  3256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.619705 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.880883 (+261178us) raft_consensus.cc:1685] finished
0709 12:24:50.880907 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.881058 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203350,"prepare.queue_time_us":314298,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":51,"raft.queue_time_us":201849,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":15762,"spinlock_wait_cycles":29984128,"thread_start_us":346084,"threads_started":2,"wal-append.queue_time_us":576635}
I20260709 12:24:50.884403 22188 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:50.884836 22133 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:50.885430 22133 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.887625 22133 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.889794 22043 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:50.890347 22043 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.892521 22043 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:50.892989 22043 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:50.893491 22043 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.896034 22043 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:50.896766 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.893147 21824 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.899623 22198 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:50.900302 22198 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.900789 22198 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:50.902755 22197 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:24:50.903790 22197 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.904428 22197 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:50.908454 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.909749 22199 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:50.909843 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 132590us
I20260709 12:24:50.909842 22044 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.910496 22199 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.910802 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.910858 22044 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.912272 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.912652 22199 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:50.913178 22199 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.913503 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:50.914548 21986 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:50.914716 22194 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:50.915012 21986 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:50.915293 22194 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:50.915900 22194 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:50.916847 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.915534 21986 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:50.923986 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:50.927249 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:50.929339 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:50.930135 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3779) took 1094 ms. Trace:
I20260709 12:24:50.930531 13727 rpcz_store.cc:276] 0709 12:24:49.835887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.908202 (+ 72315us) service_pool.cc:224] Handling call
0709 12:24:50.929956 (+1021754us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b54001a4854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:50.930069 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179922176}
I20260709 12:24:50.931494 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:50.931660 13652 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: [30.20-31.21]
I20260709 12:24:50.932423 21986 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.936954 21986 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:50.947582 21986 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:50.913638 22044 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:50.948873 22151 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.144s	user 0.006s	sys 0.000s
I20260709 12:24:50.951951 21986 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:50.952997 21986 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:50.949669 22164 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:50.953742 22164 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:50.956235 22164 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:50.950600 22188 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:50.958851 22188 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:50.961792 22188 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:50.971949 22220 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.161s	user 0.003s	sys 0.000s
I20260709 12:24:50.983173 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4397) took 1082 ms. Trace:
I20260709 12:24:50.991927 13650 rpcz_store.cc:276] 0709 12:24:49.900564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.996562 (+ 95998us) service_pool.cc:224] Handling call
0709 12:24:49.996877 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.139915 (+143038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.139954 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:50.140010 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:50.140042 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.982763 (+842721us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54001e6754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:50.982830 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.983088 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328456448}
W20260709 12:24:50.940781 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.996703 22188 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:50.998296 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:50.988869 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:51.002388 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.019073 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:51.020984 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.021905 13793 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:51.025017 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.027760 13652 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 15
I20260709 12:24:51.028183 13652 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:24:51.028488 13652 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:51.029552 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.030681 13647 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:51.036689 13793 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:51.020175 22188 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:51.020303 22164 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:51.010625 21779 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:51.068042 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.079816 13793 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:50.775592 22162 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:51.088301 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.025s	sys 0.016s
W20260709 12:24:51.088886 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.025s	sys 0.016s
I20260709 12:24:51.021481 22043 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.074860 13647 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:24:51.090585 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:51.045297 22209 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.298s	user 0.000s	sys 0.003s
W20260709 12:24:51.077239 22148 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.080s	user 0.000s	sys 0.009s
I20260709 12:24:51.120215 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.031654 21957 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.097007 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.057s	sys 0.050s
I20260709 12:24:50.913713 22199 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:51.066471 21953 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.840s	user 0.011s	sys 0.008s
I20260709 12:24:51.119831 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:51.127174 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.058s	sys 0.050s
I20260709 12:24:51.066879 22023 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.129318 13801 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:51.136435 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.136868 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.137478 13643 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.130302 13793 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:24:51.131181 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 686922us
I20260709 12:24:51.140604 13798 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:51.133132 13792 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:51.142647 13723 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:51.146706 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.147809 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:51.148582 13727 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:24:51.152571 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:51.102820 22164 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:51.155035 13794 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:51.164815 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.165189 13649 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:24:51.165562 13800 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
W20260709 12:24:51.142293 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 210844us
I20260709 12:24:51.168835 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.169492 13644 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [14.11-17.12]
I20260709 12:24:51.187722 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.189450 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.124405 22216 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:51.189647 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1767) took 1006 ms. Trace:
I20260709 12:24:51.194216 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.194621 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:51.102540 21779 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:51.201155 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.202627 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.211176 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.124096 22043 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.216181 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:51.216591 13799 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:51.219225 13723 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:51.193622 13721 rpcz_store.cc:276] 0709 12:24:50.183167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.183342 (+   175us) service_pool.cc:224] Handling call
0709 12:24:50.183800 (+   458us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:50.252134 (+ 68334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.252188 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:50.254434 (+  2246us) log.cc:844] Serialized 52 byte log entry
0709 12:24:50.545174 (+290740us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:50.749430 (+204256us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.188035 (+438605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.188730 (+   695us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.188984 (+   254us) raft_consensus.cc:1685] finished
0709 12:24:51.189004 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.189570 (+   566us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158127,"prepare.queue_time_us":146147,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":22802,"raft.queue_time_us":216883,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176,"thread_start_us":691827,"threads_started":4,"wal-append.queue_time_us":355218}
I20260709 12:24:51.221716 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.227870 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:51.234957 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82480.5us
I20260709 12:24:51.237990 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:51.260394 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.260394 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:51.261034 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.260509 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.263432 13800 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:51.263700 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:51.264575 13693 leader_election.cc:400] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:24:51.264766 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.269208 13693 leader_election.cc:403] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:51.133989 21957 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.196609 22216 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:51.269394 13649 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:51.267714 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:51.144398 22023 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.285419 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1345) took 1966 ms. Trace:
I20260709 12:24:51.227737 13727 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:51.266769 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:51.288944 13720 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:51.198666 21861 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:51.279294 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:51.289942 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:51.290578 13727 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:51.269919 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.275359 13719 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:51.276984 13795 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:24:51.294389 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.295601 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:51.276080 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:51.284351 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:51.288028 13722 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:51.290894 21861 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:51.290928 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:24:51.310290 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.311182 13651 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:24:51.312008 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.203536 22197 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:51.293294 13647 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:51.286192 14417 rpcz_store.cc:276] 0709 12:24:49.318767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.318925 (+   158us) service_pool.cc:224] Handling call
0709 12:24:49.319279 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.350334 (+ 31055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.350371 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.351046 (+   675us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.636158 (+285112us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:49.971144 (+334986us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.156783 (+185639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.156868 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.076362 (+919494us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:24:51.276482 (+200120us) raft_consensus.cc:1685] finished
0709 12:24:51.285189 (+  8707us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.285352 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14040,"prepare.queue_time_us":24114,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":164316,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":110,"spinlock_wait_cycles":11222272,"thread_start_us":723832,"threads_started":3,"wal-append.queue_time_us":552809}
I20260709 12:24:51.298568 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.315623 13651 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:51.200333 21819 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:51.312942 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.235383 22108 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:51.325162 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:51.333745 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.272300 21833 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:51.334937 13649 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:51.336180 13719 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:24:51.337764 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:51.341347 13800 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:51.273407 22201 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.292742 21743 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.345364 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.343899 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:51.343182 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 198618us
I20260709 12:24:51.358345 13727 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:24:51.366104 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77715.9us
I20260709 12:24:51.378635 13649 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:51.285584 21953 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.325955 22215 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:24:51.368749 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.267910 22187 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:51.370968 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:51.373114 13722 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:24:51.374900 13725 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:51.278244 22216 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:51.400770 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.401811 13651 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:51.402987 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.407074 21833 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:51.408627 14391 leader_election.cc:400] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
I20260709 12:24:51.429450 14391 leader_election.cc:403] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:51.411908 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:51.438640 13801 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:51.440735 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.461570 13645 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:51.342015 22197 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:24:51.309229 22193 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.087s
W20260709 12:24:51.441005 22148 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:24:51.407897 21861 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:51.383827 22211 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.514618 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.432787 22187 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:51.440428 21833 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:51.371471 22108 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:51.523887 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.543754 13651 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:51.494956 21900 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:51.563738 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:51.383230 22201 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:51.396898 21743 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:51.517053 22224 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 12:24:51.585271 22236 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
I20260709 12:24:51.648005 13726 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:24:51.586818 22190 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.587182 22225 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.381s
I20260709 12:24:51.542780 21833 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.590814 22172 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:24:51.536769 22211 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:51.537633 22165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.017s	sys 0.011s
I20260709 12:24:51.596661 21861 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:24:51.597259 22162 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:51.598361 22043 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:51.599489 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.602334 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.455561 22226 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.603037 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.396075 21953 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.586460 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.615557 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.614506 13643 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:51.754652 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:51.753413 14418 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:24:51.755884 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.756687 13643 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:51.610069 13651 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:24:51.316367 22164 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:51.621800 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:51.542665 22146 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22.
I20260709 12:24:51.554396 22188 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:51.622217 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:51.631835 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1374) took 1404 ms. Trace:
I20260709 12:24:51.781924 13647 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:51.634492 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 227790us
I20260709 12:24:51.781586 13795 rpcz_store.cc:276] 0709 12:24:50.227816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.228061 (+   245us) service_pool.cc:224] Handling call
0709 12:24:50.228457 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.307725 (+1079268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.631474 (+323749us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002cb0d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:51.631754 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410691968}
I20260709 12:24:51.784053 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.599493 22023 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:51.627758 21882 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.003s
I20260709 12:24:51.637948 22229 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:24:51.650897 21743 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:51.641374 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:51.647337 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:51.799579 13719 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:51.653748 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.671301 13649 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:24:51.800814 13801 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
W20260709 12:24:51.660255 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 392581us
I20260709 12:24:51.801753 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:51.693967 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:51.697161 21957 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:51.697930 22225 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:51.633016 21900 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.704010 13645 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:51.669113 22199 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:51.640734 22160 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.648524 22201 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:51.719261 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:51.835364 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.730824 13648 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:51.758080 14417 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:51.684267 21779 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:51.596714 22187 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:51.672931 22158 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:51.602686 21819 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:51.753364 13650 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
W20260709 12:24:51.715518 22165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.421s	user 0.018s	sys 0.011s
I20260709 12:24:51.815292 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:51.828433 21900 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:51.827564 22225 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:51.684692 22190 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.814320 22193 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:24:51.754168 22029 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.104s
W20260709 12:24:51.877140 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 158067us
I20260709 12:24:51.864399 13719 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
W20260709 12:24:51.867051 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 472036us
I20260709 12:24:51.838351 22213 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.883882 13648 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
W20260709 12:24:51.874505 21986 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.000s
I20260709 12:24:51.876227 13795 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:24:51.866925 22057 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:51.789424 22226 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:51.838975 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.016s
W20260709 12:24:51.962182 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.009s	sys 0.016s
W20260709 12:24:51.962728 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 650198us
I20260709 12:24:51.966166 22228 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:51.965790 22225 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.967063 22225 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:51.966854 22228 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:51.969115 22228 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:51.970016 22223 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 12:24:51.972736 13795 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:24:51.839479 22160 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:51.982396 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 199231us
W20260709 12:24:51.874094 21824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.013s	sys 0.010s
W20260709 12:24:51.995234 13646 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.015s	sys 0.013s
W20260709 12:24:51.996505 13646 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.016s	sys 0.013s
I20260709 12:24:51.904197 21986 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:51.865921 22158 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:51.998021 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119951us
I20260709 12:24:51.943737 22057 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:52.012517 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 258546us
W20260709 12:24:52.013513 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 133395us
W20260709 12:24:51.996387 21824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.013s	sys 0.011s
I20260709 12:24:51.887356 22213 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.022397 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.023384 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:52.024451 22218 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:24:52.024981 22245 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:52.025853 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.026609 22247 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.031296 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66450.6us
I20260709 12:24:51.988062 22230 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 12:24:52.015952 22194 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.097s	user 0.014s	sys 0.003s
I20260709 12:24:52.032399 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.065678 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.067104 13719 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 14.
I20260709 12:24:52.064029 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.074497 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.076068 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.079756 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.076897 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.063055 22230 consensus_queue.cc:1243] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:52.086215 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.088076 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.012384 22165 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:24:52.090895 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.095963 13727 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:52.097950 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:52.027119 22201 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.048s
I20260709 12:24:52.021252 22206 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
W20260709 12:24:52.080752 22198 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.179s	user 0.017s	sys 0.000s
I20260709 12:24:52.101244 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.135553 13727 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:52.102063 22240 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.383s	user 0.000s	sys 0.002s
W20260709 12:24:52.103008 22244 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 12:24:52.103844 22253 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:52.075574 22246 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:52.107753 22221 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.276s	user 0.000s	sys 0.003s
I20260709 12:24:52.059303 22188 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:24:52.119155 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.121009 22207 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.077997 21861 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:52.123597 22211 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.089756 21953 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.125674 22213 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.075680 21986 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:52.126999 22226 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.135689 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.059824 22247 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:52.134729 22216 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.006s
W20260709 12:24:52.158414 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 133267us
I20260709 12:24:52.165337 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:52.068975 22194 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:52.170146 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93295us
I20260709 12:24:52.148226 13727 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:52.183635 13727 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:52.183791 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:52.154351 22246 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.143621 22257 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:24:52.165393 22248 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:24:52.181417 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.164499 21779 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:24:52.216634 22227 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:24:52.217545 22227 consensus_queue.cc:1243] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:24:52.219018 22251 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:24:52.220463 13727 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.221947 13727 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:52.222656 13720 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:52.210376 22213 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:24:52.223878 22266 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:52.224344 22266 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:52.225291 22266 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.145238 22057 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.231061 22057 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:52.194595 22230 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:52.208331 22248 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:52.236145 22248 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:52.237529 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1376) took 1100 ms. Trace:
I20260709 12:24:52.202734 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:52.203696 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.204615 13795 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:52.177862 22245 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:52.238837 13800 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:52.237944 13801 rpcz_store.cc:276] 0709 12:24:51.136637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.826373 (+689736us) service_pool.cc:224] Handling call
0709 12:24:51.838538 (+ 12165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.002724 (+164186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:52.002767 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:52.003367 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:24:52.159567 (+156200us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:52.159628 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.159730 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.237254 (+ 77524us) raft_consensus.cc:1685] finished
0709 12:24:52.237284 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.237459 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158547,"raft.queue_time_us":158049,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":7153,"thread_start_us":124921,"threads_started":1,"wal-append.queue_time_us":215033}
I20260709 12:24:52.240232 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.202461 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:52.240778 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:52.177469 21861 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:52.249483 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:52.171057 22228 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:24:52.146203 22198 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.192317 22246 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.207914 22229 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:52.206060 21986 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:52.261883 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.205564 21953 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:52.266770 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.270428 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:52.271854 14417 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:52.274240 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.195658 22190 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.238143 22247 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.279883 13727 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:24:52.292569 13727 pending_rounds.cc:85] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:24:52.286437 22266 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:52.288021 13794 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:24:52.288228 22243 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:52.171226 22207 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.254750 22230 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:24:52.291061 13724 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:52.241914 22213 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:52.292853 13727 pending_rounds.cc:107] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:24:52.195981 22216 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.301880 13727 raft_consensus.cc:2889] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.303342 13727 pending_rounds.cc:107] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:24:52.303619 14417 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:52.304008 13727 raft_consensus.cc:2889] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.304806 22272 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:52.306951 22272 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.308637 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1377) took 1171 ms. Trace:
I20260709 12:24:52.309379 22272 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.311714 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.316689 22216 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.317935 22207 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:52.319857 22231 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.926s	user 0.009s	sys 0.003s
I20260709 12:24:52.309862 22218 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:52.321465 21986 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.280272 22226 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:24:52.309084 13799 rpcz_store.cc:276] 0709 12:24:51.136959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.838770 (+701811us) service_pool.cc:224] Handling call
0709 12:24:51.875519 (+ 36749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.875800 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:24:52.070000 (+194200us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.308000 (+238000us) spinlock_profiling.cc:243] Waited 9.65 ms on lock 0x7b180040dce0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:24:52.308251 (+   251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.308307 (+    56us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:52.308360 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.308436 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.308572 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2220,"prepare.queue_time_us":289625,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":1894,"spinlock_wait_cycles":22203264,"thread_start_us":317004,"threads_started":2,"wal-append.queue_time_us":214071}
I20260709 12:24:52.280681 21833 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.295306 22158 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.312703 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.323194 13722 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:24:52.314659 13723 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:24:52.313200 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.980s (SENT)
I20260709 12:24:52.325453 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.330576 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:52.331218 13727 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:52.280851 21953 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.334764 13719 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:24:52.336551 13719 pending_rounds.cc:85] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 13
I20260709 12:24:52.336992 13719 pending_rounds.cc:107] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:24:52.337294 13719 raft_consensus.cc:2889] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.345453 13718 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:52.349705 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.338927 22158 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:52.354858 22158 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.357415 22158 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.335968 22197 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:52.359277 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:52.358142 22197 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.364136 22197 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:52.368067 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:52.370033 13723 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:52.371760 21779 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:52.372359 21779 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.372793 21779 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.371803 22213 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.366087 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3941) took 1076 ms. Trace:
I20260709 12:24:52.377954 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.379793 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.371001 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:52.379541 13720 rpcz_store.cc:276] 0709 12:24:51.289770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.300607 (+ 10837us) service_pool.cc:224] Handling call
0709 12:24:51.342219 (+ 41612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.223125 (+880906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.365803 (+142678us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:52.366013 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2024327552}
I20260709 12:24:52.381525 22158 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:52.339010 22230 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:24:52.386377 22272 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:52.304986 21861 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.388162 13725 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:52.387020 22272 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.383445 22248 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.397841 22198 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.398622 22185 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:24:52.399858 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:52.400077 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1370) took 2209 ms (client timeout 1980 ms). Trace:
I20260709 12:24:52.362721 22273 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:52.400516 22158 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:52.385061 22164 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:52.403025 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:52.419234 13792 rpcz_store.cc:269] 0709 12:24:50.190413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.192475 (+  2062us) service_pool.cc:224] Handling call
0709 12:24:52.399809 (+2207334us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:52.399961 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927144320}
I20260709 12:24:52.330120 22245 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:52.423471 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 176943us
I20260709 12:24:52.430609 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:52.431608 22248 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.433981 22272 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.440937 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:52.439087 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:52.442764 22246 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:52.418583 22210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.543s	user 0.006s	sys 0.018s
W20260709 12:24:52.444648 22210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.569s	user 0.006s	sys 0.018s
I20260709 12:24:52.426684 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1368) took 1416 ms. Trace:
I20260709 12:24:52.427910 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.447271 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:52.447897 13727 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:52.445750 22255 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.449908 22255 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.452725 22255 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.452940 22252 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:52.453378 22252 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:52.465706 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:52.441071 22165 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:24:52.466742 13792 raft_consensus.cc:2368] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:52.418809 22185 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:24:52.468325 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.468909 22210 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:52.469805 22182 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:52.460284 21900 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.000s
I20260709 12:24:52.470717 22182 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.460534 22247 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:52.472222 22247 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:52.426914 22207 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:24:52.470790 21900 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.473891 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.447100 21833 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.473533 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3936) took 1256 ms. Trace:
I20260709 12:24:52.466871 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:52.479753 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.488092 13718 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:52.085415 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:52.481307 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.460120 13652 rpcz_store.cc:276] 0709 12:24:51.009822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.009989 (+   167us) service_pool.cc:224] Handling call
0709 12:24:51.027519 (+ 17530us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:51.074483 (+ 46964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:51.074525 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:51.075160 (+   635us) log.cc:844] Serialized 52 byte log entry
0709 12:24:51.112925 (+ 37765us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:51.113417 (+   492us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.145346 (+ 31929us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.289260 (+1143914us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b1800327520. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9
0709 12:24:52.290521 (+  1261us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800327520. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:24:52.426282 (+135761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.426370 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:52.426422 (+    52us) raft_consensus.cc:1685] finished
0709 12:24:52.426440 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.426617 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76835,"prepare.queue_time_us":11555,"prepare.run_wall_time_us":44,"raft.queue_time_us":80151,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":13768,"spinlock_wait_cycles":52929152,"thread_start_us":172931,"threads_started":2,"wal-append.queue_time_us":148714}
I20260709 12:24:52.487540 13723 rpcz_store.cc:276] 0709 12:24:51.217078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.237640 (+ 20562us) service_pool.cc:224] Handling call
0709 12:24:52.473354 (+1235714us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400155ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:52.473471 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415476608}
I20260709 12:24:52.493451 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:52.466784 22165 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:52.496765 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.496762 22165 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:52.459995 22206 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.473240 22247 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.473318 22182 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.504205 21900 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.506412 22254 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:52.507309 22254 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:24:52.486876 22243 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:52.511336 22243 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.513610 22243 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.459659 21953 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.474335 13547 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 19, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:52.468360 22185 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.515471 22272 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:24:52.454224 22252 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.516803 22252 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:52.519177 22255 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:52.519666 22255 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:52.520074 22255 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:52.520548 22206 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.500720 22198 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:52.523990 22198 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:52.524386 22198 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:52.538214 13727 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:52.538129 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.538180 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.539740 13723 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:52.540661 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.541119 13723 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:52.543262 13800 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:52.542378 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:52.476174 22246 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.541934 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.546208 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:52.479583 22210 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:52.545038 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:52.552557 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:52.557636 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.560305 14418 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:24:52.562467 22215 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:52.562439 21779 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.563213 22215 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.563722 22215 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:52.571280 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.540597 22261 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:24:52.573648 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.574337 13718 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
I20260709 12:24:52.557864 21953 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:52.559285 22272 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 32.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:24:52.566834 22146 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:52.586479 22273 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:52.548679 22207 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:52.589897 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:52.573518 22134 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:52.598140 22210 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.603547 22230 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:52.600998 22246 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:24:52.601312 22146 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:52.588986 21953 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.643174 22199 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:24:52.649224 22201 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:52.649722 22201 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:52.588253 22261 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.682524 21779 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.684089 22146 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.685222 13792 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:24:52.686530 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:52.687868 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.617455 22230 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.620190 22246 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:52.691615 22246 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.692548 22230 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.650640 22201 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.651757 22188 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.695778 22188 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.703193 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:52.706784 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.515s (SENT) [suppressed 33 similar messages]
I20260709 12:24:52.665087 22199 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.710928 22199 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:52.716240 22217 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.883s	user 0.007s	sys 0.007s
I20260709 12:24:52.720708 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.721629 13792 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:52.722589 13799 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:52.725510 22126 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:52.725919 22126 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:52.726766 22126 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.728010 22126 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:52.728543 22197 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.659221 22108 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.665937 22206 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.618695 21953 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:52.722057 22261 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:52.646159 22185 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:24:52.698242 22188 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:52.768846 22230 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.772970 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3934) took 1556 ms. Trace:
I20260709 12:24:52.773655 13724 rpcz_store.cc:276] 0709 12:24:51.216456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.216949 (+   493us) service_pool.cc:224] Handling call
0709 12:24:51.217253 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.302844 (+1085591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.772590 (+469746us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400175054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:52.772861 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468993408}
I20260709 12:24:52.769713 22230 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.736375 22199 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:52.770044 22243 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:52.614926 22207 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.795965 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:52.771162 22255 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:52.838449 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.45485e+06us
I20260709 12:24:52.778777 22188 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
W20260709 12:24:52.847843 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.57178e+06us
I20260709 12:24:52.850078 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:52.790558 22243 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:52.873778 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1367) took 1865 ms. Trace:
W20260709 12:24:52.904072 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78528.2us
I20260709 12:24:52.874305 13646 rpcz_store.cc:276] 0709 12:24:51.008477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.008640 (+   163us) service_pool.cc:224] Handling call
0709 12:24:51.008994 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.009288 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:24:51.997503 (+988215us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.873386 (+875883us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.873445 (+    59us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:24:52.873503 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.873576 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:52.873716 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":742998,"prepare.queue_time_us":26630,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":8767,"thread_start_us":1102647,"threads_started":2,"wal-append.queue_time_us":185771}
I20260709 12:24:52.922573 22146 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:52.922695 21957 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.923285 22146 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.923383 21957 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:52.819484 22022 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:24:52.926628 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:52.926692 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:52.927572 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:52.950884 13651 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:52.929010 13646 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:52.946525 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:52.925912 22146 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:52.950119 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.953060 13798 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
W20260709 12:24:53.008924 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:53.010649 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.008190 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.010118 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.011950 13647 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:24:53.015549 13652 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:52.925868 21957 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:53.042523 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 190703us
I20260709 12:24:53.039943 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.040620 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.052057 13647 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 22 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:24:53.041393 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:52.875392 22197 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:53.053517 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 203572us
I20260709 12:24:53.057988 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.059451 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.059721 14417 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:24:53.054831 13798 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:24:53.054083 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:52.861940 22241 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.061342 13649 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:53.062357 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.063059 13649 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:24:53.064560 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 48919.7us
I20260709 12:24:53.021548 13650 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:24:53.065518 13650 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:53.074157 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:53.022964 22283 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 12:24:53.082672 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.087023 13647 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:53.087980 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.044127 22022 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:53.092161 13617 leader_election.cc:400] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:53.041098 22207 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:53.083900 13652 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:53.075186 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.085587 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.108886 14422 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:24:53.094198 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.111734 13648 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:24:53.047777 22146 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:53.021587 22273 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:53.113958 13647 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:53.108388 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:53.152850 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44575us
I20260709 12:24:53.154999 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.090396 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:53.139117 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.083566 22190 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.110771 22281 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
W20260709 12:24:53.114449 22198 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.589s	user 0.014s	sys 0.001s
I20260709 12:24:53.094597 13646 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:53.148430 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:53.107259 21824 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.160378 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.092442 22022 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:53.160790 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.157284 14425 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:53.177068 14419 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:53.164866 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.150774 13650 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.141301 22106 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:53.175849 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.178042 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.186305 13650 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.149133 22146 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.185173 14424 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:53.191413 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.106760 22296 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
W20260709 12:24:53.173502 22290 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:24:53.116173 22133 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:24:53.194360 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.192296 14423 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:53.204903 14425 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:53.196395 13798 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:53.172817 21861 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:53.138332 22280 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.073s
I20260709 12:24:53.211809 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.141361 22239 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:24:53.179704 22108 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.163759 22190 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.178685 22044 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:53.217717 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.166923 21957 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:24:53.218888 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.228193 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.223354 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:53.182936 22043 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:53.183195 22106 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:53.263096 22106 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.263397 22255 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.170682 22273 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.263128 22243 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.189395 22199 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:53.267396 22273 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.267576 22199 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:53.178766 22198 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.268364 22199 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:53.200649 22146 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:53.231763 22207 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:53.270746 22198 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.271184 22207 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.271703 22207 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.234871 22188 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:24:53.272938 22188 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.273371 22188 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:24:53.236826 22286 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
W20260709 12:24:53.237915 22297 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:24:53.241145 22252 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.002s
I20260709 12:24:53.221508 22261 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:24:53.242678 22239 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:53.225188 22211 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:24:53.246376 22108 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.217445 22245 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:24:53.221383 22023 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:24:53.224414 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.316589 13652 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.283257 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.287463 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:53.318766 13793 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:53.213605 21861 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:53.291695 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 151371us
I20260709 12:24:53.231038 22044 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:53.293188 21953 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.000s
I20260709 12:24:53.296784 22239 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:53.325685 21861 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:24:53.343194 22215 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.769s	user 0.011s	sys 0.003s
W20260709 12:24:53.331153 22126 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.603s	user 0.015s	sys 0.002s
I20260709 12:24:53.348321 22190 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.324362 22261 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.354389 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:53.356276 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:53.371703 14419 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.372444 22300 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.373066 22300 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.357683 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4163) took 2343 ms. Trace:
W20260709 12:24:53.375622 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.374481 13796 rpcz_store.cc:276] 0709 12:24:51.013903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.014048 (+   145us) service_pool.cc:224] Handling call
0709 12:24:51.014344 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.263389 (+249045us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:51.292515 (+ 29126us) log.cc:844] Serialized 14 byte log entry
0709 12:24:52.126743 (+834228us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b180007e4c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:24:52.184632 (+ 57889us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:52.184689 (+    57us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:52.184747 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.357322 (+1172575us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54000e4cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:53.357397 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.357578 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21061,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":14160,"raft.queue_time_us":5194,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":16540,"spinlock_wait_cycles":587363328,"thread_start_us":57614,"threads_started":1,"wal-append.queue_time_us":968333,"wal-append.run_cpu_time_us":1258,"wal-append.run_wall_time_us":1047998}
I20260709 12:24:53.359974 13646 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:24:53.353531 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:53.366925 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:53.326009 22044 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.393074 13652 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:53.397562 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.398903 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:53.328084 21953 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.403702 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.404340 14419 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:53.408033 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.393112 21883 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:24:53.404654 22133 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:24:53.420706 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.381400 22280 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:24:53.383527 22261 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.377862 22215 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.433452 14419 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:53.405915 22304 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:53.441633 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 213555us
I20260709 12:24:53.442691 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.443787 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.446864 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.396098 22239 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.458655 22215 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.456874 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.469959 14419 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:53.445143 22280 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:53.448711 22211 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:53.449692 22108 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.452586 22245 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:53.455220 22273 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:53.486779 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:53.465200 22165 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.958s	user 0.012s	sys 0.008s
I20260709 12:24:53.498567 21743 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:24:53.486866 22302 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:24:53.500650 22160 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:53.503973 22158 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:53.512566 22261 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:53.518613 22300 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.520529 13719 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:53.521896 13693 leader_election.cc:400] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:53.553694 13693 leader_election.cc:403] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.555173 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:53.523329 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.556604 13721 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:24:53.524282 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.526294 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.523974 22211 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:53.560425 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.558378 13725 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:53.539312 22273 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.534852 13724 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:53.537985 13722 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:24:53.541071 22190 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:53.473301 22305 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:24:53.566470 22305 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:53.567196 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:53.541553 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54886.5us
I20260709 12:24:53.567837 13724 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:53.542351 22307 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:53.543817 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:53.551934 22239 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.550834 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:53.527349 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.527287 13650 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:24:53.574465 13797 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:53.532037 22245 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.477658 22188 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.562397 13646 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:24:53.552114 22158 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:53.563289 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:53.565582 21833 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:53.565526 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.538825 22226 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:24:53.560448 22243 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:53.562044 22211 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:24:53.500080 22185 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.000s
I20260709 12:24:53.542790 21824 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:53.571974 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 168339us
I20260709 12:24:53.568388 22273 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:24:53.575364 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.571821 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:53.589591 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.578685 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.544435 21743 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:53.572446 22307 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.597786 13646 raft_consensus.cc:2368] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:53.579535 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.598829 13652 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.581944 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:53.568691 13726 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:53.586494 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.601193 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:53.548488 22199 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.574657 13650 pending_rounds.cc:85] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 11
I20260709 12:24:53.602161 13650 pending_rounds.cc:107] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:24:53.602468 13650 raft_consensus.cc:2889] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:53.547468 22165 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.567385 22305 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.604149 22305 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:53.604853 22165 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.596037 13724 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:24:53.596848 14422 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:24:53.605937 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.606765 13646 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:53.578958 22245 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:53.578605 21833 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.612309 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.609689 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:53.620929 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.621613 13650 raft_consensus.cc:2376] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19. Re-sending same reply.
I20260709 12:24:53.621852 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.622582 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:53.613602 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.616410 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.612529 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.623451 13724 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:53.624701 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.625337 13646 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:53.625564 13724 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:53.577795 22261 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:24:53.568217 22190 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.630762 13650 raft_consensus.cc:2376] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19. Re-sending same reply.
I20260709 12:24:53.606475 22300 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:53.613749 22304 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.624889 21833 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.611001 22198 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:53.616364 22226 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:24:53.649621 22311 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:24:53.650604 22311 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:53.648978 13693 leader_election.cc:400] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:53.652251 13693 leader_election.cc:403] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.623559 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.679816 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.647625 22243 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:24:53.682395 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.683501 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.685964 22190 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.680418 13651 raft_consensus.cc:2376] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29. Re-sending same reply.
I20260709 12:24:53.676817 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:53.695621 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:24:53.610757 22108 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:53.607668 22307 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.617033 21819 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:53.633206 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:53.644569 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:53.626962 21953 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:53.703766 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49746.6us
W20260709 12:24:53.705417 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.018s	sys 0.038s
W20260709 12:24:53.705541 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72425.1us
I20260709 12:24:53.655215 22188 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.656224 22194 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.660944 22201 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:53.710764 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:53.662575 22262 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:53.644634 22300 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:53.712396 22262 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.712842 22262 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:53.713125 22300 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.714646 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:53.715268 13798 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:53.641278 22207 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.717821 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.718379 22207 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.719295 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:53.664456 13726 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:24:53.642503 22158 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.728214 22158 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.628851 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.650992 22311 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:53.639858 22280 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:24:53.730005 22311 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.610927 22146 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.731177 22311 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:53.667210 13721 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:53.669607 13722 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:53.672523 22252 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:53.733071 22146 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.675071 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:53.649024 22198 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:53.697377 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.733886 14419 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:53.735132 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:53.677328 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.645565 22160 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:53.737756 22252 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:53.737246 22198 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.739216 22198 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:53.746366 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:53.747413 13797 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:24:53.739198 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61959.7us
W20260709 12:24:53.751992 22188 consensus_queue.cc:1175] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 27 status { last_received { term: 27 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 27 index: 16 } }
W20260709 12:24:53.686210 22289 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.541s	user 0.002s	sys 0.000s
W20260709 12:24:53.756587 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49588.6us
I20260709 12:24:53.692931 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.698498 22106 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.637368 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.759410 13651 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:53.759943 13650 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:24:53.705803 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.019s	sys 0.038s
I20260709 12:24:53.760468 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:53.761333 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 653388us
I20260709 12:24:53.736186 14417 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:53.668944 22044 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:53.706362 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:53.763509 13693 leader_election.cc:400] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18.
I20260709 12:24:53.695061 22243 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:53.768217 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.773020 13650 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:53.874933 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.797255 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.842108 13724 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:53.775458 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:53.880050 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:53.831337 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 259574us
I20260709 12:24:53.880667 14417 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:53.873513 13798 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 18.
I20260709 12:24:53.756088 22229 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:24:53.704391 21819 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:53.828470 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122190us
W20260709 12:24:53.836879 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 153438us
I20260709 12:24:53.802732 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.825503 14419 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:53.893630 13651 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:53.825503 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.895622 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.779295 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.884289 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.842772 13646 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:53.781847 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.879626 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.774479 13693 leader_election.cc:400] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:53.817813 13797 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:24:53.857800 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.887158 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.900775 13652 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:53.883818 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:53.759811 21883 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:53.891963 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.902407 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.903062 13797 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
W20260709 12:24:53.903352 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129018us
I20260709 12:24:53.904500 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.883006 22190 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:24:53.906153 13724 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:53.894682 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.907008 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.908620 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.837914 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:53.910732 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:53.798833 22252 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.907912 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:53.912696 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.912391 14419 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:24:53.916081 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:53.840227 22317 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:53.825659 22243 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:53.922389 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.951138 13650 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:53.923863 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:53.925132 13800 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:53.965394 13647 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:53.928221 22317 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.968288 22317 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:53.921048 21861 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.551s	user 0.006s	sys 0.005s
I20260709 12:24:53.912217 22324 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:53.971009 22324 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:53.970628 21861 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.971632 22324 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:53.972641 22324 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.973436 21861 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.976503 13645 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:53.928996 22261 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:24:53.921095 21953 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:24:53.980015 22230 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:53.932396 22252 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.978034 13645 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:24:53.983671 13651 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:24:53.983707 13645 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
W20260709 12:24:53.986734 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.988109 13724 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:53.985991 13645 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:24:53.912379 22249 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:24:53.914614 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:53.994716 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80198.7us
I20260709 12:24:53.754679 22160 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.005955 22160 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:54.006644 22160 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:24:54.007112 22160 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:54.007998 22160 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.009835 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:54.011330 22314 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:24:53.771111 22108 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.147s
I20260709 12:24:53.991129 13651 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:53.920670 22194 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:54.018653 22194 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:54.019061 22218 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:53.796473 22313 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.154s
I20260709 12:24:54.021178 22044 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:54.022390 22106 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:53.971640 22210 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:54.023248 22210 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:54.024188 22210 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.025311 22210 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:53.998594 13800 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:53.923159 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.000607 21953 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:54.002893 13799 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:54.028939 22245 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.030157 13793 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:54.031071 13796 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:54.032698 21743 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.906368 21819 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:54.035491 14420 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
I20260709 12:24:54.036818 14420 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:54.037966 14420 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:54.038813 22252 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.039888 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1782) took 1680 ms. Trace:
I20260709 12:24:54.040813 22245 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:54.019660 22218 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:24:54.043360 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.040267 13718 rpcz_store.cc:276] 0709 12:24:52.359266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.612840 (+253574us) service_pool.cc:224] Handling call
0709 12:24:52.613205 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.691424 (+ 78219us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:24:52.838539 (+147115us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.106787 (+268248us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.106876 (+    89us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:24:53.106928 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.039655 (+932727us) spinlock_profiling.cc:243] Waited 78 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:54.039706 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.039830 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56308,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":2572,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6706,"spinlock_wait_cycles":179308928,"thread_start_us":411727,"threads_started":2,"wal-append.queue_time_us":349391}
I20260709 12:24:54.019657 22194 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.044840 22133 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.045542 22133 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:53.995431 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:54.053730 14417 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:24:53.935114 22131 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:24:53.936707 22300 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:24:54.056028 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133004us
W20260709 12:24:54.056556 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61207.2us
I20260709 12:24:54.069864 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.071745 13693 leader_election.cc:400] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:24:54.072935 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:54.058653 13796 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:53.920922 22318 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:54.060930 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.082924 13650 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:54.062774 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:54.063794 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:54.085575 14420 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:53.926434 22320 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:24:53.987234 22199 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.089674 22320 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:54.090197 22320 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:54.102814 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.104290 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.105212 13645 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:54.106375 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:53.996467 22321 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:24:54.109409 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.109866 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.110236 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.094563 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.110831 14417 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:24:54.111276 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.111953 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.059515 22230 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:54.065017 22207 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:54.118834 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:54.119732 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.091555 22199 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:24:54.135044 13799 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:54.082687 22318 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.128095 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.048362 22190 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
W20260709 12:24:54.129602 14418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1258) took 1881 ms (client timeout 2228 ms). Trace:
I20260709 12:24:54.094673 22252 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:24:54.139490 14420 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:54.140407 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.109400 22313 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.144181 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.192636 13645 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:54.193321 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:54.193837 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.143317 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:24:54.148018 14418 rpcz_store.cc:269] 0709 12:24:52.247762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.249153 (+  1391us) service_pool.cc:224] Handling call
0709 12:24:54.129203 (+1880050us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54007125d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:54.129488 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635343104}
I20260709 12:24:54.151367 22199 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:24:54.151612 22252 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:54.152503 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:54.177870 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.196084 13650 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:54.174330 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.196841 13649 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:54.174646 13547 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 20, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 20 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:54.176682 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4400) took 2978 ms. Trace:
W20260709 12:24:54.179544 22330 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:24:54.179934 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.147599 22318 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:54.189333 22218 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.190904 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1576) took 1070 ms. Trace:
I20260709 12:24:54.202023 14422 rpcz_store.cc:276] 0709 12:24:53.120862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.140851 (+ 19989us) service_pool.cc:224] Handling call
0709 12:24:53.141198 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.606131 (+464933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.190638 (+584507us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:54.190837 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028020096}
W20260709 12:24:54.133363 22273 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.004s
I20260709 12:24:54.194732 13795 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:54.203990 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:54.205530 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:54.197526 13643 rpcz_store.cc:276] 0709 12:24:51.198491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.789442 (+590951us) service_pool.cc:224] Handling call
0709 12:24:51.857075 (+ 67633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:52.862912 (+1005837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:52.862947 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:52.929090 (+ 66143us) log.cc:844] Serialized 26 byte log entry
0709 12:24:53.181753 (+252663us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:53.181793 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.181881 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:53.240218 (+ 58337us) raft_consensus.cc:1685] finished
0709 12:24:53.240243 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.176389 (+936146us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b5400113d50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:54.176615 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1005362,"raft.queue_time_us":1008190,"spinlock_wait_cycles":162289280,"thread_start_us":120002,"threads_started":1,"wal-append.queue_time_us":194428}
I20260709 12:24:54.210011 13643 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:24:54.210461 13643 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:54.194211 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:54.132956 22207 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.132318 22300 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:54.216477 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.217797 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:54.228179 13726 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:54.219125 13718 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:54.230183 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:54.231447 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:54.220046 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:54.195926 22304 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.002s
I20260709 12:24:54.234769 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:54.238032 13722 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.238891 13718 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:24:54.201401 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58192.6us
I20260709 12:24:54.239014 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:54.239802 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.061587 22022 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:54.241935 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 326763us
I20260709 12:24:54.256924 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:54.258046 14422 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:54.258785 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:54.211411 13643 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.252367 13799 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:54.128697 22044 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
W20260709 12:24:54.275036 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.214350 22193 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.224741 22106 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:24:54.226632 22230 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:54.234912 22207 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:54.275779 13726 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:54.194198 22241 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.071s
W20260709 12:24:54.294536 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 372374us
I20260709 12:24:54.296293 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:54.252192 22304 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:54.240897 22261 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:54.265707 21824 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:24:54.283767 22330 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:24:54.311347 13650 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:24:54.254472 22199 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:54.224802 22273 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:54.300165 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 393241us
I20260709 12:24:54.214119 22313 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.332499 13647 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:54.308156 22249 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:54.279196 22239 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:24:54.345623 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65541.7us
I20260709 12:24:54.350980 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:54.190828 22243 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:54.331305 22211 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.734s	user 0.004s	sys 0.008s
I20260709 12:24:54.273728 22305 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 12:24:54.366555 13722 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:54.372359 14421 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:54.329857 21824 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.292996 22193 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.327941 21819 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.331059 22300 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:24:54.376247 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:54.405403 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54428.3us
W20260709 12:24:54.405624 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 211382us
I20260709 12:24:54.357918 22239 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.364677 22199 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:54.409212 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113018us
W20260709 12:24:54.377256 22325 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.269s	user 0.004s	sys 0.000s
W20260709 12:24:54.413174 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 167533us
I20260709 12:24:54.409323 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:54.376737 21824 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:54.411090 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.419864 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:54.421741 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:54.411445 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 274442us
I20260709 12:24:54.362779 22332 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:24:54.416719 14416 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:24:54.407354 22108 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.423439 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.363371 22194 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 12:24:54.448588 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.445365 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:54.449461 13645 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:54.404345 22131 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:54.450451 14417 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:54.451022 22131 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.374362 22211 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.453851 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:24:54.463776 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 111401us
I20260709 12:24:54.430013 22300 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:24:54.464635 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.436286 22328 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:54.411806 22239 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:54.470036 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.410707 22043 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.038s
I20260709 12:24:54.430330 22337 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:24:54.467720 22044 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:24:54.476119 22342 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:24:54.477454 22334 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:24:54.515370 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103857us
I20260709 12:24:54.477401 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:54.419044 22261 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.421049 22226 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:24:54.479319 22158 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:54.480389 21819 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.375345 22252 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.483320 22333 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:54.455404 21883 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:54.420666 22249 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.468442 22165 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:24:54.495033 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 119562us
I20260709 12:24:54.448477 22146 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:54.504230 22043 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.503237 13796 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:54.511935 13645 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:54.420926 22199 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:54.528553 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51245.2us
I20260709 12:24:54.553440 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:54.469530 22353 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:54.460932 22332 consensus_queue.cc:1243] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:24:54.498749 22230 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.530673 22198 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.791s	user 0.013s	sys 0.002s
I20260709 12:24:54.486800 22328 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:54.443396 22108 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.535264 22350 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:24:54.541611 22300 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.576116 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:54.578189 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 159636us
I20260709 12:24:54.553589 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:54.533998 22337 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:54.542743 22339 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:24:54.568068 22165 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:54.609968 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:54.617004 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63521.2us
I20260709 12:24:54.547508 22210 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:24:54.595810 22146 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:54.540776 22333 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:54.625137 22334 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:24:54.648259 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 102951us
I20260709 12:24:54.608457 22352 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.661881 22352 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.665431 22352 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.669785 22365 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.018s
I20260709 12:24:54.618817 22194 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.571590 22043 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.620311 22339 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:54.673724 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63898.9us
I20260709 12:24:54.624454 22342 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:54.598554 22230 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.690554 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:54.650887 22350 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:54.634029 22332 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:54.651190 22273 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.619926 22300 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:54.674503 22307 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.915s	user 0.014s	sys 0.000s
I20260709 12:24:54.710035 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:54.710239 22346 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:24:54.710638 22335 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:54.711527 22347 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:54.711645 22328 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.712105 22332 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.686229 22158 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:54.713490 22350 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.716477 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:54.719085 22317 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.750s	user 0.008s	sys 0.005s
I20260709 12:24:54.703505 22219 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:54.688483 22359 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.659399 22333 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:54.724851 13720 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:54.675294 22198 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:54.729691 22165 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:54.676791 21883 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:54.627990 22362 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:24:54.743332 22043 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.732128 22201 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:54.744073 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:54.811209 14420 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:54.751458 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:54.780557 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 245930us
I20260709 12:24:54.684787 22194 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.787526 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:54.821094 13720 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:54.791505 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:54.702310 22339 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:24:55.051484 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 415376us
W20260709 12:24:54.746492 22311 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.015s	user 0.018s	sys 0.003s
W20260709 12:24:55.050374 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 298985us
I20260709 12:24:54.816016 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:55.090180 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.091032 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.783172 22347 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:24:55.105919 21833 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.452s	user 0.014s	sys 0.000s
I20260709 12:24:55.106593 21833 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.108264 13719 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:55.110540 21833 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.111949 21833 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.112668 22311 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:55.115695 21833 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:55.120898 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.117095 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.113416 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 322039us
I20260709 12:24:55.130563 22369 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:55.120970 13693 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.752s (SENT)
I20260709 12:24:55.131327 22369 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.131798 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:55.133478 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:55.133725 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:55.133841 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:55.139437 22370 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:55.141693 14419 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:24:55.143129 22370 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:55.143391 22368 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.143755 22370 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.143987 22368 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.144979 13617 leader_election.cc:400] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:55.145340 13617 leader_election.cc:403] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:24:55.142768 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.146039 13723 raft_consensus.cc:2368] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:55.147071 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.147996 21833 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:55.149360 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.153400 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.170656 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:54.812322 22219 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:55.172957 22219 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.173684 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:55.174475 13645 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:55.174536 22219 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:55.175078 13645 raft_consensus.cc:614] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:55.176178 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:55.176733 13645 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:24:54.737308 22307 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.091944 22340 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:55.163237 13723 raft_consensus.cc:2368] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
W20260709 12:24:55.169121 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81088.7us
I20260709 12:24:55.171686 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.157506 13722 raft_consensus.cc:2376] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20. Re-sending same reply.
I20260709 12:24:55.183637 13720 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:55.180691 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:54.783344 22317 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.174561 13650 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:24:54.804857 22198 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.104192 22347 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:55.194891 22356 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.390s	user 0.001s	sys 0.000s
W20260709 12:24:55.195075 22363 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:55.197255 22194 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:54.784941 22146 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:55.198442 22360 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.558s	user 0.001s	sys 0.000s
I20260709 12:24:54.735185 22337 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.746178 22335 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:54.791188 22359 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.049736 22201 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.185248 21883 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.209245 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:55.229362 22207 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.899s	user 0.013s	sys 0.000s
I20260709 12:24:55.219416 13798 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:55.178411 22108 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.223709 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:55.295672 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72057.7us
I20260709 12:24:55.171731 22368 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:55.186069 22199 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:55.214655 22347 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:24:55.233806 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:55.241585 13651 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:24:55.215260 22352 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:24:55.243988 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.331698 13723 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:55.236429 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:55.212985 22355 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:24:55.291941 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.228112 22210 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
W20260709 12:24:55.233176 22357 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.436s	user 0.000s	sys 0.001s
I20260709 12:24:55.358750 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2857) took 1245 ms. Trace:
I20260709 12:24:55.193542 22371 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:55.296753 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:55.302209 22252 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.006s	sys 0.009s
W20260709 12:24:55.304630 13690 leader_election.cc:336] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.874s (SENT)
W20260709 12:24:55.384456 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 79879.7us
I20260709 12:24:55.333458 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.386090 13720 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:24:55.329228 22368 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.207456 22340 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:55.308972 22318 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.064s	user 0.006s	sys 0.007s
I20260709 12:24:55.392433 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:55.397814 14418 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:55.401119 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:55.222213 22361 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.193s
I20260709 12:24:55.335911 22296 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:55.299221 22207 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.355046 22201 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:55.355459 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.356102 22355 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.359251 13800 rpcz_store.cc:276] 0709 12:24:54.113376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.113522 (+   146us) service_pool.cc:224] Handling call
0709 12:24:54.113938 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.114404 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.114442 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.124687 (+ 10245us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.640409 (+515722us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:54.640458 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.640542 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.358510 (+717968us) raft_consensus.cc:1685] finished
0709 12:24:55.358534 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.358682 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":720,"raft.queue_time_us":243431,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":12738,"thread_start_us":449874,"threads_started":1,"wal-append.queue_time_us":543837}
I20260709 12:24:55.359562 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4182) took 1508 ms. Trace:
I20260709 12:24:55.420746 13792 rpcz_store.cc:276] 0709 12:24:53.851335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.918989 (+ 67654us) service_pool.cc:224] Handling call
0709 12:24:53.919488 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.309014 (+389526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.309076 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.330573 (+ 21497us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.572645 (+242072us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:54.572705 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.572821 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.359346 (+786525us) raft_consensus.cc:1685] finished
0709 12:24:55.359368 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.359505 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80470,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":585503,"threads_started":2,"wal-append.queue_time_us":338036}
W20260709 12:24:55.327699 22356 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:24:55.422922 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:55.423998 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4183) took 1563 ms. Trace:
W20260709 12:24:55.380895 21824 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.942s	user 0.013s	sys 0.000s
I20260709 12:24:55.387634 22375 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:24:55.245031 22162 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:24:55.331316 22108 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:24:55.402307 14418 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:24:55.345530 13645 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
W20260709 12:24:55.383718 22252 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.006s	sys 0.009s
I20260709 12:24:55.359228 22381 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.479532 22381 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:55.386833 22360 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:55.408016 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.483996 13720 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:55.485361 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.486689 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1381) took 1372 ms. Trace:
I20260709 12:24:55.489245 22108 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:55.414783 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:55.490980 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 194343us
I20260709 12:24:55.389976 22352 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:55.374997 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 254060us
W20260709 12:24:55.437528 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 201159us
I20260709 12:24:55.403443 22190 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:24:55.396231 22249 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:55.495704 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81903.2us
I20260709 12:24:55.408806 22296 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:55.396589 22230 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:55.424382 13797 rpcz_store.cc:276] 0709 12:24:53.860319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.920606 (+ 60287us) service_pool.cc:224] Handling call
0709 12:24:53.925980 (+  5374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.247297 (+321317us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:54.300696 (+ 53399us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.517096 (+216400us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.541348 (+ 24252us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.541528 (+   180us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:54.541564 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.541670 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.423766 (+882096us) raft_consensus.cc:1685] finished
0709 12:24:55.423789 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.423935 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137702,"prepare.queue_time_us":50547,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":204427,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":28935,"thread_start_us":329536,"threads_started":2,"wal-append.queue_time_us":350458}
I20260709 12:24:55.487275 13649 rpcz_store.cc:276] 0709 12:24:54.113832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.206517 (+ 92685us) service_pool.cc:224] Handling call
0709 12:24:54.206912 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.240728 (+ 33816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.260075 (+ 19347us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.260621 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.455395 (+194774us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:54.474555 (+ 19160us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.498928 (+ 24373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.499019 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.486365 (+987346us) raft_consensus.cc:1685] finished
0709 12:24:55.486398 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.486605 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121174,"prepare.queue_time_us":19229,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":61060,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":50292,"spinlock_wait_cycles":18046208,"thread_start_us":128707,"threads_started":1,"wal-append.queue_time_us":222840}
I20260709 12:24:55.386497 22300 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:55.498322 22262 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.785s	user 0.015s	sys 0.003s
I20260709 12:24:55.493490 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:55.542438 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49044.8us
I20260709 12:24:55.544378 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:55.546008 13617 leader_election.cc:400] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:55.546227 13617 leader_election.cc:403] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:55.396517 22194 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:55.499393 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:55.559873 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.397527 22368 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.355542 22345 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.199s
I20260709 12:24:55.524547 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.394625 22318 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.465998 21824 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.526041 22193 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:55.504784 22320 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.414s	user 0.009s	sys 0.012s
W20260709 12:24:55.562357 13617 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.794s (SENT)
I20260709 12:24:55.564455 13723 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18.
I20260709 12:24:55.549608 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:55.490073 22108 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:55.551937 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.593925 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.796s (SENT)
I20260709 12:24:55.594425 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:55.557626 22230 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:24:55.589283 22199 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:24:55.509655 22190 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:55.610426 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:55.541821 22332 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.591490 13650 raft_consensus.cc:2410] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:24:55.552901 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:55.561324 22317 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:55.584744 22370 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.605470 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:55.607468 22373 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 12:24:55.608736 22108 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.562911 22252 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:24:55.631759 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.510265 22249 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.507121 22352 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.561466 22369 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.562045 22210 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:24:55.648828 13617 leader_election.cc:400] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:55.594461 22345 consensus_queue.cc:1243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:24:55.566046 22198 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:55.653059 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.557996 22262 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.561647 21953 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 12:24:55.646575 22359 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.640170 22328 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:55.669572 13797 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:24:55.659440 22370 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:55.673696 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:55.675205 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.564952 22194 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.707473 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3821) took 1066 ms. Trace:
I20260709 12:24:55.707862 13723 rpcz_store.cc:276] 0709 12:24:54.641081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.647505 (+1006424us) service_pool.cc:224] Handling call
0709 12:24:55.707402 (+ 59897us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.708425 22394 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.578676 22376 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:55.708943 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.557319 22380 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 12:24:55.598899 22320 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.714138 13801 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:55.714548 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 170144us
I20260709 12:24:55.682583 22252 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:55.716118 13692 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.003s
W20260709 12:24:55.716473 13692 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.005s	sys 0.003s
W20260709 12:24:55.701154 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 151609us
I20260709 12:24:55.660697 22336 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:24:55.707259 22207 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:55.670605 13617 leader_election.cc:403] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:24:55.709961 13723 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:24:55.578711 22387 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.138s
I20260709 12:24:55.725250 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:55.725867 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.718688 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:55.726509 13792 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
W20260709 12:24:55.720932 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.685638 22340 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.704599 22359 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:55.744828 13726 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:24:55.743386 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:55.746469 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3809) took 1111 ms. Trace:
I20260709 12:24:55.746798 13721 rpcz_store.cc:276] 0709 12:24:54.635068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.743086 (+1108018us) service_pool.cc:224] Handling call
0709 12:24:55.746405 (+  3319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.729069 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3822) took 1087 ms. Trace:
I20260709 12:24:55.748425 13723 rpcz_store.cc:276] 0709 12:24:54.641632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.708623 (+1066991us) service_pool.cc:224] Handling call
0709 12:24:55.728990 (+ 20367us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:55.751119 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102505us
I20260709 12:24:55.752360 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:55.720835 22368 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.758095 13721 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:24:55.760174 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1590) took 1069 ms. Trace:
I20260709 12:24:55.752796 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:55.694842 22397 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.763443 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1795) took 1540 ms. Trace:
I20260709 12:24:55.764300 13617 leader_election.cc:400] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:24:55.765475 13720 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:24:55.766307 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3820) took 1128 ms. Trace:
I20260709 12:24:55.763155 14417 rpcz_store.cc:276] 0709 12:24:54.691243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.691463 (+   220us) service_pool.cc:224] Handling call
0709 12:24:54.699036 (+  7573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.699431 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.699467 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.700008 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.810869 (+110861us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:54.810933 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.811063 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.759926 (+948863us) raft_consensus.cc:1685] finished
0709 12:24:55.759954 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.760107 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23804,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":8555,"thread_start_us":110087,"threads_started":1,"wal-append.queue_time_us":454313}
I20260709 12:24:55.684773 22383 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:55.609308 22384 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.129s
W20260709 12:24:55.742508 22377 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
W20260709 12:24:55.736677 22339 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.584s	user 0.007s	sys 0.004s
I20260709 12:24:55.739296 22352 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.743253 22318 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:55.759223 22376 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 16. Current term is 17.
I20260709 12:24:55.775146 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:55.743480 22262 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:55.765058 13721 rpcz_store.cc:276] 0709 12:24:54.223209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.757088 (+1533879us) service_pool.cc:224] Handling call
0709 12:24:55.757649 (+   561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.763079 (+  5430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.763388 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.766815 13720 rpcz_store.cc:276] 0709 12:24:54.638202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.484966 (+846764us) service_pool.cc:224] Handling call
0709 12:24:55.766217 (+281251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.780856 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1798) took 1556 ms. Trace:
I20260709 12:24:55.799333 13723 rpcz_store.cc:276] 0709 12:24:54.223916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.758663 (+1534747us) service_pool.cc:224] Handling call
0709 12:24:55.780006 (+ 21343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.780273 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.780313 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:55.780377 (+    64us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:55.780425 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.780500 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.780763 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.801476 13723 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:24:55.737129 22381 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:55.772490 22230 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:55.652060 22296 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.788268 22400 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.790479 13645 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
W20260709 12:24:55.802893 22395 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:24:55.783660 22378 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:55.726323 22193 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.056s
I20260709 12:24:55.772531 22368 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:55.790762 22352 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.779587 22340 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:55.804136 22355 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.810671 13721 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:55.829435 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:55.883051 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1799) took 1658 ms. Trace:
I20260709 12:24:55.883618 13721 rpcz_store.cc:276] 0709 12:24:54.224139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.809623 (+1585484us) service_pool.cc:224] Handling call
0709 12:24:55.810145 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.882636 (+ 72491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.882954 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:55.788297 22398 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:24:55.721601 22190 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:55.834400 22239 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.337s	user 0.015s	sys 0.000s
I20260709 12:24:55.786638 22339 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.835126 21824 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:55.806419 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3823) took 1160 ms. Trace:
I20260709 12:24:55.838963 22368 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.956058 13723 rpcz_store.cc:276] 0709 12:24:54.641975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.800638 (+1158663us) service_pool.cc:224] Handling call
0709 12:24:55.801147 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.801950 (+   803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.802210 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.957424 13723 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 14. Current term is 15. Ops: []
W20260709 12:24:55.815570 22386 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.247s	user 0.001s	sys 0.001s
I20260709 12:24:55.962234 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3825) took 1319 ms. Trace:
I20260709 12:24:55.811235 22405 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:55.841009 22392 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:24:55.809365 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1800) took 1386 ms. Trace:
I20260709 12:24:55.804793 22165 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:55.709075 22394 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.983476 22404 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.984036 22165 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.984125 22404 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.847711 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3808) took 1212 ms. Trace:
I20260709 12:24:55.790658 22383 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.986614 22394 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:55.986418 13722 rpcz_store.cc:276] 0709 12:24:54.634820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.815512 (+180692us) service_pool.cc:224] Handling call
0709 12:24:55.847431 (+1031919us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b54003fedd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:55.847616 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160012544}
W20260709 12:24:55.987645 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.120s (SENT)
I20260709 12:24:55.988618 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3827) took 1345 ms. Trace:
W20260709 12:24:55.932966 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.989112 13725 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:24:55.988987 13722 rpcz_store.cc:276] 0709 12:24:54.643327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.987629 (+1344302us) service_pool.cc:224] Handling call
0709 12:24:55.988045 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.988281 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.988306 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:55.988349 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:55.988394 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.988445 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.988556 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.989913 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3826) took 1347 ms. Trace:
I20260709 12:24:55.828877 22194 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.849774 22352 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.990238 13725 rpcz_store.cc:276] 0709 12:24:54.643010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.964529 (+1321519us) service_pool.cc:224] Handling call
0709 12:24:55.988878 (+ 24349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.989646 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.989855 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.839366 22296 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:55.965039 22355 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:55.993880 22355 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:55.945955 22353 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.349s	user 0.002s	sys 0.012s
W20260709 12:24:55.942138 22333 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.130s	user 0.013s	sys 0.000s
I20260709 12:24:55.996165 22355 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.997372 22333 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.788915 22252 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:55.999063 22404 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:55.834528 22393 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:24:56.002697 22401 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:24:55.954461 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 358034us
I20260709 12:24:56.003530 22401 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:55.951360 22328 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.844412 22340 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:55.942548 22387 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.777069 22397 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.959689 22368 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:55.982870 13720 rpcz_store.cc:276] 0709 12:24:54.422810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.808152 (+1385342us) service_pool.cc:224] Handling call
0709 12:24:55.808615 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.808978 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.809003 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:55.809101 (+    98us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:24:55.809134 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.809190 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.809308 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.962771 13723 rpcz_store.cc:276] 0709 12:24:54.642684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.956992 (+1314308us) service_pool.cc:224] Handling call
0709 12:24:55.957291 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.962031 (+  4740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.962183 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.007763 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:56.022220 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:55.990394 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.965358 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:56.004237 22347 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.003s
I20260709 12:24:56.027591 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:56.027770 13648 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:56.028656 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:55.993083 22352 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.988860 22383 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:55.964936 22384 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:56.032233 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:55.955577 22239 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.005425 21824 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:56.021989 22387 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.036995 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:56.019726 22340 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:56.019280 22353 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:24:56.024129 22398 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:56.042001 22332 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.051273 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:56.051858 13720 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:56.038033 22108 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:24:56.043378 22333 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.020852 22328 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.127624 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:56.060829 13645 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:56.051306 22320 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.164624 22339 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.146225 22333 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:56.168109 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:56.029247 22405 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:55.843580 22400 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.123648 14422 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:56.123545 22376 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:24:56.025983 22230 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.310693 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:56.030462 22347 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.051630 21824 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:56.313906 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 145901us
I20260709 12:24:56.327726 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:56.230067 22353 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.231142 22193 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.351145 13796 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
I20260709 12:24:56.046115 22384 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.124552 22401 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.296768 22339 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.310052 22327 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:24:56.130117 22332 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.389796 13648 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:56.325299 22194 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:56.401118 22407 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.401569 22409 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:56.338621 22252 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.228844 22387 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.294976 22333 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.402618 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:56.341200 22381 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:56.503155 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3951) took 4303 ms. Trace:
I20260709 12:24:56.373307 22353 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.512253 22252 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.396723 22339 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.523609 14423 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:24:56.405072 22194 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.524158 13727 rpcz_store.cc:276] 0709 12:24:52.199926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.549399 (+349473us) service_pool.cc:224] Handling call
0709 12:24:52.570236 (+ 20837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.655075 (+1084839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.655113 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.655179 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:53.655220 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.502848 (+2847628us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001a4854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:56.502904 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.503083 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494330752}
I20260709 12:24:56.534196 14423 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 13
I20260709 12:24:56.537140 14423 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:24:56.537465 14423 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:56.533597 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3832) took 1888 ms. Trace:
I20260709 12:24:56.542022 13720 rpcz_store.cc:276] 0709 12:24:54.644989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.532644 (+1887655us) service_pool.cc:224] Handling call
0709 12:24:56.533078 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.533274 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.533301 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:56.533347 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:56.533376 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.533430 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.533536 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:56.541280 14416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1262) took 3180 ms (client timeout 1874 ms). Trace:
W20260709 12:24:56.544325 14416 rpcz_store.cc:269] 0709 12:24:53.350697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.353223 (+  2526us) service_pool.cc:224] Handling call
0709 12:24:56.508207 (+3154984us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400600254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:56.530852 (+ 22645us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2385204608}
W20260709 12:24:56.545473 13722 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.102s	sys 0.157s
W20260709 12:24:56.545840 13722 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.102s	sys 0.157s
I20260709 12:24:56.395730 22193 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.523334 22303 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:24:56.551678 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:24:56.530493 22401 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:24:56.543843 22194 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:24:56.582036 22342 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.872s	user 0.012s	sys 0.005s
I20260709 12:24:56.530752 22407 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.597381 13727 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:24:56.607761 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1801) took 1794 ms. Trace:
I20260709 12:24:56.611982 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3833) took 1966 ms. Trace:
I20260709 12:24:56.611613 13720 rpcz_store.cc:276] 0709 12:24:54.813737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.591491 (+1777754us) service_pool.cc:224] Handling call
0709 12:24:56.607013 (+ 15522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.607277 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.607324 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:56.607388 (+    64us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:24:56.607437 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.607511 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.607675 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.613082 13727 rpcz_store.cc:276] 0709 12:24:54.645318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.579449 (+1934131us) service_pool.cc:224] Handling call
0709 12:24:56.597068 (+ 17619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.611567 (+ 14499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.611900 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.533743 22381 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.605417 22342 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:24:56.533321 22409 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.599200 22397 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.650539 13727 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:24:56.657087 14425 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:24:56.625082 22328 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:56.664808 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.666836 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.666993 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:56.665037 13727 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:24:56.671027 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 351342us
I20260709 12:24:56.626362 22400 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:56.655696 22405 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.685061 13727 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:24:56.685532 13727 pending_rounds.cc:85] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:24:56.686374 13727 pending_rounds.cc:107] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:24:56.686749 13727 raft_consensus.cc:2889] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:56.680961 22411 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 12:24:56.689801 22384 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.677498 22355 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.675539 21824 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.695863 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:56.697430 22414 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:24:56.704696 22381 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:56.706954 22413 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:56.706825 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3830) took 2062 ms. Trace:
W20260709 12:24:56.708151 22415 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:24:56.709575 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3828) took 2066 ms. Trace:
I20260709 12:24:56.687040 22400 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:56.711014 22353 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.710757 22400 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.722569 22332 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.722803 22400 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:56.723366 22339 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.724464 22252 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.709949 13722 rpcz_store.cc:276] 0709 12:24:54.643667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.990133 (+1346466us) service_pool.cc:224] Handling call
0709 12:24:55.992882 (+  2749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.994377 (+  1495us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:24:55.994508 (+   131us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.996627 (+  2119us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.546372 (+549745us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:56.546437 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.546535 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.709319 (+162784us) raft_consensus.cc:1685] finished
0709 12:24:56.709343 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.709507 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3119,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":46,"thread_start_us":547934,"threads_started":1,"wal-append.queue_time_us":592690}
I20260709 12:24:56.747443 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:56.733000 13722 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:24:56.770176 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.005s	sys 0.013s
W20260709 12:24:56.775241 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.006s	sys 0.013s
W20260709 12:24:56.776034 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55908us
W20260709 12:24:56.776396 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 729744us
W20260709 12:24:56.722105 22219 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.547s	user 0.009s	sys 0.005s
I20260709 12:24:56.707618 13725 rpcz_store.cc:276] 0709 12:24:54.644296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.991393 (+1347097us) service_pool.cc:224] Handling call
0709 12:24:55.991765 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.992163 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.992194 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.992736 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.396849 (+404113us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:56.396908 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.397040 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.706512 (+309472us) raft_consensus.cc:1685] finished
0709 12:24:56.706552 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.706740 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4720,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":403616,"threads_started":1,"wal-append.queue_time_us":543495}
I20260709 12:24:56.796136 13649 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
W20260709 12:24:56.777032 22296 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.784s	user 0.014s	sys 0.000s
I20260709 12:24:56.693789 22347 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.785434 21824 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:24:56.833662 13725 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:56.743765 22355 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:56.795166 22400 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.822469 22418 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.836437 22333 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:56.936602 22409 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:56.948897 22355 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:57.045444 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.109640 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.144101 22424 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:57.144902 22424 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.150039 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:56.853710 22400 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:56.950178 22333 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.171970 22333 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.169894 22407 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:57.170135 22370 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.464s	user 0.009s	sys 0.005s
I20260709 12:24:57.174229 22370 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:24:57.171351 22340 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.043s	user 0.007s	sys 0.007s
I20260709 12:24:57.174757 22333 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.177117 22340 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.178704 14416 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:56.861935 22418 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.196466 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3990) took 1176 ms. Trace:
I20260709 12:24:57.197013 13722 rpcz_store.cc:276] 0709 12:24:56.019560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.775535 (+755975us) service_pool.cc:224] Handling call
0709 12:24:56.775886 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.776139 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:24:56.844060 (+ 67921us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.195922 (+351862us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:57.196014 (+    92us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:57.196086 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.196188 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.196376 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20485,"prepare.queue_time_us":170218,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"thread_start_us":398393,"threads_started":2}
I20260709 12:24:57.213153 22340 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:57.216076 22401 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.626s	user 0.015s	sys 0.000s
I20260709 12:24:57.234875 22407 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.231148 22424 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:57.227468 22194 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.624s	user 0.004s	sys 0.008s
I20260709 12:24:57.242413 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:57.256065 22211 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.245596 22407 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.239768 22400 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.265290 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:57.271003 14422 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:57.263417 22409 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:57.267023 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:57.267501 22407 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.205318 22349 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:24:57.269814 22368 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.237s	user 0.004s	sys 0.012s
I20260709 12:24:57.268967 22218 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:57.293617 22368 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.297540 22368 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.270166 22158 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:24:57.275606 22409 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.270488 22418 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.263803 22194 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.288559 22349 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:57.306856 22409 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.307494 22349 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.308200 22194 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.308713 22349 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:57.308981 13727 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:24:57.310113 22158 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:57.313053 22165 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.311285 22158 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.314500 22158 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:57.320237 14422 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:57.334137 22190 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.334985 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:57.335081 22190 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.336027 22164 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:57.336778 22164 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:57.337904 22164 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.339833 22239 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.348147 22307 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:57.403470 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.406697 22349 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:57.409571 22134 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.410467 22340 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.410701 22134 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.411510 22340 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.415546 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1385) took 3299 ms. Trace:
I20260709 12:24:57.416121 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:57.416348 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:57.416296 13647 rpcz_store.cc:276] 0709 12:24:54.115427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.207545 (+ 92118us) service_pool.cc:224] Handling call
0709 12:24:54.332083 (+124538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.348554 (+ 16471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.414805 (+3066251us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001e7150. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.415248 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285599104}
I20260709 12:24:57.418125 14424 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:57.420241 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:57.421478 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4450) took 1862 ms (client timeout 1999 ms). Trace:
W20260709 12:24:57.425251 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.430665 22106 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:24:57.431411 22106 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:57.408610 22349 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:57.421993 13647 rpcz_store.cc:269] 0709 12:24:55.558883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.419412 (+1860529us) service_pool.cc:224] Handling call
0709 12:24:57.421360 (+  1948us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.436084 22134 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.438375 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:57.441509 14418 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:24:57.443209 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1400) took 1900 ms (client timeout 1529 ms). Trace:
W20260709 12:24:57.443502 14391 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 2.107s (SENT)
W20260709 12:24:57.443842 13649 rpcz_store.cc:269] 0709 12:24:55.542796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.552494 (+  9698us) service_pool.cc:224] Handling call
0709 12:24:57.442647 (+1890153us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400221fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.442884 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2657460992}
I20260709 12:24:57.446162 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:57.447371 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.447542 22307 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:57.447686 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4449) took 1889 ms (client timeout 2021 ms). Trace:
I20260709 12:24:57.444290 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:57.433660 22106 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.448643 22307 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:57.448407 13649 rpcz_store.cc:269] 0709 12:24:55.558601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.445657 (+1887056us) service_pool.cc:224] Handling call
0709 12:24:57.447623 (+  1966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.450415 22106 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:57.450831 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:57.453482 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4451) took 1894 ms (client timeout 2055 ms). Trace:
W20260709 12:24:57.453902 13649 rpcz_store.cc:269] 0709 12:24:55.559284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.450372 (+1891088us) service_pool.cc:224] Handling call
0709 12:24:57.453422 (+  3050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.455179 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3988) took 1439 ms. Trace:
I20260709 12:24:57.455710 13727 rpcz_store.cc:276] 0709 12:24:56.013654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.687799 (+674145us) service_pool.cc:224] Handling call
0709 12:24:56.771546 (+ 83747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.309390 (+537844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.451052 (+141662us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54003fd4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:57.452553 (+  1501us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235340672}
I20260709 12:24:57.456192 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:57.446992 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1791) took 3224 ms. Trace:
I20260709 12:24:57.446795 22213 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.461520 22213 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.461606 22340 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.466651 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:57.459944 13618 connection.cc:256] RPC call timeout handler was delayed by -1.9094134070001019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260709 12:24:57.467901 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.469740 14418 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:57.471871 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1269) took 4119 ms. Trace:
I20260709 12:24:57.472329 14423 rpcz_store.cc:276] 0709 12:24:53.352806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.416634 (+ 63828us) service_pool.cc:224] Handling call
0709 12:24:56.508363 (+3091729us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.538202 (+ 29839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.471515 (+933313us) spinlock_profiling.cc:243] Waited 3.08 s on lock 0x7b5400600250. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.471692 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7074752768}
I20260709 12:24:57.473265 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:57.473523 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.475755 22239 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:57.476523 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.477131 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:57.477253 22239 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.477859 13799 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:57.479601 21573 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:57.479957 13794 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:57.480317 22348 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:57.481006 22348 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:57.480569 21573 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:57.482599 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.483450 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:57.469871 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1270) took 3986 ms. Trace:
W20260709 12:24:57.470295 13618 connection.cc:256] RPC call timeout handler was delayed by -1.4558134070002779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.246s for a response
I20260709 12:24:57.472338 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:57.483507 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.462421 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4184) took 3599 ms (client timeout 2460 ms). Trace:
W20260709 12:24:57.491071 13798 rpcz_store.cc:269] 0709 12:24:53.862908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.891624 (+ 28716us) service_pool.cc:224] Handling call
0709 12:24:57.461335 (+3569711us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.461917 (+   582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3029926656}
I20260709 12:24:57.492093 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:57.492416 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:57.493813 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.493662 22371 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:57.491354 22349 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.501791 22296 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.502692 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:57.502575 22296 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.503710 13799 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:57.507190 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:57.508253 14423 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:24:57.512986 13800 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:57.487389 14419 rpcz_store.cc:276] 0709 12:24:53.483517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.914126 (+430609us) service_pool.cc:224] Handling call
0709 12:24:53.914527 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.142204 (+1227677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.469281 (+2327077us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:57.469660 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821720832}
I20260709 12:24:57.497090 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4181) took 3657 ms. Trace:
W20260709 12:24:57.499246 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.496907 22348 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.519891 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.523130 13800 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:24:57.515569 13793 rpcz_store.cc:276] 0709 12:24:53.839930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.912763 (+ 72833us) service_pool.cc:224] Handling call
0709 12:24:53.913046 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.030565 (+117519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.496358 (+3465793us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400287dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:57.496849 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268863360}
I20260709 12:24:57.477991 21743 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:57.459302 13726 rpcz_store.cc:276] 0709 12:24:54.222158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.295533 (+ 73375us) service_pool.cc:224] Handling call
0709 12:24:54.295991 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.745327 (+1449336us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.446216 (+1700889us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:57.446667 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3331541248}
I20260709 12:24:57.536008 22348 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:57.536773 21573 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:57.539300 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53447.8us
I20260709 12:24:57.540252 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.542059 13800 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:57.547750 22023 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:24:57.548813 22023 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:57.552224 22307 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.554703 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:57.552165 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.559404 13793 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:57.560882 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.561221 22193 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:57.561722 13793 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:57.562763 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.562294 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.563877 13798 raft_consensus.cc:2410] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:24:57.565089 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.567427 22165 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:57.569079 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:57.570204 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:57.570735 22273 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:24:57.571185 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:57.574555 13647 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:57.577206 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:57.580904 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.581892 13798 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:24:57.578387 13722 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:57.579797 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.595191 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.597702 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.599727 22312 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:57.602614 22376 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.581718 22023 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.603302 22376 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.605413 22023 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:57.605742 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:57.606676 13723 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:24:57.606765 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1367) took 3411 ms. Trace:
I20260709 12:24:57.608352 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:57.613135 14421 rpcz_store.cc:276] 0709 12:24:54.194825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.195119 (+   294us) service_pool.cc:224] Handling call
0709 12:24:54.195413 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.373199 (+177786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.606029 (+3232830us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400516b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:57.606574 (+   545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406318976}
I20260709 12:24:57.614806 21824 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:57.608695 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2835) took 4034 ms (client timeout 1752 ms). Trace:
I20260709 12:24:57.618104 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2814) took 4056 ms. Trace:
W20260709 12:24:57.615893 13646 rpcz_store.cc:269] 0709 12:24:53.574516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.626023 (+ 51507us) service_pool.cc:224] Handling call
0709 12:24:57.608119 (+3982096us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.608465 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241473280}
I20260709 12:24:57.602617 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:57.623101 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:57.624078 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2856) took 2028 ms (client timeout 2177 ms). Trace:
W20260709 12:24:57.624449 13646 rpcz_store.cc:269] 0709 12:24:55.595236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.622547 (+2027311us) service_pool.cc:224] Handling call
0709 12:24:57.623985 (+  1438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.625658 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:57.626219 13797 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
I20260709 12:24:57.627247 22329 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:24:57.627785 22329 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:57.629367 22329 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.631299 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2848) took 2455 ms. Trace:
I20260709 12:24:57.633000 21573 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.618535 22370 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.620299 21824 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.635426 13643 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:24:57.639498 22313 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:57.623114 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49010us
I20260709 12:24:57.650995 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:57.652884 22406 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:57.642382 13650 rpcz_store.cc:276] 0709 12:24:55.175705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.428795 (+253090us) service_pool.cc:224] Handling call
0709 12:24:57.630804 (+2202009us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.631091 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74000768}
W20260709 12:24:57.669242 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.673029 22165 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:24:57.646030 13800 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:24:57.684407 13643 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:24:57.618618 13644 rpcz_store.cc:276] 0709 12:24:53.559623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.559757 (+   134us) service_pool.cc:224] Handling call
0709 12:24:53.572815 (+ 13058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.573090 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:24:53.588563 (+ 15473us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.432447 (+3843884us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800497200. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:24:57.433532 (+  1085us) log.cc:844] Serialized 14 byte log entry
0709 12:24:57.585682 (+152150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:57.588341 (+  2659us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:24:57.592676 (+  4335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.594845 (+  2169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.615668 (+ 20823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92275,"prepare.queue_time_us":30782,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":125,"raft.queue_time_us":176707,"raft.run_cpu_time_us":665,"raft.run_wall_time_us":65167,"spinlock_wait_cycles":6009728,"thread_start_us":551971,"threads_started":3,"wal-append.queue_time_us":453637,"wal-append.run_cpu_time_us":6775,"wal-append.run_wall_time_us":1207053}
I20260709 12:24:57.690995 13794 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:57.690629 13649 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
I20260709 12:24:57.709654 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:57.711179 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.715054 13647 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:57.715453 14424 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:24:57.717111 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2860) took 1703 ms (client timeout 1716 ms). Trace:
W20260709 12:24:57.717732 13649 rpcz_store.cc:269] 0709 12:24:56.005860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.578582 (+1572722us) service_pool.cc:224] Handling call
0709 12:24:57.708733 (+130151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.722159 13799 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
W20260709 12:24:57.724507 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4454) took 2155 ms (client timeout 2000 ms). Trace:
W20260709 12:24:57.725082 13647 rpcz_store.cc:269] 0709 12:24:55.560074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.443641 (+1883567us) service_pool.cc:224] Handling call
0709 12:24:57.715862 (+272221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.724172 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1388) took 3607 ms. Trace:
I20260709 12:24:57.727610 22206 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.727808 21883 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.728792 22044 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.728467 21883 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.726532 13643 rpcz_store.cc:276] 0709 12:24:54.116280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.209509 (+ 93229us) service_pool.cc:224] Handling call
0709 12:24:54.209866 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.613559 (+3403693us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b58004d45d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b500aa 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4
0709 12:24:57.702401 (+ 88842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.724099 (+ 21698us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122971520}
I20260709 12:24:57.696688 22313 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.685878 22376 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:57.733815 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:57.729302 22044 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.689159 21573 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.687386 22337 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:57.739475 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89586.7us
I20260709 12:24:57.736294 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:57.737469 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:57.744393 13800 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:24:57.688823 22370 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.747428 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:57.749822 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:57.750591 13644 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:57.753443 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:57.753504 22335 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:57.755579 22245 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:57.756327 22245 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:57.757529 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:57.758340 13799 raft_consensus.cc:2376] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16. Re-sending same reply.
I20260709 12:24:57.753013 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:57.762014 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2862) took 1747 ms (client timeout 1891 ms). Trace:
W20260709 12:24:57.762627 13650 rpcz_store.cc:269] 0709 12:24:56.013093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.676251 (+1663158us) service_pool.cc:224] Handling call
0709 12:24:57.760760 (+ 84509us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:57.764102 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:57.764879 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 15.
W20260709 12:24:57.765412 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2859) took 1760 ms (client timeout 2037 ms). Trace:
W20260709 12:24:57.766840 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56983us
I20260709 12:24:57.767544 22304 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:24:57.768287 22304 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:57.769402 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:57.758975 22337 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:57.771497 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1364) took 3778 ms. Trace:
I20260709 12:24:57.772003 14420 rpcz_store.cc:276] 0709 12:24:53.992553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.151277 (+158724us) service_pool.cc:224] Handling call
0709 12:24:54.151626 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.811799 (+660173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.770991 (+2959192us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:57.771299 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1516369280}
I20260709 12:24:57.728227 22206 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:57.776803 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1279) took 1748 ms. Trace:
I20260709 12:24:57.777575 14425 rpcz_store.cc:276] 0709 12:24:56.028490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.028649 (+   159us) service_pool.cc:224] Handling call
0709 12:24:57.776270 (+1747621us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b5400644bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:57.776503 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420788608}
I20260709 12:24:57.779062 13646 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:57.773907 22165 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:24:57.783684 22133 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:57.757637 22245 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:57.753623 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.080s (SENT) [suppressed 21 similar messages]
I20260709 12:24:57.769855 22304 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:57.821398 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 170647us
I20260709 12:24:57.907578 13793 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:24:58.094734 13649 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:57.765789 13650 rpcz_store.cc:269] 0709 12:24:56.005456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.763566 (+1758110us) service_pool.cc:224] Handling call
0709 12:24:57.765352 (+  1786us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:57.790448 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2853) took 2210 ms (client timeout 2428 ms). Trace:
W20260709 12:24:58.039806 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.058950 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:58.096980 13647 rpcz_store.cc:269] 0709 12:24:55.549772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.732486 (+2182714us) service_pool.cc:224] Handling call
0709 12:24:57.760546 (+ 28060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.098181 13799 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
W20260709 12:24:58.098694 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4460) took 2352 ms (client timeout 2166 ms). Trace:
W20260709 12:24:58.099299 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.099187 13649 rpcz_store.cc:269] 0709 12:24:55.746625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.724969 (+1978344us) service_pool.cc:224] Handling call
0709 12:24:58.098606 (+373637us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:57.815544 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46506.9us
W20260709 12:24:57.922045 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.950635 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.098961 22312 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:57.871721 22023 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:24:57.931416 22133 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:57.784524 22296 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:24:58.094182 22324 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.328s	user 0.025s	sys 0.041s
I20260709 12:24:58.112356 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:58.112789 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 369356us
I20260709 12:24:58.111009 22023 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.111140 22133 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:58.114601 22324 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.712s	user 0.027s	sys 0.041s
I20260709 12:24:57.752400 22313 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.126003 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.119167 22023 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:24:58.119251 22133 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:24:58.133584 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2852) took 2582 ms (client timeout 2518 ms). Trace:
W20260709 12:24:58.135715 13649 rpcz_store.cc:269] 0709 12:24:55.545831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.119883 (+2574052us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:58.128278 (+  8395us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:58.134718 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.137059 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:24:58.139930 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.140518 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.132434 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4174) took 4343 ms. Trace:
I20260709 12:24:58.145541 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:58.152328 13646 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2855) took 2557 ms (client timeout 2281 ms). Trace:
I20260709 12:24:58.152889 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.155135 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3984) took 3933 ms. Trace:
I20260709 12:24:58.155234 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.155623 13724 rpcz_store.cc:276] 0709 12:24:54.221222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.221341 (+   119us) service_pool.cc:224] Handling call
0709 12:24:54.221652 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.436017 (+3214365us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b18001c15c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393becbfb 00007f5393bf764b
0709 12:24:57.437860 (+  1843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.437889 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:57.452608 (+ 14719us) log.cc:844] Serialized 26 byte log entry
0709 12:24:57.588045 (+135437us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:57.588091 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.154599 (+566508us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b54003bf454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:58.154659 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.154729 (+    70us) raft_consensus.cc:1685] finished
0709 12:24:58.154743 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.154943 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109299,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":10122,"spinlock_wait_cycles":179569920,"thread_start_us":252024,"threads_started":2,"wal-append.queue_time_us":142739}
I20260709 12:24:58.159011 13799 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:24:58.163640 13801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4177) took 4384 ms (client timeout 1956 ms). Trace:
W20260709 12:24:58.164598 22348 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.628s	user 0.016s	sys 0.000s
I20260709 12:24:58.166167 22337 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.167716 22249 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:58.178535 22022 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:58.180797 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.164193 13801 rpcz_store.cc:269] 0709 12:24:53.778761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.779548 (+   787us) service_pool.cc:224] Handling call
0709 12:24:58.163084 (+4383536us) spinlock_profiling.cc:243] Waited 1.8 s on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.163359 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4141388160}
I20260709 12:24:58.185325 13644 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:24:58.185976 13644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4457) took 2485 ms (client timeout 2211 ms). Trace:
W20260709 12:24:58.186375 13644 rpcz_store.cc:269] 0709 12:24:55.700639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.723998 (+2023359us) service_pool.cc:224] Handling call
0709 12:24:58.185897 (+461899us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.141933 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:58.152812 13646 rpcz_store.cc:269] 0709 12:24:55.594913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.625243 (+2030330us) service_pool.cc:224] Handling call
0709 12:24:58.152118 (+526875us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001e7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.152223 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351109504}
W20260709 12:24:58.195420 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4446) took 2637 ms (client timeout 2554 ms). Trace:
W20260709 12:24:58.196143 13649 rpcz_store.cc:269] 0709 12:24:55.557610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.143945 (+2586335us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:58.195323 (+ 51378us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:24:58.197160 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.142431 13795 rpcz_store.cc:276] 0709 12:24:53.777659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.824928 (+ 47269us) service_pool.cc:224] Handling call
0709 12:24:54.194525 (+369597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.214926 (+ 20401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.109177 (+3894251us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54000d2fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.120975 (+ 11798us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789419264}
I20260709 12:24:58.202746 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.206236 13722 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:58.207360 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.207876 13723 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:58.208264 13722 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:58.200522 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.205966 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.204825 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.211937 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.214691 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.217998 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.220880 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.202172 21573 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:58.222754 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:24:58.223515 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2866) took 2175 ms. Trace:
W20260709 12:24:58.223372 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46338.7us
I20260709 12:24:58.224088 13644 rpcz_store.cc:276] 0709 12:24:56.048016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.202653 (+2154637us) service_pool.cc:224] Handling call
0709 12:24:58.223455 (+ 20802us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.220471 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.239467 13718 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:58.242008 13724 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:24:58.244050 13726 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:58.246445 13646 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:58.247417 13793 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:58.248454 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:58.250828 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.252071 13646 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:58.253062 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.253304 13645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4455) took 2692 ms (client timeout 1907 ms). Trace:
W20260709 12:24:58.253756 13645 rpcz_store.cc:269] 0709 12:24:55.560327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.828505 (+268178us) service_pool.cc:224] Handling call
0709 12:24:58.252867 (+2424362us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400114254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.253106 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408295168}
I20260709 12:24:58.253764 13646 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:58.254956 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.255398 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:58.255621 13645 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:58.256223 13646 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.256743 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:58.257527 13645 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:58.258064 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.259390 21824 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.260897 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1404) took 3098 ms. Trace:
I20260709 12:24:58.261265 22261 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:24:58.262203 22261 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:58.266403 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.261371 13796 rpcz_store.cc:276] 0709 12:24:55.162288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.162405 (+   117us) service_pool.cc:224] Handling call
0709 12:24:55.162757 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.351893 (+1189136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.260334 (+1908441us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540012af54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:58.260696 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2732322560}
I20260709 12:24:58.267740 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.273175 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.273758 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.278317 13726 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:24:58.278968 22370 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:58.281613 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.274319 13800 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19. Re-sending same reply.
I20260709 12:24:58.275477 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.289438 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.286801 13652 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:58.290712 22382 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.291385 14424 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:58.291260 22382 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.292905 13722 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:58.294133 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.308126 13721 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:58.309059 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
W20260709 12:24:58.310459 22320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.040s	sys 0.122s
W20260709 12:24:58.311019 22320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.041s	sys 0.122s
I20260709 12:24:58.311805 22213 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:58.313457 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:58.314520 13726 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:58.313865 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.317998 13800 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:24:58.307996 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 3974) took 4742 ms (client timeout 1970 ms). Trace:
I20260709 12:24:58.312110 13644 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:58.319362 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2867) took 2268 ms. Trace:
I20260709 12:24:58.319864 13644 rpcz_store.cc:276] 0709 12:24:56.050868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.231104 (+2180236us) service_pool.cc:224] Handling call
0709 12:24:58.319302 (+ 88198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.319082 13801 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:24:58.320703 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.321569 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:58.319111 13719 rpcz_store.cc:269] 0709 12:24:53.565060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.565227 (+   167us) service_pool.cc:224] Handling call
0709 12:24:58.307338 (+4742111us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54003fded4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.307605 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3516147072}
I20260709 12:24:58.324275 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.325081 13723 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:58.318666 22261 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.329249 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.329274 22376 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:58.330489 13644 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:24:58.330480 22376 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:58.331036 22206 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:58.331652 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.332280 13644 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:58.332849 22376 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.333109 22239 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.336229 22190 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:24:58.337049 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.337879 22372 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:24:58.339527 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.328647 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.344964 13645 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:24:58.346752 22372 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:24:58.347221 22372 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:58.348353 22372 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:58.348892 22382 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:58.352134 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:58.326403 22261 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:58.355129 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.355547 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.346766 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.355916 13719 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:24:58.343885 22324 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.353598 22372 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:58.364884 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.352329 22313 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:58.367506 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.367959 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.367789 22324 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:58.367576 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.369637 13723 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:24:58.378518 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.379706 13801 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 17: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:58.382164 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:58.385846 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:58.399147 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:58.399451 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 275996us
I20260709 12:24:58.413312 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.413414 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.411540 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.424996 21883 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.414464 13723 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:58.443902 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:58.444924 13646 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:24:58.360888 22382 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 12:24:58.446784 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61062.7us
I20260709 12:24:58.447837 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.448804 14424 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:24:58.451352 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1802) took 3632 ms. Trace:
I20260709 12:24:58.451805 13720 rpcz_store.cc:276] 0709 12:24:54.814063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.613893 (+1799830us) service_pool.cc:224] Handling call
0709 12:24:56.625356 (+ 11463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.664915 (+ 39559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.664949 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:56.664991 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:56.665033 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.389372 (+1724339us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54003fe8d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:58.401669 (+ 12297us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.446389 (+ 44720us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90497792}
I20260709 12:24:58.454319 13718 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:24:58.454752 13718 pending_rounds.cc:85] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:24:58.411923 22324 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.456751 13726 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:58.455006 13718 pending_rounds.cc:107] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:24:58.472513 13718 raft_consensus.cc:2889] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:58.398958 22133 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.477829 22312 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:58.489265 22312 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.489678 22312 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:58.493171 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.499182 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.518107 13649 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:58.512476 13727 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:58.527144 13794 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:24:58.524783 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.532433 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.494812 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:24:58.535795 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46720.3us
I20260709 12:24:58.538515 13792 raft_consensus.cc:606] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:58.541316 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.486707 22023 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.538201 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:58.545886 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.543685 22379 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:24:58.549280 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.550056 22379 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:24:58.547744 13720 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:58.690696 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:58.562755 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.630982 14417 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:24:58.696717 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.680002 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.567160 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:58.700493 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.658145 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.567966 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.568570 13647 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:58.567540 13726 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:24:58.611559 13800 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
W20260709 12:24:58.658092 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1384) took 4542 ms (client timeout 1561 ms). Trace:
I20260709 12:24:58.550616 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.652133 13646 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:24:58.673480 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:58.756141 13651 rpcz_store.cc:269] 0709 12:24:54.115137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.118537 (+  3400us) service_pool.cc:224] Handling call
0709 12:24:58.657692 (+4539155us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.657862 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2225228544}
I20260709 12:24:58.643368 13801 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:24:58.616611 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68925us
I20260709 12:24:58.644040 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.687350 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:58.711242 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2861) took 2705 ms (client timeout 3250 ms). Trace:
I20260709 12:24:58.762482 13797 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:24:58.550621 22445 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.002s
I20260709 12:24:58.703398 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.711684 13652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2841) took 5100 ms (client timeout 3084 ms). Trace:
I20260709 12:24:58.714931 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.559084 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:24:58.762571 13649 rpcz_store.cc:269] 0709 12:24:56.006209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.205420 (+2199211us) service_pool.cc:224] Handling call
0709 12:24:58.710985 (+505565us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54002b39d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.711158 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":570111232}
W20260709 12:24:58.652884 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 84819.3us
W20260709 12:24:58.764731 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 197671us
I20260709 12:24:58.630136 22131 raft_consensus.cc:2764] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:24:58.705874 22023 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:58.550483 22379 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:24:58.696861 22402 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.016s	sys 0.007s
I20260709 12:24:58.538400 22106 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.222s
I20260709 12:24:58.673480 22243 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.775434 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:24:58.766197 13652 rpcz_store.cc:269] 0709 12:24:53.611370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.934250 (+322880us) service_pool.cc:224] Handling call
0709 12:24:58.711455 (+4777205us) spinlock_profiling.cc:243] Waited 4.28 s on lock 0x7b54001e7b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.711609 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9833804032}
W20260709 12:24:58.773515 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.774027 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.780743 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.781374 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.782487 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95225.4us
I20260709 12:24:58.782796 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.783535 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:58.783742 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:58.784650 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.784919 22043 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:24:58.785272 13651 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:24:58.786338 22106 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:24:58.786610 13645 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:24:58.786357 22296 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:24:58.787606 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.787905 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2864) took 2728 ms (client timeout 1516 ms). Trace:
I20260709 12:24:58.789165 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.788341 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:24:58.786623 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:58.788925 22397 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:24:58.789423 13648 rpcz_store.cc:269] 0709 12:24:56.029899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.231744 (+201845us) service_pool.cc:224] Handling call
0709 12:24:58.740685 (+2508941us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b5400147dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.758087 (+ 17402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140462848}
I20260709 12:24:58.782497 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:58.690955 22312 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:58.793329 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.793080 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:58.795897 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.790448 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2873) took 1165 ms. Trace:
I20260709 12:24:58.790889 14420 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
W20260709 12:24:58.775862 22402 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.019s	sys 0.014s
I20260709 12:24:58.796407 13645 rpcz_store.cc:276] 0709 12:24:57.624661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.787318 (+1162657us) service_pool.cc:224] Handling call
0709 12:24:58.790381 (+  3063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.662681 22352 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 12:24:58.772749 22133 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:58.796371 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.798987 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.801172 13652 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:58.794646 13651 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:24:58.820469 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.802104 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.802388 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:24:58.820889 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.802843 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.803030 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.804462 22044 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:58.790067 22243 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:58.785405 22043 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.808276 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.797320 22397 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.798754 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:58.815260 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.814817 22379 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:58.815925 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:24:58.815984 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2865) took 2775 ms (client timeout 2166 ms). Trace:
W20260709 12:24:58.817020 22448 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:24:58.816797 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:58.822503 13645 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:24:58.855206 13650 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:58.855470 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2878) took 1227 ms. Trace:
I20260709 12:24:58.855876 13645 rpcz_store.cc:276] 0709 12:24:57.628370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.799021 (+1170651us) service_pool.cc:224] Handling call
0709 12:24:58.855407 (+ 56386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.857124 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.851298 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.819085 22023 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:58.861102 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:24:58.861713 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2880) took 1125 ms. Trace:
I20260709 12:24:58.858173 13645 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:58.852430 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:24:58.852167 13647 rpcz_store.cc:269] 0709 12:24:56.040785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.180374 (+2139589us) service_pool.cc:224] Handling call
0709 12:24:58.815712 (+635338us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54001b6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:58.815895 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889381248}
I20260709 12:24:58.862421 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2881) took 1114 ms. Trace:
I20260709 12:24:58.853513 13726 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:24:58.854203 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 58495.2us
I20260709 12:24:58.862044 13649 rpcz_store.cc:276] 0709 12:24:57.736080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.771471 (+1035391us) service_pool.cc:224] Handling call
0709 12:24:58.861642 (+ 90171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.864070 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.863090 13645 rpcz_store.cc:276] 0709 12:24:57.747634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.856797 (+1109163us) service_pool.cc:224] Handling call
0709 12:24:58.862351 (+  5554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.864665 13727 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:24:58.865058 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:24:58.866489 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67604us
I20260709 12:24:58.867365 13646 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:58.867419 13645 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
W20260709 12:24:58.865990 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49283.5us
I20260709 12:24:58.869952 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:58.870390 13796 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:58.870720 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.872377 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.873054 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.873106 13652 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:24:58.874954 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2877) took 1247 ms. Trace:
I20260709 12:24:58.875418 13652 rpcz_store.cc:276] 0709 12:24:57.628043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.860316 (+1232273us) service_pool.cc:224] Handling call
0709 12:24:58.874868 (+ 14552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.870918 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:58.878363 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.877225 22131 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.004s
I20260709 12:24:58.881225 22133 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:24:58.882377 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 164698us
I20260709 12:24:58.882855 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.857738 22043 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:24:58.859588 22023 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:58.860520 22397 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:58.887296 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.887876 13649 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:24:58.895045 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:58.896550 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.897840 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.898495 22239 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:58.898630 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.899199 22239 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:58.899609 13794 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:58.900558 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.900396 22239 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.901407 13794 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:58.881659 22280 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:58.901943 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:58.902465 22280 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.902889 22280 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:58.903649 22406 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:58.904094 22406 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.904443 22406 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:58.906826 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:58.907061 22146 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:24:58.907449 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.908277 13718 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:58.908849 22261 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:24:58.885862 13651 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:24:58.909415 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.909722 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.910292 13718 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:58.910805 13800 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:24:58.911229 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.911778 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.909492 22261 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:58.914713 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.915426 13795 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:24:58.915241 22261 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.916455 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.917110 13801 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:24:58.917519 22312 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.908855 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:58.918479 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1599) took 1491 ms. Trace:
I20260709 12:24:58.918607 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.919060 14417 rpcz_store.cc:276] 0709 12:24:57.427071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.427337 (+   266us) service_pool.cc:224] Handling call
0709 12:24:57.428121 (+   784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.753308 (+1325187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.918192 (+164884us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54004b6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:58.918395 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2600984960}
I20260709 12:24:58.919982 22321 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:24:58.920646 22321 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:58.913944 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:58.889725 22023 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:58.918049 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.922585 22412 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:24:58.924015 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:58.912065 22379 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:58.916606 22243 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:58.925693 14422 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
W20260709 12:24:58.926399 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.877661 22044 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:58.930214 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:58.930714 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.935278 13647 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:24:58.911932 13718 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:24:58.936906 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2874) took 1312 ms. Trace:
I20260709 12:24:58.934808 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.938498 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.937282 13647 rpcz_store.cc:276] 0709 12:24:57.624966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.863078 (+1238112us) service_pool.cc:224] Handling call
0709 12:24:58.936836 (+ 73758us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.947165 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.949091 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.950356 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:58.947705 13796 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:24:58.957352 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.930855 22044 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:58.955478 13794 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:24:58.962890 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:24:58.961807 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51293us
I20260709 12:24:58.967909 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:58.969480 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:58.970731 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:58.971052 22369 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:24:58.971807 21957 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.972223 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.972486 13647 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:24:58.972332 21957 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:58.973201 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2854) took 3421 ms. Trace:
I20260709 12:24:58.972864 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:24:58.971524 13649 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:24:59.053529 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1399) took 3513 ms. Trace:
I20260709 12:24:59.054059 13649 rpcz_store.cc:276] 0709 12:24:55.540504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.970746 (+3430242us) service_pool.cc:224] Handling call
0709 12:24:58.971190 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.052995 (+ 81805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.053413 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.054839 13726 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:24:59.056751 13649 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:24:59.057679 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4456) took 3468 ms. Trace:
I20260709 12:24:58.946620 22412 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:59.058941 22412 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:24:59.059512 22412 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:59.059787 22133 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:24:58.993885 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63776.6us
I20260709 12:24:59.062306 14425 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:24:59.062837 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:59.063961 22193 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:59.064239 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:59.065110 22193 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:24:58.937562 22450 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:24:59.066177 22450 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:24:59.066612 22450 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:24:58.918514 22339 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:24:58.991732 22273 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:24:59.072832 22273 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:59.005841 22249 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.970008 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.074399 22273 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.075412 22243 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:24:59.075147 22249 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:59.075901 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 106074us
I20260709 12:24:58.973672 13647 rpcz_store.cc:276] 0709 12:24:55.551598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.971745 (+3420147us) service_pool.cc:224] Handling call
0709 12:24:58.972197 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.972923 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.973109 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.002103 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3824) took 4359 ms. Trace:
I20260709 12:24:59.079550 13721 rpcz_store.cc:276] 0709 12:24:54.642327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.951113 (+1308786us) service_pool.cc:224] Handling call
0709 12:24:58.307568 (+2356455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.309174 (+  1606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.001868 (+692694us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b54003fded0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:59.002037 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5395089664}
W20260709 12:24:58.993202 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.075927 22243 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:59.010916 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:58.989893 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:59.087009 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:24:59.087474 13724 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:24:58.983559 22106 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:24:59.013391 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.005584 13643 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: [21.16-23.17]
I20260709 12:24:59.008877 22329 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:59.001202 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:59.023090 13648 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
W20260709 12:24:59.096165 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 165520us
I20260709 12:24:59.097913 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:59.027052 13799 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:59.028941 22376 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.027056 22331 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 12:24:59.100971 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2857) took 3505 ms. Trace:
I20260709 12:24:59.076030 22273 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:24:59.101816 22305 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:24:59.102802 22405 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:59.096737 13720 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:24:59.103010 22376 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.103503 22405 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.105479 22376 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:59.105695 22405 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:59.106240 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1398) took 3566 ms. Trace:
W20260709 12:24:59.030651 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70664.2us
I20260709 12:24:59.106654 13648 rpcz_store.cc:276] 0709 12:24:55.540195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.929643 (+3389448us) service_pool.cc:224] Handling call
0709 12:24:59.022823 (+ 93180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.105932 (+ 83109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.106157 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.101477 13647 rpcz_store.cc:276] 0709 12:24:55.595505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.078667 (+3483162us) service_pool.cc:224] Handling call
0709 12:24:59.079106 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.100502 (+ 21396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.100542 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.100632 (+    90us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:59.100680 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.100756 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.100902 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12949,"raft.queue_time_us":18292,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126}
I20260709 12:24:59.094501 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4453) took 3534 ms. Trace:
I20260709 12:24:59.108007 13643 rpcz_store.cc:276] 0709 12:24:55.559820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.004566 (+3444746us) service_pool.cc:224] Handling call
0709 12:24:59.005139 (+   573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:59.094096 (+ 88957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.094414 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:58.977025 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:58.980805 13650 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:24:59.031576 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45418.2us
I20260709 12:24:59.035555 13547 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 13, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 13 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.038651 13652 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
W20260709 12:24:59.038816 13541 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 234) took 59915 ms (client timeout 15000 ms). Trace:
I20260709 12:24:59.039392 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:24:59.040992 21957 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:59.042583 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:58.921933 22321 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.047892 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:59.002907 22044 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:59.051815 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:59.082047 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.088549 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:59.087010 13796 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:24:59.058092 13649 rpcz_store.cc:276] 0709 12:24:55.589083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.055329 (+3466246us) service_pool.cc:224] Handling call
0709 12:24:59.056390 (+  1061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.057303 (+   913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.057577 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.095503 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:24:59.093489 13793 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:24:59.026180 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4452) took 3466 ms. Trace:
I20260709 12:24:59.132438 13651 rpcz_store.cc:276] 0709 12:24:55.559575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.997235 (+3437660us) service_pool.cc:224] Handling call
0709 12:24:58.997724 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.997962 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.997986 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:58.998033 (+    47us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:24:58.998086 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.998140 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.026072 (+ 27932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.083856 22243 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:24:59.100798 22164 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:59.113230 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1397) took 3573 ms. Trace:
I20260709 12:24:59.111436 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4447) took 3553 ms. Trace:
I20260709 12:24:59.137622 13650 rpcz_store.cc:276] 0709 12:24:55.557976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.979904 (+3421928us) service_pool.cc:224] Handling call
0709 12:24:58.980382 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.111166 (+130784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.111379 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.121079 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.122891 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:59.134588 13726 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:24:59.141287 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.093195 22106 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:59.145190 13652 rpcz_store.cc:276] 0709 12:24:55.539851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.888500 (+3348649us) service_pool.cc:224] Handling call
0709 12:24:59.038370 (+149870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.112983 (+ 74613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.113177 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.143503 22211 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:24:59.148010 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 69843us
I20260709 12:24:59.156546 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.157536 13652 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: [22.14-23.15]
I20260709 12:24:59.158335 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2863) took 3132 ms. Trace:
I20260709 12:24:59.158288 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:59.158802 13652 rpcz_store.cc:276] 0709 12:24:56.026462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.156720 (+3130258us) service_pool.cc:224] Handling call
0709 12:24:59.157224 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:59.158029 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.158276 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.159586 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:24:59.074302 22023 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.160276 13643 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:59.160739 13798 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: []
I20260709 12:24:59.136883 22164 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.166177 22164 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:59.162319 22249 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:59.168255 13643 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 12:24:59.168993 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2870) took 1545 ms. Trace:
I20260709 12:24:59.169380 22362 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:24:59.169348 13643 rpcz_store.cc:276] 0709 12:24:57.623786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.167616 (+1543830us) service_pool.cc:224] Handling call
0709 12:24:59.168013 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.168687 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.168918 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.169898 22362 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.170316 22362 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:24:59.178328 22372 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.015s
I20260709 12:24:59.178717 13547 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 14 to 18, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.180596 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:59.181192 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 129489us
I20260709 12:24:59.181592 13649 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:59.180856 13547 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 19, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.166703 13648 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:59.161139 22211 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.182168 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.182636 22211 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:59.182744 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4458) took 3436 ms. Trace:
I20260709 12:24:59.183547 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.183327 13648 rpcz_store.cc:276] 0709 12:24:55.746047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.122022 (+3375975us) service_pool.cc:224] Handling call
0709 12:24:59.166525 (+ 44503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.182467 (+ 15942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.182683 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46592}
I20260709 12:24:59.184461 21957 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:24:59.185221 22404 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:24:59.185647 22404 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.166750 13647 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:24:59.143410 21953 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:24:59.186030 22404 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:59.190714 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4461) took 3426 ms. Trace:
I20260709 12:24:59.200512 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.201036 13647 rpcz_store.cc:276] 0709 12:24:55.761220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.137401 (+3376181us) service_pool.cc:224] Handling call
0709 12:24:59.166516 (+ 29115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.186926 (+ 20410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.187470 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19328}
I20260709 12:24:59.198119 13645 raft_consensus.cc:606] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.202165 13645 raft_consensus.cc:614] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:59.188719 22331 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:59.199698 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:59.200079 14420 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:24:59.206066 13650 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:24:59.200735 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:24:59.206382 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:59.120850 22280 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.209034 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4462) took 3420 ms. Trace:
I20260709 12:24:59.209573 13650 rpcz_store.cc:276] 0709 12:24:55.788344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.153013 (+3364669us) service_pool.cc:224] Handling call
0709 12:24:59.203364 (+ 50351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.208020 (+  4656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.208958 (+   938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.210949 13547 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:59.207046 13649 raft_consensus.cc:2376] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24. Re-sending same reply.
I20260709 12:24:59.214666 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.188719 22382 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:24:59.200263 22347 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:24:59.117645 13541 rpcz_store.cc:269] 0709 12:23:59.122919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.123315 (+   396us) service_pool.cc:224] Handling call
0709 12:24:59.038401 (+59915086us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:41.855087 (+     0us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f539762916c
0709 12:24:42.169876 (+314789us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:42.170618 (+   742us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:42.170685 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:24:42.170708 (+    23us) tablet.cc:662] Decoding operations
0709 12:24:42.172055 (+  1347us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:42.172182 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:42.172217 (+    35us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:24:42.192471 (+ 20254us) tablet.cc:701] Row locks acquired
0709 12:24:42.192528 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:24:42.192743 (+   215us) write_op.cc:270] Start()
0709 12:24:42.193034 (+   291us) write_op.cc:276] Timestamp: P: 1783599882192667 usec, L: 0
0709 12:24:42.193084 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:24:42.331669 (+138585us) log.cc:844] Serialized 4781 byte log entry
0709 12:24:42.811926 (+480257us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393befe28 00007f539762a581 00007f5397629d58 00007f539762e9f1 00007f539762e9a9 00007f539762e939 00007f539762e901 00007f539762db2d 0000000000388634 0000000000388479
0709 12:24:43.301997 (+490071us) op_driver.cc:464] REPLICATION: finished
0709 12:24:43.675364 (+373367us) write_op.cc:301] APPLY: starting
0709 12:24:43.692886 (+ 17522us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:44.142424 (+449538us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:44.143361 (+   937us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:44.599362 (+456001us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:44.615600 (+ 16238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 12:24:44.615650 (+    50us) write_op.cc:312] APPLY: finished
0709 12:24:44.616068 (+   418us) log.cc:844] Serialized 113 byte log entry
0709 12:24:45.718115 (+1102047us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b
0709 12:24:46.492063 (+773948us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:46.502766 (+ 10703us) write_op.cc:454] Released schema lock
0709 12:24:46.516599 (+ 13833us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:46.684930 (+168331us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:46.692229 (+  7299us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":979894,"reported_tablets":50,"spinlock_wait_cycles":37632,"child_traces":[["op",{"apply.queue_time_us":321084,"apply.run_cpu_time_us":73978,"apply.run_wall_time_us":2932911,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54383,"num_ops":12,"prepare.queue_time_us":944290,"prepare.run_cpu_time_us":25771,"prepare.run_wall_time_us":642325,"raft.queue_time_us":160337,"raft.run_cpu_time_us":13150,"raft.run_wall_time_us":298962,"replication_time_us":1089386,"spinlock_wait_cycles":76104064,"thread_start_us":3258733,"threads_started":5,"wal-append.queue_time_us":2306526,"wal-append.run_cpu_time_us":32987,"wal-append.run_wall_time_us":2719319}]]}
I20260709 12:24:59.191340 22396 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:59.265606 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:59.266908 13652 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:24:59.268496 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.275820 14419 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:24:59.278709 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:24:59.291707 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77128.8us
I20260709 12:24:59.264988 22305 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:59.295850 13647 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:24:59.301945 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2875) took 1676 ms. Trace:
I20260709 12:24:59.302960 13647 rpcz_store.cc:276] 0709 12:24:57.625643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.260283 (+1634640us) service_pool.cc:224] Handling call
0709 12:24:59.278323 (+ 18040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.301532 (+ 23209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.301851 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.197126 22450 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.272086 22396 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.272580 22347 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:24:59.321075 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 120648us
I20260709 12:24:59.297513 22405 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:24:59.316547 22372 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:24:59.331558 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.318226 22396 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:59.324312 22347 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:59.282590 22335 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:24:59.325558 13547 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 20, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 20 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.328804 22230 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:24:59.329857 22405 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.350637 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.348145 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4463) took 1057 ms. Trace:
I20260709 12:24:59.348218 13547 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.351939 13647 rpcz_store.cc:276] 0709 12:24:58.290769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.304001 (+1013232us) service_pool.cc:224] Handling call
0709 12:24:59.323554 (+ 19553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.331571 (+  8017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.331612 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.337633 (+  6021us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:24:59.347783 (+ 10150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.347891 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.348063 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.343148 22352 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:59.325380 22397 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:59.362747 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.364276 13725 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:24:59.349936 22405 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:59.331730 22406 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:59.373698 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 203087us
W20260709 12:24:59.370621 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.387548 14417 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:24:59.344599 22043 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.376689 13547 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.360651 22352 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:24:59.376957 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:59.364337 22335 consensus_queue.cc:1243] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:24:59.402233 14422 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:24:59.393309 13647 pending_rounds.cc:85] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 19
I20260709 12:24:59.404853 13647 pending_rounds.cc:107] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 29.20
I20260709 12:24:59.405241 13647 raft_consensus.cc:2889] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:59.406037 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57971us
I20260709 12:24:59.406493 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4465) took 1115 ms. Trace:
I20260709 12:24:59.408015 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.408639 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.374588 22239 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:24:59.406914 13650 rpcz_store.cc:276] 0709 12:24:58.291393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.365138 (+1073745us) service_pool.cc:224] Handling call
0709 12:24:59.388495 (+ 23357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.406069 (+ 17574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.406100 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.406153 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:59.406198 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.406298 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.406430 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.413205 22280 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:59.361723 22044 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.427471 22261 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:59.428095 22261 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:59.429502 22044 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:24:59.429186 22261 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.430763 22261 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:59.433570 22335 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:24:59.435416 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47876.3us
I20260709 12:24:59.401158 22352 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:24:59.474653 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.361335 22361 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.196s
I20260709 12:24:59.359262 22394 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.209s
I20260709 12:24:59.408298 21953 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:24:59.410471 22023 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:59.414845 22230 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:24:59.414740 22239 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:59.497460 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:59.531668 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2884) took 1756 ms. Trace:
I20260709 12:24:59.483788 13547 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 18, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 18 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:59.535473 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2871) took 1864 ms. Trace:
W20260709 12:24:59.537640 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61903.5us
I20260709 12:24:59.483189 22043 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:24:59.480049 21953 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:59.494043 22239 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:24:59.490722 22230 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:59.538684 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:24:59.475755 22412 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.532241 13652 rpcz_store.cc:276] 0709 12:24:57.775928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.277317 (+1501389us) service_pool.cc:224] Handling call
0709 12:24:59.302083 (+ 24766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.318777 (+ 16694us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:24:59.384415 (+ 65638us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.531283 (+146868us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.531337 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:24:59.531387 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.531459 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.531604 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53003,"prepare.queue_time_us":93945,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":436,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":11781,"thread_start_us":98907,"threads_started":2}
W20260709 12:24:59.537226 22461 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:24:59.550228 13793 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:24:59.546916 13547 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:24:59.551775 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45268.3us
W20260709 12:24:59.552824 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71590.5us
I20260709 12:24:59.554554 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:59.557646 22466 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:24:59.545910 22379 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:24:59.546386 22230 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:24:59.567378 22463 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:24:59.570398 22023 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:24:59.572759 22304 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:24:59.579942 22469 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.145s	user 0.003s	sys 0.000s
I20260709 12:24:59.583245 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 3993) took 3560 ms. Trace:
W20260709 12:24:59.583621 22462 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.286s	user 0.001s	sys 0.007s
I20260709 12:24:59.586352 22476 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:24:59.587980 22406 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:59.590334 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:59.594106 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:24:59.570684 22465 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:24:59.596432 13725 rpcz_store.cc:276] 0709 12:24:56.022633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.859951 (+837318us) service_pool.cc:224] Handling call
0709 12:24:57.138865 (+278914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.376808 (+2237943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.582963 (+206155us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54001a5754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:59.583181 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4771088256}
I20260709 12:24:59.604656 13547 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 16, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.554356 22450 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:59.608551 13721 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:24:59.633455 13649 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:59.620841 13650 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:24:59.614758 14417 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:24:59.648818 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:24:59.649821 13793 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
W20260709 12:24:59.622088 22273 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.005s
I20260709 12:24:59.611879 13547 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 17, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 17 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.631726 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:59.656890 13725 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:24:59.657546 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1409) took 1087 ms. Trace:
I20260709 12:24:59.658494 13649 rpcz_store.cc:276] 0709 12:24:58.559819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.632883 (+1073064us) service_pool.cc:224] Handling call
0709 12:24:59.633231 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.646524 (+ 13293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.646749 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.660038 13547 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 19, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 19 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.661821 14419 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:24:59.662165 22367 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:24:59.662909 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68909.7us
I20260709 12:24:59.663750 22368 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:24:59.626734 22472 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 12:24:59.663852 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.637281 22461 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:24:59.652532 22329 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.002s
I20260709 12:24:59.685093 13547 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:24:59.673393 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 84962.5us
I20260709 12:24:59.675060 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:59.704404 13644 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:24:59.610424 22335 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:24:59.677241 22367 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:24:59.715718 22164 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 12:24:59.716060 22367 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:24:59.717249 22367 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.722791 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:24:59.716328 22164 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.724050 13718 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:59.725692 22164 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:59.728765 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
W20260709 12:24:59.731269 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.681576 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:24:59.629292 22397 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:24:59.735268 13649 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:24:59.736006 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4477) took 1174 ms. Trace:
I20260709 12:24:59.736172 22335 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:24:59.736397 13649 rpcz_store.cc:276] 0709 12:24:58.561701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.699197 (+1137496us) service_pool.cc:224] Handling call
0709 12:24:59.734870 (+ 35673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.735730 (+   860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.735944 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:59.738399 22394 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:24:59.741043 22404 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:24:59.675155 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 136599us
I20260709 12:24:59.707803 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.705055 22305 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.736748 22335 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:59.760730 13649 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:24:59.742600 13645 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:24:59.699606 13721 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:24:59.743000 13547 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 24, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 24 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:59.598603 22243 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.762843 13547 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 20, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 20 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.614557 21957 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.634482 22106 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.765290 22347 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.700516 22412 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:24:59.773844 22211 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.005s
I20260709 12:24:59.774104 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:59.785318 13645 raft_consensus.cc:606] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.785651 13645 raft_consensus.cc:614] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:59.774555 14424 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:24:59.774652 21957 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.775914 22106 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:24:59.800205 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92500.5us
I20260709 12:24:59.537154 13648 rpcz_store.cc:276] 0709 12:24:57.624070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.212596 (+1588526us) service_pool.cc:224] Handling call
0709 12:24:59.274657 (+ 62061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.334390 (+ 59733us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:24:59.396206 (+ 61816us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.487524 (+ 91318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.488156 (+   632us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:24:59.488244 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.488365 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.488618 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1481,"prepare.queue_time_us":78630,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":29,"raft.queue_time_us":61696,"thread_start_us":118182,"threads_started":2}
I20260709 12:24:59.778708 22385 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:24:59.778497 22335 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.779453 22487 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:59.779953 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:24:59.759307 22362 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.588s	user 0.013s	sys 0.000s
I20260709 12:24:59.784052 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4472) took 1403 ms. Trace:
I20260709 12:24:59.806141 13650 rpcz_store.cc:276] 0709 12:24:58.380299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.476109 (+1095810us) service_pool.cc:224] Handling call
0709 12:24:59.496286 (+ 20177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.655794 (+159508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.783710 (+127916us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400197b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:24:59.783973 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285964032}
I20260709 12:24:59.799933 13719 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:24:59.706635 22405 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:59.819551 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49281.5us
I20260709 12:24:59.768314 22305 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.824239 13646 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:24:59.827145 13547 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 13, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 13 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:24:59.827572 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 222232us
I20260709 12:24:59.808213 22487 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:24:59.838181 13649 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:24:59.840170 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:59.843037 13547 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 21. New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.855789 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:59.803074 22477 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:24:59.803809 22211 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.868633 22193 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:24:59.868811 22249 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:24:59.888829 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 80307.8us
I20260709 12:24:59.889734 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.805760 22352 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.899776 13792 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:24:59.895972 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:59.894155 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:59.818580 22362 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:24:59.870728 22372 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:24:59.897397 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:24:59.897330 13547 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.932973 22164 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:24:59.933669 22164 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:24:59.933281 13547 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 21, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.934933 22164 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:24:59.935187 13547 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.936358 22475 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:24:59.936627 13547 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 14 to 17, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 17 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:24:59.938476 22474 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:24:59.898900 22477 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:24:59.854391 22487 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:24:59.900801 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.904906 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1408) took 1416 ms. Trace:
W20260709 12:24:59.925684 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.940109 13648 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:24:59.941372 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:59.943084 14424 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:24:59.942081 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1812) took 1509 ms. Trace:
I20260709 12:24:59.938123 13547 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:24:59.955093 13727 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:24:59.953394 13646 rpcz_store.cc:276] 0709 12:24:58.491967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.870058 (+378091us) service_pool.cc:224] Handling call
0709 12:24:59.904693 (+1034635us) spinlock_profiling.cc:243] Waited 953 ms on lock 0x7b54001b5654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:59.904824 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190911744}
I20260709 12:24:59.959976 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:59.850281 22239 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:24:59.963605 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62901.9us
I20260709 12:24:59.970350 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:24:59.963967 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 192780us
I20260709 12:24:59.954087 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4475) took 1422 ms. Trace:
I20260709 12:24:59.971500 13646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "a4188193a10c4d50a555d76ea85c60b7"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:24:59.972236 13646 raft_consensus.cc:606] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Received request to transfer leadership to TS a4188193a10c4d50a555d76ea85c60b7
I20260709 12:24:59.972821 13646 raft_consensus.cc:614] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:59.887089 22379 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:24:59.903441 22296 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:24:59.976087 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:24:59.924865 22385 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:24:59.938984 22476 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:24:59.888669 22339 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:24:59.949550 22335 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:24:59.879694 22321 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.087s
W20260709 12:24:59.982335 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.973237 13799 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:24:59.974107 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:24:59.955374 13723 rpcz_store.cc:276] 0709 12:24:58.432888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.450691 (+ 17803us) service_pool.cc:224] Handling call
0709 12:24:58.755324 (+304633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.782515 (+ 27191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.782568 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.783193 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.961433 (+178240us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:24:58.961549 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.961622 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.656549 (+694927us) raft_consensus.cc:1685] finished
0709 12:24:59.656574 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.904208 (+247634us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54001a5250. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:24:59.941985 (+ 37777us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5589,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":142,"spinlock_wait_cycles":495453952,"thread_start_us":177780,"threads_started":1,"wal-append.queue_time_us":363143}
W20260709 12:24:59.985699 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 149385us
I20260709 12:24:59.969717 22249 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:24:59.900116 22450 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:24:59.894888 22478 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.078s
W20260709 12:24:59.990401 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86437.4us
I20260709 12:24:59.965103 22331 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:24:59.992241 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:24:59.977957 13547 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 26, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 26 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:25:00.003425 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.004215 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103277us
I20260709 12:25:00.001405 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:24:59.979693 22385 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.981617 22329 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:00.028054 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:00.028812 13646 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:24:59.998368 22304 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:24:59.992842 22474 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:00.043200 13547 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:00.042567 13645 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:00.045428 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:00.046232 13645 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:00.004436 22372 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:00.024609 22385 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:00.048043 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107243us
W20260709 12:25:00.071965 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81350.3us
I20260709 12:25:00.072986 13645 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:00.029502 22498 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:00.039044 22304 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:00.070358 13547 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:00.027938 22329 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.035243 22193 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:00.047106 22347 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.047279 22243 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.038774 22474 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:00.082041 13645 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:00.001905 21953 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.082610 22304 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:00.051885 22372 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:00.094802 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62413.7us
I20260709 12:25:00.090444 13646 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:00.099807 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.077200 22498 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:00.083567 22329 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:00.090942 22405 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.073916 22406 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:25:00.099663 13547 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 16, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 16 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:00.076581 22476 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:25:00.114727 13646 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:25:00.094560 22502 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:00.121141 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.119786 13646 pending_rounds.cc:85] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 10
I20260709 12:25:00.107172 22273 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.113691 22243 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.108074 22498 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:00.080857 22379 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.162604 22305 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.163376 22305 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:00.163753 22305 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:00.174093 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.109820 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:00.103041 22478 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
W20260709 12:25:00.182188 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72467.6us
I20260709 12:25:00.145620 13547 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:00.121783 22193 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.128939 22502 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.107654 22249 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.142201 22352 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.142418 22243 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:00.152657 21957 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.146697 22362 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.116631 22211 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:00.104049 22339 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.031s
W20260709 12:25:00.162431 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46612.1us
I20260709 12:25:00.125543 13646 pending_rounds.cc:107] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:25:00.165135 22367 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:00.107322 22493 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.117209 22280 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:00.184142 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:00.217725 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.217497 22502 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:00.145740 22500 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:25:00.182631 22489 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 12:25:00.184126 22305 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:00.219189 22496 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
W20260709 12:25:00.219048 22495 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:25:00.227290 13646 raft_consensus.cc:2889] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:00.218722 22503 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:25:00.251907 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 277877us
I20260709 12:25:00.225842 13547 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 9 to 12, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 12 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:00.218564 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 194905us
W20260709 12:25:00.233036 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.235689 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:00.255960 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4484) took 1068 ms. Trace:
I20260709 12:25:00.143807 22273 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.223372 22243 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:00.242705 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 121662us
I20260709 12:25:00.253841 13547 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 15, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:00.225008 22239 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.233319 22280 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.237332 22493 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:00.251207 22488 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.199s	user 0.000s	sys 0.002s
I20260709 12:25:00.266566 22280 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:00.270426 22517 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:00.271235 22517 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:00.272192 22517 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.273057 22273 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:00.279875 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:00.182065 22474 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:00.256589 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.257568 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:00.260814 13547 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 14 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 16 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:00.290542 14422 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:25:00.256928 22503 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.256404 13646 rpcz_store.cc:276] 0709 12:24:59.187360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.108677 (+921317us) service_pool.cc:224] Handling call
0709 12:25:00.109106 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.255644 (+146538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.255886 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:00.257177 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73124.4us
I20260709 12:25:00.260793 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.240470 22412 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:25:00.262897 22396 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.913s	user 0.014s	sys 0.000s
W20260709 12:25:00.281195 22230 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.001s
I20260709 12:25:00.253129 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:00.306476 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:00.339574 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1414) took 1379 ms. Trace:
I20260709 12:25:00.337651 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4482) took 1159 ms. Trace:
W20260709 12:25:00.318084 22261 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.887s	user 0.015s	sys 0.000s
I20260709 12:25:00.318691 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:00.301376 22329 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.323589 22491 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:00.324862 21953 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:24:59.971728 13652 rpcz_store.cc:276] 0709 12:24:58.531771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.549000 (+1017229us) service_pool.cc:224] Handling call
0709 12:24:59.559434 (+ 10434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.565056 (+  5622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.565083 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.651559 (+ 86476us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.880791 (+229232us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:24:59.953413 (+ 72622us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.953703 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.953789 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.953837 (+    48us) raft_consensus.cc:1685] finished
0709 12:24:59.953852 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.954011 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87337,"raft.queue_time_us":155,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3723,"thread_start_us":209030,"threads_started":1,"wal-append.queue_time_us":201586}
I20260709 12:25:00.332242 22474 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.354825 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:00.333292 22522 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:00.338912 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:00.309063 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:00.313165 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.341463 13648 rpcz_store.cc:276] 0709 12:24:59.177863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.966154 (+788291us) service_pool.cc:224] Handling call
0709 12:24:59.992690 (+ 26536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.076357 (+ 83667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.076394 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.098688 (+ 22294us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.243057 (+144369us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:00.243104 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.243180 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.337397 (+ 94217us) raft_consensus.cc:1685] finished
0709 12:25:00.337422 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.337579 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25464,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":14160,"thread_start_us":98672,"threads_started":1,"wal-append.queue_time_us":202975}
I20260709 12:25:00.341265 13650 rpcz_store.cc:276] 0709 12:24:58.960547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.828022 (+867475us) service_pool.cc:224] Handling call
0709 12:24:59.869115 (+ 41093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.879164 (+ 10049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.879204 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.951463 (+ 72259us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.240781 (+289318us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:00.240828 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.240913 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.339169 (+ 98256us) raft_consensus.cc:1685] finished
0709 12:25:00.339191 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.339503 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1177,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":19811,"thread_start_us":221818,"threads_started":1,"wal-append.queue_time_us":342375}
I20260709 12:25:00.358601 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:00.283102 22385 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.349153 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:00.314801 22503 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:00.359203 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:00.344651 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 91636.4us
I20260709 12:25:00.366611 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:00.340966 22404 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:00.380113 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67078.5us
W20260709 12:25:00.382017 13644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1405) took 2059 ms (client timeout 1667 ms). Trace:
W20260709 12:25:00.382498 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43670.4us
I20260709 12:25:00.369225 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4464) took 2068 ms. Trace:
I20260709 12:25:00.345803 22412 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:00.382429 13652 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:00.349509 22491 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:00.362929 13648 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:00.384033 13643 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:25:00.382470 13644 rpcz_store.cc:269] 0709 12:24:58.323039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.332973 (+  9934us) service_pool.cc:224] Handling call
0709 12:25:00.381800 (+2048827us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:00.381941 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137274240}
I20260709 12:25:00.388756 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:00.383385 13647 rpcz_store.cc:276] 0709 12:24:58.291093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.353902 (+1062809us) service_pool.cc:224] Handling call
0709 12:24:59.361914 (+  8012us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:59.531254 (+169340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.531289 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:59.542192 (+ 10903us) log.cc:844] Serialized 78 byte log entry
0709 12:24:59.804051 (+261859us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:24:59.820236 (+ 16185us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.820495 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.820577 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.319913 (+499336us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b18000abe20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:00.353844 (+ 33931us) raft_consensus.cc:1685] finished
0709 12:25:00.359742 (+  5898us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.359942 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27716,"prepare.queue_time_us":15964,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":3611,"raft.queue_time_us":63204,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":8206,"spinlock_wait_cycles":10762112,"thread_start_us":214920,"threads_started":1,"wal-append.queue_time_us":390002}
I20260709 12:25:00.390049 13650 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: [27.21-27.21]
I20260709 12:25:00.381704 13719 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:00.392575 22501 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:00.411429 13646 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:00.341065 22396 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.378708 22522 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:00.434849 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.436437 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.416064 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1416) took 1146 ms. Trace:
I20260709 12:25:00.418375 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:00.412295 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4485) took 1224 ms. Trace:
I20260709 12:25:00.438283 13648 rpcz_store.cc:276] 0709 12:24:59.187825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.358050 (+1170225us) service_pool.cc:224] Handling call
0709 12:25:00.358427 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.404712 (+ 46285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.412228 (+  7516us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.439754 13648 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:25:00.362673 22518 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:00.365373 22329 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.428210 22385 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.343215 22230 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.429970 22493 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.431402 13647 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:00.348203 22394 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.056s
I20260709 12:25:00.367538 22106 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:00.433596 13644 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:00.465603 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4487) took 1089 ms. Trace:
I20260709 12:25:00.466089 13644 rpcz_store.cc:276] 0709 12:24:59.376242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.405383 (+1029141us) service_pool.cc:224] Handling call
0709 12:25:00.405841 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.465215 (+ 59374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.465516 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.463364 22239 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.353693 22280 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.422155 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:00.442520 13651 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:00.467600 22239 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.470155 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4483) took 1292 ms. Trace:
I20260709 12:25:00.470664 13651 rpcz_store.cc:276] 0709 12:24:59.178132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.259784 (+ 81652us) service_pool.cc:224] Handling call
0709 12:25:00.469986 (+1210202us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001b5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:00.470095 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2304509696}
I20260709 12:25:00.437894 13650 rpcz_store.cc:276] 0709 12:24:59.269963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.389191 (+1119228us) service_pool.cc:224] Handling call
0709 12:25:00.389709 (+   518us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.415624 (+ 25915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.415970 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:00.446475 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:00.446889 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 201457us
W20260709 12:25:00.472615 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50563us
I20260709 12:25:00.472908 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.395565 22261 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:00.395649 22368 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.472090 13652 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:00.446388 22331 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:00.477425 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:00.479158 13651 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:25:00.481310 13649 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 19.
I20260709 12:25:00.481359 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:00.482344 14417 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:00.487186 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:00.490974 22515 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:25:00.492242 22515 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:00.492637 22515 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:00.493433 22515 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.494885 13647 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:25:00.495030 22230 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.491932 22304 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.478758 22339 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:00.495267 22493 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:00.488852 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:00.499804 22239 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.500599 22352 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:00.505844 13651 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:25:00.476981 22518 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:00.542557 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57970.3us
W20260709 12:25:00.542989 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54231.2us
I20260709 12:25:00.543622 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.513983 13650 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:00.544157 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:00.547395 13645 raft_consensus.cc:606] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.547900 13645 raft_consensus.cc:614] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:00.506672 22321 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:00.510886 22339 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.566681 22339 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.555266 22230 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:00.479732 22368 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.561538 22304 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.573941 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:00.575949 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.489969 22529 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.059s
I20260709 12:25:00.510933 22493 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:00.604249 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49264.5us
I20260709 12:25:00.546618 22352 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:00.552249 22402 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:00.572399 22515 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.613576 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:00.547395 22522 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.570549 22230 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.506778 22305 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.623991 22368 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.638165 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:00.607699 22493 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.646286 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.502243 22491 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.612641 22352 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:00.627439 22526 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:00.513221 22412 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.640265 22230 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.659587 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:00.642599 22394 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:00.653309 22321 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:00.655427 22379 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:00.655426 22402 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:00.657142 22526 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:00.706506 22526 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:00.709108 22531 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.258s	user 0.000s	sys 0.004s
I20260709 12:25:00.668865 22211 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:00.715775 22211 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.717767 22211 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:00.720645 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.722046 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4467) took 2429 ms. Trace:
I20260709 12:25:00.722698 22479 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:00.723321 22479 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.679917 22478 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:00.725585 22479 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.730399 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1421) took 1189 ms. Trace:
I20260709 12:25:00.731200 13795 rpcz_store.cc:276] 0709 12:24:59.541197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.541407 (+   210us) service_pool.cc:224] Handling call
0709 12:24:59.541740 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.368403 (+826663us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:00.368434 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.368481 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:00.368515 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.704368 (+335853us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800183220. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:25:00.705559 (+  1191us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b540012b454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:00.705622 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.705802 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18892,"raft.queue_time_us":19133,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":31900,"spinlock_wait_cycles":1881287424}
I20260709 12:25:00.670876 22396 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:00.735889 22498 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.005s
I20260709 12:25:00.736372 22498 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.667661 22321 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:00.738792 22321 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.743484 22507 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:00.744102 22507 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:00.746651 22396 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:00.747844 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83296.8us
I20260709 12:25:00.747474 22396 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.748709 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:00.749609 22404 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:00.749960 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.750733 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:00.722494 13643 rpcz_store.cc:276] 0709 12:24:58.292177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.366012 (+1073835us) service_pool.cc:224] Handling call
0709 12:25:00.383594 (+1017582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.404800 (+ 21206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.721787 (+316987us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54002e4ad0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:00.721948 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290568576}
I20260709 12:25:00.752014 22396 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:00.753911 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:00.756986 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:00.757823 22502 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 12:25:00.758335 22502 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.759660 14425 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:00.745200 22507 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.762415 22507 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:00.766218 22406 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:00.766726 22406 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:00.766816 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:00.767802 22406 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.750005 22404 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.770025 22404 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:00.775624 22394 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:00.776245 22394 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.681813 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:00.685336 22491 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:00.665443 22518 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.686681 22164 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:25:00.692353 22487 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.719s	user 0.006s	sys 0.007s
I20260709 12:25:00.692734 22339 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.693023 22230 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.670322 22379 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.698441 22519 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.701678 22305 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.677198 22402 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:00.778057 22273 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.806892 13801 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:25:00.780781 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.808004 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.808876 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.782279 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:00.810950 13800 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:00.792452 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:00.733489 22372 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.784340 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:00.797905 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 116174us
I20260709 12:25:00.800757 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:00.814356 13643 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:25:00.802947 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:00.801261 13645 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 20. (term mismatch)
I20260709 12:25:00.815435 13645 pending_rounds.cc:85] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 19
I20260709 12:25:00.815794 13645 pending_rounds.cc:107] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 25.20
I20260709 12:25:00.816103 13645 raft_consensus.cc:2889] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:00.784266 22164 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:00.818080 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:00.823076 13797 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:00.824837 22339 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:00.803462 22379 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:00.802037 22131 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:25:00.793874 22362 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:25:00.830683 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:00.800168 22499 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:00.802678 22211 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.834945 22211 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:00.834753 22131 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:00.836452 22498 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.838376 22475 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:00.840207 22501 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:00.845438 22493 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.783845 22243 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:00.848934 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54580.6us
I20260709 12:25:00.831902 13650 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 20: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:25:00.857487 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48869.2us
I20260709 12:25:00.863930 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:00.883073 13646 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 17: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:00.807140 22396 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:00.904351 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:00.798228 22397 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:25:00.909422 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:00.798743 22487 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.924691 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:00.848601 22335 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.096s
I20260709 12:25:00.848875 22273 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:00.807302 22519 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.900755 22474 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:25:00.936424 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:00.969779 22280 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:00.971763 22499 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:00.973347 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:00.931210 22478 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:25:00.974974 22478 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:00.975412 22478 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:00.976303 22478 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:00.979161 13798 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:00.973232 22503 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.002s
W20260709 12:25:00.987653 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.974155 13794 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:00.987275 22402 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:00.993360 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:00.996264 13798 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:00.977617 22478 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:00.994916 13795 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:01.010993 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86374.8us
I20260709 12:25:01.014475 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:01.017784 13800 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
W20260709 12:25:01.022974 14390 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.064s	sys 0.117s
I20260709 12:25:01.032752 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.033572 13799 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:01.034746 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:01.032734 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.037526 13794 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:01.038609 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:01.039665 14390 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.064s	sys 0.117s
W20260709 12:25:01.040520 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 605763us
I20260709 12:25:01.044312 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:01.046706 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4493) took 1050 ms. Trace:
I20260709 12:25:01.047331 13648 rpcz_store.cc:276] 0709 12:24:59.996734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.459782 (+463048us) service_pool.cc:224] Handling call
0709 12:25:00.460259 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.464449 (+  4190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.464487 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:00.478017 (+ 13530us) log.cc:844] Serialized 26 byte log entry
0709 12:25:00.664217 (+186200us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:00.664265 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.664342 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.046373 (+382031us) raft_consensus.cc:1685] finished
0709 12:25:01.046396 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.046608 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3693,"raft.queue_time_us":14998,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6040,"thread_start_us":149804,"threads_started":1,"wal-append.queue_time_us":409201}
I20260709 12:25:00.992520 22503 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.062585 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:00.986568 22164 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.067225 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:01.075992 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:01.081755 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.037923 22402 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:01.077167 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:01.089906 13794 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:01.091298 13797 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
W20260709 12:25:01.040403 22517 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.048s	sys 0.056s
I20260709 12:25:01.055524 22515 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:01.096832 14424 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:25:01.094584 13799 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:25:01.055435 22280 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:01.051789 22368 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:01.092294 22537 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:01.125044 22537 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.125581 22537 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:01.068349 22305 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:01.126832 22379 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.127844 22404 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.129352 22379 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.074219 22397 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:25:01.103094 22362 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:25:01.045756 22321 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.130869 22397 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:01.108580 22304 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:01.091951 22230 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:01.131826 22503 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.110945 22402 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.112846 22526 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.114491 22243 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:01.156070 22404 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.156637 22406 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:01.132298 22230 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:25:01.114595 22502 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.088780 22492 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:01.099833 22493 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.131249 22397 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:01.175071 22397 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.176620 22397 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:01.136863 13793 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:01.179028 22526 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.131851 22304 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:01.117674 22394 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.180729 22304 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.152350 22532 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:01.130048 22379 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 12:25:01.099798 22517 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.049s	sys 0.056s
I20260709 12:25:01.182682 22519 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:01.183305 22379 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:01.183717 22379 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:01.184594 22379 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.185055 22321 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:01.161264 22487 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.136147 22515 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.163457 22273 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.089036 22522 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:25:01.171274 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.172791 13798 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:25:01.172139 22492 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.150473 22352 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.190996 22368 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:01.204129 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.216351 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.201714 22515 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.224195 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:01.225253 22164 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:01.187222 22532 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:01.182649 22394 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:01.231096 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 259729us
I20260709 12:25:01.229228 14424 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:01.232818 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:01.237787 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:01.242965 14417 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:25:01.251206 14419 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:01.227241 22379 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:01.225255 22515 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:01.271529 14420 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:01.201310 22368 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:01.275153 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 475659us
I20260709 12:25:01.284363 22552 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:01.277259 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:01.286558 22532 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.265241 22475 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:25:01.288021 22532 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:01.289847 22394 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:25:01.291694 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.292387 22261 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:01.292887 22478 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:01.293490 22478 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:01.295063 22352 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.214944 22522 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.301120 22522 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:01.303134 22368 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.284226 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:01.310168 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.310714 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:01.310820 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 221294us
I20260709 12:25:01.312332 13793 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:01.315797 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:01.313644 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 236612us
W20260709 12:25:01.316834 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.317826 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:01.322033 13692 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.009s	sys 0.004s
I20260709 12:25:01.322909 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:01.323431 13692 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.009s	sys 0.004s
I20260709 12:25:01.322288 13798 pending_rounds.cc:85] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 14
I20260709 12:25:01.324771 13798 pending_rounds.cc:107] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:25:01.325035 13798 raft_consensus.cc:2889] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:01.326674 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:01.319177 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:01.333050 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:01.337347 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:01.335958 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 203673us
I20260709 12:25:01.335440 14422 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:01.304812 22492 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:01.345955 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:01.337076 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:01.356092 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.356722 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:01.362028 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.321597 22368 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:01.375553 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:01.362843 13649 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:01.359220 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.381423 13727 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:01.381810 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:01.366493 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.383062 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:01.338423 22280 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:01.384516 13723 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:01.367638 13725 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:01.367437 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:01.386066 13722 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:01.314896 22522 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.046s
I20260709 12:25:01.389416 13724 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:01.390188 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:01.311789 22538 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.069s
I20260709 12:25:01.372591 22519 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.415548 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.418421 13799 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:01.414871 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.410487 13722 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:01.419330 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.399627 13726 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:25:01.420401 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:01.409715 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.319839 22555 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.077s
I20260709 12:25:01.424465 13722 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:01.410220 14422 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:01.425174 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.426430 13799 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:01.419916 13645 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:01.428371 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.429852 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.430641 13794 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:01.431700 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:01.434996 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.436426 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:01.437536 13725 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 19: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:01.438052 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.441363 13801 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:01.412312 22559 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.444648 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.452693 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:01.447849 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5506431660001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 12:25:01.448556 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.450404 13727 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
W20260709 12:25:01.451377 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 89513.8us
I20260709 12:25:01.463603 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.397074 22397 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:25:01.462641 13645 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:01.410535 22280 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.470219 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.471550 13646 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:01.388484 22537 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:01.463286 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5497431660000984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 12:25:01.487073 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5209431660000519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:25:01.493480 13693 connection.cc:256] RPC call timeout handler was delayed by -2.5056244130000778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3073s for a response
W20260709 12:25:01.489897 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 170754us
I20260709 12:25:01.424328 22519 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.380291 22478 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:01.500003 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.494159 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4648782829999618s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.48730000000000007s for a response
W20260709 12:25:01.508035 22548 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:01.506978 13648 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:01.498567 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:01.523151 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:01.526059 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.531189 13724 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:01.532713 22537 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:01.536818 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.547048 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:01.550125 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:01.492040 22557 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:01.580564 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.584129 13719 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:01.562333 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.582502 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.590840 13649 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:01.591014 13724 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:01.557310 22537 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.461706 22559 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.538456 22529 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:25:01.594614 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90333.8us
I20260709 12:25:01.607084 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:01.615978 13795 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:01.616755 13794 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
W20260709 12:25:01.632804 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.611232 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:01.602459 13726 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 12:25:01.640913 13726 pending_rounds.cc:85] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 9
I20260709 12:25:01.641316 13726 pending_rounds.cc:107] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:25:01.641623 13726 raft_consensus.cc:2889] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:01.626432 14420 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:01.682377 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:01.692649 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.693645 14422 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:01.694155 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:01.694433 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 195922us
I20260709 12:25:01.694996 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:01.696030 13721 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:01.696096 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:01.602766 22541 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:25:01.700695 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:01.595885 22563 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:25:01.596379 22560 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:25:01.713268 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.560s (SENT)
I20260709 12:25:01.713981 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:01.718641 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:01.719385 13798 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:01.723628 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.723651 13693 connection.cc:256] RPC call timeout handler was delayed by -2.4318431659999078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260709 12:25:01.727221 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.728858 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.728698 13690 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:01.720978 22574 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:01.731048 13719 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 12:25:01.733489 13798 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:01.606470 22559 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.599710 22519 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:01.626763 22557 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:01.747200 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:01.778105 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48534.1us
I20260709 12:25:01.777765 13649 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:01.779956 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.608599 22402 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.755429 13721 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:01.780802 13721 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:01.765372 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.772456 13720 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:01.595377 22537 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:01.749080 22164 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.001s
I20260709 12:25:01.789988 13719 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:25:01.792011 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78801.8us
W20260709 12:25:01.794342 13693 connection.cc:256] RPC call timeout handler was delayed by -2.40844316599987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1882s for a response
I20260709 12:25:01.820922 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.744302 22529 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.823175 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:01.831845 13795 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:25:01.828008 13649 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:01.834654 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.835431 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:01.767602 22405 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:01.767006 22522 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.776333 22492 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:01.820879 22560 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:25:01.867225 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.679s (SENT)
W20260709 12:25:01.846673 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.798667 22304 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:01.867142 13793 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:01.794118 22402 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.861687 13721 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:01.843070 22561 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.303s	user 0.003s	sys 0.000s
I20260709 12:25:01.842558 22397 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:01.839773 22566 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:01.919029 13693 connection.cc:256] RPC call timeout handler was delayed by -2.3715244129998609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32220000000000004s for a response
W20260709 12:25:01.915310 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.919483 13693 connection.cc:256] RPC call timeout handler was delayed by -2.34632441299982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.325s for a response
I20260709 12:25:01.831909 22574 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:01.917354 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.923668 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:01.918800 22397 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:01.926015 13721 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:01.874995 22492 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.911495 22405 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:01.943400 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1987758239997675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:25:01.944221 13693 connection.cc:256] RPC call timeout handler was delayed by -2.16763866600013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
I20260709 12:25:01.853184 22529 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:01.934132 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:25:01.953716 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:01.959270 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:01.959852 13724 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:01.926748 22567 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:01.961092 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.961247 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.935190 22397 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:01.964573 13718 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:01.941635 22492 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:01.874142 22522 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:01.967350 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:01.969530 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4494) took 1972 ms. Trace:
I20260709 12:25:01.971169 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.972010 13649 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:01.968171 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:01.969956 13644 rpcz_store.cc:276] 0709 12:24:59.997135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.467270 (+470135us) service_pool.cc:224] Handling call
0709 12:25:00.467668 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.634780 (+167112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.634815 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.651754 (+ 16939us) log.cc:844] Serialized 52 byte log entry
0709 12:25:01.091037 (+439283us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:01.185917 (+ 94880us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.186251 (+   334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.215200 (+ 28949us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b18002e6780. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:25:01.261448 (+ 46248us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400095954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:01.261511 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.969275 (+707764us) raft_consensus.cc:1685] finished
0709 12:25:01.969300 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.969464 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45068,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":6642,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":14671,"spinlock_wait_cycles":377348096,"thread_start_us":402101,"threads_started":1,"wal-append.queue_time_us":690421}
I20260709 12:25:01.977476 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:01.978165 13644 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:01.954825 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1305134300000645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
W20260709 12:25:01.979203 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0871034030001283s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
W20260709 12:25:01.981703 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0295431659997121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2303s for a response
W20260709 12:25:01.982272 13693 connection.cc:256] RPC call timeout handler was delayed by -1.840940737999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3004s for a response
W20260709 12:25:01.982730 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8232068029997208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2528s for a response
W20260709 12:25:01.983510 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5205407379999087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.336s for a response
W20260709 12:25:01.990438 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2303431659997841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3191s for a response
I20260709 12:25:01.973421 13652 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:01.996694 22537 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.961786 22576 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.962504 22475 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
W20260709 12:25:01.999588 22406 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.004s
I20260709 12:25:02.001497 22529 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:01.974853 22567 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:02.004453 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.023644 13798 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:25:02.005038 22579 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:25:02.007601 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.007401 13649 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:02.029874 13721 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:25:02.030426 13721 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:25:02.030792 13721 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:25:02.031124 13721 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:02.009557 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1558220849997269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3168s for a response
I20260709 12:25:02.010349 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:02.014899 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.925964 22566 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:02.041857 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 168680us
I20260709 12:25:02.046363 13652 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:02.048082 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.048861 13652 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:02.049615 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:02.050964 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.051823 13652 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:02.053205 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.054435 13652 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
I20260709 12:25:02.056160 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:02.055545 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:02.058229 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:02.055620 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:02.061530 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:02.065251 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:01.946002 22557 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.079896 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.080380 13727 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:25:02.080897 13721 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:25:01.968146 22574 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.082824 13644 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
W20260709 12:25:02.073352 22582 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:25:02.087244 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.022744 22576 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.084175 13652 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:02.101351 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.099025 13721 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:02.099992 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.106851 13725 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:02.108191 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.109416 13721 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:02.111517 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133987us
I20260709 12:25:02.114683 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.115581 13724 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:02.027303 22405 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.119136 13649 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:02.131707 22304 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:02.132898 22397 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.145871 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.159576 13724 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 16: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:02.148702 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.160471 13725 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:02.162662 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.165035 13724 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:02.166597 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:02.167657 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50020.6us
W20260709 12:25:02.144789 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 307073us
I20260709 12:25:02.167332 13650 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
W20260709 12:25:02.155809 22588 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:02.176999 13647 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:02.182510 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.195043 13725 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:02.198922 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.184976 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.165539 22402 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.168781 22592 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.107796 22567 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.155562 22304 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.182184 22584 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.175534 22131 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:25:02.271567 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.271891 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.271272 13793 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:02.288061 13724 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:02.292021 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:02.293020 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 234853us
I20260709 12:25:02.181365 22529 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.074s
I20260709 12:25:02.217618 22522 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.298514 13724 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:25:02.301636 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.193430 22412 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.102s
I20260709 12:25:02.255622 22402 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:02.306965 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.267329 22372 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:02.311992 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.269248 22570 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.268627 22594 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:02.315897 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:02.269074 22475 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:02.322940 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.323887 13724 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:02.333747 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.336477 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:02.339022 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.337553 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:02.340975 13718 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:02.343009 13646 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:02.343880 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4511) took 1003 ms. Trace:
I20260709 12:25:02.344285 13646 rpcz_store.cc:276] 0709 12:25:01.340578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.321099 (+980521us) service_pool.cc:224] Handling call
0709 12:25:02.342817 (+ 21718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.343613 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.343822 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.345800 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.267004 22592 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.348767 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.349879 13725 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
W20260709 12:25:02.351603 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.352367 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.284812 22584 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.354620 13724 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:25:02.359047 13646 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:02.359167 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.359854 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.356146 22566 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:02.339280 22475 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.360071 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.394121 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.396139 13725 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
W20260709 12:25:02.397315 22591 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 12:25:02.399322 22602 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:02.399128 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.400489 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.401027 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.400518 22596 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.401887 13727 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:02.401306 13722 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:02.402238 22596 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.351229 22131 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:02.403505 22131 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:02.364008 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4512) took 1023 ms. Trace:
I20260709 12:25:02.402462 13724 raft_consensus.cc:2376] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19. Re-sending same reply.
I20260709 12:25:02.397866 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.405122 22131 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.415444 13725 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:02.322741 22570 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.420178 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.415069 13646 rpcz_store.cc:276] 0709 12:25:01.340874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.345061 (+1004187us) service_pool.cc:224] Handling call
0709 12:25:02.358843 (+ 13782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.360738 (+  1895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.363913 (+  3175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.433832 22609 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:02.434581 22609 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:02.435937 22609 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.437183 13727 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:02.437687 22609 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:02.420964 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.447345 13722 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:02.394097 22475 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:02.405861 22596 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.448902 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4504) took 1120 ms. Trace:
I20260709 12:25:02.450016 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.458853 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4508) took 1128 ms. Trace:
W20260709 12:25:02.457619 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 181705us
I20260709 12:25:02.454063 13651 rpcz_store.cc:276] 0709 12:25:01.328527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.338287 (+  9760us) service_pool.cc:224] Handling call
0709 12:25:01.364075 (+ 25788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.594680 (+230605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.594723 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:01.776967 (+182244us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.113948 (+336981us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:02.113988 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.114067 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.448630 (+334563us) raft_consensus.cc:1685] finished
0709 12:25:02.448654 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.448833 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14591,"raft.queue_time_us":222985,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7539,"thread_start_us":482541,"threads_started":2,"wal-append.queue_time_us":397325}
I20260709 12:25:02.441362 22397 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.461973 13644 rpcz_store.cc:276] 0709 12:25:01.330032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.097413 (+767381us) service_pool.cc:224] Handling call
0709 12:25:02.097969 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.105809 (+  7840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.105858 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.106500 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.300061 (+193561us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:02.456302 (+156241us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.458498 (+  2196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.458586 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.458627 (+    41us) raft_consensus.cc:1685] finished
0709 12:25:02.458639 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.458790 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9214,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5614,"thread_start_us":254230,"threads_started":2,"wal-append.queue_time_us":282965}
I20260709 12:25:02.450657 13718 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:02.466349 13644 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:02.467096 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4516) took 1125 ms. Trace:
I20260709 12:25:02.467453 13644 rpcz_store.cc:276] 0709 12:25:01.342031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.464813 (+1122782us) service_pool.cc:224] Handling call
0709 12:25:02.465180 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.466841 (+  1661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.467040 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.468657 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:02.467696 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:02.469936 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4518) took 1109 ms. Trace:
I20260709 12:25:02.470578 13644 rpcz_store.cc:276] 0709 12:25:01.359885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.468491 (+1108606us) service_pool.cc:224] Handling call
0709 12:25:02.468890 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.469287 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.469329 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.469406 (+    77us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:02.469456 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.469529 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.469697 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.359157 22522 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.100s
I20260709 12:25:02.476233 13651 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:25:02.479836 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4514) took 1138 ms. Trace:
I20260709 12:25:02.480293 13644 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:25:02.480270 13651 rpcz_store.cc:276] 0709 12:25:01.341435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.461673 (+1120238us) service_pool.cc:224] Handling call
0709 12:25:02.476044 (+ 14371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.479278 (+  3234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.479772 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.481325 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1432) took 1121 ms. Trace:
I20260709 12:25:02.481719 13644 rpcz_store.cc:276] 0709 12:25:01.360208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.471590 (+1111382us) service_pool.cc:224] Handling call
0709 12:25:02.480042 (+  8452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.480816 (+   774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.481270 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:02.482946 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61395.2us
I20260709 12:25:02.462616 22612 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:02.486681 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
W20260709 12:25:02.490728 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 188551us
I20260709 12:25:02.491776 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.486129 22612 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.475358 22397 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:25:02.496054 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4509) took 1165 ms. Trace:
I20260709 12:25:02.499559 13652 rpcz_store.cc:276] 0709 12:25:01.334640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.111798 (+777158us) service_pool.cc:224] Handling call
0709 12:25:02.135471 (+ 23673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.135744 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.135768 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.136166 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.265039 (+128873us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:02.265076 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.265155 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.495612 (+230457us) raft_consensus.cc:1685] finished
0709 12:25:02.495719 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.495964 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":128533,"threads_started":1,"wal-append.queue_time_us":306127}
I20260709 12:25:02.495663 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.501869 13652 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:25:02.507341 22614 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:02.507912 22614 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:02.509061 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:02.509632 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:02.510650 13645 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 12:25:02.496384 22243 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260709 12:25:02.496932 22522 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:02.499778 22612 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:02.517720 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.510972 22475 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:02.521816 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:02.516263 22611 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:25:02.513265 22379 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:02.531173 22584 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:02.519359 22261 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:02.521538 22394 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:02.535620 22576 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.538626 22394 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:02.538676 22261 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:02.540017 22620 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:02.539882 22394 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:02.513051 22617 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:25:02.516834 22243 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.537217 22592 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.546612 22475 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:02.547164 22475 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:02.548100 22475 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.548529 22478 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:02.549419 22475 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:02.546643 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:02.551119 22394 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:02.537086 22614 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.551635 22394 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:02.552384 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:02.552769 22394 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.553292 22243 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.554620 22394 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.556324 22243 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.558753 14417 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:02.559099 22570 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.541589 22620 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:02.552544 22614 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:02.563571 22620 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.564829 22517 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:02.565559 22620 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:02.565802 22517 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:02.567152 22193 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:02.567552 22193 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:02.568216 22517 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:02.568507 22193 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.569656 22517 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.571012 14425 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:02.541195 22576 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.573140 14420 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:25:02.573916 22396 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:02.574848 22576 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.576946 22592 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:25:02.578223 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4517) took 1235 ms (client timeout 1586 ms). Trace:
I20260709 12:25:02.576467 22594 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:02.580607 22594 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:02.580516 13648 rpcz_store.cc:269] 0709 12:25:01.343832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.356295 (+ 12463us) service_pool.cc:224] Handling call
0709 12:25:02.578016 (+1221721us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54002229d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:02.578151 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343501696}
I20260709 12:25:02.581638 22594 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.583169 22594 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:02.570976 22517 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:02.588806 22559 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.589116 22280 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.594846 13793 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:02.590551 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4021) took 1086 ms. Trace:
W20260709 12:25:02.593585 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.596487 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.595718 13726 rpcz_store.cc:276] 0709 12:25:01.501845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.501983 (+   138us) service_pool.cc:224] Handling call
0709 12:25:01.532330 (+ 30347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.642201 (+109871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.587848 (+945647us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b5400173754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:02.588062 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160138112}
W20260709 12:25:02.598135 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.579818 22492 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
I20260709 12:25:02.561267 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:02.608281 14424 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:02.612705 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1434) took 1090 ms. Trace:
I20260709 12:25:02.589463 22559 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.612383 22503 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.616871 22503 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.618348 22559 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.605875 22396 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:02.627669 22396 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.629022 22612 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.631682 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:02.630995 22492 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.634070 14425 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:02.635933 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.640707 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:02.613157 13651 rpcz_store.cc:276] 0709 12:25:01.522141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.481355 (+959214us) service_pool.cc:224] Handling call
0709 12:25:02.491452 (+ 10097us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.491868 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.491898 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.492419 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.609254 (+116835us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:02.609318 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.609429 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.612438 (+  3009us) raft_consensus.cc:1685] finished
0709 12:25:02.612462 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.612636 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2443,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2316,"thread_start_us":98848,"threads_started":1,"wal-append.queue_time_us":109361}
I20260709 12:25:02.636451 22576 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:02.645867 22505 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:02.650695 22505 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:02.651144 22505 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:02.631311 22243 raft_consensus.cc:2764] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:25:02.635746 22396 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:02.642859 22368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.232s	user 0.004s	sys 0.020s
I20260709 12:25:02.670596 21953 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:02.671114 21953 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:02.672003 21953 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.673095 21953 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:02.680265 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1436) took 1037 ms. Trace:
W20260709 12:25:02.682390 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.680974 13644 rpcz_store.cc:276] 0709 12:25:01.643187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.483602 (+840415us) service_pool.cc:224] Handling call
0709 12:25:02.494103 (+ 10501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.494651 (+   548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.494693 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.495204 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.637469 (+142265us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:02.637511 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.637582 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.679961 (+ 42379us) raft_consensus.cc:1685] finished
0709 12:25:02.679985 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.680163 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":2744,"thread_start_us":122062,"threads_started":1,"wal-append.queue_time_us":167447}
I20260709 12:25:02.640089 22609 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.703060 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141873us
I20260709 12:25:02.698288 14425 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
W20260709 12:25:02.683816 22368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.283s	user 0.004s	sys 0.020s
I20260709 12:25:02.687137 22491 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:02.710743 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4036) took 1002 ms. Trace:
I20260709 12:25:02.720392 22503 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.708762 22592 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.709556 22609 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.711495 22368 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:02.738343 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:02.711752 22379 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:02.736649 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:02.715037 13723 rpcz_store.cc:276] 0709 12:25:01.708676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.741314 (+ 32638us) service_pool.cc:224] Handling call
0709 12:25:01.741833 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.756818 (+ 14985us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:02.564091 (+807273us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b1800353bc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:25:02.612328 (+ 48237us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.710208 (+ 97880us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.710317 (+   109us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:02.710387 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.710486 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.710662 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1600,"prepare.queue_time_us":168017,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":9888,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":42291,"spinlock_wait_cycles":7932160,"thread_start_us":104139,"threads_started":2,"wal-append.queue_time_us":83314}
I20260709 12:25:02.748447 22304 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.754374 22609 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.756634 22592 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.769901 22368 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.771257 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.772200 22405 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:02.772588 22592 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.772780 22405 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:02.773288 22405 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:02.781118 22505 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.784685 13719 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:02.784914 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:02.761162 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.761893 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.768415 22243 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:02.738466 22491 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:02.795367 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.797806 13719 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:25:02.809916 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4245) took 1113 ms. Trace:
I20260709 12:25:02.811026 14420 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:02.795715 13793 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
W20260709 12:25:02.790169 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.817589 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.815229 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.819896 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:02.819957 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.822098 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.823791 22379 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:02.824008 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.824806 13792 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:02.825485 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.828909 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.829329 13800 raft_consensus.cc:2410] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:25:02.829643 13724 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:02.830850 22594 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.831441 22594 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:02.832944 22623 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:02.833822 22131 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:02.834204 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.834884 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:02.835188 13797 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:02.835645 13798 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:02.793918 22549 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.385s	user 0.003s	sys 0.000s
W20260709 12:25:02.839440 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107478us
I20260709 12:25:02.790354 22321 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.840456 22243 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:02.840736 22321 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:02.843292 22321 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.794122 22362 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:02.843997 22362 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:02.794138 22551 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:25:02.834532 22412 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:02.851688 22412 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.852125 22412 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:02.853732 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:02.850206 22491 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:02.858595 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.858847 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.838187 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.859758 13718 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:25:02.864161 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.864938 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:02.865849 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:02.868062 13792 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:25:02.868353 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:02.869210 13797 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:02.869627 22606 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:02.867552 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.873543 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.873929 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.874396 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:02.796700 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.875814 13723 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:02.876081 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:02.877007 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.877415 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.877959 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.878952 13724 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:25:02.879710 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.810513 13794 rpcz_store.cc:276] 0709 12:25:01.696078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.696252 (+   174us) service_pool.cc:224] Handling call
0709 12:25:01.723640 (+ 27388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.777561 (+ 53921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.777607 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.679710 (+902103us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b180027cee0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a
0709 12:25:02.680546 (+   836us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:25:02.681323 (+   777us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.758529 (+ 77206us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:02.758568 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.758642 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.807911 (+ 49269us) raft_consensus.cc:1685] finished
0709 12:25:02.807951 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.809828 (+  1877us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35638,"raft.queue_time_us":53641,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":8957,"spinlock_wait_cycles":34400256,"thread_start_us":76790,"threads_started":1,"wal-append.queue_time_us":86972}
I20260709 12:25:02.885565 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:02.886176 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.888046 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:02.888981 13651 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:02.890117 13644 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:02.881481 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:02.905012 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.866307 22612 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:02.906024 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.910171 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.910957 13720 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:02.911322 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:02.872882 22592 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:02.881982 22556 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.351s	user 0.011s	sys 0.024s
I20260709 12:25:02.907612 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:02.926620 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.943605 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.946501 13767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.586s (SENT) [suppressed 36 similar messages]
I20260709 12:25:02.975072 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.967370 22614 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:02.896313 22321 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:02.894099 22304 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:03.002388 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:02.885054 22569 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:03.004093 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96635.2us
I20260709 12:25:03.017071 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:03.004968 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58530.3us
I20260709 12:25:02.872630 22491 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 12:25:02.903220 22478 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:02.902818 22567 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:25:02.976845 22556 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.473s	user 0.011s	sys 0.024s
I20260709 12:25:02.908849 22576 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 12:25:02.968572 22526 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:02.967797 22609 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:02.969668 22592 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.069864 22592 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:02.903820 22517 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
I20260709 12:25:03.073330 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.043596 22567 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.076103 22567 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:03.079294 22607 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.608s	user 0.006s	sys 0.000s
I20260709 12:25:03.052371 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.046366 22515 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:03.082726 22515 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.010898 21953 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:02.967945 22529 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.099s
I20260709 12:25:03.068064 22526 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.069059 22609 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:03.080819 22564 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.564s	user 0.005s	sys 0.000s
I20260709 12:25:03.060420 22396 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:03.019059 22304 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:03.084899 13644 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:02.903146 22620 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.136s
I20260709 12:25:03.001967 22614 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.036285 22519 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 12:25:03.096484 22526 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.096076 22609 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.099436 22396 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:03.134959 13718 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:25:03.140385 22565 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.620s	user 0.000s	sys 0.020s
I20260709 12:25:03.110404 22304 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:03.142458 22273 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:03.145694 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1630) took 1837 ms. Trace:
I20260709 12:25:03.118845 22614 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:03.171427 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1434) took 1861 ms. Trace:
I20260709 12:25:03.173238 13793 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:25:03.177776 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2905) took 1713 ms. Trace:
I20260709 12:25:03.178992 13649 rpcz_store.cc:276] 0709 12:25:01.464238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.055035 (+590797us) service_pool.cc:224] Handling call
0709 12:25:03.177438 (+1122403us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:03.177677 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140101376}
I20260709 12:25:03.176597 13800 rpcz_store.cc:276] 0709 12:25:01.310289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.310404 (+   115us) service_pool.cc:224] Handling call
0709 12:25:03.170980 (+1860576us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:03.171304 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3489468672}
I20260709 12:25:03.156850 14419 rpcz_store.cc:276] 0709 12:25:01.307968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.308282 (+   314us) service_pool.cc:224] Handling call
0709 12:25:01.308761 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.309194 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.309238 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:01.318044 (+  8806us) log.cc:844] Serialized 26 byte log entry
0709 12:25:01.438618 (+120574us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:01.438679 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.438772 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.139746 (+1700974us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:03.145231 (+  5485us) raft_consensus.cc:1685] finished
0709 12:25:03.145268 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.145599 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":706,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":7736,"spinlock_wait_cycles":42878208,"thread_start_us":120237,"threads_started":1,"wal-append.queue_time_us":173020}
I20260709 12:25:03.152083 22396 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.203200 22193 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:03.224486 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3045585669997308s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16410000000000002s for a response
W20260709 12:25:03.225626 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2180840500000158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
I20260709 12:25:03.228381 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:03.239114 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.256875 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:03.246733 22529 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:03.254077 22515 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:03.272569 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 478051us
I20260709 12:25:03.271920 22529 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:03.277428 14424 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1303) took 1896 ms (client timeout 1594 ms). Trace:
I20260709 12:25:03.303954 22529 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:03.280884 14424 rpcz_store.cc:269] 0709 12:25:01.380561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.381536 (+   975us) service_pool.cc:224] Handling call
0709 12:25:03.277154 (+1895618us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54004c3e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:03.277319 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2764319616}
I20260709 12:25:03.259096 22405 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.265743 22396 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.324198 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:03.291378 22478 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:03.337306 22405 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:03.337067 22592 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.339080 13796 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:03.339538 22592 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.340157 22396 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.342674 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.343672 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:03.342078 13794 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:03.346786 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.350946 22478 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:03.374064 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52004us
I20260709 12:25:03.379052 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:03.370872 22396 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.329816 22412 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:03.385812 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 159497us
I20260709 12:25:03.345861 22567 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:03.391764 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1359993490000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 12:25:03.392364 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.824s (SENT)
I20260709 12:25:03.392760 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:03.393687 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1172840500003076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 12:25:03.385048 22394 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.383718 22243 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:03.403384 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.407668 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.408558 13645 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:03.409026 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.409690 13649 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:03.406778 13647 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:03.410557 13645 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:03.380277 22304 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.411243 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:03.413626 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.413650 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.414494 13651 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:03.414609 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:03.399705 22368 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:03.406183 22537 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.405828 22515 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.388484 22396 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.406020 22569 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:03.429106 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:03.430169 13648 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:03.413633 22572 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.439968 13651 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
W20260709 12:25:03.440840 22643 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:03.410017 22394 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.452664 22394 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.453858 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.454849 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:03.456020 22396 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.456670 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:03.444969 22569 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.457572 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.458616 13651 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
W20260709 12:25:03.459436 22635 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:25:03.460391 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.435889 22537 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.463647 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:03.417290 22638 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:03.438171 22131 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:03.478425 22633 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:03.486967 22581 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.625s	user 0.005s	sys 0.004s
I20260709 12:25:03.488145 22492 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:03.489116 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4502) took 2161 ms. Trace:
I20260709 12:25:03.435693 22515 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.490629 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.489809 13643 rpcz_store.cc:276] 0709 12:25:01.327827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.330365 (+  2538us) service_pool.cc:224] Handling call
0709 12:25:01.354847 (+ 24482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.497610 (+142763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.497647 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:01.498347 (+   700us) log.cc:844] Serialized 26 byte log entry
0709 12:25:01.612269 (+113922us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:01.612314 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.612388 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.474322 (+1861934us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:03.488562 (+ 14240us) raft_consensus.cc:1685] finished
0709 12:25:03.488586 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.489029 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7563,"raft.queue_time_us":167147,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":35630,"spinlock_wait_cycles":8906112,"thread_start_us":220360,"threads_started":2,"wal-append.queue_time_us":204296}
I20260709 12:25:03.491663 13651 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:03.462481 22572 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.495004 22594 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:03.500789 22637 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.165s	user 0.003s	sys 0.000s
I20260709 12:25:03.501050 22526 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.477497 22405 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:03.475725 22243 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:03.510774 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:03.514410 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.506762 22243 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:03.522361 22304 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:03.523500 22243 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:03.526901 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68635.4us
I20260709 12:25:03.496758 22609 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.534729 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:03.535589 13724 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.536047 22537 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:03.536573 22555 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:03.538367 22362 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.539741 22362 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.548763 22505 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:03.525524 22594 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:03.550547 22594 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.550990 22594 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:03.555121 22412 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:03.480036 22131 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:03.556344 22131 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:03.560693 22569 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:03.573549 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:03.575160 14424 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:03.577086 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:03.578764 14419 raft_consensus.cc:2410] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:25:03.581171 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.584237 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4513) took 2243 ms. Trace:
I20260709 12:25:03.584728 13646 rpcz_store.cc:276] 0709 12:25:01.341154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.432227 (+1091073us) service_pool.cc:224] Handling call
0709 12:25:03.177640 (+745413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.178089 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:03.178133 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:03.178196 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:03.178250 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.178355 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.584031 (+405676us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400097750. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:03.584165 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4455,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":5473,"spinlock_wait_cycles":1665522048}
I20260709 12:25:03.530963 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:03.506168 22526 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.595161 13651 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:03.557816 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.473292 22529 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.059s
I20260709 12:25:03.596428 13650 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:03.617987 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.902s (SENT)
I20260709 12:25:03.619089 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.622509 22570 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:03.625806 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.626577 13651 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:25:03.636078 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:03.636827 13646 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:03.638715 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.013s (SENT)
W20260709 12:25:03.645462 13617 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.098s (SENT)
I20260709 12:25:03.646282 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.648437 13724 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:25:03.650584 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.111s (SENT)
I20260709 12:25:03.651453 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.647454 22570 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:03.669775 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:03.670998 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:03.674307 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:03.698383 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 100950us
I20260709 12:25:03.675136 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:03.699323 14422 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:03.700069 13650 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:03.678028 22623 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:03.680809 22492 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:03.682394 22614 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.715040 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:03.716815 13724 raft_consensus.cc:2376] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22. Re-sending same reply.
I20260709 12:25:03.718353 22280 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:25:03.684988 22537 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:25:03.718858 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68911us
I20260709 12:25:03.719559 22537 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:03.719966 22537 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:03.720881 22537 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:03.721549 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 199770us
W20260709 12:25:03.686105 22646 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
W20260709 12:25:03.723140 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.091s (SENT)
I20260709 12:25:03.723724 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:03.723840 22569 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:03.724471 22569 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.695631 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:03.726648 13722 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:03.708628 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:03.729116 13721 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:03.730482 22304 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:03.628159 22321 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:03.731638 22321 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.732234 22321 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.734858 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:03.735816 13727 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:03.739790 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:03.741513 22379 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:03.750495 22526 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:03.648710 22609 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.760953 22131 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:03.766922 14425 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:03.766996 22606 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:03.767820 22606 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:03.768514 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75814.3us
I20260709 12:25:03.771533 22592 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:03.772326 22478 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:03.772850 22478 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.774133 22567 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:03.771824 22396 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:03.709816 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:03.767017 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.781070 13646 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:03.778157 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:03.786955 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 89748.8us
I20260709 12:25:03.789592 13720 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:03.799357 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:03.766010 22193 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.005s
I20260709 12:25:03.830585 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:03.831657 13722 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:03.824733 14425 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:25:03.834650 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51708.8us
I20260709 12:25:03.722205 22537 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:03.845100 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.847012 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:03.808571 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:03.842459 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:03.850226 13721 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:03.854048 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:03.855167 13722 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:03.775103 22478 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.858240 13727 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:03.785771 22515 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:03.878310 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:03.858841 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:03.814446 22379 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:25:03.821568 22606 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.838745 22592 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.838553 22273 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:03.855507 22609 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:03.814738 22396 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.823343 22280 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:25:03.828444 22555 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:03.915571 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.894071 22538 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:03.885561 22618 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:25:03.911348 22362 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:03.936393 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:03.826525 22620 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.095s
W20260709 12:25:03.921818 22612 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.015s	sys 0.027s
I20260709 12:25:03.954684 13646 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:25:03.916633 22492 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:03.929602 22405 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:03.900735 22592 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:03.928057 22569 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:03.953106 22362 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.915225 22609 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:03.970494 22612 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.015s	sys 0.027s
W20260709 12:25:04.001448 22650 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:25:04.004781 22396 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.006664 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:04.020042 22592 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:03.969663 22538 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:04.023685 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.024647 13651 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.008787 22609 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.016390 22614 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.028241 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.029886 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.030849 13646 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:04.037400 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.038537 13724 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:25:03.988371 22362 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:04.059026 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 243721us
I20260709 12:25:04.021071 22131 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.061894 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:04.063194 13724 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.030808 22559 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.992102 22412 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.067s
I20260709 12:25:04.062920 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:04.081244 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:04.085642 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 277140us
I20260709 12:25:04.093689 13646 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:04.100687 13724 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:04.113375 13651 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:04.073336 22576 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:04.120767 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:04.073483 22273 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:04.127660 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:04.128244 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:04.128456 13800 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.082909 22379 consensus_queue.cc:1243] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:04.131003 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:04.132186 13795 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:04.087544 22620 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:04.133878 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:04.136426 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:04.137475 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:04.141836 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:04.144109 13800 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:04.111512 22396 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:04.152828 13798 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:04.076337 22559 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.155180 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:04.155938 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:04.162181 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:04.163779 14420 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:25:04.158840 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:04.165470 13800 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:04.069629 22555 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.186743 13801 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:04.111296 22405 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
W20260709 12:25:04.183889 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 358123us
I20260709 12:25:04.087212 22592 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.065s
I20260709 12:25:04.136241 22620 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.191274 13798 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:25:04.192421 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:04.202855 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:04.166044 22623 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:04.182310 13795 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
W20260709 12:25:04.210902 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.173002 22515 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:25:04.218007 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.130429 22576 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.184612 22570 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:04.220304 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:04.215637 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:04.190591 22396 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:04.201891 22620 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:04.111807 22538 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.240891 13720 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:25:04.245780 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 398854us
I20260709 12:25:04.196416 22304 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:04.243193 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:04.190750 22567 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:25:04.217743 22492 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:04.259675 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:04.254662 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:04.215451 22623 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.257027 13722 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:04.224539 22592 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:04.243634 22515 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:04.289202 22572 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.247080 22661 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.260946 22567 consensus_queue.cc:1243] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:04.253703 22660 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:04.260027 22492 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:04.277443 22623 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:04.302582 22655 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:25:04.303099 22576 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.299525 22661 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.316208 22396 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.318830 22661 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.319759 22661 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:04.306336 22660 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:04.306756 22656 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:25:04.256227 22412 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:04.235920 22273 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.259181 22304 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:04.320680 22594 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.769s	user 0.007s	sys 0.009s
I20260709 12:25:04.337172 22304 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.337392 22594 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.340380 22594 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:04.320765 22321 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.588s	user 0.016s	sys 0.000s
I20260709 12:25:04.320252 22661 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.310581 22671 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:04.354431 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.523s (SENT)
W20260709 12:25:04.332623 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 79733.2us
I20260709 12:25:04.360543 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:04.381278 22526 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.322913 22639 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:04.325177 22660 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:04.386300 13801 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:04.342423 22537 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.358446 22670 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:04.353480 22538 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.329535 22576 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:04.341547 22515 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.432749 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:04.359999 22661 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:04.442093 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87733.1us
W20260709 12:25:04.432816 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173288us
I20260709 12:25:04.351092 22412 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.450430 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:04.357577 22321 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:04.357887 22659 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:25:04.360349 22671 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:04.443976 22567 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:04.450654 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:04.432360 22592 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:04.453955 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:04.465315 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:04.431828 22526 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.469882 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.454988 22608 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:04.479866 22412 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.480535 22680 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:04.484239 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.486088 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.486508 22592 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.488756 13647 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: [29.22-29.22]
I20260709 12:25:04.500097 13644 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:25:04.489799 22657 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:04.484324 22526 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.493548 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.385318 22492 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.459267 22559 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.460242 22671 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.469084 22612 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:04.511201 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.472071 22678 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:04.515542 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.447877 22537 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.518128 13652 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:04.518931 13719 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 16.
I20260709 12:25:04.448951 22538 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.520087 22612 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:04.521446 22538 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.517486 22680 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:04.525977 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 432578us
I20260709 12:25:04.512290 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.527109 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:04.527631 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:25:04.521106 22612 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.458297 22576 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.471379 22679 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:04.498759 22608 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:04.532297 22492 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:04.536875 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:04.571707 22609 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:04.572386 22609 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.572820 22609 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:04.542553 22608 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:04.548604 22679 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:04.587949 22679 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.587991 22405 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:04.590529 22405 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:04.591591 22405 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:04.593716 22675 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:25:04.599228 22537 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:04.598591 22620 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.600987 13799 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27.
I20260709 12:25:04.585547 22671 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.628324 13652 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:25:04.621898 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94858.7us
W20260709 12:25:04.621003 22362 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.553s	user 0.013s	sys 0.002s
I20260709 12:25:04.637367 22362 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.640774 22362 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.657670 22671 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.629563 22620 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:04.666947 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.667750 13719 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
W20260709 12:25:04.677278 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.663643 22505 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.015s	sys 0.010s
I20260709 12:25:04.602325 22665 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.124s
I20260709 12:25:04.662159 22567 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:04.701573 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.703764 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:04.700852 22684 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:25:04.664376 22304 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:04.709494 13727 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
W20260709 12:25:04.698961 22505 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.015s	sys 0.011s
I20260709 12:25:04.716166 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:04.721594 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 208430us
I20260709 12:25:04.739379 22633 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:04.748172 22620 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.748986 22671 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.749981 22590 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.752653 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.790133 13644 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:04.757786 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.791285 13652 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:04.758723 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.759730 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:04.770320 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:04.713984 22538 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:04.793414 13643 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:04.794476 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:04.776778 22633 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:04.771484 22412 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:04.784960 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.805094 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:04.787624 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:04.808089 13719 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:04.788753 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:04.808928 13723 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:04.784104 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.797545 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:04.810653 13725 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:04.789832 22505 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:04.813166 22685 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:04.812783 22505 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.810025 13721 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:04.820225 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:04.823148 13718 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:25:04.710738 22680 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.718361 22567 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.795198 22321 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:04.801424 22412 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.718524 22304 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.800035 22538 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.842406 13652 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:04.846074 13722 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:04.847453 13643 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:04.791517 22661 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.796308 22590 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.792905 22671 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:04.793409 22620 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.863459 13644 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:04.868844 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:04.852259 22412 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:04.880467 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.049983 13793 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:25:05.026721 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:05.194525 13644 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:04.956537 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:05.195990 13792 raft_consensus.cc:606] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:05.196432 13792 raft_consensus.cc:614] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:04.969864 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.124100 13723 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:04.969053 13795 pending_rounds.cc:85] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 12
I20260709 12:25:04.991124 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.198340 13795 pending_rounds.cc:107] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:25:04.956441 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.199185 13795 raft_consensus.cc:2889] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:05.199705 13801 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:05.200857 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.033627 13643 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:05.203013 13798 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:25:04.892455 13693 connection.cc:256] RPC call timeout handler was delayed by -1.984789229999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 12:25:04.964674 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:04.857668 22538 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:05.230569 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9370892299998559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 12:25:04.864099 22576 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:05.233114 13693 connection.cc:256] RPC call timeout handler was delayed by -1.9109892299998137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 12:25:05.249054 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8677892299997438s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 12:25:05.080711 22699 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 12:25:05.095242 22702 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:25:04.959851 22696 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.872534 22618 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.381s
I20260709 12:25:05.250547 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:04.969617 22689 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 12:25:05.246821 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 389442us
I20260709 12:25:05.014510 13719 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
W20260709 12:25:05.259790 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 256779us
I20260709 12:25:05.261898 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.262764 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:04.953927 22682 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:04.970458 22690 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:25:05.131047 22304 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.172684 22671 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:05.250633 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8497892299997147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 12:25:05.326046 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63232.1us
I20260709 12:25:05.051554 22567 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.133121 22590 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.266068 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:05.422451 13644 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:04.904747 22321 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:05.050676 22701 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.328332 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 201861us
I20260709 12:25:05.496315 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:05.335296 22696 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:04.900193 22694 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:05.371086 22305 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:04.955868 22633 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.360496 13643 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:05.234237 22620 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.177867 22555 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:05.232849 22405 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:05.536782 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:05.537714 14420 raft_consensus.cc:2376] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19. Re-sending same reply.
I20260709 12:25:05.362804 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.364944 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:05.366876 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.540926 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:05.007494 22537 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.320s
W20260709 12:25:05.367861 13650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.023s	sys 0.034s
I20260709 12:25:05.076510 22505 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.250165 22704 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:04.963027 22572 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.365s
I20260709 12:25:05.546644 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:05.547430 13643 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:05.418807 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:05.371578 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 174675us
W20260709 12:25:05.420020 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8272892299996784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 12:25:05.549341 13693 connection.cc:256] RPC call timeout handler was delayed by -1.809289230000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:25:05.549806 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7948892300000807s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 12:25:05.550191 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7138892299999497s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
W20260709 12:25:05.550588 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4861892300000363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20950000000000002s for a response
W20260709 12:25:05.550957 13693 connection.cc:256] RPC call timeout handler was delayed by -1.412389229999917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
W20260709 12:25:05.551504 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3934892299998864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 12:25:05.551937 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3511892299998181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
W20260709 12:25:05.552302 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3187892299997657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22810000000000002s for a response
W20260709 12:25:05.552697 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1324892299999192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24880000000000002s for a response
W20260709 12:25:05.553066 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1054892299998755s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25179999999999997s for a response
W20260709 12:25:05.366097 22660 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.922s	user 0.014s	sys 0.000s
I20260709 12:25:05.156414 22661 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.266201 22700 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.970494 22639 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:05.523337 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:05.561918 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:05.332042 22612 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:05.543450 13650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.023s	sys 0.034s
I20260709 12:25:05.565245 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4542) took 1099 ms. Trace:
I20260709 12:25:05.496590 22321 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.555536 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:05.575191 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 156448us
I20260709 12:25:05.516896 22694 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.542906 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:05.553939 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:05.521914 22305 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:05.565716 13650 rpcz_store.cc:276] 0709 12:25:04.466000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.466186 (+   186us) service_pool.cc:224] Handling call
0709 12:25:04.508360 (+ 42174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.508700 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:04.711985 (+203285us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.564728 (+852743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.564801 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:05.564873 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.564973 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.565178 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70648,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":962273,"threads_started":2,"wal-append.queue_time_us":830154}
W20260709 12:25:05.555927 22705 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
I20260709 12:25:05.559494 22243 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:05.622740 22243 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:05.623381 22492 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.018s	user 0.012s	sys 0.002s
I20260709 12:25:05.624171 22708 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:05.624675 22708 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:05.625077 22708 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:05.625149 22243 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.578487 22612 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:05.626915 22709 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.444s	user 0.001s	sys 0.000s
I20260709 12:25:05.628125 22612 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:05.543710 22555 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:05.629658 22688 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.847s	user 0.002s	sys 0.000s
I20260709 12:25:05.630638 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:05.631527 13793 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 19: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:05.631470 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4548) took 1158 ms. Trace:
W20260709 12:25:05.631739 22714 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.516s	user 0.001s	sys 0.000s
I20260709 12:25:05.632000 13645 rpcz_store.cc:276] 0709 12:25:04.473415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.480023 (+  6608us) service_pool.cc:224] Handling call
0709 12:25:04.480453 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:04.481116 (+   663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.481142 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:04.481973 (+   831us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.607286 (+125313us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:04.607450 (+   164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.607551 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.628696 (+1021145us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:05.631058 (+  2362us) raft_consensus.cc:1685] finished
0709 12:25:05.631082 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.631394 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":229,"raft.queue_time_us":2265,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":60,"spinlock_wait_cycles":6411776,"thread_start_us":123808,"threads_started":1,"wal-append.queue_time_us":212704}
I20260709 12:25:05.579304 22614 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:05.632647 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.618803 22576 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:05.636642 22576 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.636739 13793 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:05.637110 22576 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:05.584167 22663 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:05.638619 22663 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:05.639547 22663 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.640559 22663 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:05.640856 22623 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.320s	user 0.014s	sys 0.000s
I20260709 12:25:05.641371 22623 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.642086 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:05.620394 22696 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.546742 22515 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 12:25:05.635597 22707 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:05.574903 22639 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:05.619853 22608 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.037s	user 0.011s	sys 0.002s
W20260709 12:25:05.644551 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 90745.5us
I20260709 12:25:05.658751 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:05.676440 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.690615 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:05.691334 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:05.677354 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.661072 22661 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:05.563779 22711 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:05.564208 22660 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:05.646134 22609 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.043s	user 0.014s	sys 0.003s
I20260709 12:25:05.683337 22614 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:05.684906 22697 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:05.685982 22555 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:05.579771 22321 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:05.581112 22694 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:05.546978 22405 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.692281 13643 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:05.692638 13793 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:05.692348 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:05.525920 22701 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.702593 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:05.709270 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:05.564170 22704 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.722797 13721 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:25:05.769075 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:05.769515 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.770202 13719 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:05.773892 13723 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:05.695911 22243 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:05.776497 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:05.779039 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:05.780056 13793 raft_consensus.cc:2368] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24.
I20260709 12:25:05.782955 13798 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:25:05.783900 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:05.785881 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.774307 22721 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:05.789309 13723 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:25:05.577193 22700 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.794286 13719 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:25:05.715055 22696 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:05.714733 22620 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.775027 22707 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:25:05.711107 22661 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:05.806913 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:05.805812 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.022s	sys 0.042s
W20260709 12:25:05.827529 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.022s	sys 0.043s
I20260709 12:25:05.827615 13719 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
W20260709 12:25:05.827991 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 965022us
I20260709 12:25:05.789364 22721 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:05.831578 22673 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.532s	user 0.000s	sys 0.024s
I20260709 12:25:05.832063 22721 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.783360 22505 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:05.790727 22697 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:05.697322 22608 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.833870 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4261) took 1718 ms. Trace:
I20260709 12:25:05.699981 22609 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.837690 13794 rpcz_store.cc:276] 0709 12:25:04.115093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.115444 (+   351us) service_pool.cc:224] Handling call
0709 12:25:04.115959 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.127501 (+ 11542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.127559 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.136905 (+  9346us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.251293 (+114388us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:04.251338 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.251418 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.830815 (+1579397us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:05.833181 (+  2366us) raft_consensus.cc:1685] finished
0709 12:25:05.833202 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.833781 (+   579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34154,"raft.queue_time_us":12950,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":18609,"spinlock_wait_cycles":2675584,"thread_start_us":62072,"threads_started":1,"wal-append.queue_time_us":139188}
I20260709 12:25:05.853492 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.711292 22711 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:05.844969 22708 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.822525 22696 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:05.848029 22515 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.822634 22620 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.850389 22478 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:05.373301 22682 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.871025 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:05.860805 22412 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.959s	user 0.009s	sys 0.006s
W20260709 12:25:05.852165 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 316074us
I20260709 12:25:05.853394 22505 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:05.863300 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 171044us
I20260709 12:25:05.859800 22502 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.868160 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:05.880456 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:05.880992 13652 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 15.
W20260709 12:25:05.878612 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.883512 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4264) took 1757 ms. Trace:
I20260709 12:25:05.884688 13794 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:05.884478 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:05.884333 13796 rpcz_store.cc:276] 0709 12:25:04.126374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.128085 (+  1711us) service_pool.cc:224] Handling call
0709 12:25:05.883048 (+1754963us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54000d2fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:05.883184 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661913984}
I20260709 12:25:05.856494 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:05.875406 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:05.889282 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:05.901795 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:05.872069 22701 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.872234 22515 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.916143 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:05.890832 22697 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:05.894773 22716 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:05.951208 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1655) took 1639 ms. Trace:
I20260709 12:25:05.951531 22716 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.913760 22677 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:05.952345 22677 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:05.951821 14419 rpcz_store.cc:276] 0709 12:25:04.311608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.311783 (+   175us) service_pool.cc:224] Handling call
0709 12:25:04.312285 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.463545 (+151260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.463581 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.463629 (+    48us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:04.463662 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.950903 (+1487241us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:05.950962 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.951132 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347277696}
I20260709 12:25:05.921367 22515 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.928113 22701 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:05.952024 22716 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:05.963488 22687 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.261s	user 0.000s	sys 0.010s
I20260709 12:25:05.928788 22721 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:05.964586 22694 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.965837 13797 raft_consensus.cc:2368] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:25:05.966759 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4546) took 1498 ms. Trace:
I20260709 12:25:05.967177 13648 rpcz_store.cc:276] 0709 12:25:04.468262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.468437 (+   175us) service_pool.cc:224] Handling call
0709 12:25:04.468761 (+   324us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:04.481054 (+ 12293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.481085 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:04.481973 (+   888us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.513174 (+ 31201us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:04.513226 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.513302 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.962772 (+1449470us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:05.966339 (+  3567us) raft_consensus.cc:1685] finished
0709 12:25:05.966365 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.966684 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3240,"raft.queue_time_us":13723,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":109,"spinlock_wait_cycles":15317248,"thread_start_us":30601,"threads_started":1,"wal-append.queue_time_us":138373}
W20260709 12:25:05.970155 22658 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.709s	user 0.004s	sys 0.009s
I20260709 12:25:05.970335 22576 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.973517 13800 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:05.928624 22321 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:05.974012 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4262) took 1858 ms (client timeout 2235 ms). Trace:
I20260709 12:25:05.975239 13800 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: [29.16-29.16]
I20260709 12:25:05.862797 22614 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.975723 22321 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:05.927738 22538 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.002s
I20260709 12:25:05.935581 22682 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.936101 22700 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:05.936446 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.000519 13723 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:05.938529 22505 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.941195 22704 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.941922 22609 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.889571 22696 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.942704 22620 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.893988 22661 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:05.943249 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:05.943545 22708 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.943722 22623 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:05.943846 22712 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.523s	user 0.002s	sys 0.000s
I20260709 12:25:05.944655 22660 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.945721 22405 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:05.946399 22711 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.032397 22682 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:06.032995 22682 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.996374 22671 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:25:06.035243 22682 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:05.889885 22502 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.036082 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:05.882244 22412 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:05.952971 22677 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:05.974722 13799 rpcz_store.cc:269] 0709 12:25:04.115567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.120183 (+  4616us) service_pool.cc:224] Handling call
0709 12:25:05.973673 (+1853490us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:05.973933 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096330624}
W20260709 12:25:05.980341 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64558.3us
I20260709 12:25:05.935084 22608 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:05.930944 22724 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:05.961927 13792 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:05.985536 13800 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:06.036533 22725 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:05.987758 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.045933 14419 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:25:05.974120 22576 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:06.008188 22606 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:05.991091 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.608s (SENT)
W20260709 12:25:06.048177 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57182us
W20260709 12:25:06.048894 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.641s (SENT)
I20260709 12:25:06.049263 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:05.997227 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.049325 13723 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:06.015988 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4279) took 1050 ms. Trace:
I20260709 12:25:06.022606 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:06.025324 13721 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:06.050788 14425 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:06.029841 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:05.996063 22538 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.036748 14416 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:06.039220 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.071842 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:05.973353 22639 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.042887 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:06.052066 13799 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:06.076835 13799 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:06.077028 14422 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:06.054281 13792 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:25:06.055748 13800 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:25:06.051069 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:06.064021 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.079350 14423 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:06.068840 22694 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:06.079099 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:06.065487 22606 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.050529 13795 rpcz_store.cc:276] 0709 12:25:04.965830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.968216 (+  2386us) service_pool.cc:224] Handling call
0709 12:25:04.968742 (+   526us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:05.329099 (+360357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.329148 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:05.330073 (+   925us) log.cc:844] Serialized 78 byte log entry
0709 12:25:05.789693 (+459620us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:05.833429 (+ 43736us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.968773 (+135344us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b18004a2480. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:25:05.969070 (+   297us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.999661 (+ 30591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.999790 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.015577 (+ 15787us) raft_consensus.cc:1685] finished
0709 12:25:06.015615 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.015896 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171106,"prepare.queue_time_us":183029,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":44135,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":45889,"spinlock_wait_cycles":14911104,"thread_start_us":368720,"threads_started":1,"wal-append.queue_time_us":502796}
I20260709 12:25:06.100025 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:06.100875 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49890.2us
I20260709 12:25:06.101161 13722 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:06.079908 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:06.102576 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.102428 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:06.084762 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:06.084946 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:06.087512 13617 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.720s (SENT)
W20260709 12:25:06.088886 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 196574us
I20260709 12:25:06.108335 22273 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:06.106180 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.098994 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.083701 14418 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:06.116214 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:06.119216 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.746s (SENT)
I20260709 12:25:06.129732 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:06.132899 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.135421 13796 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:25:06.137590 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:06.130834 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.098134 22708 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:06.074421 22724 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:06.100149 22576 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:06.128530 22623 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:06.152158 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.158708 13725 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:06.153489 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:06.162531 22590 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:06.163132 22590 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.163573 22590 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:06.173123 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:06.154156 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 126172us
I20260709 12:25:06.155155 13722 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:06.150399 13721 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:25:06.176441 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.177333 13721 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:06.179399 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:06.184723 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:06.196782 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.223346 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50322.8us
I20260709 12:25:06.225744 13645 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:06.148423 22708 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.229751 13645 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:06.231184 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.232319 13651 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:25:06.236722 13644 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:25:06.070955 22725 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.236883 13650 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
W20260709 12:25:06.226904 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47664.1us
I20260709 12:25:06.235893 13651 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:06.223681 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103416us
W20260709 12:25:06.245257 13617 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.774s (SENT)
W20260709 12:25:06.246417 13617 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.929s (SENT)
I20260709 12:25:06.246834 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:06.247159 13725 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:06.250053 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:06.251072 13651 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:25:06.237843 22708 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:06.240012 22614 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:06.257340 13651 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:06.259632 13650 raft_consensus.cc:1217] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 21.13->[22.14-23.15]   Dedup: 22.14->[23.15-23.15]
I20260709 12:25:06.262501 13650 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: [22.14-23.15]
W20260709 12:25:06.244477 22680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.008s	sys 0.009s
I20260709 12:25:06.260053 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:06.261881 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.268429 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.226644 22620 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:06.258980 22618 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.309844 22618 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.310823 22576 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.248839 22670 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:06.260387 22724 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.311586 22576 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.276552 22725 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:06.277722 22538 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:06.278648 22718 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:06.253741 22623 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:06.230782 22734 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:06.287102 22614 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:06.288041 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4560) took 1530 ms. Trace:
I20260709 12:25:06.316533 13649 rpcz_store.cc:276] 0709 12:25:04.757110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.822899 (+ 65789us) service_pool.cc:224] Handling call
0709 12:25:04.849963 (+ 27064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.882137 (+ 32174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.882186 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.882245 (+    59us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:04.882314 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.287734 (+1405420us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b5400114254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:06.287795 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.287968 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73390848}
I20260709 12:25:06.291766 22716 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.260895 13721 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
W20260709 12:25:06.268579 22680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.008s	sys 0.009s
I20260709 12:25:06.318604 22620 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:06.305915 14421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1652) took 2629 ms (client timeout 2008 ms). Trace:
I20260709 12:25:06.325773 22620 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:06.326699 14421 rpcz_store.cc:269] 0709 12:25:03.668635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.673955 (+  5320us) service_pool.cc:224] Handling call
0709 12:25:06.297702 (+2623747us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:06.297981 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709575040}
I20260709 12:25:06.308604 22304 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:06.328070 22304 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:06.328539 22304 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:06.311988 22670 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:06.330822 22305 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.332422 22305 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:06.334914 22614 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.273701 22639 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:06.340978 22718 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:06.342149 22718 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.314467 14425 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:06.321259 22618 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:06.316133 22576 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:06.352006 22716 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:06.359465 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
W20260709 12:25:06.361943 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.376675 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 226878us
I20260709 12:25:06.381213 22708 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.384261 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:06.390698 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:06.391556 13651 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:06.392457 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.394394 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.385010 13650 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:06.397812 13795 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:06.396112 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.399708 13651 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:25:06.400429 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.400585 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.399184 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.402612 13643 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:06.403090 13645 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:06.406481 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.407130 13644 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:06.411815 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:06.412523 13795 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:06.412227 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.418711 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:06.376551 22677 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.428656 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:06.429140 13649 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:25:06.408931 22661 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.435604 13649 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 10
W20260709 12:25:06.435137 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4277) took 1479 ms (client timeout 1966 ms). Trace:
I20260709 12:25:06.442312 13649 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:25:06.442768 13649 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:06.447150 13795 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:06.440840 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:06.439239 13643 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:06.451311 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:06.442426 13797 rpcz_store.cc:269] 0709 12:25:04.956232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.962372 (+  6140us) service_pool.cc:224] Handling call
0709 12:25:06.426117 (+1463745us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54002cc4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:06.435062 (+  8945us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1765772032}
I20260709 12:25:06.454525 22590 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.456813 22590 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:06.456897 22304 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.422351 22362 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:06.458127 22362 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.459153 22362 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.462214 22708 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.399230 22670 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.470224 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:06.473855 14419 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:06.471314 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.477854 13651 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:06.434501 22678 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:06.487186 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:06.480381 22609 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:06.503230 22670 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:06.490721 22716 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:06.518826 22716 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.519291 22716 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:06.519913 22677 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:06.528965 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:06.534970 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43713.2us
I20260709 12:25:06.551564 14419 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:06.563196 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.491868 22678 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.463305 22661 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.485625 22590 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.074s
W20260709 12:25:06.607650 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44606.4us
I20260709 12:25:06.559478 22660 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:06.612655 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.560828 22609 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:06.619220 22661 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.621317 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.615824 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.623800 22609 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:06.630772 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.602784 22515 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:06.633008 22701 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:06.635134 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2929) took 1383 ms. Trace:
I20260709 12:25:06.637377 13793 rpcz_store.cc:276] 0709 12:25:05.252083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.785571 (+533488us) service_pool.cc:224] Handling call
0709 12:25:06.628796 (+843225us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54002891d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:06.635056 (+  6260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405088128}
I20260709 12:25:06.610788 22671 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:06.661830 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.637432 22701 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.667614 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.668782 13721 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:06.668960 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.639961 22304 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.639585 22678 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.625176 22660 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.651643 22697 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:06.653041 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 543050us
I20260709 12:25:06.654045 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4072) took 1867 ms. Trace:
I20260709 12:25:06.685122 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:06.686728 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.686968 13690 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:06.655413 22502 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:06.658816 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4556) took 1903 ms. Trace:
I20260709 12:25:06.691368 13646 rpcz_store.cc:276] 0709 12:25:04.755515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.755664 (+   149us) service_pool.cc:224] Handling call
0709 12:25:04.756015 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.864890 (+108875us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.864936 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.865010 (+    74us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:04.865055 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.658218 (+1793163us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400112954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:06.658284 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.658733 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249626368}
I20260709 12:25:06.659888 22670 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.659777 22492 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:06.660759 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.638154 22515 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:06.651492 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 735454us
I20260709 12:25:06.639520 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:06.699329 22745 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.415s	user 0.004s	sys 0.000s
I20260709 12:25:06.700698 22670 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.702656 22692 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:06.703264 22692 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:06.703965 22670 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.704275 22692 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.690779 22243 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:06.705986 22243 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.672456 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.630867 22730 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:06.707414 13719 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:06.707509 22730 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.708480 22515 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.694216 13793 raft_consensus.cc:2410] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:25:06.695835 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:06.710427 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:06.711228 13797 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:06.699424 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:06.711783 22243 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.712316 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:06.712745 22708 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:06.712915 13797 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:06.713423 22708 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.715870 22708 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.718609 22492 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.688239 13726 rpcz_store.cc:276] 0709 12:25:04.786408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.788237 (+  1829us) service_pool.cc:224] Handling call
0709 12:25:04.802243 (+ 14006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.802588 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:04.811411 (+  8823us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.979059 (+1167648us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b18003ef8c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:25:06.653612 (+674553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.653678 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:06.653731 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.653832 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.653978 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":555243,"prepare.queue_time_us":2358,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":541,"raft.queue_time_us":542294,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":12199808,"thread_start_us":135646,"threads_started":1,"wal-append.queue_time_us":1297358}
I20260709 12:25:06.710090 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.651546 22729 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:06.723126 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:06.724830 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.720403 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.726601 14417 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:06.726855 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.731906 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.732633 13722 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:06.673578 22701 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:06.732064 13795 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:25:06.683701 22663 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:06.684156 22691 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:06.762863 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:06.684583 22671 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.763310 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:06.763839 13649 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:06.753082 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.764133 13643 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:06.709977 22730 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.747082 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.735252 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:06.745784 13721 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:06.712949 22243 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:06.720470 22677 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:06.734898 22321 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:06.735731 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.762967 22661 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:06.770886 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:06.806852 13651 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:06.735157 22590 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:25:06.789768 13647 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:06.783838 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:06.808326 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:06.795077 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:06.799326 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2924) took 1548 ms (client timeout 1607 ms). Trace:
I20260709 12:25:06.805614 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:06.770206 13719 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:06.800544 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.795656 13646 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:25:06.791286 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:06.803591 13726 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:06.793380 13797 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:06.821934 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:06.820515 13727 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:06.824384 13721 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:06.839944 13725 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:06.822122 13798 rpcz_store.cc:269] 0709 12:25:05.250785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.364507 (+113722us) service_pool.cc:224] Handling call
0709 12:25:06.799051 (+1434544us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54001004d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:06.799239 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559079168}
I20260709 12:25:06.827632 13649 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:06.829893 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:06.806092 22321 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.788729 22576 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:06.815909 13722 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:06.849747 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:06.799896 22677 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.863507 22677 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:06.851838 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:06.867165 13798 raft_consensus.cc:606] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:06.867666 13798 raft_consensus.cc:614] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:06.807477 22661 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:06.874902 13793 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:06.878196 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:06.856696 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 157358us
I20260709 12:25:06.857308 13646 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:25:06.858376 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.855142 22576 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:06.886148 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:06.789791 22663 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:06.826709 22492 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:06.887595 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:06.789652 22691 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:06.931355 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43909us
I20260709 12:25:06.938028 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:06.951673 13727 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:06.853502 22680 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:06.901166 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:07.053362 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152246us
W20260709 12:25:07.067656 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129749us
I20260709 12:25:07.068934 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:07.054301 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:07.091483 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 404588us
I20260709 12:25:07.092622 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:06.808389 22697 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:07.066232 22716 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.530s	user 0.014s	sys 0.000s
I20260709 12:25:06.890743 22576 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:07.051803 22620 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:07.065745 22243 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:07.054757 22679 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:07.046226 22721 raft_consensus.cc:2764] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election decision vote started in defunct term 27: won
I20260709 12:25:06.879649 22362 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.224s
I20260709 12:25:06.839888 22671 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.052584 22680 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.092175 22729 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:07.109196 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:06.887277 22505 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.218s
I20260709 12:25:07.095317 22556 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:07.103184 22594 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:07.102183 22670 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:07.088980 22692 raft_consensus.cc:2764] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:25:07.124825 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:07.121876 22612 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:25:07.157773 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48692.7us
I20260709 12:25:07.107561 22716 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.127450 22660 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:07.162277 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:07.127792 22680 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:07.172621 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.139420 22556 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:07.131174 22679 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:07.160077 22670 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:07.140575 22762 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:07.197397 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:07.188722 22612 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:07.132973 22492 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:25:07.158821 22594 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:07.199782 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:07.115914 22321 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.194028 22556 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:07.218727 22405 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:07.219149 22691 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:07.220228 22670 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.222653 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4099) took 1081 ms. Trace:
I20260709 12:25:07.223743 22612 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:07.223999 22660 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.224591 22660 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:07.226964 22758 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:25:07.219425 22760 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:07.223117 13723 rpcz_store.cc:276] 0709 12:25:06.141764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.142100 (+   336us) service_pool.cc:224] Handling call
0709 12:25:06.142603 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.660475 (+517872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.660509 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.660969 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.889395 (+228426us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:06.889443 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.889524 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.222365 (+332841us) raft_consensus.cc:1685] finished
0709 12:25:07.222395 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.222581 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":517345,"raft.queue_time_us":523513,"spinlock_wait_cycles":22758144,"thread_start_us":228102,"threads_started":1,"wal-append.queue_time_us":556670}
W20260709 12:25:07.240617 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 388192us
I20260709 12:25:07.127240 22661 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:07.206310 22759 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 12:25:07.251554 13646 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:07.258034 13646 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:07.256062 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.261231 13646 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.262492 13646 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:07.269038 22761 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:07.233664 22243 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:25:07.270422 22405 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:07.270709 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4092) took 1191 ms. Trace:
I20260709 12:25:07.271518 22594 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.272540 22660 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:07.272817 22716 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:07.264279 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:07.274142 22679 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.285295 13693 leader_election.cc:400] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:25:07.285732 13693 leader_election.cc:403] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:25:07.286389 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78283us
I20260709 12:25:07.288676 13646 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:07.292135 13647 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:07.292616 13647 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:07.204803 22721 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:07.283663 13718 rpcz_store.cc:276] 0709 12:25:06.078858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.079775 (+   917us) service_pool.cc:224] Handling call
0709 12:25:06.128987 (+ 49212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.656455 (+527468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.656488 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.656981 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.058490 (+401509us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:07.058558 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.058662 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.270457 (+211795us) raft_consensus.cc:1685] finished
0709 12:25:07.270478 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.270634 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":526964,"raft.queue_time_us":528241,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":55,"thread_start_us":401018,"threads_started":1,"wal-append.queue_time_us":485339}
W20260709 12:25:07.263439 22764 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:07.318010 22243 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22.
I20260709 12:25:07.232313 22766 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:07.316642 13647 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.271502 22405 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:07.324934 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.237802 22692 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:25:07.332615 22606 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:07.326056 22729 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:25:07.335901 22729 consensus_queue.cc:1243] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:07.336516 22729 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:07.322302 22762 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:25:07.321971 22691 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:25:07.372057 13647 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:07.343081 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4096) took 1222 ms. Trace:
I20260709 12:25:07.362028 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.347761 13723 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:07.348750 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.387777 13720 rpcz_store.cc:276] 0709 12:25:06.121148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.121386 (+   238us) service_pool.cc:224] Handling call
0709 12:25:06.121807 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.654716 (+532909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.654750 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.655290 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.852270 (+196980us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:06.852317 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.852403 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.342804 (+490401us) raft_consensus.cc:1685] finished
0709 12:25:07.342828 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.342993 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":532397,"raft.queue_time_us":535249,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":196179,"threads_started":1,"wal-append.queue_time_us":450152}
I20260709 12:25:07.391404 13721 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:25:07.343585 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.393172 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.394380 13727 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:25:07.396528 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.399276 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.398573 13721 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26. Re-sending same reply.
I20260709 12:25:07.402124 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:07.408023 13720 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:07.406538 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:07.408708 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.409969 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.391422 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.411409 13725 raft_consensus.cc:2410] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:25:07.415017 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:07.343030 22692 consensus_queue.cc:1243] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:07.433954 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.423481 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.438755 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.440796 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.438898 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.343922 22304 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.336858 22729 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:07.340152 22537 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:07.447968 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:07.447330 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.399103 22691 consensus_queue.cc:1243] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:25:07.453857 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.452589 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.336764 22556 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:07.460700 13721 raft_consensus.cc:2368] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
W20260709 12:25:07.463696 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.438752 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.450505 13795 raft_consensus.cc:606] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:07.464572 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.451968 22537 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.466857 22556 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:07.467447 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2931) took 2214 ms. Trace:
W20260709 12:25:07.469054 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2930) took 2216 ms (client timeout 2428 ms). Trace:
I20260709 12:25:07.470522 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4577) took 1081 ms. Trace:
I20260709 12:25:07.471822 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:07.469630 13792 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:07.473317 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.361989 22405 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.484385 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:07.485566 13801 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:07.458284 22677 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:07.449381 22304 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:08.005935 13618 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.009s
I20260709 12:25:08.006319 13725 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26. Re-sending same reply.
W20260709 12:25:07.954031 13651 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.019s
I20260709 12:25:08.008466 13650 rpcz_store.cc:276] 0709 12:25:06.389512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.402400 (+ 12888us) service_pool.cc:224] Handling call
0709 12:25:06.402684 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.640858 (+238174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.640896 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:06.640947 (+    51us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:06.640982 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.470222 (+829240us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54001e6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:07.470321 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.470458 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30,"raft.queue_time_us":494,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":96,"spinlock_wait_cycles":546792192}
W20260709 12:25:08.009604 13651 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.005s	sys 0.019s
W20260709 12:25:07.537799 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80284us
W20260709 12:25:08.008888 13618 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.004s	sys 0.009s
W20260709 12:25:08.010720 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 535090us
W20260709 12:25:08.006326 13794 rpcz_store.cc:269] 0709 12:25:05.252320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.853198 (+600878us) service_pool.cc:224] Handling call
0709 12:25:07.468824 (+1615626us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:07.468980 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54170496}
W20260709 12:25:07.805341 22680 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.004s
I20260709 12:25:08.000279 22661 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.471678 22696 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:07.463593 22576 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:07.527582 22729 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:07.523896 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:07.557029 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.525250 13647 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:08.012137 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:08.020443 22405 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:08.015501 22680 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.063877 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:08.164685 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:08.165138 22680 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:08.165685 13721 raft_consensus.cc:2368] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23.
I20260709 12:25:08.019342 22661 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.039594 13727 raft_consensus.cc:2368] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24.
I20260709 12:25:08.036243 22729 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:08.050639 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:08.014599 22780 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:25:07.701805 22701 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.889s	user 0.016s	sys 0.000s
W20260709 12:25:08.013835 22786 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:25:07.966279 22412 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.015633 22537 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.217545 22701 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.263402 22792 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:08.026324 22677 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.280678 22792 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.338761 22697 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.351377 22747 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.894s	user 0.012s	sys 0.017s
W20260709 12:25:08.391868 22781 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
W20260709 12:25:08.393280 22785 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:25:08.395349 22790 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:25:08.400763 22792 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:08.402174 22787 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:25:08.398041 22701 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:08.447901 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4598) took 1189 ms. Trace:
W20260709 12:25:08.040830 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.449265 13644 rpcz_store.cc:276] 0709 12:25:07.257722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.276975 (+ 19253us) service_pool.cc:224] Handling call
0709 12:25:07.277286 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.297563 (+ 20277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.297615 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:07.298341 (+   726us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.654325 (+355984us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:07.654399 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.654533 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.446699 (+792166us) raft_consensus.cc:1685] finished
0709 12:25:08.446878 (+   179us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.447396 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34931,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":4664,"thread_start_us":355415,"threads_started":1,"wal-append.queue_time_us":976389}
I20260709 12:25:08.456120 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.459445 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 2908) took 2237 ms. Trace:
I20260709 12:25:08.460135 13648 rpcz_store.cc:276] 0709 12:25:06.221732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.222616 (+   884us) service_pool.cc:224] Handling call
0709 12:25:06.223053 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.223334 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.223365 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.238423 (+ 15058us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.372084 (+133661us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:06.372124 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.372223 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.264085 (+1891862us) spinlock_profiling.cc:243] Waited 7.81 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:08.458565 (+194480us) raft_consensus.cc:1685] finished
0709 12:25:08.458620 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.459172 (+   552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17969536,"thread_start_us":102671,"threads_started":1,"wal-append.queue_time_us":158942}
W20260709 12:25:08.040079 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.215314 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:08.216898 13725 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:08.508616 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3928) took 1083 ms. Trace:
I20260709 12:25:08.509192 13725 rpcz_store.cc:276] 0709 12:25:07.424563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.011797 (+587234us) service_pool.cc:224] Handling call
0709 12:25:08.508282 (+496485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.028002 22304 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:08.212286 13726 raft_consensus.cc:2368] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28.
I20260709 12:25:08.512164 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3906) took 1162 ms. Trace:
I20260709 12:25:08.512689 13726 rpcz_store.cc:276] 0709 12:25:07.349943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.050177 (+700234us) service_pool.cc:224] Handling call
0709 12:25:08.511886 (+461709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.006801 13796 rpcz_store.cc:276] 0709 12:25:05.252591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.886793 (+634202us) service_pool.cc:224] Handling call
0709 12:25:07.467213 (+1580420us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54000e4cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:07.467378 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532087552}
I20260709 12:25:08.508591 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1461) took 2124 ms. Trace:
W20260709 12:25:08.540813 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.541680 22321 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.015s	sys 0.000s
I20260709 12:25:08.166882 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:08.533589 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:08.546015 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3924) took 1120 ms. Trace:
I20260709 12:25:08.508598 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:08.531185 13799 rpcz_store.cc:276] 0709 12:25:06.383769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.383998 (+   229us) service_pool.cc:224] Handling call
0709 12:25:06.384317 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.396867 (+ 12550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.396897 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.406564 (+  9667us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.279211 (+872647us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b18001df9e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:25:07.459898 (+180687us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:07.459936 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.460010 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.507492 (+1047482us) raft_consensus.cc:1685] finished
0709 12:25:08.507545 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.508159 (+   614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13935,"raft.queue_time_us":14507,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":5153,"spinlock_wait_cycles":3359104,"thread_start_us":175945,"threads_started":1,"wal-append.queue_time_us":1352980}
I20260709 12:25:08.553362 13726 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:08.554852 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3903) took 1205 ms. Trace:
W20260709 12:25:08.555397 13651 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.010s	sys 0.013s
I20260709 12:25:08.555348 13726 rpcz_store.cc:276] 0709 12:25:07.348969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.535386 (+1186417us) service_pool.cc:224] Handling call
0709 12:25:08.552892 (+ 17506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.554342 (+  1450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.554742 (+   400us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:08.555994 13651 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.010s	sys 0.013s
W20260709 12:25:08.559419 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.006s
W20260709 12:25:08.560038 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.004s	sys 0.006s
I20260709 12:25:08.562975 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.563740 13725 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:25:08.564242 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3921) took 1140 ms. Trace:
I20260709 12:25:08.564404 13646 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:08.564669 13725 rpcz_store.cc:276] 0709 12:25:07.423329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.532812 (+1109483us) service_pool.cc:224] Handling call
0709 12:25:08.564158 (+ 31346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.563089 13721 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:08.570947 13725 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:08.572132 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3904) took 1222 ms. Trace:
I20260709 12:25:08.572757 13725 rpcz_store.cc:276] 0709 12:25:07.349295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.565807 (+1216512us) service_pool.cc:224] Handling call
0709 12:25:08.570413 (+  4606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.571587 (+  1174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.572009 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.572830 13648 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:25:08.548122 13727 rpcz_store.cc:276] 0709 12:25:07.423882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.214920 (+791038us) service_pool.cc:224] Handling call
0709 12:25:08.544212 (+329292us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:08.576761 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.575206 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:08.594749 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4590) took 1826 ms. Trace:
I20260709 12:25:08.367192 22697 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.595798 13725 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:25:08.596489 13725 pending_rounds.cc:85] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 11
I20260709 12:25:08.596861 13725 pending_rounds.cc:107] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:25:08.597259 13725 raft_consensus.cc:2889] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:08.598731 22791 raft_consensus.cc:2764] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
W20260709 12:25:08.599522 22806 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:25:08.599644 13727 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:08.600459 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3923) took 1176 ms. Trace:
I20260709 12:25:08.600919 13727 rpcz_store.cc:276] 0709 12:25:07.423719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.598363 (+1174644us) service_pool.cc:224] Handling call
0709 12:25:08.599063 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.600123 (+  1060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.600365 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.601853 22803 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.597186 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.602816 22803 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.595465 13651 rpcz_store.cc:276] 0709 12:25:06.768441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.807594 (+ 39153us) service_pool.cc:224] Handling call
0709 12:25:06.807955 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.808221 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:07.132237 (+324016us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.556956 (+1424719us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.594010 (+ 37054us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.594114 (+   104us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:08.594232 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.594375 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.594664 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19528,"prepare.queue_time_us":606510,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":130,"raft.queue_time_us":797969,"raft.run_cpu_time_us":478,"raft.run_wall_time_us":3385,"thread_start_us":1723829,"threads_started":4,"wal-append.queue_time_us":375924,"wal-append.run_cpu_time_us":1741,"wal-append.run_wall_time_us":1083321}
I20260709 12:25:08.606679 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:08.607883 13796 raft_consensus.cc:606] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:08.613016 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4285) took 1941 ms. Trace:
W20260709 12:25:08.614385 14389 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.010s
I20260709 12:25:08.616398 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3920) took 1193 ms. Trace:
I20260709 12:25:08.617143 13725 rpcz_store.cc:276] 0709 12:25:07.422744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.583992 (+1161248us) service_pool.cc:224] Handling call
0709 12:25:08.595014 (+ 11022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.598068 (+  3054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.616306 (+ 18238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.618741 22304 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.619815 22304 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.622848 13727 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:08.598160 22661 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:08.623407 13727 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:08.602609 22677 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:08.613613 13801 rpcz_store.cc:276] 0709 12:25:06.671395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.671541 (+   146us) service_pool.cc:224] Handling call
0709 12:25:07.468431 (+796890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.007109 (+538678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.610293 (+603184us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54000e4cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:08.612250 (+  1957us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54000e4cd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:08.612609 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1870570112}
I20260709 12:25:08.598148 22697 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.626832 22803 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:08.643503 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 2.344e+06us
W20260709 12:25:08.645152 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.815s (SENT) [suppressed 29 similar messages]
I20260709 12:25:08.650169 13727 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.651948 13726 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:25:08.652987 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3936) took 1163 ms. Trace:
I20260709 12:25:08.653353 13726 rpcz_store.cc:276] 0709 12:25:07.490079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.649568 (+1159489us) service_pool.cc:224] Handling call
0709 12:25:08.651495 (+  1927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.652676 (+  1181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.652918 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.654428 22304 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.651991 13727 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:08.656173 22697 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:08.656446 22802 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.656888 22576 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:08.657083 22697 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.657232 22802 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.660187 22802 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.660290 22697 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:08.664007 22739 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 2.389s	user 0.012s	sys 0.026s
I20260709 12:25:08.665151 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:08.678480 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.664011 22405 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.014s
I20260709 12:25:08.691577 22697 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:08.696998 22697 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.699888 22697 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:08.702562 13617 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.306s (SENT)
I20260709 12:25:08.703299 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:08.711927 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4287) took 2037 ms. Trace:
I20260709 12:25:08.712455 13792 rpcz_store.cc:276] 0709 12:25:06.674687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.674858 (+   171us) service_pool.cc:224] Handling call
0709 12:25:07.469430 (+794572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.007598 (+538168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.711441 (+703843us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54000d2ad0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:08.711713 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1775312128}
I20260709 12:25:08.728494 22502 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:08.729291 22502 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.730398 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4585) took 1974 ms. Trace:
I20260709 12:25:08.730830 13645 rpcz_store.cc:276] 0709 12:25:06.755897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.756819 (+   922us) service_pool.cc:224] Handling call
0709 12:25:06.757153 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.757380 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:25:06.791115 (+ 33735us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.793913 (+  2798us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.699579 (+1905666us) spinlock_profiling.cc:243] Waited 916 us on lock 0x7b18003dfba0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f53937aad70 00007f539379e293 00007f5393793cea 00007f539379423a 00007f5393bfea2a 00007f5393c0e0bf 00007f5393c0e038
0709 12:25:08.729631 (+ 30052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.729729 (+    98us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:08.729811 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.729928 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.730187 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49322,"prepare.queue_time_us":371699,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":93,"raft.queue_time_us":3463,"spinlock_wait_cycles":10687104}
I20260709 12:25:08.732071 22502 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.738018 22537 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:08.739223 22791 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:08.740126 22791 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.740589 22537 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.743381 22537 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.745324 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.749357 22502 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.750492 22502 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:08.753750 22502 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:08.754804 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.756168 13645 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:08.759552 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:08.760465 13792 raft_consensus.cc:606] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:08.760959 13792 raft_consensus.cc:614] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:08.762449 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 97586.6us
I20260709 12:25:08.764778 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:08.727735 13542 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 17 to 19, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 19 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:08.767122 22788 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:08.676532 22576 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:08.773270 14418 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:08.773531 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:08.774576 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:08.780015 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.780718 13648 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:25:08.781148 13651 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:08.781195 13643 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 12:25:08.782374 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 584816us
I20260709 12:25:08.783326 13650 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:08.785709 13650 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:25:08.787449 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:08.788410 13645 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:08.792894 22321 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.793659 22321 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.794395 22537 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:08.811018 22766 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:08.811655 22766 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:08.812984 22766 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:08.813725 13801 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:08.817600 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:08.818902 13796 raft_consensus.cc:2410] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:25:08.821595 22766 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:08.821482 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:08.823109 22775 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.486s	user 0.005s	sys 0.022s
W20260709 12:25:08.823534 13693 connection.cc:256] RPC call timeout handler was delayed by -1.397275167999851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
W20260709 12:25:08.824113 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3900751679998393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 12:25:08.824505 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3819751679998262s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:25:08.824860 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3477751679997709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 12:25:08.825179 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2991751679996923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 12:25:08.825490 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1416751679998924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 12:25:08.825788 13693 connection.cc:256] RPC call timeout handler was delayed by -1.05977516799976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 12:25:08.768582 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:08.836599 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:08.826229 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0480751679997411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 12:25:08.843041 13721 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:08.850473 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.851366 13723 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:25:08.851760 13650 raft_consensus.cc:2410] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:25:08.827286 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4594) took 2032 ms. Trace:
I20260709 12:25:08.852748 13649 rpcz_store.cc:276] 0709 12:25:06.796238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.881900 (+ 85662us) service_pool.cc:224] Handling call
0709 12:25:06.886228 (+  4328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.886489 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.886525 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.887026 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.203057 (+316031us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:07.203128 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.203228 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.821805 (+1618577us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:08.826618 (+  4813us) raft_consensus.cc:1685] finished
0709 12:25:08.826648 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.827024 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79289216,"thread_start_us":102246,"threads_started":1,"wal-append.queue_time_us":265280}
I20260709 12:25:08.837598 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:08.855064 13796 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:08.855280 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.855739 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.859355 14418 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:08.860617 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:08.861187 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:08.862691 13651 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:08.863183 13651 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:08.863718 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:08.801527 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:08.864701 13651 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:08.866519 22740 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 2.585s	user 0.068s	sys 0.138s
I20260709 12:25:08.867491 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:08.869226 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3922) took 1445 ms. Trace:
I20260709 12:25:08.869510 13646 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:08.869745 13723 rpcz_store.cc:276] 0709 12:25:07.423554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.446941 (+ 23387us) service_pool.cc:224] Handling call
0709 12:25:08.868938 (+1421997us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54003fd4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:08.869144 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3184000256}
I20260709 12:25:08.871392 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.861512 14424 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:08.873641 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.802940 22697 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 12:25:08.874866 13643 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:08.896378 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.897441 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:08.874225 13722 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:08.875386 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.901180 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.876420 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:08.903322 13647 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:25:08.950834 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.878113 13725 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:09.036029 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3933) took 1580 ms. Trace:
I20260709 12:25:09.036652 13725 rpcz_store.cc:276] 0709 12:25:07.455554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.618411 (+1162857us) service_pool.cc:224] Handling call
0709 12:25:08.877495 (+259084us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:25:08.877791 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.035663 (+157872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.035942 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15062784}
I20260709 12:25:08.882241 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:09.010358 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.969453 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3932) took 1488 ms. Trace:
W20260709 12:25:08.880424 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 112038us
I20260709 12:25:08.879354 13649 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:25:08.979100 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3934) took 1512 ms. Trace:
I20260709 12:25:09.044735 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
W20260709 12:25:08.938565 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 154619us
I20260709 12:25:09.044011 13721 rpcz_store.cc:276] 0709 12:25:07.457067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.166617 (+709550us) service_pool.cc:224] Handling call
0709 12:25:08.939988 (+773371us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.027550 13727 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:25:08.904850 13645 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17.
I20260709 12:25:09.041771 13722 rpcz_store.cc:276] 0709 12:25:07.451294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.606834 (+1155540us) service_pool.cc:224] Handling call
0709 12:25:08.873286 (+266452us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:25:08.873673 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:08.899459 (+ 25786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.939568 (+ 40109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39597312}
I20260709 12:25:09.047945 13727 pending_rounds.cc:85] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:25:08.866150 13651 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:09.048842 13727 pending_rounds.cc:107] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:25:09.050526 13727 raft_consensus.cc:2889] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:09.053525 13648 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:25:09.055094 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.059512 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:09.062649 13646 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:25:09.063032 14423 raft_consensus.cc:2376] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27. Re-sending same reply.
I20260709 12:25:08.796659 22321 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:09.063710 22700 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.888829 22633 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:09.065240 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3925) took 1641 ms. Trace:
I20260709 12:25:09.068394 13727 rpcz_store.cc:276] 0709 12:25:07.424045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.601926 (+1177881us) service_pool.cc:224] Handling call
0709 12:25:08.622472 (+ 20546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.064946 (+442474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.065175 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.066155 13792 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 12:25:08.943456 22770 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.603s	user 0.016s	sys 0.014s
I20260709 12:25:08.886945 22537 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:09.076640 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.884415 22697 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:08.890849 22663 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:09.080220 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 197496us
W20260709 12:25:09.088819 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.888176 22721 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:25:09.092727 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62068.5us
I20260709 12:25:09.100013 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:09.101608 22671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.735s	user 0.020s	sys 0.030s
I20260709 12:25:09.101740 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:09.102205 22671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.760s	user 0.020s	sys 0.030s
W20260709 12:25:09.102967 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 257) took 40698 ms (client timeout 15000 ms). Trace:
I20260709 12:25:09.106886 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:09.108273 22729 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.892s	user 0.017s	sys 0.007s
I20260709 12:25:09.080974 22321 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.109534 22321 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.112982 22321 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:09.103012 14424 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:09.091934 22721 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:09.086915 22697 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:09.126051 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:09.130971 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.088526 22700 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.135190 22663 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:09.155982 14424 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:09.166510 22680 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:25:09.168813 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4103) took 2458 ms. Trace:
I20260709 12:25:09.155929 22809 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:09.170841 22243 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:25:09.170902 22809 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.179172 22721 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:09.103559 13547 rpcz_store.cc:269] 0709 12:24:28.404932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.215868 (+10810936us) service_pool.cc:224] Handling call
0709 12:25:09.102614 (+29886746us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:01.106653 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:01.107340 (+   687us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:01.107399 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:25:01.107417 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:01.109806 (+  2389us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:01.109923 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:01.109949 (+    26us) tablet.cc:685] Acquiring locks for 32 operations
0709 12:25:01.122893 (+ 12944us) tablet.cc:701] Row locks acquired
0709 12:25:01.122941 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:25:01.123202 (+   261us) write_op.cc:270] Start()
0709 12:25:01.123376 (+   174us) write_op.cc:276] Timestamp: P: 1783599901123131 usec, L: 0
0709 12:25:01.123410 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:25:01.124446 (+  1036us) log.cc:844] Serialized 12487 byte log entry
0709 12:25:02.277514 (+1153068us) op_driver.cc:464] REPLICATION: finished
0709 12:25:02.539589 (+262075us) spinlock_profiling.cc:243] Waited 954 us on lock 0x7b180024b680. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:02.539777 (+   188us) write_op.cc:301] APPLY: starting
0709 12:25:02.539903 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:02.717711 (+177808us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:02.717759 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:03.172309 (+454550us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:03.173146 (+   837us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=0,mrs_lookups=0
0709 12:25:03.173269 (+   123us) write_op.cc:312] APPLY: finished
0709 12:25:03.186727 (+ 13458us) log.cc:844] Serialized 276 byte log entry
0709 12:25:03.322306 (+135579us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:03.332194 (+  9888us) write_op.cc:454] Released schema lock
0709 12:25:03.335417 (+  3223us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:03.335989 (+   572us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:03.336020 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7873210,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":152563,"apply.run_cpu_time_us":96227,"apply.run_wall_time_us":796188,"cfile_cache_hit":130,"cfile_cache_hit_bytes":138143,"cfile_init":1,"lbm_read_time_us":439,"lbm_reads_lt_1ms":4,"num_ops":32,"prepare.queue_time_us":434218,"prepare.run_cpu_time_us":28301,"prepare.run_wall_time_us":241674,"raft.queue_time_us":178887,"raft.run_cpu_time_us":8034,"raft.run_wall_time_us":270595,"replication_time_us":1090807,"spinlock_wait_cycles":2193152,"thread_start_us":823784,"threads_started":5,"wal-append.queue_time_us":498945,"wal-append.run_cpu_time_us":30174,"wal-append.run_wall_time_us":2738810}]]}
I20260709 12:25:09.180603 22721 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:09.169350 13724 rpcz_store.cc:276] 0709 12:25:06.666008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.666177 (+   169us) service_pool.cc:224] Handling call
0709 12:25:06.698082 (+ 31905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.097055 (+398973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:07.097090 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.097138 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:07.097169 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.078660 (+1981491us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b5400026cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:09.084131 (+  5471us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.124617 (+ 40486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917013760}
W20260709 12:25:09.188158 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.191376 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:09.193056 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86340.9us
I20260709 12:25:09.194101 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:09.197445 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:09.199009 22789 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.921s	user 0.001s	sys 0.015s
I20260709 12:25:09.202113 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:09.203397 22697 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.205055 22697 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:09.212504 22783 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.932s	user 0.004s	sys 0.016s
I20260709 12:25:09.215770 22696 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:09.216504 22696 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:09.219256 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 158) took 22336 ms (client timeout 15000 ms). Trace:
I20260709 12:25:09.220206 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4115) took 2427 ms. Trace:
I20260709 12:25:09.220728 13719 rpcz_store.cc:276] 0709 12:25:06.792953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.824642 (+ 31689us) service_pool.cc:224] Handling call
0709 12:25:06.824944 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.825167 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.825191 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.830171 (+  4980us) log.cc:844] Serialized 26 byte log entry
0709 12:25:07.313322 (+483151us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:07.313359 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.313448 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:08.893768 (+1580320us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:09.219571 (+325803us) raft_consensus.cc:1685] finished
0709 12:25:09.219597 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.220029 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12164864,"thread_start_us":482792,"threads_started":1,"wal-append.queue_time_us":510023}
I20260709 12:25:09.222764 13718 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:09.224563 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3897) took 1897 ms. Trace:
I20260709 12:25:09.224974 13718 rpcz_store.cc:276] 0709 12:25:07.327132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.351250 (+ 24118us) service_pool.cc:224] Handling call
0709 12:25:09.222385 (+1871135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.223286 (+   901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.224322 (+  1036us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54001f0750. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:09.224480 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4278434176}
I20260709 12:25:09.234524 13720 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 12:25:09.236152 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3913) took 1855 ms (client timeout 2106 ms). Trace:
W20260709 12:25:09.236580 13720 rpcz_store.cc:269] 0709 12:25:07.380436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.409915 (+ 29479us) service_pool.cc:224] Handling call
0709 12:25:09.235875 (+1825960us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54003fded4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:09.236077 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3767885440}
I20260709 12:25:09.244658 22791 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:09.246963 13764 connection.cc:256] RPC call timeout handler was delayed by -1.2778044250003404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 12:25:09.248050 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.251103 22809 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:09.251010 13542 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 17 to 19, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 19 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:09.189285 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48018.1us
W20260709 12:25:09.220014 13547 rpcz_store.cc:269] 0709 12:24:46.883442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.218715 (+22335273us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:09.219083 (+   368us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:25:09.152859 22730 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:09.264268 22730 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:09.274865 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.266300 22670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.734s	user 0.017s	sys 0.024s
W20260709 12:25:09.276284 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.282173 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.278235 13651 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:25:09.282821 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:09.286182 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75995.6us
I20260709 12:25:09.258098 22405 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.287997 14424 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
W20260709 12:25:09.293392 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.294090 13617 connection.cc:256] RPC call timeout handler was delayed by -2.3938947870001357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 12:25:09.294602 13617 connection.cc:256] RPC call timeout handler was delayed by -2.3907149490000847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 12:25:09.293437 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:09.302418 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 285932us
W20260709 12:25:09.276050 22670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.937s	user 0.018s	sys 0.025s
I20260709 12:25:09.284845 22802 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:09.302507 22492 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:09.306190 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:09.265795 22730 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.287110 22729 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:09.316634 22594 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:09.312666 22791 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:09.318794 22362 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:09.319479 22791 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:09.319976 22791 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:09.304899 22663 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:09.322932 22663 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:09.325292 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 127977us
I20260709 12:25:09.326556 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:09.327601 22576 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:09.327430 13693 leader_election.cc:400] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24.
I20260709 12:25:09.327787 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.328456 22576 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:09.325696 22663 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.303962 22809 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:09.340821 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:09.345425 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.749s (SENT)
I20260709 12:25:09.345981 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:09.290614 22405 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:09.353335 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 105510us
I20260709 12:25:09.339347 22766 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:09.342650 22697 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.362457 22396 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.004s
I20260709 12:25:09.366338 22809 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.368026 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.368839 22809 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.369827 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.370352 22537 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:09.370554 22766 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.371423 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.371815 22576 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 30, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.364933 22803 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:09.380277 14424 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:09.386250 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.386559 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:09.387038 13718 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:25:09.406916 22816 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:25:09.409170 22766 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.407087 13797 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:09.392086 22396 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:09.365342 22680 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 12:25:09.417127 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89437.6us
W20260709 12:25:09.418361 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82813.3us
W20260709 12:25:09.422176 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92299.4us
I20260709 12:25:09.412607 22700 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:09.426707 13720 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:09.411670 22697 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:09.437392 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.123s (SENT)
W20260709 12:25:09.440286 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 113066us
W20260709 12:25:09.441871 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.441911 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:09.443249 22694 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:09.439330 13719 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:25:09.448966 22537 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:25:09.451135 22815 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.148s	user 0.002s	sys 0.001s
I20260709 12:25:09.456399 22243 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:09.457773 22243 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:09.460855 22766 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:09.461428 22766 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:09.462756 22766 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.464223 22405 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.461477 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4297) took 2059 ms. Trace:
W20260709 12:25:09.450139 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80426.5us
I20260709 12:25:09.465221 13797 rpcz_store.cc:276] 0709 12:25:07.402116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.402250 (+   134us) service_pool.cc:224] Handling call
0709 12:25:07.402583 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.422827 (+2020244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.461154 (+ 38327us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:09.461410 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4608590720}
I20260709 12:25:09.435449 22818 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:09.472925 13718 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:09.464294 22766 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:09.481343 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.474753 22405 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.486444 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.500085 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.484934 22809 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:25:09.490402 22818 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.482651 22700 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.495504 22791 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.497026 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.506206 22700 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.506870 22809 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:09.507433 22809 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:09.509078 22700 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.511974 22818 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.508739 22809 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:09.524240 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.528385 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:09.528800 13651 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:25:09.529520 13644 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:09.529618 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.530663 13727 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:25:09.532891 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:09.535593 22694 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:09.535655 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:09.536520 22694 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.537418 22526 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.498167 22537 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:25:09.528693 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:09.539112 13648 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:09.539721 22694 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:09.540009 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108300us
I20260709 12:25:09.540426 22396 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:09.540666 13617 connection.cc:256] RPC call timeout handler was delayed by -2.2378638039999714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31980000000000003s for a response
W20260709 12:25:09.541074 13617 connection.cc:256] RPC call timeout handler was delayed by -2.2252638039999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32120000000000004s for a response
W20260709 12:25:09.541421 13617 connection.cc:256] RPC call timeout handler was delayed by -2.2093947870002921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260709 12:25:09.541172 22396 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.538030 22526 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.544060 22396 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:09.541751 13617 connection.cc:256] RPC call timeout handler was delayed by -2.0239947870004471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2377s for a response
W20260709 12:25:09.544689 13617 connection.cc:256] RPC call timeout handler was delayed by -1.9798947870003758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24260000000000004s for a response
I20260709 12:25:09.549701 13650 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:09.550190 13650 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:09.551407 13650 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.550132 13646 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:25:09.552524 13650 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:09.553766 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.554647 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:09.555404 13652 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:09.556736 13643 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:09.556974 22694 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:09.557245 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.557971 22671 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:09.558503 22671 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:09.559015 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:09.559998 22594 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:09.559870 22671 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:09.560671 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63794.5us
I20260709 12:25:09.560850 22594 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:09.561442 13724 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:09.561599 22671 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:09.562171 22537 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:09.561514 22594 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:09.563807 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:09.563865 22594 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:09.564388 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3966947870003423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3074s for a response
W20260709 12:25:09.565454 13617 connection.cc:256] RPC call timeout handler was delayed by -1.2910638040002596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.42500000000000004s for a response
I20260709 12:25:09.563866 14424 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
W20260709 12:25:09.567371 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1455947870003911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33530000000000004s for a response
I20260709 12:25:09.514163 22809 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:09.589553 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.619062 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87306.8us
I20260709 12:25:09.621784 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:09.625154 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63511.7us
I20260709 12:25:09.562955 22537 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:25:09.594708 22803 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.642827 22803 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:09.647938 22803 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.649928 13725 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:09.655043 13722 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:09.659413 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.660533 13727 raft_consensus.cc:2376] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25. Re-sending same reply.
W20260709 12:25:09.663133 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:09.667137 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 108242us
I20260709 12:25:09.604264 22405 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.687733 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.688856 13721 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:25:09.688385 22697 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.697916 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:09.698292 22791 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:09.685750 22537 consensus_queue.cc:1243] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:09.712431 13724 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:25:09.712934 22697 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:09.713640 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.713822 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.727877 13724 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:09.714509 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:25:09.728799 22696 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.512s	user 0.015s	sys 0.000s
I20260709 12:25:09.729233 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.729328 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.730023 13726 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:09.730185 13724 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:25:09.731587 13724 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:09.731856 22405 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:09.732592 22776 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:09.733505 13648 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:09.714828 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:09.736037 13617 leader_election.cc:400] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:25:09.715372 22828 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:09.668591 22396 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 12:25:09.738970 22828 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:09.715588 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.740440 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.705049 13764 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:09.742429 13792 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:09.742961 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.743660 22803 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:09.717366 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.744052 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.718248 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:09.718739 22526 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:09.720988 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.721071 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.722128 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:09.736375 13617 leader_election.cc:403] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:09.728202 13725 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:25:09.746095 13722 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:09.712450 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:09.756173 13719 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:09.760141 13644 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:09.745702 13723 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:09.770337 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:09.773207 14417 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:09.767243 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:09.769515 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.780390 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.744064 22803 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:09.787096 13725 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:09.789870 13724 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:09.797001 22730 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:09.797729 13794 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:09.798593 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:09.798969 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.799671 13794 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:25:09.799737 22677 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:09.811069 13726 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:09.816524 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.773052 22680 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:09.832469 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.833321 13727 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:25:09.836789 13798 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:09.837714 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.838560 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:09.843096 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:09.846683 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:09.839735 13799 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:09.839804 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.729254 22696 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:09.867828 13720 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:25:09.837960 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.864302 13722 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 31 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:09.786217 22405 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:09.895442 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:09.891564 13721 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:09.818424 22677 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:09.907302 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.849439 22766 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.912353 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62384.3us
I20260709 12:25:09.914942 13796 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:09.915656 13720 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:25:09.912853 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135562us
I20260709 12:25:09.785805 22776 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.115s
W20260709 12:25:09.914098 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.853645 22362 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.060s
I20260709 12:25:09.924463 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:09.904908 22677 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:09.932385 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 220013us
I20260709 12:25:09.934902 13800 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
W20260709 12:25:09.960409 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.999927 13718 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
W20260709 12:25:10.005752 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.377s (SENT)
I20260709 12:25:09.919214 22766 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:10.007746 14417 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:10.003875 13718 pending_rounds.cc:85] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:25:10.011921 13718 pending_rounds.cc:107] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:25:10.012271 13718 raft_consensus.cc:2889] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:10.016427 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.017673 13720 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:25:10.018437 13793 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:25:10.019469 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43493.4us
W20260709 12:25:10.020219 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.022346 13652 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:10.021397 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:10.036159 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.055459 13693 leader_election.cc:400] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:25:10.056181 13693 leader_election.cc:403] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:10.057507 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.426s (SENT)
I20260709 12:25:10.066510 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:10.002135 22556 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.276s
W20260709 12:25:10.065917 22671 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.504s	user 0.016s	sys 0.000s
W20260709 12:25:10.082907 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.080587 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:10.089453 13718 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:10.091578 13794 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:25:10.074640 22791 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:25:10.077904 22809 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20.
I20260709 12:25:10.078727 22730 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:10.112345 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.040058 22803 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:10.126904 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 81309.2us
W20260709 12:25:10.141758 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.145998 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.146834 13718 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:10.147061 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 119188us
I20260709 12:25:10.159780 22766 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:10.162320 22792 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:10.163100 22792 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:10.164206 22792 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:10.169014 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.198s (SENT)
I20260709 12:25:10.169557 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:10.170671 22850 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:10.170779 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:10.171113 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.212s (SENT)
I20260709 12:25:10.171584 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:10.172387 22850 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:25:10.173390 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.353s (SENT)
I20260709 12:25:10.138765 22696 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:10.191618 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 245417us
I20260709 12:25:10.210412 13721 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:25:10.240097 22833 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 12:25:10.267645 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.271620 13793 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:25:10.297201 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.322569 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 149204us
W20260709 12:25:10.327473 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121810us
W20260709 12:25:10.321220 22837 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.252s	user 0.000s	sys 0.002s
I20260709 12:25:10.228466 22766 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 12:25:10.347867 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.358814 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:25:10.326674 22841 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:10.350248 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:10.361047 13723 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:10.360736 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.361384 13725 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 31 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
W20260709 12:25:10.356271 13617 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 4.250s (SENT)
W20260709 12:25:10.338507 22839 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.244s	user 0.000s	sys 0.002s
W20260709 12:25:10.350911 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 518522us
I20260709 12:25:10.349447 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.365690 13726 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:10.362049 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.367851 13718 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:10.362046 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:10.377652 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:10.364766 22842 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 12:25:10.345034 22537 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
W20260709 12:25:10.375077 22838 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.259s	user 0.002s	sys 0.000s
I20260709 12:25:10.388577 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.391110 13726 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:25:10.385107 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:10.374734 22576 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:10.369974 22850 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:10.379978 22766 consensus_queue.cc:1243] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:25:10.405582 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106921us
I20260709 12:25:10.407801 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:10.392578 22405 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:10.381083 22841 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:10.423058 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47447.7us
I20260709 12:25:10.434520 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:10.417604 22396 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:10.439217 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82994.2us
I20260709 12:25:10.443935 13726 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:10.464794 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.465798 13650 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:10.466588 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.467128 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.414606 22850 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.471061 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:10.459558 22845 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
I20260709 12:25:10.475231 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:10.478507 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:10.475844 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.473848 13652 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:10.479121 13645 raft_consensus.cc:2376] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 17. Re-sending same reply.
W20260709 12:25:10.471666 22840 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.229s	user 0.002s	sys 0.000s
W20260709 12:25:10.486339 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.467897 22396 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:10.471544 22853 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:25:10.512193 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:10.494626 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.485745 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.513721 22853 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:25:10.497253 22858 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:25:10.528482 22803 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:10.497539 22867 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:25:10.471781 22872 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:25:10.502924 22850 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:10.522640 22697 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.535276 22697 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.503242 22841 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:10.489745 22869 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:10.507813 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:10.538911 22869 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:10.539911 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.539597 22697 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:10.540082 22869 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:10.508090 22396 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.509668 14417 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:25:10.509863 14424 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:25:10.510198 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63353.9us
W20260709 12:25:10.486435 22861 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.153s	user 0.002s	sys 0.000s
I20260709 12:25:10.514390 13649 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:25:10.495859 22855 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.002s
W20260709 12:25:10.492126 22863 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:25:10.545045 22680 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:10.559208 22680 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:10.559688 22864 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:25:10.547735 22862 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
W20260709 12:25:10.563325 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.597994 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.597594 13644 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:10.560362 22680 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:10.606434 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95063us
I20260709 12:25:10.602000 22850 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:25:10.580412 22841 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.032s
W20260709 12:25:10.641738 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 249627us
W20260709 12:25:10.642444 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.645743 22405 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:10.646638 22405 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.646713 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.647204 22405 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:10.650449 22677 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.006s
I20260709 12:25:10.650914 22677 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:10.646101 22697 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.654134 22697 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:10.653113 22677 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:10.658824 13644 raft_consensus.cc:1217] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Deduplicated request from leader. Original: 15.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 12:25:10.659416 13644 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:10.666618 22846 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:25:10.667049 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.667188 22846 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:10.668366 22846 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:10.670228 22877 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:10.666667 22526 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:25:10.673519 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.676085 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.676782 13649 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:10.677487 13645 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:10.680052 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.680792 13645 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:10.681886 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.683320 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.684545 22697 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:10.684376 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.670177 22846 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:10.690116 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.691650 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.692747 13649 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:10.695272 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.695973 13649 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:10.696975 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.699275 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.699913 22526 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:10.705206 13643 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:25:10.700284 13645 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:25:10.662432 22405 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:10.708915 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:10.713348 22853 raft_consensus.cc:2764] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:25:10.713562 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:10.717855 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.725458 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.726809 13720 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:25:10.726912 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:10.727105 13800 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 27. Ops: []
I20260709 12:25:10.729537 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.729964 13795 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:10.730111 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.728199 13720 pending_rounds.cc:85] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:25:10.731519 13720 pending_rounds.cc:107] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:25:10.731793 13720 raft_consensus.cc:2889] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:10.732609 22697 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:10.733187 22697 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:10.734018 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.728374 13644 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
W20260709 12:25:10.736156 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70356.7us
W20260709 12:25:10.797168 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.762897 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.803927 22888 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.019s
I20260709 12:25:10.805527 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:10.806394 22526 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:10.807054 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:10.809267 22881 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:10.814183 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.796427 22834 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.781s	user 0.006s	sys 0.000s
I20260709 12:25:10.793254 22405 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.821980 22405 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.825407 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4630) took 1177 ms. Trace:
I20260709 12:25:10.744282 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4302) took 1001 ms. Trace:
I20260709 12:25:10.754565 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.830184 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.828687 13796 rpcz_store.cc:276] 0709 12:25:09.742547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.742711 (+   164us) service_pool.cc:224] Handling call
0709 12:25:10.744054 (+1001343us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:10.744214 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387648896}
W20260709 12:25:10.803550 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.780583 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.804818 22853 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:10.794588 22880 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:25:10.835227 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.836535 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.837597 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.737003 22697 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:10.839815 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.788471 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.825929 13643 rpcz_store.cc:276] 0709 12:25:09.647523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.788357 (+140834us) service_pool.cc:224] Handling call
0709 12:25:09.850786 (+ 62429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.705652 (+854866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.824986 (+119334us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:10.825301 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964024576}
I20260709 12:25:10.766181 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.831072 13799 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:10.738196 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:10.843868 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:10.756048 22766 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.738559 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.858124 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:10.859658 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:10.860805 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:10.869917 13800 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:10.825452 22405 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:10.872795 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:10.879943 14423 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:10.883896 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:10.890820 22671 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:10.891343 22880 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:25:10.919200 22362 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:10.919735 22362 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:10.920579 14417 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:10.902302 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:10.891084 22405 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.878823 22766 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.892433 22526 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.011s
I20260709 12:25:10.904172 22868 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:25:10.908249 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4614) took 1360 ms. Trace:
I20260709 12:25:10.889025 22888 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 15, Time since last communication: 0.013s
W20260709 12:25:10.910717 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:10.926155 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.893982 22537 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:10.922422 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.923978 13724 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:10.924887 13651 rpcz_store.cc:276] 0709 12:25:09.547751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.558005 (+ 10254us) service_pool.cc:224] Handling call
0709 12:25:09.708547 (+150542us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:25:09.708753 (+   206us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:09.709172 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.709203 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:09.710158 (+   955us) log.cc:844] Serialized 52 byte log entry
0709 12:25:09.792956 (+ 82798us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b480033c558. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:09.868698 (+ 75742us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:09.868762 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.868871 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.906920 (+1038049us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b2400196e68. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b45fb0 00007f53937bc488 00007f53937bcbe8 00007f5393794398 00007f539378f76a 00007f539378ebb6
0709 12:25:10.907967 (+  1047us) raft_consensus.cc:1685] finished
0709 12:25:10.907990 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.908168 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10067,"raft.run_cpu_time_us":3239,"raft.run_wall_time_us":73897,"spinlock_wait_cycles":104310784,"thread_start_us":141865,"threads_started":1,"wal-append.queue_time_us":334837}
I20260709 12:25:10.932885 13724 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:10.934415 13693 connection.cc:256] RPC call timeout handler was delayed by -1.647118483000213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 12:25:10.934866 13693 connection.cc:256] RPC call timeout handler was delayed by -1.6012184830001388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 12:25:10.935231 13693 connection.cc:256] RPC call timeout handler was delayed by -1.5922184830001243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
W20260709 12:25:10.935524 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3780184830002327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18020000000000003s for a response
W20260709 12:25:10.935823 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2133184829999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 12:25:10.936152 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0486184830001548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 12:25:10.933920 22362 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.937611 22537 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.938681 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:10.939500 13649 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:10.940070 22537 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:10.938601 22362 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:10.941538 13647 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:10.941694 22412 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:25:10.942801 22857 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:10.943193 22857 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:10.944458 22857 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.23, Last appended by leader: 23, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.945364 13648 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:10.938094 13650 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:10.945919 13644 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:10.947010 14424 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:10.913888 22671 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:25:10.947010 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:10.951309 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61152.3us
I20260709 12:25:10.956099 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:10.957021 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:10.957500 22696 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:10.958055 13720 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:25:10.957103 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.958600 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.959086 13727 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
W20260709 12:25:10.960660 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.933233 22880 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:10.932585 22697 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.959254 13718 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30. Re-sending same reply.
I20260709 12:25:10.946204 22857 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:10.925176 22405 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:10.977324 22890 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.977994 22890 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:10.955041 22853 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:10.984839 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:10.980729 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:10.952862 22412 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:25:10.982903 22849 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:25:10.992754 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.988173 22594 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:25:10.992803 22696 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:10.993397 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.024309 13727 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:25:10.994757 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:11.025044 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70228.9us
I20260709 12:25:11.025166 13644 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:10.981181 22526 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:25:10.996694 22870 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:25:11.000159 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.966826 22880 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:10.981454 13724 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.038595 22880 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:11.038941 13724 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:11.003880 14424 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:25:11.005292 22890 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:11.010949 22853 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:10.973893 22697 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:11.015776 22405 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:11.060935 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.016912 13800 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:25:11.017223 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:11.017527 22677 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:11.018540 22868 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:11.019338 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.063310 13718 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:25:11.021430 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:11.022083 22537 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:11.022768 22766 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:11.028684 13649 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:25:11.036943 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60405.7us
I20260709 12:25:11.048801 13792 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:25:11.017927 22694 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
W20260709 12:25:11.066627 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.026575 22696 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:11.072343 22405 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:11.072692 22696 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:11.073120 22405 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:11.075585 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.076386 13720 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:25:11.047700 22881 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:11.071308 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:11.080101 13646 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:11.057672 22853 consensus_queue.cc:1243] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:11.017951 22776 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 12:25:11.105122 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.054662 22671 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:25:11.043196 22849 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:25:11.108518 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.055826 22841 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:25:11.121801 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104667us
I20260709 12:25:11.080929 22694 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:25:11.129710 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.137915 14420 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:11.136006 13644 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:11.138036 13720 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:25:11.144814 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 123487us
I20260709 12:25:11.145906 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.146826 13720 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:25:11.150599 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79401.2us
I20260709 12:25:11.151857 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:11.081290 22862 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:25:11.138213 22694 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:11.175603 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95227.1us
I20260709 12:25:11.190097 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.190577 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.202772 13725 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:11.191635 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.192000 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.205549 13726 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:11.192656 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.207926 13722 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31.
I20260709 12:25:11.196029 13718 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:25:11.209272 13718 pending_rounds.cc:85] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 10
I20260709 12:25:11.209587 13718 pending_rounds.cc:107] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:25:11.209856 13718 raft_consensus.cc:2889] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:11.193292 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.226426 13724 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:25:11.227106 13724 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 13
I20260709 12:25:11.227413 13724 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:25:11.227741 13724 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:11.192325 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:11.239449 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.254241 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.211215 13719 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:11.230954 13727 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:25:11.258517 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.260334 13720 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:11.186707 22694 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:11.277740 13726 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: [17.9-19.10]
W20260709 12:25:11.278987 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 220181us
W20260709 12:25:11.280668 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 235795us
I20260709 12:25:11.159714 22881 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.103s
W20260709 12:25:11.278332 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 133317us
I20260709 12:25:11.288933 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.191370 22697 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:11.292085 13723 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:25:11.187314 22911 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:11.291468 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:11.188791 22526 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:11.305716 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:11.307641 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 250256us
W20260709 12:25:11.308921 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.297592 22809 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:25:11.328433 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.331905 13652 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:11.336122 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:11.337306 13724 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30. Re-sending same reply.
I20260709 12:25:11.338492 13726 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:11.339448 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48116.4us
I20260709 12:25:11.188522 22905 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.124s
I20260709 12:25:11.341065 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:11.311156 22841 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.343675 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45038.2us
I20260709 12:25:11.289630 22853 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.042s
I20260709 12:25:11.359227 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.360137 13724 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:11.289382 22910 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:25:11.363210 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:11.363677 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.367146 13726 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:25:11.368566 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.347813 22556 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:11.373792 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.376051 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:11.380597 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:11.381531 13727 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 17.
I20260709 12:25:11.371999 22841 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:11.348013 22677 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:25:11.387214 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:11.398240 22803 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.399109 22803 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:11.402072 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.402777 22803 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:11.405514 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54301.9us
W20260709 12:25:11.406062 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65110.7us
I20260709 12:25:11.406977 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:11.354835 22730 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.026s
I20260709 12:25:11.377624 22556 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:11.420517 13721 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:11.374758 22850 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.037s
I20260709 12:25:11.415256 22556 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:11.420665 22396 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 12:25:11.425730 22396 consensus_queue.cc:1243] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:11.394239 22405 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:11.397032 22915 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:11.424310 22897 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:25:11.422222 22901 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:11.473098 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:11.455022 22915 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:11.484031 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77116.3us
I20260709 12:25:11.491247 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:11.493548 22827 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.461915 22920 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
W20260709 12:25:11.500975 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.502599 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.460259 22901 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:11.513666 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126601us
I20260709 12:25:11.525406 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:11.487452 22915 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:11.529353 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:11.564110 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:11.511824 22870 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:11.495770 22935 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:11.610901 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46897.1us
I20260709 12:25:11.618614 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:11.612035 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:11.645257 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:11.564261 22890 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.516640 22901 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:11.594491 22870 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.587991 22925 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:11.620153 22935 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:11.494190 22827 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:11.667531 22841 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:11.726562 22697 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:11.755510 22920 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:11.747588 22405 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.770352 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.011s	sys 0.036s
W20260709 12:25:11.772286 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.759258 22870 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:11.776427 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.011s	sys 0.037s
W20260709 12:25:11.777601 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 619925us
I20260709 12:25:11.779898 13690 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:11.780799 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.776907 22925 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:11.781620 22697 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:11.781823 22923 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 12:25:11.763494 22890 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.782130 22697 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.782648 22894 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:11.783262 22894 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.784181 22890 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:11.785120 13617 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.012s	sys 0.018s
W20260709 12:25:11.785466 13617 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.013s	sys 0.018s
I20260709 12:25:11.785408 22894 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:11.787519 22696 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.714s	user 0.015s	sys 0.000s
I20260709 12:25:11.788028 22696 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:11.792394 22913 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 12:25:11.785925 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 593158us
I20260709 12:25:11.773069 22909 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:11.796808 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.805809 22901 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:11.838371 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.801901 22928 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
W20260709 12:25:11.820313 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.825083 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 299723us
I20260709 12:25:11.839876 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:11.843364 14424 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:11.845135 13727 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:11.762801 22935 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:11.809515 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.749s (SENT)
W20260709 12:25:11.851209 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 232743us
W20260709 12:25:11.806087 22880 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.001s
W20260709 12:25:11.810776 22900 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 12:25:11.782668 22697 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:11.814777 22868 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:11.802541 22696 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:11.838141 22901 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:11.844264 22890 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.843319 22894 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.853528 22857 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.881s	user 0.011s	sys 0.005s
I20260709 12:25:11.807639 22766 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:11.887861 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.906162 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:11.907204 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:11.824216 22405 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.837966 22915 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.796775 22870 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:11.885601 22670 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:25:11.824858 22909 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.864809 22880 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:11.905473 22849 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:25:11.912180 22890 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.912701 22894 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.908743 22943 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 12:25:11.962541 13718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.006s	sys 0.005s
I20260709 12:25:11.926128 22766 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:11.974726 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85647.4us
I20260709 12:25:11.975580 22405 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:11.976274 13800 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:11.981139 22556 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.002s
I20260709 12:25:11.981289 22935 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:11.982312 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
W20260709 12:25:11.996526 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.004s	sys 0.008s
I20260709 12:25:11.990551 22944 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.015260 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 205763us
W20260709 12:25:11.992384 13718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.006s	sys 0.006s
W20260709 12:25:11.994377 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 214574us
W20260709 12:25:11.982594 22694 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.685s	user 0.006s	sys 0.008s
W20260709 12:25:12.002689 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.005s	sys 0.008s
I20260709 12:25:12.002753 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:12.026693 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 631560us
W20260709 12:25:12.008030 13725 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.008s	sys 0.008s
I20260709 12:25:11.967976 22888 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:12.026928 13727 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:12.027649 13725 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.008s
W20260709 12:25:12.031080 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.033506 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.030086 13724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.008s	sys 0.022s
I20260709 12:25:12.042950 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:12.042212 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.046922 13724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.009s	sys 0.022s
W20260709 12:25:12.044939 13649 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.020s	sys 0.027s
W20260709 12:25:12.052053 13649 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.020s	sys 0.027s
W20260709 12:25:12.052860 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59022.6us
I20260709 12:25:12.054803 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:12.056022 14418 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:12.032342 22888 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:12.091069 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.106s (SENT)
I20260709 12:25:12.094646 13727 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:25:12.096364 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:12.107510 13651 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:12.110387 13792 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:12.112659 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:12.107249 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:12.107249 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:12.007308 22556 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:12.118301 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67126.9us
I20260709 12:25:12.122599 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:12.035367 22694 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.060967 22405 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:12.069466 22911 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.125942 14423 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:12.100051 22888 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:12.097793 22945 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:12.118687 22827 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:12.032812 22944 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.159778 13796 pending_rounds.cc:85] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 15
I20260709 12:25:12.162461 13796 pending_rounds.cc:107] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:25:12.162740 13796 raft_consensus.cc:2889] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.161031 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:12.165431 13797 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:12.166628 13794 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:12.131686 22362 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:25:12.150902 22912 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:25:12.211171 14420 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:25:12.185871 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:12.223943 13650 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: [29.22-30.23]
W20260709 12:25:12.236027 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.265969 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.265929 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 174895us
I20260709 12:25:12.264039 22362 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:12.269501 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4636) took 1341 ms. Trace:
I20260709 12:25:12.139219 22929 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 12:25:12.270830 22929 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:12.271256 22929 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:12.272207 22929 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:12.273658 22929 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:12.274780 22932 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.024s
I20260709 12:25:12.269694 22362 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:12.277313 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:12.268486 22915 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:12.275123 22894 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:12.269920 22949 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:12.282006 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:12.269934 13643 rpcz_store.cc:276] 0709 12:25:10.927571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.930388 (+  2817us) service_pool.cc:224] Handling call
0709 12:25:10.931427 (+  1039us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.931817 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.931855 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.937862 (+  6007us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.113365 (+175503us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b180039df60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:25:11.370011 (+256646us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:11.370054 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.370124 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.269057 (+898933us) raft_consensus.cc:1685] finished
0709 12:25:12.269084 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.269418 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2294,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":2353408,"thread_start_us":256408,"threads_started":1,"wal-append.queue_time_us":953591}
W20260709 12:25:12.297598 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.132s (SENT)
I20260709 12:25:12.298286 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:12.294323 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115386us
I20260709 12:25:12.303458 22939 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:25:12.153688 22911 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:12.312667 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 446905us
I20260709 12:25:12.292601 22870 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.308892 22730 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:12.293978 22890 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.320986 22952 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:12.321586 22946 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:25:12.364467 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:12.364662 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4660) took 1352 ms. Trace:
I20260709 12:25:12.365468 14420 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:12.365628 13649 rpcz_store.cc:276] 0709 12:25:11.012711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.077630 (+ 64919us) service_pool.cc:224] Handling call
0709 12:25:11.111206 (+ 33576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.111496 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:12.052540 (+941044us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.364135 (+311595us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:12.364214 (+    79us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:12.364282 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.364389 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.364578 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36452,"prepare.queue_time_us":66758,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"thread_start_us":1178050,"threads_started":2}
I20260709 12:25:12.337577 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:12.323872 22952 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:12.325726 22935 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:12.396057 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58578.1us
I20260709 12:25:12.368460 22730 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.391319 22870 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.411238 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:12.415822 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 148157us
I20260709 12:25:12.399979 22594 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:12.399935 22556 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:12.420644 13617 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.136s (SENT)
I20260709 12:25:12.406503 22694 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:12.445164 22944 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:12.440181 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:12.422755 22956 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:12.464337 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.163s (SENT)
I20260709 12:25:12.441617 22945 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:12.490170 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50899.8us
I20260709 12:25:12.415305 22870 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.410599 22730 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:12.464777 22911 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.462509 22944 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:12.491127 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:12.514649 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.452023 22412 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:12.454879 22594 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:12.493239 22857 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.480350 22827 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:12.554378 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.553807 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.451346 22959 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.256s
I20260709 12:25:12.507809 22868 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:12.507943 22537 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:12.612633 22948 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.613859 22937 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.852s	user 0.002s	sys 0.000s
W20260709 12:25:12.618539 22950 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:25:12.620404 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1914) took 1259 ms. Trace:
I20260709 12:25:12.620674 22730 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.605129 22956 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:12.559124 22857 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.621214 22909 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.623230 22870 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.624655 22857 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.620851 13725 rpcz_store.cc:276] 0709 12:25:11.360526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.360682 (+   156us) service_pool.cc:224] Handling call
0709 12:25:11.377077 (+ 16395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.392328 (+ 15251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.392380 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.413128 (+ 20748us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.032855 (+619727us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:12.032920 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.033029 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.620165 (+587136us) raft_consensus.cc:1685] finished
0709 12:25:12.620189 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.620334 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2127,"raft.queue_time_us":3561,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":14002,"thread_start_us":606741,"threads_started":1,"wal-append.queue_time_us":887609}
I20260709 12:25:12.561121 22412 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.627081 22412 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.553717 22933 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.234s
W20260709 12:25:12.635962 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.600068 22958 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:12.560226 22888 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.604435 22537 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:12.712994 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.711572 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 3966) took 1336 ms. Trace:
I20260709 12:25:12.738940 13726 rpcz_store.cc:276] 0709 12:25:11.377577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.387205 (+  9628us) service_pool.cc:224] Handling call
0709 12:25:11.387631 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.395923 (+  8292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.395978 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.422183 (+ 26205us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.483292 (+ 61109us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:11.483661 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.484025 (+   364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.484119 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.613002 (+1128883us) spinlock_profiling.cc:243] Waited 7.04 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:12.631500 (+ 18498us) raft_consensus.cc:1685] finished
0709 12:25:12.711167 (+ 79667us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.711481 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28656,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":8454,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":14158,"spinlock_wait_cycles":16201088,"thread_start_us":8664,"threads_started":1,"wal-append.queue_time_us":107856}
I20260709 12:25:12.600425 22827 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:12.712682 22973 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
W20260709 12:25:12.766932 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 249086us
I20260709 12:25:12.713827 22921 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:12.629726 22594 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:12.803810 22951 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
W20260709 12:25:12.827811 22955 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.364s	user 0.002s	sys 0.000s
W20260709 12:25:12.829614 22967 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:25:12.830950 22919 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:12.831719 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:12.860590 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.870735 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:12.879209 22980 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:25:12.880923 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:12.881336 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1915) took 1520 ms. Trace:
I20260709 12:25:12.881719 13724 rpcz_store.cc:276] 0709 12:25:11.360814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.361840 (+  1026us) service_pool.cc:224] Handling call
0709 12:25:11.362488 (+   648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.378179 (+ 15691us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:11.385612 (+  7433us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.101184 (+715572us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.130053 (+ 28869us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.130229 (+   176us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:12.130279 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.130367 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.881133 (+750766us) raft_consensus.cc:1685] finished
0709 12:25:12.881155 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.881276 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61392,"prepare.queue_time_us":12566,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":134549,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":36966,"thread_start_us":643198,"threads_started":1,"wal-append.queue_time_us":884774}
W20260709 12:25:12.883206 22927 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.135s	user 0.004s	sys 0.000s
W20260709 12:25:12.889488 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 430374us
W20260709 12:25:12.891311 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 422543us
I20260709 12:25:12.895977 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:12.900429 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.227s (SENT)
I20260709 12:25:12.901242 13645 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:12.803122 22921 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:12.613668 22948 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:12.533630 22945 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:12.594528 22868 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.717737 22958 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:12.912412 22970 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
I20260709 12:25:12.493530 22952 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.846127 22919 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:12.956923 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66693.7us
W20260709 12:25:12.957979 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.262s (SENT)
I20260709 12:25:12.958551 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:12.964164 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:12.961359 22966 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
W20260709 12:25:12.964335 22697 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.073s	user 0.011s	sys 0.006s
I20260709 12:25:12.963510 22984 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:12.964444 22888 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:12.976679 22537 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.022567 22994 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.035362 22994 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.035998 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4163) took 1129 ms. Trace:
I20260709 12:25:13.037810 22994 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:13.038872 22977 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:25:12.979733 22697 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:12.989286 22827 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.042176 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4161) took 1136 ms. Trace:
I20260709 12:25:13.042805 13721 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:13.043931 22697 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.045158 22919 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.047144 22948 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.022890 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.042659 13718 rpcz_store.cc:276] 0709 12:25:11.906205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.283223 (+377018us) service_pool.cc:224] Handling call
0709 12:25:12.286310 (+  3087us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.292150 (+  5840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.292183 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:12.305309 (+ 13126us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.488420 (+183111us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:12.557507 (+ 69087us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.561536 (+  4029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.561678 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.041867 (+480189us) raft_consensus.cc:1685] finished
0709 12:25:13.041904 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.042097 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5825,"prepare.queue_time_us":95907,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":3306,"raft.queue_time_us":45057,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":664,"spinlock_wait_cycles":21115136,"thread_start_us":157385,"threads_started":1,"wal-append.queue_time_us":438114}
I20260709 12:25:13.036586 13725 rpcz_store.cc:276] 0709 12:25:11.906780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.625524 (+718744us) service_pool.cc:224] Handling call
0709 12:25:12.625949 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.631232 (+  5283us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:12.718617 (+ 87385us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.952247 (+233630us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.953030 (+   783us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.953202 (+   172us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:12.953537 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.953804 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.953919 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.035660 (+ 81741us) raft_consensus.cc:1685] finished
0709 12:25:13.035688 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.035912 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6978,"prepare.queue_time_us":139334,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":1813,"raft.queue_time_us":13490,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":56927,"thread_start_us":140912,"threads_started":1,"wal-append.queue_time_us":178700}
W20260709 12:25:13.026391 22993 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 12:25:13.088044 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97903.2us
I20260709 12:25:13.089759 13725 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
W20260709 12:25:13.079516 22929 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.805s	user 0.020s	sys 0.000s
I20260709 12:25:13.094090 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.090229 13725 pending_rounds.cc:85] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:25:13.095783 13725 pending_rounds.cc:107] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 12:25:13.096064 13725 raft_consensus.cc:2889] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:13.102895 22958 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.093943 22922 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.106407 22922 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.109212 22922 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.113502 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4167) took 1205 ms. Trace:
I20260709 12:25:13.115092 22888 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:13.114235 13724 rpcz_store.cc:276] 0709 12:25:11.907790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.884392 (+976602us) service_pool.cc:224] Handling call
0709 12:25:12.884763 (+   371us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:12.887067 (+  2304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.887099 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:12.897657 (+ 10558us) log.cc:844] Serialized 78 byte log entry
0709 12:25:12.989420 (+ 91763us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:12.989498 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.989607 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.113102 (+123495us) raft_consensus.cc:1685] finished
0709 12:25:13.113135 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.113396 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1900,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":8449,"thread_start_us":53357,"threads_started":1,"wal-append.queue_time_us":197315}
I20260709 12:25:13.098150 22919 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.023s
I20260709 12:25:13.117843 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4171) took 1207 ms. Trace:
I20260709 12:25:13.118381 13724 rpcz_store.cc:276] 0709 12:25:11.921249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.116743 (+1195494us) service_pool.cc:224] Handling call
0709 12:25:13.117153 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.117390 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.117427 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.117494 (+    67us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:13.117545 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.117620 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.117766 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:13.084506 22970 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:25:13.119953 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4172) took 1198 ms. Trace:
I20260709 12:25:13.120294 13724 rpcz_store.cc:276] 0709 12:25:11.921627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.119220 (+1197593us) service_pool.cc:224] Handling call
0709 12:25:13.119492 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.119660 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.119683 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.119722 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:13.119753 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.119798 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.119904 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.124104 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4169) took 1214 ms. Trace:
I20260709 12:25:13.124547 13725 rpcz_store.cc:276] 0709 12:25:11.909774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.064025 (+1154251us) service_pool.cc:224] Handling call
0709 12:25:13.089337 (+ 25312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.123826 (+ 34489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.124045 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.125883 13724 pending_rounds.cc:85] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:25:13.126329 13724 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:25:13.126667 13724 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:13.127473 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.564s (SENT)
W20260709 12:25:13.134452 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.121533 22986 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:13.159389 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1926) took 1203 ms. Trace:
I20260709 12:25:13.120427 22935 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:13.170979 22890 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:13.170852 22935 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:13.166759 22999 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:25:13.174146 13725 rpcz_store.cc:276] 0709 12:25:11.955691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.125281 (+1169590us) service_pool.cc:224] Handling call
0709 12:25:13.158534 (+ 33253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.159009 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.159041 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.159089 (+    48us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:13.159124 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.159186 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.159318 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:13.182138 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 159354us
I20260709 12:25:13.183926 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:13.184839 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4164) took 1277 ms. Trace:
W20260709 12:25:13.186318 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.185321 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:13.186759 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:13.192018 14424 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 23. (index mismatch)
I20260709 12:25:13.192416 14420 raft_consensus.cc:2410] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 32 because replica has last-logged OpId of term: 30 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
W20260709 12:25:13.194367 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56532.5us
I20260709 12:25:13.173650 22945 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.179054 22956 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:13.185274 13726 rpcz_store.cc:276] 0709 12:25:11.907020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.755467 (+848447us) service_pool.cc:224] Handling call
0709 12:25:12.756039 (+   572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.756501 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:12.877115 (+120614us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.974726 (+ 97611us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.975360 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.976333 (+   973us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:12.976671 (+   338us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.172477 (+195806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.172563 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.184581 (+ 12018us) raft_consensus.cc:1685] finished
0709 12:25:13.184604 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.184766 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199178,"prepare.queue_time_us":61285,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":7092,"raft.queue_time_us":397410,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":11696,"thread_start_us":150912,"threads_started":2,"wal-append.queue_time_us":168547}
I20260709 12:25:13.201989 22956 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.202546 22956 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:13.185741 22890 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:13.204478 22890 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.204895 22890 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.195686 22959 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:13.207008 22959 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.207440 22959 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:13.210076 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:13.196044 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:13.180313 22921 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:13.239490 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45570.3us
W20260709 12:25:13.228085 23002 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:13.187670 22935 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.173871 22986 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.268074 22986 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.193579 22929 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.268874 22959 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.223656 22697 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:13.259266 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63788us
W20260709 12:25:13.271801 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144365us
W20260709 12:25:13.272621 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.815s (SENT)
I20260709 12:25:13.273166 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:13.253932 13644 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:25:13.269080 22929 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.274225 22909 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:25:13.276932 22929 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.270951 22921 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.008s
I20260709 12:25:13.279439 22697 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.280002 22697 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:13.283623 22968 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:13.284149 22968 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:13.285077 22968 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.286566 22899 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:13.286597 22968 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:13.287277 22899 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:13.288672 22899 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.289763 22899 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:13.290011 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1490) took 1226 ms. Trace:
I20260709 12:25:13.290442 13650 rpcz_store.cc:276] 0709 12:25:12.063578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.063694 (+   116us) service_pool.cc:224] Handling call
0709 12:25:12.064085 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.251828 (+187743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.289705 (+1037877us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001b5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:13.289941 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366736128}
I20260709 12:25:13.290961 22697 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.260710 22922 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:13.293247 22922 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.293639 22922 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:13.294219 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1927) took 1329 ms. Trace:
I20260709 12:25:13.295032 13725 rpcz_store.cc:276] 0709 12:25:11.965173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.176961 (+1211788us) service_pool.cc:224] Handling call
0709 12:25:13.293673 (+116712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.293870 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.293898 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.293946 (+    48us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:13.293983 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.294052 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.294157 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.272238 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:13.314071 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4168) took 1404 ms. Trace:
I20260709 12:25:13.313637 22827 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:13.317950 22959 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.318715 22697 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.314922 13718 rpcz_store.cc:276] 0709 12:25:11.909454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.063540 (+1154086us) service_pool.cc:224] Handling call
0709 12:25:13.085775 (+ 22235us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:13.173454 (+ 87679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.173491 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:13.180172 (+  6681us) log.cc:844] Serialized 52 byte log entry
0709 12:25:13.310813 (+130641us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:13.310872 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.310973 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.313131 (+  2158us) raft_consensus.cc:1685] finished
0709 12:25:13.313170 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.313972 (+   802us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87066,"raft.queue_time_us":89459,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":1181,"thread_start_us":111797,"threads_started":1,"wal-append.queue_time_us":116454}
I20260709 12:25:13.318718 22959 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:13.324266 22959 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.324811 22959 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:13.326877 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54307.5us
W20260709 12:25:13.327678 13617 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.868s (SENT)
I20260709 12:25:13.328099 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:13.329015 22827 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:13.329137 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 1932) took 1169 ms. Trace:
W20260709 12:25:13.329746 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.332861 22868 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.334713 13725 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:25:13.329566 13721 rpcz_store.cc:276] 0709 12:25:12.159755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.258604 (+ 98849us) service_pool.cc:224] Handling call
0709 12:25:13.328826 (+1070222us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54003fe8d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:13.329064 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1758327680}
I20260709 12:25:13.349361 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4174) took 1378 ms. Trace:
I20260709 12:25:13.355171 13726 rpcz_store.cc:276] 0709 12:25:11.971043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.202236 (+1231193us) service_pool.cc:224] Handling call
0709 12:25:13.202649 (+   413us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:13.203074 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.203100 (+    26us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:13.203858 (+   758us) log.cc:844] Serialized 78 byte log entry
0709 12:25:13.271920 (+ 68062us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:13.347756 (+ 75836us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.348954 (+  1198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.349048 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.349097 (+    49us) raft_consensus.cc:1685] finished
0709 12:25:13.349111 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.349284 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44515,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":4661,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":67628,"threads_started":1,"wal-append.queue_time_us":58789}
W20260709 12:25:13.353314 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.233s (SENT)
I20260709 12:25:13.357755 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:13.360826 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1929) took 1331 ms. Trace:
I20260709 12:25:13.362119 13726 rpcz_store.cc:276] 0709 12:25:12.029920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.358569 (+1328649us) service_pool.cc:224] Handling call
0709 12:25:13.359752 (+  1183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.360094 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.360129 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.360190 (+    61us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:13.360553 (+   363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.360619 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.360753 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:13.368500 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.357229 22921 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.369482 22921 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.373595 13726 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:13.382305 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1930) took 1251 ms. Trace:
I20260709 12:25:13.382787 13726 rpcz_store.cc:276] 0709 12:25:12.131101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.363245 (+1232144us) service_pool.cc:224] Handling call
0709 12:25:13.363575 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.381985 (+ 18410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.382215 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:13.384794 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.354910 22945 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.399062 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.407536 13650 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:13.369408 22959 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.403607 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:13.408576 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:13.404031 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.408835 13799 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:13.404346 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:13.404686 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.406183 13795 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:25:13.409210 13645 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.409994 13643 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:13.421008 13650 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:13.426878 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.427140 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:13.387552 22971 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.436244 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64311.9us
I20260709 12:25:13.395860 22697 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.436554 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4176) took 1451 ms. Trace:
I20260709 12:25:13.438320 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1928) took 1437 ms. Trace:
I20260709 12:25:13.444036 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:13.443192 13718 rpcz_store.cc:276] 0709 12:25:11.984900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.320363 (+1335463us) service_pool.cc:224] Handling call
0709 12:25:13.320724 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.320981 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:13.321710 (+   729us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.435865 (+114155us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.436080 (+   215us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.436152 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:13.436225 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.436333 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.436492 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":360,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":76,"raft.queue_time_us":208,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":195,"spinlock_wait_cycles":1408,"thread_start_us":113382,"threads_started":1}
I20260709 12:25:13.444742 13801 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:13.446408 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:13.447633 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:13.443790 13721 rpcz_store.cc:276] 0709 12:25:12.000509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.348215 (+1347706us) service_pool.cc:224] Handling call
0709 12:25:13.362057 (+ 13842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.362332 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:13.394638 (+ 32306us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.437799 (+ 43161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.437892 (+    93us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:13.437961 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.438061 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.438219 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23318,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":103,"thread_start_us":42690,"threads_started":1}
I20260709 12:25:13.458076 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4173) took 1536 ms. Trace:
I20260709 12:25:13.461810 13724 rpcz_store.cc:276] 0709 12:25:11.922415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.121386 (+1198971us) service_pool.cc:224] Handling call
0709 12:25:13.121694 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:13.174106 (+ 52412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.174157 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:13.178386 (+  4229us) log.cc:844] Serialized 52 byte log entry
0709 12:25:13.308586 (+130200us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:13.308660 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.308782 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.405715 (+ 96933us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:13.430371 (+ 24656us) raft_consensus.cc:1685] finished
0709 12:25:13.430401 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.457996 (+ 27595us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43864,"raft.queue_time_us":45690,"raft.run_cpu_time_us":1799,"raft.run_wall_time_us":226990,"spinlock_wait_cycles":2422016,"thread_start_us":109867,"threads_started":1,"wal-append.queue_time_us":119250}
I20260709 12:25:13.465698 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:13.467662 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 240182us
I20260709 12:25:13.379139 22956 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.468833 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:13.470455 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:13.478210 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:13.479995 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:13.481721 13795 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:13.409729 22890 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.485050 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.485812 13649 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:13.490067 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.490801 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:13.460968 22697 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.493897 13718 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:13.431077 22945 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:13.504303 22965 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.049s	user 0.003s	sys 0.000s
I20260709 12:25:13.513696 13645 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:13.483790 22948 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:13.515887 13799 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:13.516464 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1489) took 1453 ms. Trace:
I20260709 12:25:13.520218 13649 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 27. Ops: []
I20260709 12:25:13.524003 22680 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:13.524696 22922 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.518875 22809 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:13.518901 13647 rpcz_store.cc:276] 0709 12:25:12.063225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.063342 (+   117us) service_pool.cc:224] Handling call
0709 12:25:12.107264 (+ 43922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.121731 (+ 14467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.121765 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.125824 (+  4059us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.313499 (+187675us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:12.367913 (+ 54414us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.368935 (+  1022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.369011 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.486408 (+1117397us) spinlock_profiling.cc:243] Waited 52.6 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:13.507173 (+ 20765us) raft_consensus.cc:1685] finished
0709 12:25:13.516226 (+  9053us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.516391 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14781,"prepare.queue_time_us":55160,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":3287,"raft.queue_time_us":24450,"spinlock_wait_cycles":144209664,"thread_start_us":165908,"threads_started":1,"wal-append.queue_time_us":261741}
I20260709 12:25:13.447270 22971 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.528049 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4175) took 1556 ms. Trace:
I20260709 12:25:13.529421 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.537436 13725 rpcz_store.cc:276] 0709 12:25:11.971906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.304131 (+1332225us) service_pool.cc:224] Handling call
0709 12:25:13.334383 (+ 30252us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.335206 (+   823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.527790 (+192584us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54003fe8d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:13.527975 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36643200}
W20260709 12:25:13.532377 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75469.3us
W20260709 12:25:13.534878 13693 connection.cc:256] RPC call timeout handler was delayed by -1.597541179000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
W20260709 12:25:13.540899 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4256411790001948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
W20260709 12:25:13.541553 13693 connection.cc:256] RPC call timeout handler was delayed by -1.412141179000173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 12:25:13.542037 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3671411790001002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1804s for a response
W20260709 12:25:13.542487 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1493411790002028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 12:25:13.542949 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0854411790000995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 12:25:13.543416 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0215411789999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
W20260709 12:25:13.544416 13693 connection.cc:256] RPC call timeout handler was delayed by -1.00534117899997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 12:25:13.545845 13718 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:13.527741 22809 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.532374 22827 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:13.551637 22827 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:13.552703 22827 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:13.550030 22809 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:13.555645 22890 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.556445 22890 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.556960 22890 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.558586 22921 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.560225 22956 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.560951 22956 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:13.562808 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.563014 22959 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.563334 23024 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:13.563524 13725 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:13.563796 23024 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:13.564697 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:13.564958 22922 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.564814 23024 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.567576 22945 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.565388 13725 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:13.572410 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.580101 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:13.526202 22963 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.036s	user 0.000s	sys 0.003s
I20260709 12:25:13.581560 22945 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:13.584081 22809 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.584401 22697 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.586344 22943 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.588496 22971 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.517426 22948 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.595522 22890 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.589902 22766 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:13.601918 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.620253 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74432.3us
I20260709 12:25:13.622181 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.579547 22959 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.606652 22948 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:13.648367 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.653364 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.658221 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.662448 13725 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:13.665824 13801 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:13.665889 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:13.667586 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.651082 22766 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:13.667562 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1479) took 1539 ms. Trace:
I20260709 12:25:13.673197 13793 rpcz_store.cc:276] 0709 12:25:12.128175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.128481 (+   306us) service_pool.cc:224] Handling call
0709 12:25:12.128962 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.129362 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.129386 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.146830 (+ 17444us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.311550 (+164720us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:12.315515 (+  3965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.315583 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.525496 (+1209913us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:13.656669 (+131173us) raft_consensus.cc:1685] finished
0709 12:25:13.656693 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.667473 (+ 10780us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71896,"raft.queue_time_us":37375,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11047,"spinlock_wait_cycles":56396160,"thread_start_us":164253,"threads_started":1,"wal-append.queue_time_us":256844}
I20260709 12:25:13.676324 22670 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.677178 22860 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:13.677757 22860 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:13.684859 22972 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.225s	user 0.003s	sys 0.000s
I20260709 12:25:13.676966 22670 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.630554 22943 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.671710 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.664261 22890 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:13.667511 22971 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:13.689776 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4159) took 1794 ms. Trace:
I20260709 12:25:13.691751 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.691450 13722 rpcz_store.cc:276] 0709 12:25:11.895510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.895728 (+   218us) service_pool.cc:224] Handling call
0709 12:25:11.919886 (+ 24158us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.170148 (+250262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.170182 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.170746 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.387898 (+217152us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:12.399450 (+ 11552us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.400595 (+  1145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.400711 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.684055 (+1283344us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:13.687133 (+  3078us) raft_consensus.cc:1685] finished
0709 12:25:13.689513 (+  2380us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.689705 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11495,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":114367,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":10045,"spinlock_wait_cycles":163510656,"thread_start_us":368970,"threads_started":2,"wal-append.queue_time_us":183889}
I20260709 12:25:13.651422 22959 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.647214 23030 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:25:13.680346 22860 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:13.694429 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60706.2us
I20260709 12:25:13.672765 22697 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:13.696132 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.699463 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:13.700968 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:13.700877 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:13.693997 22968 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:13.695937 22766 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:25:13.703922 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.525s (SENT)
I20260709 12:25:13.706125 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:13.707278 23033 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:13.707434 22670 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.709311 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4177) took 1528 ms. Trace:
I20260709 12:25:13.709681 13726 rpcz_store.cc:276] 0709 12:25:12.180738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.383573 (+1202835us) service_pool.cc:224] Handling call
0709 12:25:13.383954 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.388120 (+  4166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.388154 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.399903 (+ 11749us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.623022 (+223119us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:13.623071 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.623148 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.709130 (+ 85982us) raft_consensus.cc:1685] finished
0709 12:25:13.709147 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.709256 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3724,"raft.queue_time_us":21741,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":157780,"threads_started":1,"wal-append.queue_time_us":251823}
I20260709 12:25:13.711153 23038 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:13.711865 13726 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:13.718335 13726 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:13.723050 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.723677 13643 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:13.724673 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.705536 22697 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:13.704730 22971 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:13.730046 22803 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.733843 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:13.735858 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.605s (SENT)
I20260709 12:25:13.736482 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:13.732134 22803 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.717337 22909 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:13.740768 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.717391 22809 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.747098 22909 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:13.748080 22990 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.920s	user 0.000s	sys 0.003s
I20260709 12:25:13.749979 22909 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:13.750708 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4162) took 1844 ms. Trace:
I20260709 12:25:13.751331 13720 rpcz_store.cc:276] 0709 12:25:11.906529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.576854 (+670325us) service_pool.cc:224] Handling call
0709 12:25:12.577274 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.600867 (+ 23593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.600901 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.632914 (+ 32013us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.802366 (+169452us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:12.802433 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.802600 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.747370 (+944770us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:13.750353 (+  2983us) raft_consensus.cc:1685] finished
0709 12:25:13.750374 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.750634 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23095,"raft.queue_time_us":25545,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4893,"spinlock_wait_cycles":12847872,"thread_start_us":46536,"threads_started":1,"wal-append.queue_time_us":112425}
I20260709 12:25:13.753602 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.743355 22945 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:13.754658 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1918) took 2373 ms. Trace:
I20260709 12:25:13.755071 13723 rpcz_store.cc:276] 0709 12:25:11.380993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.383385 (+  2392us) service_pool.cc:224] Handling call
0709 12:25:11.405183 (+ 21798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.422238 (+ 17055us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:11.636491 (+214253us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.403389 (+766898us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.409987 (+  6598us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.410175 (+   188us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:12.410215 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.410318 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.752441 (+1342123us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b1800330820. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:13.754310 (+  1869us) raft_consensus.cc:1685] finished
0709 12:25:13.754426 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.754593 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1353,"prepare.queue_time_us":32437,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":9782,"raft.queue_time_us":79283,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":38834,"spinlock_wait_cycles":4708224,"thread_start_us":843747,"threads_started":3,"wal-append.queue_time_us":636134}
I20260709 12:25:13.757277 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.758962 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:13.759559 13720 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:25:13.760749 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.763931 13723 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:13.764693 22915 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:13.767215 22915 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:13.767616 22915 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:13.768549 22915 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.771713 22915 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:13.771605 23024 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:13.773138 13725 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:25:13.742355 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.788893 13723 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:13.791337 22803 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.732524 22959 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.792248 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.793867 13726 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
W20260709 12:25:13.826689 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.817814 13649 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:13.831267 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.831799 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.835108 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.839721 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.840864 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.845645 22803 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:13.793711 22860 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.846752 22860 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.849381 22860 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.841517 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 15.
I20260709 12:25:13.853130 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.853268 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.860934 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.832758 22919 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:13.861994 13726 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:25:13.863651 13726 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:25:13.870850 23043 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.051s	user 0.000s	sys 0.008s
W20260709 12:25:13.872727 22975 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.287s	user 0.000s	sys 0.003s
W20260709 12:25:13.875900 22983 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.050s	user 0.002s	sys 0.000s
I20260709 12:25:13.858777 22803 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.850379 22946 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:25:13.905046 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4326) took 1835 ms. Trace:
I20260709 12:25:13.905556 13798 rpcz_store.cc:276] 0709 12:25:12.069572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.072524 (+  2952us) service_pool.cc:224] Handling call
0709 12:25:12.123048 (+ 50524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.330206 (+207158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.330241 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.330800 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.502032 (+171232us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:12.502092 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.502208 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.875233 (+1373025us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:13.904651 (+ 29418us) raft_consensus.cc:1685] finished
0709 12:25:13.904677 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.904954 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68261,"raft.queue_time_us":48782,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":40653,"spinlock_wait_cycles":6045056,"thread_start_us":198037,"threads_started":2,"wal-append.queue_time_us":372582}
I20260709 12:25:13.920980 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4321) took 2478 ms. Trace:
I20260709 12:25:13.921510 13800 rpcz_store.cc:276] 0709 12:25:11.442947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.453480 (+ 10533us) service_pool.cc:224] Handling call
0709 12:25:11.473897 (+ 20417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.995401 (+521504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.920621 (+1925220us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54000d25d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:13.920897 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133768064}
W20260709 12:25:13.923480 13690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.714s (SENT) [suppressed 32 similar messages]
I20260709 12:25:13.930302 22526 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:13.931018 22526 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:13.932080 22526 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.932648 22920 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:13.933061 22920 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:13.936256 22537 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:13.937527 22910 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:13.937922 22910 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:13.938813 22910 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:13.939724 22803 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:13.939849 22910 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:13.862357 22919 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.947203 22948 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:13.952802 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:13.953964 14419 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:13.954641 14424 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 12:25:13.955181 23045 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 12:25:13.957301 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:13.958163 13722 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:13.952939 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:13.958493 14418 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:13.961402 14424 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:13.978010 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:13.979374 14419 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:13.983615 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:13.984570 13721 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:13.988880 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:13.990590 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:13.991422 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:13.960805 22526 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:13.993897 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1480) took 1865 ms. Trace:
I20260709 12:25:13.992102 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:13.965088 22952 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:14.018793 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:14.011289 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:14.012653 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:14.020937 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:14.020918 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 97481.2us
W20260709 12:25:14.022109 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.025372 22948 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:14.020282 14420 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:13.994628 13796 rpcz_store.cc:276] 0709 12:25:12.128854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.130997 (+  2143us) service_pool.cc:224] Handling call
0709 12:25:12.159501 (+ 28504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:12.183610 (+ 24109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.183804 (+   194us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:12.187383 (+  3579us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.417822 (+230439us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:12.549367 (+131545us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.761635 (+212268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.761764 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.872271 (+1110507us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b2400227378. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b4609d 00007f539377810e 00007f539378f88c 00007f539378ebb6 00007f539378e7d8 00007f5393798221
0709 12:25:13.977819 (+105548us) raft_consensus.cc:1685] finished
0709 12:25:13.993649 (+ 15830us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.993830 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50309,"prepare.queue_time_us":192390,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":74402,"raft.queue_time_us":37725,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":72469,"spinlock_wait_cycles":10349568,"thread_start_us":400565,"threads_started":3,"wal-append.queue_time_us":235907}
I20260709 12:25:13.988162 22905 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:14.042667 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.065125 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73072.1us
I20260709 12:25:14.050688 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:14.070444 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:14.072767 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:13.934186 22920 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:14.042075 22948 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:14.061781 22919 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:14.071422 22929 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:14.084012 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 95225.7us
I20260709 12:25:14.029366 22952 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:14.074221 22537 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:14.073866 22803 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:14.077041 22946 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:14.080123 22945 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:14.089887 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:14.100816 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:14.100327 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:14.117480 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:14.106932 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:14.095868 22946 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.093318 22537 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:14.067509 22905 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:14.084661 22948 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:14.104481 22920 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.129401 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:14.107268 22925 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:14.119983 22929 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:14.111639 23053 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:14.124573 22946 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.124130 22850 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:14.133985 22537 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:14.140100 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:14.137605 22925 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:14.142438 22945 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:14.144897 23053 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:14.144620 23052 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:14.186133 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.171485 22803 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:14.185580 22925 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:14.187883 23052 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.187498 23053 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:14.195657 22948 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:14.217933 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:14.176403 22850 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:25:14.197625 22803 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:14.224164 22850 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:14.225140 22880 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:14.251655 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:14.220153 23050 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:14.215093 23052 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:14.248653 22850 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:14.261113 22850 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:14.249842 22803 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:14.244568 22905 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:14.293121 23024 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.501s	user 0.014s	sys 0.000s
I20260709 12:25:14.298925 22850 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:14.287467 22952 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:14.327845 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.341058 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:14.345273 13651 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:14.353062 13651 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:25:14.353826 22925 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:14.355067 23053 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:14.364063 22841 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:14.364631 22841 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:14.365474 22841 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:14.374147 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:14.367439 22841 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:14.378227 13652 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:25:14.367033 23053 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:14.378959 22953 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 2.050s	user 0.003s	sys 0.002s
I20260709 12:25:14.387039 22946 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:14.391940 22946 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:14.393138 22960 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.904s	user 0.002s	sys 0.002s
I20260709 12:25:14.408273 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:14.408684 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:14.409675 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:14.409927 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:14.410377 13725 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:14.411442 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:14.411665 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:14.411868 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:14.412253 13722 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:14.412357 13725 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:14.413396 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:14.415083 13721 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:25:14.415728 13721 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 16
I20260709 12:25:14.415989 13721 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:25:14.416213 13721 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:14.421787 22925 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:14.423429 22943 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:14.429540 22803 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:14.434888 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.436297 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:14.432248 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:14.441442 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:14.442709 14417 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:14.440876 14424 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:14.443624 13721 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.439924 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:14.450126 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:14.452272 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.455127 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.455246 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:14.456525 13652 raft_consensus.cc:2410] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:25:14.456888 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.457789 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:14.469451 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.474300 13647 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:14.475322 13800 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:14.450917 22694 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.478788 22694 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:14.481339 22694 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:14.483131 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.483067 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.483891 13799 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:14.484815 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:14.485030 23057 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:25:14.486017 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.493777 13721 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:14.498163 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.507925 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.510751 14417 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:14.512113 13793 raft_consensus.cc:1217] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Deduplicated request from leader. Original: 21.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:25:14.512720 13793 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:25:14.514307 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:14.516369 13800 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:14.518596 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.518668 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.520068 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.522644 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:14.530019 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.573060 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.580466 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.588327 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:14.509054 22537 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:14.630592 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:14.632129 13800 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:14.628134 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:14.629798 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.628082 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.640311 13644 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:14.628700 13652 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:25:14.642652 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.647130 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:14.647490 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.648243 13652 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.649132 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.658062 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:14.658658 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.659029 13648 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:14.661026 13792 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:14.669445 13792 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:14.673236 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.681304 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.681746 13648 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:14.682638 22881 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:14.683684 22880 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:14.688088 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:14.641000 23052 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.612425 13650 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:15.614643 23052 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:14.686982 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.735750 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:14.691255 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.764220 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2974) took 1075 ms. Trace:
I20260709 12:25:15.764623 13792 rpcz_store.cc:276] 0709 12:25:14.688400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.690994 (+  2594us) service_pool.cc:224] Handling call
0709 12:25:15.764111 (+1073117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.691704 13800 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:15.768141 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4342) took 1209 ms. Trace:
I20260709 12:25:15.768507 13800 rpcz_store.cc:276] 0709 12:25:14.559282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.629124 (+ 69842us) service_pool.cc:224] Handling call
0709 12:25:15.768077 (+1138953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.769550 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:14.699894 22526 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 12:25:15.772917 22526 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:15.773340 22526 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:14.713541 22920 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.009s
I20260709 12:25:14.686980 22850 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:14.685359 23000 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.559s	user 0.000s	sys 0.003s
I20260709 12:25:14.824118 23053 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:14.829886 13766 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:15.785094 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 955215us
I20260709 12:25:14.908231 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:15.328732 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:15.340504 23058 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.189s	user 0.005s	sys 0.000s
I20260709 12:25:15.359329 22803 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:15.843752 13794 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:15.394902 22964 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 2.939s	user 0.016s	sys 0.006s
I20260709 12:25:15.843914 23077 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:15.844368 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2983) took 1030 ms. Trace:
I20260709 12:25:15.430312 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:15.433787 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4160) took 3537 ms. Trace:
I20260709 12:25:15.844759 13794 rpcz_store.cc:276] 0709 12:25:14.813829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.327886 (+514057us) service_pool.cc:224] Handling call
0709 12:25:15.844282 (+516396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.845674 13651 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 32 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:15.477528 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2967) took 1023 ms. Trace:
I20260709 12:25:15.483139 22945 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.844583 23077 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:15.846403 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 4720) took 1364 ms (client timeout 1814 ms). Trace:
I20260709 12:25:15.846642 22945 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:15.847312 23077 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.849488 22945 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:15.511402 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.846431 13793 rpcz_store.cc:276] 0709 12:25:14.454106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.454251 (+   145us) service_pool.cc:224] Handling call
0709 12:25:14.466477 (+ 12226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.513062 (+ 46585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.477185 (+964123us) spinlock_profiling.cc:243] Waited 45.4 ms on lock 0x7b54000e60d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:15.477438 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104392960}
I20260709 12:25:15.591440 23019 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:15.604606 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4713) took 1244 ms. Trace:
I20260709 12:25:15.618474 22556 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:15.620908 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:15.622448 23024 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:15.624215 22405 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:15.627977 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.278s (SENT)
I20260709 12:25:15.863476 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:15.633558 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:15.629969 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.643242 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.864427 13649 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:15.642807 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:15.865140 13690 leader_election.cc:336] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.568s (SENT)
I20260709 12:25:15.644578 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.660777 13650 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:15.678381 23069 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.692868 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:15.866515 13796 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:25:15.651108 13652 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.999s	user 0.018s	sys 0.000s
I20260709 12:25:15.867259 13648 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
W20260709 12:25:15.867340 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 224660us
I20260709 12:25:15.707653 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.714986 23071 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:15.719445 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54798.3us
I20260709 12:25:15.721783 23072 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:15.719765 13797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.084s	user 0.036s	sys 0.036s
I20260709 12:25:15.727643 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:15.732458 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003793c0 after lost signal to thread 12472
I20260709 12:25:15.867264 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4346) took 1176 ms. Trace:
I20260709 12:25:15.732178 23013 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 12:25:15.730436 22537 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:15.729446 22412 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.868690 22405 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.739912 22956 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:15.780323 13801 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
W20260709 12:25:15.883870 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 255940us
I20260709 12:25:15.884212 22956 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.884729 22956 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:15.884328 22405 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:15.886013 22405 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:15.886471 13801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2963) took 1437 ms (client timeout 1835 ms). Trace:
I20260709 12:25:15.886547 22405 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:15.887135 13801 rpcz_store.cc:269] 0709 12:25:14.449529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.449776 (+   247us) service_pool.cc:224] Handling call
0709 12:25:15.886218 (+1436442us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54003d4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:15.886405 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3056260864}
I20260709 12:25:15.888468 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:15.889158 22405 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:15.781175 22850 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:15.874722 22969 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:15.890713 22852 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.852753 23019 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:15.765980 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:15.789181 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:15.790453 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54802.4us
I20260709 12:25:15.787753 22526 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.796653 22920 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:15.801849 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4165) took 3894 ms. Trace:
I20260709 12:25:15.830708 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:15.836764 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1937) took 2249 ms. Trace:
I20260709 12:25:15.842046 23056 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:15.363391 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.846206 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:15.846887 13651 rpcz_store.cc:269] 0709 12:25:14.481657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.429978 (+948321us) service_pool.cc:224] Handling call
0709 12:25:15.846316 (+416338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.518590 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4709) took 1189 ms. Trace:
I20260709 12:25:15.943306 13644 rpcz_store.cc:276] 0709 12:25:14.328792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.328946 (+   154us) service_pool.cc:224] Handling call
0709 12:25:14.329321 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.640874 (+311553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.518153 (+877279us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:15.518488 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714626944}
I20260709 12:25:15.553077 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:15.572957 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.852017 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1485) took 3801 ms. Trace:
I20260709 12:25:15.852926 13647 rpcz_store.cc:276] 0709 12:25:14.360326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.360471 (+   145us) service_pool.cc:224] Handling call
0709 12:25:14.360833 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.475039 (+114206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.604209 (+1129170us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400095454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:15.604518 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255593216}
W20260709 12:25:15.843557 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 935340us
I20260709 12:25:15.854674 23074 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:15.953200 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.845381 13727 rpcz_store.cc:276] 0709 12:25:11.896459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.172466 (+276007us) service_pool.cc:224] Handling call
0709 12:25:12.172952 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.389699 (+216747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.389745 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.407872 (+ 18127us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.491800 (+ 83928us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:15.431393 (+2939593us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b180044ae20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:25:15.432280 (+   887us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.433279 (+   999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.433371 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.433403 (+    32us) raft_consensus.cc:1685] finished
0709 12:25:15.433526 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.433709 (+   183us) inbound_call.cc:177] Queueing success response
0709 12:25:15.501862 (+ 68153us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
Metrics: {"mutex_wait_us":62834,"prepare.queue_time_us":139483,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"raft.queue_time_us":184372,"raft.run_cpu_time_us":12468,"raft.run_wall_time_us":3210592,"spinlock_wait_cycles":16216704,"thread_start_us":232210,"threads_started":4,"wal-append.queue_time_us":161782,"wal-append.run_cpu_time_us":4022,"wal-append.run_wall_time_us":1333798}
W20260709 12:25:15.855676 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 156) took 80558 ms (client timeout 15000 ms). Trace:
I20260709 12:25:15.856715 13643 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:15.858098 13547 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.863701 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.706718 22958 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:15.707474 23070 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:15.868952 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2958) took 1290 ms (client timeout 1703 ms). Trace:
W20260709 12:25:15.869630 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2953) took 1377 ms (client timeout 1798 ms). Trace:
W20260709 12:25:15.869686 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.869980 13796 rpcz_store.cc:276] 0709 12:25:14.690587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.690895 (+   308us) service_pool.cc:224] Handling call
0709 12:25:15.867184 (+1176289us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:15.870544 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080034b7c0 after lost signal to thread 13584
I20260709 12:25:15.867512 13652 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
W20260709 12:25:15.960209 13652 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2954) took 1464 ms (client timeout 1626 ms). Trace:
I20260709 12:25:15.868695 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2975) took 1061 ms. Trace:
W20260709 12:25:15.960959 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080019fee0 after lost signal to thread 13676
W20260709 12:25:15.876614 13797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.243s	user 0.037s	sys 0.037s
W20260709 12:25:15.961637 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800358020 after lost signal to thread 13677
I20260709 12:25:15.863248 22556 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.877365 23071 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:15.748257 22984 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.875296 23069 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:15.868196 23024 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:15.882477 23072 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.786477 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:16.013258 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226970us
I20260709 12:25:15.878135 22412 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.911940 13649 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:15.939365 13645 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:15.938674 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:15.937558 13723 rpcz_store.cc:276] 0709 12:25:13.587644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.791302 (+203658us) service_pool.cc:224] Handling call
0709 12:25:13.794204 (+  2902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.794460 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:13.844508 (+ 50048us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.966884 (+122376us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.966948 (+    64us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:13.967008 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.967313 (+   305us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.835349 (+1868036us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b18004992a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4
0709 12:25:15.836507 (+  1158us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b5400174b50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:15.836678 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5105,"prepare.queue_time_us":72797,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":74,"raft.queue_time_us":88,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":5771,"spinlock_wait_cycles":13926656,"thread_start_us":121916,"threads_started":1,"wal-append.queue_time_us":178541,"wal-append.run_cpu_time_us":1465,"wal-append.run_wall_time_us":1001478}
I20260709 12:25:15.950497 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2981) took 1138 ms. Trace:
I20260709 12:25:15.951364 23082 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:15.956844 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2976) took 1149 ms. Trace:
I20260709 12:25:15.952754 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2982) took 1139 ms. Trace:
I20260709 12:25:15.958166 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2987) took 1142 ms. Trace:
W20260709 12:25:15.959363 13648 rpcz_store.cc:269] 0709 12:25:14.491015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.692618 (+201603us) service_pool.cc:224] Handling call
0709 12:25:15.868176 (+1175558us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:15.959153 13650 rpcz_store.cc:269] 0709 12:25:14.576286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.687784 (+111498us) service_pool.cc:224] Handling call
0709 12:25:15.866232 (+1178448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.964872 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:15.965785 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.059628 14424 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:15.958926 22958 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:15.960563 13652 rpcz_store.cc:269] 0709 12:25:14.496106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.642944 (+146838us) service_pool.cc:224] Handling call
0709 12:25:15.960145 (+1317201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.955766 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:16.066286 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2986) took 1251 ms. Trace:
I20260709 12:25:16.066857 13793 rpcz_store.cc:276] 0709 12:25:14.815097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.851807 (+1036710us) service_pool.cc:224] Handling call
0709 12:25:16.066153 (+214346us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.067361 13645 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2961) took 1410 ms (client timeout 1872 ms). Trace:
I20260709 12:25:15.961035 13795 rpcz_store.cc:276] 0709 12:25:14.807332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.707202 (+899870us) service_pool.cc:224] Handling call
0709 12:25:15.868630 (+161428us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.067796 13645 rpcz_store.cc:269] 0709 12:25:14.657439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.830373 (+1172934us) service_pool.cc:224] Handling call
0709 12:25:16.067290 (+236917us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.069015 13794 rpcz_store.cc:276] 0709 12:25:14.813516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.845869 (+1032353us) service_pool.cc:224] Handling call
0709 12:25:15.952669 (+106800us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.063606 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2959) took 1425 ms (client timeout 1871 ms). Trace:
W20260709 12:25:16.070034 13649 rpcz_store.cc:269] 0709 12:25:14.638644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.633089 (+994445us) service_pool.cc:224] Handling call
0709 12:25:16.063529 (+430440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.000905 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.066977 13792 rpcz_store.cc:276] 0709 12:25:14.812818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.765696 (+952878us) service_pool.cc:224] Handling call
0709 12:25:15.950408 (+184712us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.072009 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:15.967831 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4330) took 1485 ms. Trace:
I20260709 12:25:15.945724 13646 rpcz_store.cc:276] 0709 12:25:12.050479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.050717 (+   238us) service_pool.cc:224] Handling call
0709 12:25:12.051116 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.126779 (+ 75663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.126849 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.146837 (+ 19988us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.314969 (+168132us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:12.315022 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.315104 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.394052 (+3078948us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:15.844877 (+450825us) raft_consensus.cc:1685] finished
0709 12:25:15.844903 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.845151 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21098,"raft.queue_time_us":23815,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5725,"spinlock_wait_cycles":159299328,"thread_start_us":167684,"threads_started":1,"wal-append.queue_time_us":252426}
I20260709 12:25:16.073769 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.879475 22803 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:16.077517 22803 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.076983 23084 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.078610 14417 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:16.078492 22803 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.079222 23084 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.080672 22412 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.081640 14420 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:15.966920 23075 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:15.968367 22841 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.004590 14419 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:25:16.005867 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:15.928355 23079 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:15.966461 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080005da40 after lost signal to thread 13751
I20260709 12:25:16.008405 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:16.010684 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.087392 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800386b80 after lost signal to thread 13752
I20260709 12:25:16.085456 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.088444 13652 raft_consensus.cc:2410] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:25:16.086010 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.087311 14418 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:16.014472 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:15.962518 22537 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:15.959357 22945 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:15.965132 22920 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:16.101224 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86846.5us
I20260709 12:25:16.102150 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:15.940246 13719 rpcz_store.cc:276] 0709 12:25:11.907298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.829728 (+922430us) service_pool.cc:224] Handling call
0709 12:25:12.830145 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.830578 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.830607 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.831206 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.953690 (+122484us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:12.973542 (+ 19852us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.168762 (+195220us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b1800434ce0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:25:13.169018 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.169114 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.684738 (+1515624us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:15.801623 (+1116885us) raft_consensus.cc:1685] finished
0709 12:25:15.801648 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.801784 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2417,"prepare.queue_time_us":20508,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":212369,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":19947,"spinlock_wait_cycles":6149760,"thread_start_us":122058,"threads_started":1,"wal-append.queue_time_us":188124}
W20260709 12:25:16.062436 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 123862us
I20260709 12:25:16.062906 13798 rpcz_store.cc:276] 0709 12:25:14.815441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.552652 (+737211us) service_pool.cc:224] Handling call
0709 12:25:15.958090 (+405438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.068940 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:15.956349 13547 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 21, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 21 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.059571 22868 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:16.006804 23080 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:15.895558 22852 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.081776 22809 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:16.085368 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.138859 13646 raft_consensus.cc:2410] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:25:16.139484 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2960) took 1495 ms. Trace:
I20260709 12:25:16.139920 13646 rpcz_store.cc:276] 0709 12:25:14.644148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.085033 (+1440885us) service_pool.cc:224] Handling call
0709 12:25:16.139410 (+ 54377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.066030 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:16.141567 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4718) took 1693 ms. Trace:
I20260709 12:25:16.141870 13644 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:16.142028 13646 rpcz_store.cc:276] 0709 12:25:14.447808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.140614 (+1692806us) service_pool.cc:224] Handling call
0709 12:25:16.140995 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.141199 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.141231 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.141289 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:16.141325 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.141382 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.141510 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.144147 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4719) took 1696 ms. Trace:
I20260709 12:25:16.084316 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.017279 22670 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
W20260709 12:25:16.004238 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 138574us
W20260709 12:25:16.089188 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003d1240 after lost signal to thread 13825
I20260709 12:25:15.889672 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2980) took 1079 ms. Trace:
I20260709 12:25:16.089299 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.147657 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.148303 14423 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:16.090749 13649 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:16.094123 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2950) took 1642 ms. Trace:
I20260709 12:25:16.013777 22849 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.092s
W20260709 12:25:15.954830 13542 rpcz_store.cc:269] 0709 12:23:55.295874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.296300 (+   426us) service_pool.cc:224] Handling call
0709 12:25:15.818229 (+80521929us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:09.542093 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:09.542905 (+   812us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:09.542961 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:25:09.542979 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:09.543519 (+   540us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:09.543617 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:09.543649 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:09.544122 (+   473us) tablet.cc:701] Row locks acquired
0709 12:25:09.544165 (+    43us) write_op.cc:260] PREPARE: finished
0709 12:25:09.544368 (+   203us) write_op.cc:270] Start()
0709 12:25:09.544525 (+   157us) write_op.cc:276] Timestamp: P: 1783599909544280 usec, L: 0
0709 12:25:09.544564 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:25:09.545384 (+   820us) log.cc:844] Serialized 913 byte log entry
0709 12:25:10.795352 (+1249968us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:10.953507 (+158155us) op_driver.cc:464] REPLICATION: finished
0709 12:25:11.123906 (+170399us) write_op.cc:301] APPLY: starting
0709 12:25:11.140935 (+ 17029us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:11.795441 (+654506us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:11.795489 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:11.796937 (+  1448us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:11.818988 (+ 22051us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:25:11.819081 (+    93us) write_op.cc:312] APPLY: finished
0709 12:25:11.819505 (+   424us) log.cc:844] Serialized 33 byte log entry
0709 12:25:11.819776 (+   271us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:11.819896 (+   120us) write_op.cc:454] Released schema lock
0709 12:25:11.819966 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:11.821388 (+  1422us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:11.821425 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":750857,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":103379,"apply.run_cpu_time_us":8956,"apply.run_wall_time_us":737155,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":111857,"prepare.run_cpu_time_us":25481,"prepare.run_wall_time_us":264721,"raft.queue_time_us":103256,"raft.run_cpu_time_us":9493,"raft.run_wall_time_us":126751,"replication_time_us":1378364,"spinlock_wait_cycles":14281472,"thread_start_us":361404,"threads_started":4,"wal-append.queue_time_us":352669,"wal-append.run_cpu_time_us":15769,"wal-append.run_wall_time_us":2920437}]]}
I20260709 12:25:16.106649 22905 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:16.099449 13793 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 33: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:16.105839 13547 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:16.070741 13799 rpcz_store.cc:276] 0709 12:25:14.807816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.363050 (+555234us) service_pool.cc:224] Handling call
0709 12:25:15.956768 (+593718us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.107131 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:15.967564 23074 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:15.955834 23056 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.008450 22984 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.006242 22946 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 12:25:16.166180 13644 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:16.113368 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:16.107000 22841 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.099548 22850 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.119266 22920 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.120014 22556 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.003280 23070 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.120463 23079 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.121408 13795 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:16.123759 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.083125 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:16.113899 13677 heartbeater.cc:646] Failed to heartbeat to 127.12.45.126:44051 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.45.126:44051 timed out after 15.000s (SENT)
I20260709 12:25:16.178341 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2977) took 1370 ms. Trace:
I20260709 12:25:15.952070 22969 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.070s
I20260709 12:25:16.144512 13646 rpcz_store.cc:276] 0709 12:25:14.448143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.143167 (+1695024us) service_pool.cc:224] Handling call
0709 12:25:16.143500 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.143813 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.143841 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.143883 (+    42us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:16.143926 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.143982 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.144092 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.089730 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2985) took 1275 ms. Trace:
W20260709 12:25:16.149030 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003557a0 after lost signal to thread 14449
I20260709 12:25:16.147933 13801 rpcz_store.cc:276] 0709 12:25:14.811870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.888140 (+1076270us) service_pool.cc:224] Handling call
0709 12:25:15.889404 (+  1264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.149138 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2963) took 1244 ms. Trace:
I20260709 12:25:16.141495 14424 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:16.148849 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.149407 13652 rpcz_store.cc:276] 0709 12:25:14.451549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.085148 (+1633599us) service_pool.cc:224] Handling call
0709 12:25:16.094047 (+  8899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17664}
W20260709 12:25:16.094655 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 367079us
I20260709 12:25:16.110951 22945 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.073236 13797 rpcz_store.cc:276] 0709 12:25:14.482714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.486578 (+  3864us) service_pool.cc:224] Handling call
0709 12:25:14.509823 (+ 23245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.510079 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:14.523087 (+ 13008us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.967429 (+1444342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.967494 (+    65us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:15.967573 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.967647 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.967770 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5355,"prepare.queue_time_us":12992,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":51,"raft.queue_time_us":38307,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":7502,"thread_start_us":1328197,"threads_started":1,"wal-append.queue_time_us":120300,"wal-append.run_cpu_time_us":1508,"wal-append.run_wall_time_us":1002309}
W20260709 12:25:16.157845 13690 leader_election.cc:336] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.714s (SENT)
W20260709 12:25:16.159121 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.164739 13648 raft_consensus.cc:2410] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:25:16.196578 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2984) took 1382 ms. Trace:
I20260709 12:25:16.197063 13795 rpcz_store.cc:276] 0709 12:25:14.814107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.068509 (+1254402us) service_pool.cc:224] Handling call
0709 12:25:16.196507 (+127998us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.113206 23075 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.167847 13800 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
W20260709 12:25:16.168206 13644 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2957) took 1645 ms (client timeout 2100 ms). Trace:
I20260709 12:25:16.184731 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4721) took 1689 ms. Trace:
W20260709 12:25:16.200593 13644 rpcz_store.cc:269] 0709 12:25:14.522543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.065477 (+1542934us) service_pool.cc:224] Handling call
0709 12:25:16.168142 (+102665us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.201889 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4722) took 1704 ms. Trace:
I20260709 12:25:16.165086 13547 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 24, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 24 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.202939 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4723) took 1692 ms. Trace:
W20260709 12:25:16.203421 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080033db00 after lost signal to thread 14450
I20260709 12:25:16.081902 23082 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:16.204468 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf9e0 after lost signal to thread 22898
I20260709 12:25:16.204291 13547 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:16.205379 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800216b60 after lost signal to thread 22972
I20260709 12:25:16.205578 13727 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:16.145365 23052 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:16.102196 22537 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.179021 13796 rpcz_store.cc:276] 0709 12:25:14.808233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.014352 (+1206119us) service_pool.cc:224] Handling call
0709 12:25:16.178239 (+163887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.095082 22880 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.224089 14416 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 23. Preceding OpId from leader: term: 31 index: 23. (term mismatch)
I20260709 12:25:16.228211 14416 pending_rounds.cc:85] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 22
I20260709 12:25:16.230507 14416 pending_rounds.cc:107] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 30.23
I20260709 12:25:16.230873 14416 raft_consensus.cc:2889] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.163156 22935 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 12:25:16.205778 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6060 after lost signal to thread 22978
W20260709 12:25:16.237725 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080036eac0 after lost signal to thread 23032
I20260709 12:25:16.182689 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:16.202981 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45188.2us
I20260709 12:25:16.138296 22809 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.147501 22868 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:16.206169 13547 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:16.202384 13646 rpcz_store.cc:276] 0709 12:25:14.497873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.200907 (+1703034us) service_pool.cc:224] Handling call
0709 12:25:16.201281 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.201483 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.201522 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.201566 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:16.201599 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.201675 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.201823 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.207653 13649 rpcz_store.cc:276] 0709 12:25:14.905219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.071726 (+1166507us) service_pool.cc:224] Handling call
0709 12:25:16.149071 (+ 77345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.216709 13792 rpcz_store.cc:276] 0709 12:25:14.814429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.073192 (+1258763us) service_pool.cc:224] Handling call
0709 12:25:16.089653 (+ 16461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.200654 13645 rpcz_store.cc:276] 0709 12:25:14.495850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.168759 (+1672909us) service_pool.cc:224] Handling call
0709 12:25:16.184190 (+ 15431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.184389 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.184414 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.184460 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:16.184492 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.184543 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.184669 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.203608 13644 rpcz_store.cc:276] 0709 12:25:14.511001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.201771 (+1690770us) service_pool.cc:224] Handling call
0709 12:25:16.202163 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.202438 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.202486 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.202552 (+    66us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:16.202604 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.202684 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.202845 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.244714 13617 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.807s (SENT)
I20260709 12:25:16.256109 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.223315 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:16.231848 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2951) took 1764 ms (client timeout 2229 ms). Trace:
W20260709 12:25:16.258754 13648 rpcz_store.cc:269] 0709 12:25:14.467831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.083471 (+1615640us) service_pool.cc:224] Handling call
0709 12:25:16.231783 (+148312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.156253 23084 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.178311 22869 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:16.233251 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.240607 13650 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:25:16.156826 22958 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:16.220714 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45795.2us
I20260709 12:25:16.186877 22803 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.129805 23019 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.174983 22850 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.254097 13547 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 14, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.289271 13547 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 20, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 20 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.257128 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:16.265770 22537 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.266503 23087 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:16.267920 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.268666 23085 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.270097 14418 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:16.271267 13646 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:16.281949 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48834.8us
I20260709 12:25:16.282110 13643 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:25:16.181277 22860 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:25:16.292972 13547 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 18, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 18 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:16.261978 22809 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:16.294895 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 171436us
I20260709 12:25:16.334400 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.299197 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.301282 13648 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:25:16.336109 14417 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:16.302011 23077 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:16.314502 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2962) took 1626 ms (client timeout 1630 ms). Trace:
I20260709 12:25:16.340670 13547 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 21, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 21 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:25:16.341527 13643 rpcz_store.cc:269] 0709 12:25:14.688130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.620527 (+932397us) service_pool.cc:224] Handling call
0709 12:25:16.314340 (+693813us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.342587 13547 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 17, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 17 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.312806 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.344755 13547 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 28, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 28 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.275852 22869 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:16.292959 22857 raft_consensus.cc:2764] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:25:16.360558 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.309389 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.332331 13646 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:16.339749 13645 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:16.346660 13547 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:25:16.347764 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46102.7us
I20260709 12:25:16.296065 23071 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:16.367475 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58182.6us
I20260709 12:25:16.366963 13547 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 24, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 24 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:16.364061 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.353035 23092 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.279263 23084 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.252452 22880 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:16.355734 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2952) took 1884 ms (client timeout 2401 ms). Trace:
I20260709 12:25:16.377421 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.286468 22803 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:16.377233 13650 rpcz_store.cc:269] 0709 12:25:14.471610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.084028 (+1612418us) service_pool.cc:224] Handling call
0709 12:25:16.355506 (+271478us) spinlock_profiling.cc:243] Waited 94.9 ms on lock 0x7b5400198054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:16.355666 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218261760}
I20260709 12:25:16.349124 23097 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.284801 23013 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:25:16.368595 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:16.373890 13547 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 26, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 26 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.387326 13547 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 26, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 26 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:16.307202 23095 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:16.393005 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46999.1us
I20260709 12:25:16.299234 22850 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.397742 22537 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:16.397961 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.400557 13547 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:16.396307 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.405896 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:16.408872 13547 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:16.410333 13547 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 10 to 20, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 20 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.411980 13547 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:16.412631 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.413465 13547 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 18, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 18 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.375944 23018 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:16.373039 22846 raft_consensus.cc:2764] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:25:16.388931 23071 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:16.421079 13547 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 23, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 23 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:16.422309 14417 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:16.423413 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.424261 23094 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:16.329592 23087 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.415045 22670 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:16.329910 23085 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.468011 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.386945 23084 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.394776 23069 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.458068 13547 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:16.460266 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:16.470860 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 73021.7us
I20260709 12:25:16.447290 14423 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:16.488056 14416 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:16.420132 23093 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:16.488281 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.495618 13547 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:16.501279 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.457496 23018 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:16.506840 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.389106 23092 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.502080 14423 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:16.406958 22803 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:16.510951 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 231963us
I20260709 12:25:16.418586 22537 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.395896 23097 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.465816 22670 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:16.469390 22849 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:16.391688 22868 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.523288 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:16.516175 22956 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.627s	user 0.014s	sys 0.000s
I20260709 12:25:16.530849 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:16.532802 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 474657us
I20260709 12:25:16.501384 23071 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.493644 22984 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.418097 22869 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.572163 23098 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:16.569130 14416 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:16.570386 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.548019 22670 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:16.571646 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.585872 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.547097 23074 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.568079 22841 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.600841 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.597911 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.602557 23070 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.603869 23092 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.572616 23056 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.570839 23082 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.568423 23102 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:16.596990 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:16.598850 23100 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:25:16.598917 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.608372 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.590250 22945 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.631850 14421 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:16.607187 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:16.573884 23079 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.621747 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.622175 22850 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.623775 22803 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.549625 23093 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:25:16.624037 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.569947 23071 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.627270 23085 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.628441 23084 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.630062 23097 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.569581 22956 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:16.678443 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.637202 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:16.643882 23099 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:16.645628 22776 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:16.649169 23019 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:16.620244 23018 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.656093 22670 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.690089 23093 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:16.690585 23093 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:16.689785 23080 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:16.691397 22956 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.691748 23080 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:16.692235 23080 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:16.664764 14423 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:16.621114 23077 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:16.692806 23077 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.693233 23077 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:16.682960 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 348674us
W20260709 12:25:16.671954 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 275725us
I20260709 12:25:16.701520 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.703461 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:16.685300 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48216.6us
I20260709 12:25:16.694957 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:16.669100 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 424391us
I20260709 12:25:16.705555 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:16.721849 13646 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 32 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:16.723274 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:16.724234 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.725279 22956 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:16.725883 22956 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:16.727885 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 156330us
I20260709 12:25:16.730446 13646 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:16.732784 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.687215 23099 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.740864 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.748438 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:16.746348 23123 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:16.701001 23110 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:25:16.741106 22809 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.757467 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:16.750169 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:16.687493 22776 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:16.763900 13720 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:16.763515 22776 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.755201 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.800343 13724 raft_consensus.cc:2410] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 32 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 15.
I20260709 12:25:16.766170 22776 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:16.800624 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45539us
W20260709 12:25:16.801816 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51746.6us
I20260709 12:25:16.752617 23099 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:16.819296 23114 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.092s
I20260709 12:25:16.798863 22976 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:16.722028 23112 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:16.820909 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.799853 22803 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:16.820078 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.827558 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:16.836614 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.842052 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.756093 23070 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:16.751924 23106 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:16.856560 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.846777 13718 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:16.801191 23052 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.844934 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.833451 23082 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:16.820078 23079 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:16.857235 23074 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:16.691411 23093 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.821861 23097 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 32.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.049s
I20260709 12:25:16.844064 23117 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:16.824476 22956 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.862751 23084 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:16.893225 23097 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:16.893392 23084 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:16.893661 23097 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:16.893875 23084 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:16.846101 22803 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:16.894642 23097 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.896265 22803 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.896394 23097 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:16.898756 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:16.900315 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:16.900913 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:16.901798 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:16.824128 22881 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:16.866834 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:16.910712 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43940.8us
I20260709 12:25:16.911492 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.911813 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.912554 23101 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:16.912911 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:16.886126 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.913573 14416 raft_consensus.cc:2376] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21. Re-sending same reply.
I20260709 12:25:16.914492 22405 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:16.866645 23116 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:16.908743 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:16.919262 13723 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:16.920589 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.886979 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:16.907533 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.868795 23092 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:16.913050 23101 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:16.885975 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.923378 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.868880 22976 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:16.917696 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:16.929854 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 181511us
W20260709 12:25:16.932500 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.935881 23115 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:16.925523 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.949306 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:16.952702 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.927052 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.948756 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.958528 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1509) took 1009 ms. Trace:
I20260709 12:25:16.959383 13648 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:16.959767 13646 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 16: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:16.890551 23082 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:16.904335 23079 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:16.959005 13647 rpcz_store.cc:276] 0709 12:25:15.948867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.345593 (+396726us) service_pool.cc:224] Handling call
0709 12:25:16.345884 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.346195 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:16.688752 (+342557us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.958143 (+269391us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.958203 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:16.958248 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.958337 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.958466 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37644,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"thread_start_us":515593,"threads_started":2}
I20260709 12:25:16.964540 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.965739 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.908835 23074 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:16.870035 23007 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:25:16.973555 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.977027 13646 raft_consensus.cc:2410] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:25:16.978053 13724 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:25:16.923722 23116 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.884800 23052 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.983883 23082 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:16.985648 23052 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.925598 22852 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:16.972908 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:16.982677 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.982962 23071 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:16.981777 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.987994 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.988674 13646 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:16.989624 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:16.986277 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:17.016422 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:17.017486 13647 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:16.951210 23101 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:16.966928 23093 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:17.019124 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.966604 22881 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:16.962774 23092 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:17.035434 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.023751 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:17.024065 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 73187.7us
W20260709 12:25:17.024186 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.030418 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:16.974603 23053 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:17.042062 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:17.043757 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.045953 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:17.047530 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 140112us
I20260709 12:25:17.045553 14421 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:17.049057 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.051591 13801 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:17.051859 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.019131 23115 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:17.057785 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 105920us
I20260709 12:25:16.974045 23069 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:25:16.987406 23116 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:17.059391 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:17.088937 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.094606 13799 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
W20260709 12:25:17.056775 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 117358us
I20260709 12:25:17.032883 23071 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.034073 23052 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:17.087378 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:16.978596 22677 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.065s
I20260709 12:25:17.057960 22976 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.085343 23130 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:25:17.101353 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.116708 23069 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:17.105479 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4724) took 1274 ms. Trace:
I20260709 12:25:17.111482 22925 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:17.105216 23079 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.110497 23072 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.137889 13652 rpcz_store.cc:276] 0709 12:25:15.830636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.224910 (+394274us) service_pool.cc:224] Handling call
0709 12:25:16.262788 (+ 37878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.441845 (+179057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.441880 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.441925 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:16.441968 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.105226 (+663258us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400114c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:17.105284 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.105414 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411293568}
W20260709 12:25:17.152930 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 93638.4us
I20260709 12:25:17.158742 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.144647 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.088366 22670 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:17.085760 23056 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:17.165719 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.097826 22954 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:17.100016 23052 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:17.171702 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68387.7us
I20260709 12:25:17.178737 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.139870 22925 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.185525 22925 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.159729 23069 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.187641 23072 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:17.188266 23072 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.158314 22809 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:17.189955 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86879.7us
I20260709 12:25:17.166304 23007 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:17.191169 13693 connection.cc:256] RPC call timeout handler was delayed by -2.1379184000002169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:25:17.187197 22925 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:17.150099 22405 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:17.185525 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.179388 23052 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:17.198383 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0208943400002681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:25:17.205508 13693 connection.cc:256] RPC call timeout handler was delayed by -2.0172943400002623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 12:25:17.152750 23074 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.104365 23071 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.197034 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:17.206604 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:17.202064 23072 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:17.201288 23069 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:17.209858 22677 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:17.189639 22881 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.221903 23132 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.236s	user 0.002s	sys 0.001s
I20260709 12:25:17.224221 13723 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:25:17.276150 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 69656.2us
W20260709 12:25:17.229035 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.230710 23084 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.277812 23136 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:17.232300 22869 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:17.202116 22809 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:17.238654 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2978) took 2430 ms (client timeout 1799 ms). Trace:
I20260709 12:25:17.182796 23092 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:17.247696 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8894943400000557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 12:25:17.249985 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46718.5us
W20260709 12:25:17.287792 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8552943400004551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 12:25:17.288321 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8507943400004478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:25:17.288836 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8426943400004347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 12:25:17.289954 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8138943400003882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 12:25:17.290437 13693 connection.cc:256] RPC call timeout handler was delayed by -1.8129943400003867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
I20260709 12:25:17.259789 23007 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:17.260532 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51859.2us
W20260709 12:25:17.238657 23131 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:17.173796 23056 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.277113 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.278586 23136 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.273451 23138 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.011s
I20260709 12:25:17.251699 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 16.
W20260709 12:25:17.291395 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7805943400003343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260709 12:25:17.288942 22809 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.285207 13800 rpcz_store.cc:269] 0709 12:25:14.808706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.788827 (+980121us) service_pool.cc:224] Handling call
0709 12:25:17.238212 (+1449385us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54003d4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:17.238486 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518098176}
W20260709 12:25:17.306473 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7805943400003343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 12:25:17.308326 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7427943400002732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 12:25:17.308677 13693 connection.cc:256] RPC call timeout handler was delayed by -1.7085943400002179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
W20260709 12:25:17.308971 13693 connection.cc:256] RPC call timeout handler was delayed by -1.542386383000121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 12:25:17.309286 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4496863829999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 12:25:17.309538 13693 connection.cc:256] RPC call timeout handler was delayed by -1.4259943400002157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 12:25:17.309777 13693 connection.cc:256] RPC call timeout handler was delayed by -1.3839863829998649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 12:25:17.310303 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2711943400004202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24009999999999998s for a response
W20260709 12:25:17.310712 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2327863830000751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
W20260709 12:25:17.311024 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2318863830000737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
W20260709 12:25:17.311280 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1784943400002703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2504s for a response
W20260709 12:25:17.311510 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0266863830001967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 12:25:17.312448 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:17.313659 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.300885 22954 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:25:17.298628 23007 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.330083 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.281414 23099 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.331276 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54097.4us
I20260709 12:25:17.340601 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:17.346935 13725 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:17.349349 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.332901 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.353847 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.359596 23082 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:17.350519 22412 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.362246 22412 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.363132 23142 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:17.363865 22809 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.364584 22412 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:17.367551 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4732) took 1279 ms. Trace:
I20260709 12:25:17.367969 13645 rpcz_store.cc:276] 0709 12:25:16.088250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.362428 (+274178us) service_pool.cc:224] Handling call
0709 12:25:16.362750 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.507210 (+144460us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:16.573837 (+ 66627us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.734464 (+160627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.734578 (+   114us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:16.734650 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.904049 (+169399us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b480033c558. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:17.367269 (+463220us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400199454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:17.367330 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.367482 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66414,"prepare.queue_time_us":80400,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":22056,"raft.run_cpu_time_us":2445,"raft.run_wall_time_us":161441,"spinlock_wait_cycles":338475136,"thread_start_us":141967,"threads_started":1,"wal-append.queue_time_us":151520}
I20260709 12:25:17.371372 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1446) took 1045 ms. Trace:
I20260709 12:25:17.371251 22954 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:17.372053 22954 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.371868 14418 rpcz_store.cc:276] 0709 12:25:16.326228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.328619 (+  2391us) service_pool.cc:224] Handling call
0709 12:25:17.371152 (+1042533us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54006be4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:17.371274 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102427008}
I20260709 12:25:17.373219 22954 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.373818 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.359740 22852 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:17.375204 23094 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:17.375633 23094 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:17.375976 23094 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:17.377301 23095 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:17.377732 23095 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:17.379172 22526 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.329135 23144 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:17.424549 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.424222 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 74980.5us
I20260709 12:25:17.307016 23114 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.075s
I20260709 12:25:17.429517 13725 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:17.430866 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.435277 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.439958 14418 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:17.437122 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:17.425064 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71326.4us
I20260709 12:25:17.441468 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.443075 13793 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:25:17.440799 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.448254 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.383522 23084 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.449743 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.452265 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.448649 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:17.532914 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 84344.1us
W20260709 12:25:17.509776 23019 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.816s	user 0.010s	sys 0.004s
I20260709 12:25:17.499228 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.329895 22405 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.438697 22954 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:17.508880 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:17.434206 23138 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:17.520013 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79308.7us
I20260709 12:25:17.350955 23056 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.552484 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.515249 23095 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.450749 23116 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:17.527617 23145 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:25:17.528290 23071 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:17.530071 23080 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.837s	user 0.014s	sys 0.004s
I20260709 12:25:17.455495 14419 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: [31.23-31.23]
I20260709 12:25:17.448083 22803 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:17.479789 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.544377 22881 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:25:17.536293 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 189380us
I20260709 12:25:17.432200 23072 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:17.504173 22956 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:17.539680 14416 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
W20260709 12:25:17.547371 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.570955 23144 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:17.578444 23106 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.003s
W20260709 12:25:17.537343 23093 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.503s	user 0.014s	sys 0.001s
I20260709 12:25:17.563118 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.550925 23066 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:17.595026 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.581655 22803 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.592181 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.595916 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:17.597551 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.558923 14424 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:17.580226 22881 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.608803 13651 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:17.570564 22846 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:25:17.606338 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1353) took 1005 ms. Trace:
I20260709 12:25:17.612525 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:17.614862 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.616065 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:17.616694 23093 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:17.617113 23093 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:17.617530 23093 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:17.623785 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.625141 23084 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:17.614703 14420 rpcz_store.cc:276] 0709 12:25:16.600637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.602054 (+  1417us) service_pool.cc:224] Handling call
0709 12:25:17.592059 (+990005us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b5800011198. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:25:17.606115 (+ 14056us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3974656}
I20260709 12:25:17.627233 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.628579 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.604624 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:17.535673 23136 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.593868 23095 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:17.591315 23087 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.596968 23106 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:17.594732 23066 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:17.611279 23101 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.006s
I20260709 12:25:17.578994 23080 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.609201 22956 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:17.630924 23137 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.010s
I20260709 12:25:17.612982 22881 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:17.631460 23079 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:17.661247 13793 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:25:17.639501 23144 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:17.638131 23134 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:17.668394 14418 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:17.703950 23155 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:17.711614 23116 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:17.718231 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 113738us
I20260709 12:25:17.705487 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.720532 13800 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:25:17.720659 23134 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:17.725111 13719 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:17.673872 23099 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:17.726115 13796 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:17.728242 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:17.654326 23101 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.722697 23087 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.005s
I20260709 12:25:17.723250 23153 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:17.744349 13718 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
W20260709 12:25:17.750725 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93391.3us
I20260709 12:25:17.751921 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.753916 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.751741 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:17.705232 23089 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.033s
W20260709 12:25:17.757014 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.758908 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:17.762378 23147 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
W20260709 12:25:17.764467 23115 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.001s
I20260709 12:25:17.764917 23115 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:17.740765 23153 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.768195 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:17.738144 23084 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:25:17.774101 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.779397 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:17.757014 23071 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:17.771112 23134 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:17.786176 13794 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:17.771806 23092 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:17.786929 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51678.5us
I20260709 12:25:17.787783 13725 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:25:17.787189 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:17.791568 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:17.774927 23116 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:17.755080 22868 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:25:17.798149 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:17.780223 23161 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:17.803232 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:17.805055 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4727) took 1973 ms. Trace:
I20260709 12:25:17.782648 22956 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:17.766242 23093 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.805464 13644 rpcz_store.cc:276] 0709 12:25:15.831657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.265474 (+433817us) service_pool.cc:224] Handling call
0709 12:25:16.265813 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.266045 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:16.379838 (+113793us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.399329 (+ 19491us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.242929 (+843600us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b18002a0880. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:17.804502 (+561573us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b18002a0880. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:25:17.804593 (+    91us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:17.804635 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:17.804673 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.804764 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.804974 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11352,"prepare.queue_time_us":117746,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":7614,"raft.queue_time_us":177724,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":24883,"spinlock_wait_cycles":24099584,"thread_start_us":231429,"threads_started":3,"wal-append.queue_time_us":26773,"wal-append.run_cpu_time_us":3203,"wal-append.run_wall_time_us":1218568}
W20260709 12:25:17.810977 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.792562 23072 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:17.812844 23087 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.813392 23087 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.815804 23087 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.807533 22984 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:17.823338 22868 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:17.823825 22868 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.824465 22958 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:17.825003 22868 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.828218 23080 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:17.808079 23101 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:17.820860 23150 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.188s	user 0.001s	sys 0.001s
I20260709 12:25:17.810149 22956 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:17.847337 22880 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:17.854012 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43898.7us
W20260709 12:25:17.839874 23052 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.005s
I20260709 12:25:17.835144 23161 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:17.871703 13652 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:17.871598 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:17.874392 13648 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:25:17.874574 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:17.836367 22803 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.843029 22956 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:17.843392 22868 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:17.806502 23153 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.892621 23153 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:17.895035 13649 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:17.895083 23161 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:17.898953 23163 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:25:17.901589 23093 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:17.902801 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.904109 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.906104 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:17.907408 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.908492 23137 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.908584 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:17.909307 23137 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.911765 23137 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:17.912554 13796 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
W20260709 12:25:17.913831 23094 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.003s
I20260709 12:25:17.916639 23066 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:17.968108 23066 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:17.968423 14424 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1341) took 1545 ms (client timeout 1699 ms). Trace:
W20260709 12:25:17.969151 14424 rpcz_store.cc:269] 0709 12:25:16.423041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.423157 (+   116us) service_pool.cc:224] Handling call
0709 12:25:17.968036 (+1544879us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54005965d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:17.968266 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2553191680}
W20260709 12:25:17.927436 23077 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.227s	user 0.011s	sys 0.006s
I20260709 12:25:17.931113 23108 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:17.971048 23108 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:17.938678 23056 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:17.938906 22869 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.002s
I20260709 12:25:17.941754 22537 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.877420 23052 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:17.954280 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.962375 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.964325 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4734) took 1737 ms. Trace:
I20260709 12:25:17.991578 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:17.993319 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:17.923501 23075 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:17.969645 22526 consensus_queue.cc:1175] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 26 status { last_received { term: 23 index: 18 } last_committed_idx: 18 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: [25.19-25.19]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:17.972452 14416 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:25:18.006690 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:17.975468 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.979969 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:17.983899 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:17.966490 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.009428 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.010406 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:25:18.012501 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:17.992468 13652 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:17.999353 13797 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:17.988651 13650 rpcz_store.cc:276] 0709 12:25:16.226996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.390921 (+163925us) service_pool.cc:224] Handling call
0709 12:25:16.391297 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.499241 (+107944us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:16.517012 (+ 17771us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.591225 (+ 74213us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.692653 (+101428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.692712 (+    59us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:16.692774 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.922182 (+1229408us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b18002ba5c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5397ac858f 00007f5390b07571 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd
0709 12:25:17.964032 (+ 41850us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001b5654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:17.964089 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.964242 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174385,"prepare.queue_time_us":1377968,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":1019,"raft.queue_time_us":150272,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":27672,"spinlock_wait_cycles":251074048,"thread_start_us":25677,"threads_started":1,"wal-append.queue_time_us":25916,"wal-append.run_cpu_time_us":3293,"wal-append.run_wall_time_us":1293512}
I20260709 12:25:18.002739 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:17.936926 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.984638 22935 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.013763 13726 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:18.020279 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:17.985227 23093 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:18.029101 23024 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.032491 14422 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
W20260709 12:25:18.069425 23146 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.542s	user 0.001s	sys 0.001s
I20260709 12:25:18.072144 23114 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:18.073487 23114 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:18.077183 23075 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:18.077807 23154 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:18.077910 23075 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.080627 22537 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.081408 22537 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.057730 23093 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.085047 23093 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.087410 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.088266 23160 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.088919 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.089198 23160 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:17.987918 22670 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.111s
I20260709 12:25:18.089741 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:18.100940 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.078675 23154 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:18.102654 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:18.103449 13643 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:18.103708 23154 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.105738 23154 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.106914 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:18.107656 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:17.982235 22881 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.109400 14423 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:18.110102 22537 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.110915 22881 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.112082 23108 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.113891 23108 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:17.986378 23094 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.123291 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:18.145777 14424 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:18.098969 22935 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.132555 22670 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:18.055924 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:18.136819 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4202) took 1377 ms. Trace:
I20260709 12:25:18.143512 13652 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
W20260709 12:25:18.149978 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.121824 23052 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.154969 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:18.155391 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99543.7us
I20260709 12:25:18.156620 13652 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:18.157460 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.161576 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:18.163843 13721 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:18.163674 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.093017 23075 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:17.984596 23077 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:18.165966 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.166383 14417 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:18.162264 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16.
I20260709 12:25:18.173091 13649 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:18.174577 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.173841 23124 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.158353 22670 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:18.179271 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:18.186333 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 120468us
I20260709 12:25:18.186820 13795 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:18.187045 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.181062 13644 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:18.166661 23144 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:18.025697 22956 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.189217 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:18.180528 14424 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:18.185063 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:18.182837 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 220871us
I20260709 12:25:18.193179 13722 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:18.194832 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.196218 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.194833 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:18.199879 13617 connection.cc:256] RPC call timeout handler was delayed by -1.4322979450003004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25930000000000003s for a response
I20260709 12:25:18.181318 23024 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:18.188221 23077 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.219527 22935 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.220753 22670 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.223730 22956 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.189003 23124 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.226755 23094 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.202419 23135 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:18.229359 23135 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:18.229770 23135 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:18.233631 14417 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1342) took 1766 ms (client timeout 1626 ms). Trace:
I20260709 12:25:18.204497 23114 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:18.238966 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:18.239387 23114 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:18.242825 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2965) took 1322 ms. Trace:
I20260709 12:25:18.243322 13651 rpcz_store.cc:276] 0709 12:25:16.920024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.920318 (+   294us) service_pool.cc:224] Handling call
0709 12:25:18.242647 (+1322329us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b5400198a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:18.242759 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1580950784}
I20260709 12:25:18.205327 22869 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:25:18.247391 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:18.200388 22405 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:18.204432 23144 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:18.240538 23114 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.263726 13799 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:18.265776 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.266666 13726 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:18.267196 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.267645 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.268074 23108 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:18.268384 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.268962 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.269554 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.242874 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.272387 13795 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:18.272528 13725 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:18.274185 13801 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
W20260709 12:25:18.275986 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.040s (SENT)
I20260709 12:25:18.275144 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.276585 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:18.274122 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:18.279109 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:18.234169 14417 rpcz_store.cc:269] 0709 12:25:16.467548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.467714 (+   166us) service_pool.cc:224] Handling call
0709 12:25:18.233347 (+1765633us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54004998d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:18.233486 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3710404480}
I20260709 12:25:18.278965 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:18.282048 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77004.8us
I20260709 12:25:18.282307 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:18.282931 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:18.283257 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.254s (SENT)
I20260709 12:25:18.283721 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:18.285529 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83495.6us
I20260709 12:25:18.268641 23108 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.283722 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:18.289830 13725 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:18.291761 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.292456 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:18.289853 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.295763 13651 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:18.298791 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:18.299371 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.299736 14424 raft_consensus.cc:2410] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:25:18.300726 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.301484 13726 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:25:18.305559 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.298547 23084 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:18.305495 23075 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:18.309665 13723 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:25:18.313895 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:18.290329 23094 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:18.339076 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.330968 23160 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:18.324029 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:18.332515 23124 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:18.154359 13727 rpcz_store.cc:276] 0709 12:25:16.759338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.762361 (+  3023us) service_pool.cc:224] Handling call
0709 12:25:16.762652 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.762876 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:16.845681 (+ 82805us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.944464 (+1098783us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b1800319cc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:25:18.133764 (+189300us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.136406 (+  2642us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:18.136474 (+    68us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:18.136534 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.136615 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.136745 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":258388,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":98,"raft.queue_time_us":40504,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":14557,"spinlock_wait_cycles":28801408,"thread_start_us":445021,"threads_started":4,"wal-append.queue_time_us":140496}
I20260709 12:25:18.289593 23144 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:18.342669 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.339233 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4218) took 1557 ms. Trace:
I20260709 12:25:18.348311 13721 rpcz_store.cc:276] 0709 12:25:16.781418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.156271 (+374853us) service_pool.cc:224] Handling call
0709 12:25:17.156537 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.164288 (+1007751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.338980 (+174692us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001563d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:18.339167 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316128512}
I20260709 12:25:18.300417 22405 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:18.348894 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:18.347267 14424 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:25:18.351600 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.352962 13726 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:25:18.310544 23092 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:18.310338 23077 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:18.353718 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:18.368633 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.369585 13646 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:18.368794 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.373486 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.373855 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.375986 13793 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:25:18.376159 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:18.381381 13799 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:18.344430 23094 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.387624 13647 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:18.367439 22956 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:18.370182 23093 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.369905 22803 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.369958 23024 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:18.354871 23075 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:18.392560 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.370839 22405 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:18.366973 23092 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:18.356572 13725 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:25:18.365828 23101 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:25:18.379042 23160 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:18.397934 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:18.407789 13727 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:18.410419 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.411093 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:18.390730 23094 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:18.420907 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:18.423977 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.611s (SENT)
I20260709 12:25:18.424933 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:18.375650 23161 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
W20260709 12:25:18.429090 13617 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.776s (SENT)
I20260709 12:25:18.429951 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:18.397431 22956 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:18.401937 23092 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:18.401356 22405 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:18.403877 23160 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.443048 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4210) took 1677 ms. Trace:
I20260709 12:25:18.443186 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:18.443832 13647 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:18.443536 13719 rpcz_store.cc:276] 0709 12:25:16.765946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.931539 (+165593us) service_pool.cc:224] Handling call
0709 12:25:16.931830 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.731091 (+799261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.442773 (+711682us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54001577d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:18.442979 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1823838464}
I20260709 12:25:18.447181 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.395546 22803 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.397747 23093 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.435402 23052 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.431726 23077 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:18.438103 22537 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:18.435894 23099 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:18.442095 22405 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:18.448354 23160 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:18.454054 13617 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.593s (SENT)
I20260709 12:25:18.535349 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:18.535398 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:18.536616 13645 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 32
W20260709 12:25:18.537310 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.539498 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:18.543936 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 89954.1us
W20260709 12:25:18.557996 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 206475us
I20260709 12:25:18.504856 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.523519 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 2997) took 1420 ms. Trace:
I20260709 12:25:18.533773 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:18.573383 13644 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
W20260709 12:25:18.574291 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.518146 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:18.521945 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:18.575109 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.575376 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:18.575709 13650 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:25:18.528004 13726 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:25:18.458137 13799 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:25:18.498879 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.563024 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:18.577299 13720 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:18.513388 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:18.493367 14422 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1345) took 1919 ms (client timeout 1710 ms). Trace:
I20260709 12:25:18.497321 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.580675 13801 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32.
I20260709 12:25:18.581746 13727 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:18.573423 13797 rpcz_store.cc:276] 0709 12:25:17.103050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.181800 (+ 78750us) service_pool.cc:224] Handling call
0709 12:25:18.523248 (+1341448us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b5400129b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:18.523456 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1696367872}
W20260709 12:25:18.517010 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.461746 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:18.579308 14422 rpcz_store.cc:269] 0709 12:25:16.573649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.573807 (+   158us) service_pool.cc:224] Handling call
0709 12:25:18.493097 (+1919290us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:18.493249 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2862182656}
W20260709 12:25:18.584662 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66629.8us
I20260709 12:25:18.588374 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:18.588681 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 127027us
I20260709 12:25:18.591213 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.592213 13720 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:18.449648 23108 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:18.480556 23164 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.698s	user 0.002s	sys 0.004s
I20260709 12:25:18.611236 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:18.552563 22537 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:18.491447 23148 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.960s	user 0.004s	sys 0.001s
W20260709 12:25:18.668067 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78160.7us
I20260709 12:25:18.475834 23093 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.633136 23019 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:18.673871 23108 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:18.597152 23153 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 12:25:18.685317 23153 consensus_queue.cc:1243] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 25.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:18.687031 13644 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:18.688675 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.481276 23099 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.589531 23162 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:18.689859 23099 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:18.520119 22803 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.489969 22956 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.691505 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.691820 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.692493 13721 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:18.476496 23052 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.692751 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.693400 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.694748 23052 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:18.695428 13800 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:18.696332 14422 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:18.697466 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:18.695464 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:18.704474 13651 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:25:18.704635 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.637980 23104 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:18.706421 22526 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.706826 23104 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.707029 22526 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.708753 22670 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:18.709254 23104 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.709386 22526 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:18.710994 13651 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:25:18.679739 23156 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.925s	user 0.002s	sys 0.004s
I20260709 12:25:18.680683 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4221) took 1841 ms. Trace:
I20260709 12:25:18.716338 23108 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.722934 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.719734 13724 rpcz_store.cc:276] 0709 12:25:16.839344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.193781 (+354437us) service_pool.cc:224] Handling call
0709 12:25:17.194171 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.194602 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.194630 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.195185 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.307789 (+112604us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:17.337931 (+ 30142us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.338151 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.338223 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:17.358509 (+ 20286us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b1800445060. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:18.395901 (+1037392us) spinlock_profiling.cc:243] Waited 4.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:18.680440 (+284539us) raft_consensus.cc:1685] finished
0709 12:25:18.680468 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.680609 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77563,"prepare.queue_time_us":34023,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":21916,"raft.run_cpu_time_us":525,"raft.run_wall_time_us":106119,"spinlock_wait_cycles":45559936,"thread_start_us":105167,"threads_started":1,"wal-append.queue_time_us":179295}
I20260709 12:25:18.730309 13721 raft_consensus.cc:2368] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32.
I20260709 12:25:18.559425 23135 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.739089 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.747473 13721 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:18.637908 23166 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:18.752846 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 1962) took 1663 ms. Trace:
I20260709 12:25:18.753332 13718 rpcz_store.cc:276] 0709 12:25:17.089440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.262992 (+173552us) service_pool.cc:224] Handling call
0709 12:25:17.263297 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.744852 (+481555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.752515 (+1007663us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:18.752762 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105806720}
I20260709 12:25:18.754797 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.754799 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.756048 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.756048 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.693950 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.732890 23024 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:18.711933 23144 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 12:25:18.731066 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:18.780498 13724 raft_consensus.cc:2376] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 16. Re-sending same reply.
I20260709 12:25:18.757248 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:18.757755 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 69183.7us
I20260709 12:25:18.782891 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:18.757179 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:18.765641 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:18.783420 13647 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:18.785005 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.779224 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.790108 23024 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.791800 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:18.794342 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.795783 13718 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:18.803156 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:18.803869 22776 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:18.804742 22776 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:18.756345 13648 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:18.753685 22809 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:18.810930 22809 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:18.812582 13726 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:18.813174 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:18.821364 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1718) took 1385 ms. Trace:
I20260709 12:25:18.821677 14416 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:25:18.737536 23108 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:18.823098 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 260219us
I20260709 12:25:18.825793 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 2991) took 1776 ms. Trace:
I20260709 12:25:18.745052 23099 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.826206 13794 rpcz_store.cc:276] 0709 12:25:17.049458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.208766 (+159308us) service_pool.cc:224] Handling call
0709 12:25:17.209555 (+   789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.790771 (+581216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.825535 (+1034764us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54000e65d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:18.825727 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322065280}
I20260709 12:25:18.559273 23075 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.859916 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:18.812646 22809 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:18.958813 13722 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:18.961807 23056 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:18.962612 23056 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:18.966724 23056 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:18.967629 22905 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.968242 22905 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.821998 14425 rpcz_store.cc:276] 0709 12:25:17.436040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.436886 (+   846us) service_pool.cc:224] Handling call
0709 12:25:17.540704 (+103818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.541257 (+   553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.541303 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.541979 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.653040 (+111061us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:17.653100 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.653195 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.572596 (+919401us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:18.820104 (+247508us) raft_consensus.cc:1685] finished
0709 12:25:18.820139 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.821235 (+  1096us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":168,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":16224,"spinlock_wait_cycles":2875648,"thread_start_us":110616,"threads_started":1,"wal-append.queue_time_us":189792}
I20260709 12:25:18.806234 22776 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:18.962522 13726 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:25:18.981180 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121376us
I20260709 12:25:18.988919 13722 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:25:18.989305 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.531s (SENT) [suppressed 23 similar messages]
I20260709 12:25:18.990188 22905 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:18.993080 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:18.864404 23191 raft_consensus.cc:2764] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:25:19.000921 13726 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:25:18.991894 23179 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.274s	user 0.000s	sys 0.002s
W20260709 12:25:19.036892 23181 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:25:19.039004 23184 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 12:25:19.069921 23094 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.000s
I20260709 12:25:19.070886 23094 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.071697 23108 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:19.074591 23094 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.076711 23099 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.057719 23135 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:19.083284 23092 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.003s
I20260709 12:25:19.084074 23092 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:19.087934 23160 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.004s
I20260709 12:25:19.091344 23160 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.089315 23092 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:19.163499 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.168753 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.830992 13618 leader_election.cc:400] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32.
I20260709 12:25:19.170889 13618 leader_election.cc:403] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:19.172205 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 341431us
I20260709 12:25:19.172618 13651 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:19.176709 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4758) took 1276 ms. Trace:
I20260709 12:25:19.179136 13650 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:18.852682 13645 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:25:19.181373 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 2982) took 1126 ms. Trace:
I20260709 12:25:19.182058 13645 rpcz_store.cc:276] 0709 12:25:18.054928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.059606 (+  4678us) service_pool.cc:224] Handling call
0709 12:25:18.535208 (+475602us) spinlock_profiling.cc:243] Waited 36.5 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:25:19.180970 (+645762us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83990272}
I20260709 12:25:19.184396 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:19.188982 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 397097us
I20260709 12:25:19.193193 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4230) took 1381 ms. Trace:
I20260709 12:25:19.193941 13723 rpcz_store.cc:276] 0709 12:25:17.811957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.311565 (+499608us) service_pool.cc:224] Handling call
0709 12:25:18.311853 (+   288us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:18.317701 (+  5848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.317735 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:18.345951 (+ 28216us) log.cc:844] Serialized 52 byte log entry
0709 12:25:18.724326 (+378375us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:18.824453 (+100127us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.846919 (+ 22466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.847043 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.192356 (+345313us) raft_consensus.cc:1685] finished
0709 12:25:19.192441 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.192924 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40760,"prepare.queue_time_us":286927,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":933,"raft.queue_time_us":129217,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":13426,"thread_start_us":407183,"threads_started":2,"wal-append.queue_time_us":380723}
I20260709 12:25:19.198045 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:19.215865 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4351) took 2359 ms (client timeout 1690 ms). Trace:
W20260709 12:25:19.216547 13792 rpcz_store.cc:269] 0709 12:25:16.856038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.856230 (+   192us) service_pool.cc:224] Handling call
0709 12:25:19.215323 (+2359093us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:19.215566 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4905369472}
I20260709 12:25:19.199867 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 3996) took 1410 ms. Trace:
I20260709 12:25:19.218521 13725 rpcz_store.cc:276] 0709 12:25:17.789453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.305266 (+515813us) service_pool.cc:224] Handling call
0709 12:25:19.199102 (+893836us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b5400155ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:19.199480 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113399040}
I20260709 12:25:19.177359 13649 rpcz_store.cc:276] 0709 12:25:17.900535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.902192 (+  1657us) service_pool.cc:224] Handling call
0709 12:25:17.902620 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.185126 (+282506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.175936 (+990810us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:19.176453 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621426432}
I20260709 12:25:19.223426 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.224570 13794 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:19.228062 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:19.202200 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4241) took 1254 ms. Trace:
I20260709 12:25:19.230574 13724 rpcz_store.cc:276] 0709 12:25:17.947120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.809514 (+862394us) service_pool.cc:224] Handling call
0709 12:25:19.199468 (+389954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.200189 (+   721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.200214 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:19.200294 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:19.200405 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.200523 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.201748 (+  1225us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400155ed0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:19.201940 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895534464}
I20260709 12:25:19.232319 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:19.240302 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.242240 13800 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:19.244254 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:19.246433 13795 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:19.255429 22537 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32.
I20260709 12:25:19.280238 23196 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:19.298671 23196 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.300139 22868 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:19.300751 22868 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.301137 22868 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:19.303098 22920 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:19.304876 22809 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:19.305968 22809 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.306972 23104 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:19.309057 23160 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:19.305902 22920 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:25:19.267057 23144 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:19.396445 23104 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:19.397135 23104 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:19.398500 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:19.448967 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 221305us
W20260709 12:25:19.461081 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 267564us
I20260709 12:25:19.467741 22526 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.007s
I20260709 12:25:19.469542 22526 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:19.462499 23075 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:19.470045 22526 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:19.475054 22537 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:19.496317 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 239793us
W20260709 12:25:19.502451 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.504415 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:19.504702 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.505780 14424 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:19.489982 22537 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.507829 23108 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.002s
I20260709 12:25:19.508262 22809 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:19.478883 23075 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.510478 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.510727 23196 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:19.509238 22537 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:19.512348 14417 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:19.520470 22776 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:19.507275 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:19.518805 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:19.522886 23108 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:19.546240 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.544842 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.529659 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.542826 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:19.541136 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.535259 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:19.538689 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:19.558449 22677 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:19.541466 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.556067 23124 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.555276 23075 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:19.581555 14425 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:19.582916 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:19.582796 22860 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:19.585098 23190 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.549237 22526 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.586036 23190 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.588133 23094 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.588958 23094 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.589690 23190 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:19.593375 23094 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.596434 13724 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:19.597705 23124 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.599623 14417 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:19.599998 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:19.604085 22526 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:19.619593 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141675us
I20260709 12:25:19.571261 23108 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:19.630713 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 148607us
I20260709 12:25:19.606304 22809 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:19.620950 23196 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:19.640934 13725 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
I20260709 12:25:19.640959 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:19.630975 23075 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:25:19.642138 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 518372us
I20260709 12:25:19.645962 22677 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:25:19.654031 13723 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:19.665966 23209 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:19.666883 23209 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.655150 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:19.655022 13718 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:19.669392 23209 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.669889 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:19.653671 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79068.9us
I20260709 12:25:19.633437 23094 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:19.646658 23075 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:19.652875 22405 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.161s	user 0.015s	sys 0.017s
I20260709 12:25:19.693003 13795 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:19.692530 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.685114 23075 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:19.686450 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:19.706413 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:19.707334 13727 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:19.710384 13726 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:25:19.713668 13652 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:19.695986 23196 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.716709 23196 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:19.716754 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:19.717834 13723 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
W20260709 12:25:19.718941 22670 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.010s	user 0.012s	sys 0.019s
I20260709 12:25:19.703804 22809 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:19.725595 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:19.727658 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:19.728809 23206 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:19.692240 23094 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.731985 23094 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.676734 23124 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.736413 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:19.738502 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.740324 14416 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:19.741737 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.743069 14425 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:19.745200 14420 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:25:19.746649 23161 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:19.753715 23129 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:25:19.759140 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.759747 22868 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:19.768101 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 124747us
I20260709 12:25:19.770072 22809 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:19.696406 23108 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.771807 23108 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:19.777302 23104 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.769701 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:19.783946 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.798148 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:19.801366 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:19.802996 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 138309us
W20260709 12:25:19.815946 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46381.1us
I20260709 12:25:19.820039 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:19.702654 22405 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.832612 23094 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.833627 23094 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.836201 23094 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.836862 23094 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:19.837252 23094 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:19.837760 23094 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:19.842360 23191 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:19.843101 23191 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:19.843514 23018 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:19.844812 23018 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:19.844069 13541 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 16 to 19, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:19.845212 23018 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:19.843757 23129 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:25:19.846422 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:19.847030 23129 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:19.847350 14423 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:19.847433 23129 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:19.847955 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.848639 13800 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:19.851059 22670 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:19.810518 23190 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:19.852417 22670 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:19.852819 23190 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.852950 22670 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:19.851567 23129 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.854141 23129 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:19.854734 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.855386 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:19.856474 13719 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:19.857613 22405 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:19.858702 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4001) took 1990 ms. Trace:
I20260709 12:25:19.859406 13719 rpcz_store.cc:276] 0709 12:25:17.868583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.451416 (+582833us) service_pool.cc:224] Handling call
0709 12:25:18.451916 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.856930 (+1405014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.858305 (+  1375us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000280d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:19.858526 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3229324032}
I20260709 12:25:19.861714 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:19.862640 13643 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 16.
W20260709 12:25:19.865114 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.866284 22846 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.866425 22776 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:19.866962 22846 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.868039 22776 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:19.870424 23137 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:19.861687 13541 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 11 to 18, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.872555 13801 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:25:19.874240 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.875337 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:19.876391 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:19.876617 22868 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:19.853370 23190 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:19.877401 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:19.876415 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.881312 22670 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.877446 14418 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:19.882673 22846 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:19.883001 22670 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:19.888748 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.889587 13795 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:25:19.844348 23191 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:19.873133 13541 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.915118 23104 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:19.917318 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92489.1us
I20260709 12:25:19.919834 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:19.930480 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:19.931574 13645 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:19.925853 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:19.935174 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:19.952389 13541 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 13 to 14, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 14 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:19.954077 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:19.945781 22405 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.960243 22405 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.852818 23018 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:19.966100 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:19.970613 22868 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:19.971021 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74485.6us
I20260709 12:25:19.995255 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:19.971590 22809 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:19.972087 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:19.986454 23211 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:19.997247 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61302.2us
I20260709 12:25:19.998687 22860 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:19.998850 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:19.999387 22860 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.999872 22860 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:20.001556 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:20.001595 22809 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:20.002485 13795 raft_consensus.cc:606] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:20.006500 22846 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:20.010644 13645 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:20.013208 22405 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.014286 22537 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:20.009891 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:20.024971 13727 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:20.020340 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92518.2us
I20260709 12:25:20.019528 13541 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 12 to 15, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 15 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.031032 13541 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.991132 22868 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:20.023613 23206 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:20.033983 13541 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 23 to 29, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 29 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:20.043711 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:20.044715 13643 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:20.047266 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:20.048188 13649 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:20.041656 22868 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:20.056867 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.050885 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:20.058866 23099 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:20.059366 23099 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:20.059763 23099 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:20.061093 23108 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:20.061537 23108 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.061924 23108 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:20.067416 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.068293 13541 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 20 to 22, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 22 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.046314 23018 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:20.072954 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 120738us
I20260709 12:25:20.075258 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:20.075603 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:20.076462 22537 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:20.078594 22809 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:20.079592 22809 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:20.080442 23094 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.081312 22809 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:20.081146 22860 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:20.082624 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:20.082489 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:20.085366 13795 raft_consensus.cc:606] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:20.085820 13795 raft_consensus.cc:614] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:20.089771 13726 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:20.077339 22537 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:20.091420 13643 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:20.092789 13727 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:20.091487 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:20.093253 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 215945us
I20260709 12:25:20.056743 23206 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:20.114816 23215 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:25:20.101590 23104 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 12:25:20.101055 22537 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.106117 13541 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 14 to 16, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 16 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:20.100865 23129 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:20.057958 23190 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.119879 22537 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:20.145072 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.149802 13541 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.110883 23124 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:20.120222 22677 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.164109 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:20.110643 22776 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 20, Time since last communication: 0.039s
W20260709 12:25:20.176749 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 341946us
I20260709 12:25:20.200608 22860 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:20.157888 23094 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.166388 22920 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:20.201408 23019 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:20.174343 23137 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.220568 22920 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:20.223053 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173774us
I20260709 12:25:20.223443 22869 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.224032 22869 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.225605 23190 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.226522 23190 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:20.227948 23190 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:20.228541 23190 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.230443 23206 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.232616 22869 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.235339 23209 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:20.235919 23209 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:20.236321 23209 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:20.276194 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.220709 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.556s (SENT)
I20260709 12:25:20.249821 13541 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 19 to 21, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.168010 22677 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.216403 23019 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:20.319463 13726 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:20.306716 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.322103 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:20.257663 22920 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.343768 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54361b2742ab4613b55749cbb699113b"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:20.344825 13795 raft_consensus.cc:606] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:20.353996 14417 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:20.334385 23094 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:20.289297 23137 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:20.388892 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71506.5us
I20260709 12:25:20.355913 13541 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.321801 23211 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:20.401448 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 243697us
I20260709 12:25:20.391947 23190 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:20.427546 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.432646 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 268602us
I20260709 12:25:20.431182 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:20.444015 13795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:20.442078 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.357112 23019 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.471061 13795 raft_consensus.cc:606] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:20.475118 13795 raft_consensus.cc:614] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:20.466377 23075 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.695s	user 0.003s	sys 0.012s
I20260709 12:25:20.480399 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:20.481256 13800 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:20.483526 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:20.484242 13794 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:20.405212 23211 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:20.491478 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.637s (SENT)
I20260709 12:25:20.475664 23007 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:20.486680 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 264015us
I20260709 12:25:20.488651 13541 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 13 to 19, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:20.501497 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 423381us
I20260709 12:25:20.480145 23190 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:25:20.504618 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101323us
I20260709 12:25:20.630798 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:20.557111 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.487615 22846 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:20.635421 23232 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:20.641240 22868 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.540s	user 0.006s	sys 0.005s
W20260709 12:25:20.592991 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 517804us
W20260709 12:25:20.643198 23226 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:25:20.596900 13541 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 13 to 20, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 20 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:20.646528 23227 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:25:20.648233 23137 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:20.648510 23019 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:20.647866 13541 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 21 to 23, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 23 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:25:20.603928 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 258) took 19991 ms (client timeout 15000 ms). Trace:
I20260709 12:25:20.598054 22677 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.653609 23129 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.656297 23161 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:20.656741 23161 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:20.657914 23099 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.598s	user 0.007s	sys 0.005s
I20260709 12:25:20.486954 23075 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.649930 13541 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.476574 23196 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:20.635015 22846 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:20.582791 13617 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.791s (SENT)
I20260709 12:25:20.633996 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:20.665234 13800 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
W20260709 12:25:20.673738 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 178423us
I20260709 12:25:20.654363 23129 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.578459 23007 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.641855 22868 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.657629 23161 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:20.658223 23108 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.005s
I20260709 12:25:20.695526 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:20.696280 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:20.698572 13541 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 19 to 21, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:20.702010 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:20.675320 23099 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.687060 23094 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:20.722779 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 236233us
W20260709 12:25:20.650599 13547 rpcz_store.cc:269] 0709 12:25:00.494799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.292931 (+8798132us) service_pool.cc:224] Handling call
0709 12:25:09.802840 (+509909us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393bfcf3a 00007f53999b2e9c 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:20.480279 (+10677439us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:16.741625 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:16.742317 (+   692us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:16.742367 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:25:16.742388 (+    21us) tablet.cc:662] Decoding operations
0709 12:25:16.744565 (+  2177us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:16.744672 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:16.744712 (+    40us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:25:16.747119 (+  2407us) tablet.cc:701] Row locks acquired
0709 12:25:16.747141 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:25:16.747328 (+   187us) write_op.cc:270] Start()
0709 12:25:16.747463 (+   135us) write_op.cc:276] Timestamp: P: 1783599916747260 usec, L: 0
0709 12:25:16.747487 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:25:16.878571 (+131084us) log.cc:844] Serialized 9350 byte log entry
0709 12:25:17.530289 (+651718us) op_driver.cc:464] REPLICATION: finished
0709 12:25:17.755144 (+224855us) write_op.cc:301] APPLY: starting
0709 12:25:17.759821 (+  4677us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:18.120774 (+360953us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:18.120891 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:19.044605 (+923714us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:19.045174 (+   569us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:25:19.045245 (+    71us) write_op.cc:312] APPLY: finished
0709 12:25:19.046004 (+   759us) log.cc:844] Serialized 212 byte log entry
0709 12:25:19.323158 (+277154us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:19.324062 (+   904us) write_op.cc:454] Released schema lock
0709 12:25:19.324210 (+   148us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:19.325084 (+   874us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:19.325128 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1614,"reported_tablets":50,"spinlock_wait_cycles":505316736,"child_traces":[["op",{"apply.queue_time_us":209648,"apply.run_cpu_time_us":51431,"apply.run_wall_time_us":1577911,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104639,"num_ops":24,"prepare.queue_time_us":151580,"prepare.run_cpu_time_us":13609,"prepare.run_wall_time_us":197880,"raft.queue_time_us":253011,"raft.run_cpu_time_us":49631,"raft.run_wall_time_us":135494,"replication_time_us":782565,"thread_start_us":597840,"threads_started":5,"wal-append.queue_time_us":169176,"wal-append.run_cpu_time_us":17559,"wal-append.run_wall_time_us":2411053}]]}
I20260709 12:25:20.729660 13541 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 13 to 19, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:20.734237 23099 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:20.717248 23108 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.736853 23108 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.737586 23235 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:20.738152 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:20.738790 13800 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:20.739702 23007 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.742631 23075 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.743486 14422 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:20.745476 22846 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:20.746948 23229 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:25:20.749161 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:20.734330 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:20.750597 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:20.752033 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.675009 23104 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
W20260709 12:25:20.756557 13643 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.000s
I20260709 12:25:20.748342 23094 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.757366 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:20.762543 13795 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:20.759763 13643 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:20.730396 23209 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.766760 13800 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:20.714808 23137 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:20.737155 23018 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.003s
I20260709 12:25:20.730029 23232 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:25:20.786805 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 152959us
I20260709 12:25:20.778990 13541 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:25:20.747001 23221 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:20.748327 22868 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.729676 23190 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:20.748291 23129 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:20.808171 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:20.791463 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.819077 23222 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
W20260709 12:25:20.819378 23223 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:25:20.716197 23196 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:20.784355 22776 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:20.819541 23233 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:20.821537 13792 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 27. Ops: []
I20260709 12:25:20.792065 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:20.827379 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.767956 22670 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:20.833369 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:20.837551 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
I20260709 12:25:20.790838 23224 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:25:20.838804 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 334064us
I20260709 12:25:20.839790 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:20.827101 23161 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:20.851019 23196 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:20.851584 23190 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:20.852078 23018 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.852275 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.838729 22776 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.881480 22776 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.855278 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.854918 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.854864 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:20.858167 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.858559 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:20.846800 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57568.3us
I20260709 12:25:20.831257 23233 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.791234 23137 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.861379 22670 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.903098 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:20.903575 23137 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.903769 22670 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.887442 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:20.905134 23094 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:20.906664 23220 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:25:20.842555 23129 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.910806 23129 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.888250 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:20.913594 14425 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:20.915295 14421 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:20.791458 23235 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:20.916505 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:20.891141 23209 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:20.917348 23094 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:20.894563 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.690s (SENT)
I20260709 12:25:20.918720 23235 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:20.894153 23018 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.899374 23190 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:20.928769 23190 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:20.899461 23233 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:20.861995 23224 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:20.932626 23224 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:20.933384 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:20.934464 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49323.7us
I20260709 12:25:20.825344 23104 consensus_queue.cc:1243] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:20.940721 13795 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
W20260709 12:25:20.952095 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 130905us
I20260709 12:25:20.958910 14419 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:20.965495 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:20.962164 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94639.4us
W20260709 12:25:20.968425 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 465885us
I20260709 12:25:20.969450 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.980821 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:20.979897 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.983114 14419 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:25:20.983985 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67568us
I20260709 12:25:20.981959 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:20.966593 23232 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:20.976369 22537 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.817s	user 0.012s	sys 0.003s
I20260709 12:25:21.008337 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:21.011261 13794 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.004s
I20260709 12:25:21.011695 13794 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:21.013108 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:21.016742 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.979s (SENT)
I20260709 12:25:21.017270 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:20.973994 22846 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:25:21.018217 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:21.024376 22670 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:21.025012 22670 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:21.025379 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:21.025995 22670 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.036218 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.037097 14424 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:21.040417 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.046296 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:21.042946 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44821.8us
I20260709 12:25:21.050791 23211 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:21.051004 23075 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.051548 23211 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:21.051849 23075 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.052872 23211 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.022912 23232 raft_consensus.cc:2764] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:25:21.004264 22537 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.057504 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.056428 13646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:21.096702 13646 raft_consensus.cc:606] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:21.097451 13646 raft_consensus.cc:614] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:21.057214 14425 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:21.059094 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.099159 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.099455 14418 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:21.050845 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:21.080951 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68392.8us
I20260709 12:25:21.100461 14420 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:21.057595 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.098450 14425 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:21.108487 14416 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:21.109862 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.110950 14421 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:21.039285 23094 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:21.113735 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.114485 13725 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:21.116461 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.117201 13727 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:21.118395 13792 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:21.118664 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:21.040833 23007 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:21.135649 13797 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:25:21.137082 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:21.138370 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87637.3us
I20260709 12:25:21.055755 22677 raft_consensus.cc:993] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:25:21.140069 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.147101 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.148078 22526 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:21.148501 22526 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.148823 22526 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:21.139024 22860 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:21.146739 14425 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.153127 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:21.166320 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 345043us
W20260709 12:25:21.166838 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.988s (SENT)
I20260709 12:25:21.068681 23018 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.170408 23234 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:25:21.170579 23244 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
W20260709 12:25:21.170931 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 230132us
I20260709 12:25:21.193990 22537 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.179591 23137 raft_consensus.cc:1081] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:25:21.196707 13690 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.197216 23239 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:21.173182 14425 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:21.197885 23239 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:21.158020 23108 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:21.198756 23108 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:21.199004 23239 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.200484 23239 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:21.176824 23007 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:21.201619 23007 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.173838 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.206138 13792 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:21.206221 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:21.207129 13794 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:21.208235 23235 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:21.208756 23235 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.181341 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:21.209084 23235 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:21.190474 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:21.177806 23237 raft_consensus.cc:993] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
W20260709 12:25:21.162055 23099 consensus_queue.cc:1175] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 26 status { last_received { term: 26 index: 17 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []" } } last_received_current_leader { term: 26 index: 17 } }
I20260709 12:25:21.161345 23124 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:21.199855 23108 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.23, Last appended by leader: 23, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.191535 23233 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:21.210708 13617 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.029s (SENT)
I20260709 12:25:21.214130 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:21.219170 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:21.215315 14418 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:25:21.215440 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50286.1us
I20260709 12:25:21.217101 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:21.223313 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:21.225436 22852 raft_consensus.cc:1081] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:25:21.226300 14424 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:21.218417 23124 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.259095 14425 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:25:21.305104 14425 pending_rounds.cc:85] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 16
W20260709 12:25:21.304993 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79360.3us
I20260709 12:25:21.236804 22860 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:21.236747 23209 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:21.222111 22537 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:25:21.302912 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.234143 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:21.305549 14425 pending_rounds.cc:107] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:25:21.313176 14425 raft_consensus.cc:2889] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:21.314435 23019 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:25:21.314391 13643 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:21.314801 13794 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:21.311136 23124 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:21.323647 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:21.312724 23190 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.331820 23190 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.333483 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:21.311774 23211 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.335245 23211 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.320600 23129 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:21.325258 23075 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.347816 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.296835 22809 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 12:25:21.330051 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 96031.4us
I20260709 12:25:21.328599 23256 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:21.350575 13723 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:21.350056 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.318691 23224 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:25:21.355971 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.358176 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.355257 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:21.361920 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.373857 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:21.321854 22526 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.379272 23232 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.379981 23232 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.381464 22526 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:21.362083 23161 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.500s	user 0.014s	sys 0.000s
W20260709 12:25:21.383663 23258 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.354s	user 0.004s	sys 0.000s
I20260709 12:25:21.386914 22526 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.387686 22526 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.390436 23209 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.391124 23209 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.392594 22526 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.395843 22537 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.396607 22537 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.399734 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.400564 13795 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:21.361876 13718 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54361b2742ab4613b55749cbb699113b"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:25:21.402930 22537 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.403506 13718 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:21.403909 13718 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:21.407307 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 196564us
I20260709 12:25:21.411013 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.351214 23007 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:21.358342 23233 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:21.444063 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.357041 23256 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:21.370517 23211 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.451870 13726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a8b916ed93d478c990f668818f43952"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:25:21.452633 13726 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:21.444418 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.453030 13726 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:21.370303 23190 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.374313 23075 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:21.441993 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.458132 14424 raft_consensus.cc:2376] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30. Re-sending same reply.
W20260709 12:25:21.460850 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92963.5us
I20260709 12:25:21.432377 22860 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:21.439363 23007 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.451886 23232 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.451948 23256 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:21.545579 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83973.8us
I20260709 12:25:21.451623 22920 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:21.538118 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.539623 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:21.550439 13725 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 27
W20260709 12:25:21.548791 13617 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.067s (SENT)
W20260709 12:25:21.569305 23257 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.398s	user 0.001s	sys 0.001s
I20260709 12:25:21.538805 23007 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:21.579675 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 119147us
I20260709 12:25:21.587723 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:21.553193 13726 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.548133 23161 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.610188 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:21.613533 14424 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:25:21.589391 22809 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:21.549621 23249 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.619243 23249 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.611646 23108 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.549610 23075 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:21.637636 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.549994 23209 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:21.644310 13652 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:21.648394 13725 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:21.578400 23250 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:21.651701 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.653496 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.654696 13726 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.651223 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.657068 23255 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.657985 23255 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.659708 14418 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:25:21.660339 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.650370 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:21.613118 23191 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.658017 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.661088 13649 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:21.661813 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.675175 13646 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:21.675328 13651 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:21.626101 23252 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.623365 23249 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.631197 23108 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:21.692021 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 257053us
I20260709 12:25:21.686326 23191 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.673982 23099 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:21.621532 22860 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.755393 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:21.796670 14418 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:21.742805 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:21.797504 13797 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:21.743772 13793 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
W20260709 12:25:21.767596 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 117412us
I20260709 12:25:21.743203 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.737334 13718 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:21.796686 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 345088us
I20260709 12:25:21.702394 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.772222 14417 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:21.647778 23161 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.661935 23250 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:21.676221 23224 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:21.790380 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:21.804911 13719 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:21.715143 23206 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.824389 13718 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.773581 23209 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:25:21.715646 22670 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.833271 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.838423 13651 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:21.848542 13800 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:21.850095 13645 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:21.790241 22526 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.853179 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.854789 13649 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:21.856491 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66439.6us
I20260709 12:25:21.796748 23211 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.702137 23007 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.805099 22809 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:21.842166 23209 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.876070 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.883858 13645 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:25:21.883915 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.847467 23255 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:21.891134 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.892079 13645 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:21.743142 23075 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.860472 22537 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.898547 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.901233 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.902076 13645 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:21.771760 23252 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.903792 14418 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:21.908620 13797 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:21.871472 23211 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.939561 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.852829 22670 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.909644 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.910141 13798 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:21.901114 23255 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.945405 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.844478 23206 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.886062 23209 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:25:21.916760 23235 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.707s	user 0.015s	sys 0.000s
W20260709 12:25:21.918442 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.940332 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
W20260709 12:25:21.917711 23124 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.002s
I20260709 12:25:21.928550 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:21.944435 23274 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:21.947011 13650 raft_consensus.cc:2368] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:25:21.864507 22526 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.945188 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:21.943809 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.740s (SENT)
I20260709 12:25:21.910741 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.953523 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:21.962795 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.964659 23252 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:21.973168 13801 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 34 index: 25. (index mismatch)
I20260709 12:25:21.969420 13644 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:21.973536 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.969800 13647 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 32: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:21.975375 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.919471 22537 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:21.977339 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.986617 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:21.976071 13650 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:21.969178 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:21.987421 13643 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:21.985061 13645 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:21.895917 23250 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.995487 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.996920 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:21.967687 23266 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:22.003412 13647 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:22.008049 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.966691 23286 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:22.010154 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 131761us
I20260709 12:25:21.950748 23255 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.013733 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.965456 23235 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:21.986517 23252 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:21.976461 23099 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:22.027841 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.029062 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:21.966755 23124 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.033088 13645 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:25:22.034894 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.031884 13719 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:22.007882 23094 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:22.040027 13726 raft_consensus.cc:2410] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:25:21.959194 22809 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.048967 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:22.025287 23286 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.050195 23007 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:22.050377 14418 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:25:22.050525 23190 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:25:22.051308 23108 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.031800 23161 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:22.074661 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:22.052861 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:22.054865 23280 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:25:22.055895 23285 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:25:22.056365 23287 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:22.056952 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:22.063941 22670 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:22.064301 22537 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.077196 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.085487 23286 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.122562 13798 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:25:22.051352 23206 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.123006 22526 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.077653 23224 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:25:22.125415 23124 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:22.094779 22852 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:22.126720 23287 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.128006 23232 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:25:22.128679 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.131579 23099 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:22.132736 23256 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.002s
I20260709 12:25:22.133195 23256 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.030215 23252 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.136170 23252 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.136772 23252 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:22.137277 23252 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:22.139602 23289 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:25:22.070931 23191 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.096535 23108 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.141165 23108 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.143376 23108 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.147729 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.148232 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:22.149385 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:22.149463 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.150667 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.151561 13727 raft_consensus.cc:2368] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24.
I20260709 12:25:22.152637 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:22.154836 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:22.078907 13651 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:22.027760 23266 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:22.081456 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.082896 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.175834 13719 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
W20260709 12:25:22.089704 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 157398us
I20260709 12:25:22.092654 13645 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:22.105196 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.178610 13797 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:22.102854 13647 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:25:22.158281 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:22.201835 13646 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.004s
W20260709 12:25:22.202116 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 180653us
I20260709 12:25:22.202556 13646 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 17.
W20260709 12:25:22.188441 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.006s	sys 0.022s
W20260709 12:25:22.204766 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.006s	sys 0.022s
W20260709 12:25:22.205020 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 568109us
I20260709 12:25:22.203135 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.233700 13645 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: [21.18-21.18]
I20260709 12:25:22.175889 23007 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.196327 23287 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.173557 13644 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:25:22.247081 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 169910us
I20260709 12:25:22.248054 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:22.251006 13645 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:25:22.252156 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:22.254971 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:22.265374 13649 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:25:22.268702 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.273269 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.275166 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:22.242439 23256 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:22.256806 23239 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.039s	user 0.015s	sys 0.000s
I20260709 12:25:22.234308 23255 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:22.295859 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92826.1us
I20260709 12:25:22.278591 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.230142 23094 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.037s
I20260709 12:25:22.268102 23209 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.282591 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.283385 23300 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.255697 23237 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:22.284827 13794 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:22.295972 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:22.256964 22846 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:22.297161 13723 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:22.280718 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.238320 23108 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.039s
I20260709 12:25:22.267391 23161 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:22.303172 23129 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:22.297089 23287 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:22.304229 23300 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.304979 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.305652 13801 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:22.233392 23296 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:25:22.313568 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.250802 23007 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.317834 14420 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:25:22.319065 13617 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.515s (SENT)
I20260709 12:25:22.284158 23252 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.366441 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:22.313905 23237 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.361027 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.301493 23124 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:22.361681 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.376777 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.414101 13792 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:25:22.370347 13648 raft_consensus.cc:2368] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36.
I20260709 12:25:22.377923 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.391263 13799 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:25:22.377547 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:22.405622 13800 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:22.325093 23161 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:22.414458 13719 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:22.417531 13795 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:22.422626 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:22.426200 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 113757us
W20260709 12:25:22.427616 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.599s (SENT)
I20260709 12:25:22.432230 13793 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:22.416553 23161 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:22.417052 23237 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:22.464043 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:22.467856 23099 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:22.469202 13795 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:22.469269 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:22.470887 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.746s (SENT)
I20260709 12:25:22.472339 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.474936 13719 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:22.481647 13719 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:22.476471 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 154956us
I20260709 12:25:22.483812 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.410200 23287 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.448342 23235 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:22.322737 23266 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.491460 13795 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:25:22.492623 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.498934 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:22.512157 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.512545 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.513924 13723 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:22.506345 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:22.495750 23299 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:22.509064 13719 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:22.523265 23314 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:22.525118 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.526401 23287 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:22.527952 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.530860 23007 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:22.535357 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.537233 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.542788 13719 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:22.527880 23300 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:22.551640 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:22.552834 23224 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:22.554795 23224 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:22.555295 23224 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.555701 23224 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:22.558313 23239 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.558935 23239 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:22.560164 23302 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:25:22.560664 23019 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:22.561231 23019 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:22.520325 22852 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:22.563267 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.564306 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:22.565553 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52909.9us
W20260709 12:25:22.521600 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51096.8us
I20260709 12:25:22.499903 23108 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:22.585191 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.774s (SENT)
I20260709 12:25:22.585794 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.586692 23239 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:22.587661 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:22.588603 13795 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 36
I20260709 12:25:22.567591 23007 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:22.607265 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:22.608139 13644 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.568233 23299 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:22.618023 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.615270 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:22.591342 23129 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:22.628512 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:22.630558 14424 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:22.633986 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:22.634768 13645 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:22.641279 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:22.642066 14425 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:22.569924 23019 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.642788 14420 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:25:22.604377 23295 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:25:22.644553 23295 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:22.615881 23261 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:25:22.644923 23295 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:22.645366 23211 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.647936 13767 leader_election.cc:400] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36.
I20260709 12:25:22.647761 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.647256 23299 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:22.647781 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62661.7us
W20260709 12:25:22.649245 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.807s (SENT)
I20260709 12:25:22.649636 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:22.650954 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.826s (SENT)
I20260709 12:25:22.651387 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.652189 23108 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:22.652854 23108 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:22.645885 23295 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.603976 23296 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:22.660269 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.655085 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.661937 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:22.648264 13767 leader_election.cc:403] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:22.654810 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:22.658726 13652 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:22.657626 13645 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:25:22.661158 13718 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:22.675076 13644 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:22.655681 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.869s (SENT)
I20260709 12:25:22.734504 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.735442 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:22.738885 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111371us
W20260709 12:25:22.742872 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66821.4us
I20260709 12:25:22.747788 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.657294 23295 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:22.759176 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.766472 13724 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:22.767849 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:22.766315 23255 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.769948 23298 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.020s
I20260709 12:25:22.770169 23255 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:22.771555 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.773082 23255 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:22.774499 23211 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:22.779120 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:22.780027 13796 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:22.780423 13796 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:22.773932 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.736979 23300 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:22.768900 13652 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:22.781514 13796 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.783635 13795 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:25:22.654223 23161 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:22.788537 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.756633 23209 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:22.795616 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.791574 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:22.800024 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:22.801394 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:22.784981 23300 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.808249 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.788498 23282 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36.
I20260709 12:25:22.805263 23211 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:25:22.811599 23300 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.815755 23211 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:22.811468 23293 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:22.822718 23293 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:22.823139 23293 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:22.824990 23313 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
W20260709 12:25:22.826558 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 170903us
W20260709 12:25:22.827358 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.871s (SENT)
I20260709 12:25:22.827790 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:22.832293 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.050s (SENT)
I20260709 12:25:22.821521 23211 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.805132 23255 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:25:22.815665 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:22.835644 13693 leader_election.cc:400] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:25:22.843765 13693 leader_election.cc:403] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:25:22.835937 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:22.838912 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:22.842816 13719 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:25:22.844730 13719 pending_rounds.cc:85] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 11
I20260709 12:25:22.846658 13719 pending_rounds.cc:107] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:25:22.847599 13719 raft_consensus.cc:2889] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.848652 13652 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:22.849037 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.844738 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.850456 22846 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:22.852834 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:22.853312 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47168.9us
I20260709 12:25:22.853523 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:22.854457 23108 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:22.854646 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:22.855304 14424 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:22.855420 22846 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:22.836023 23298 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:22.861835 23249 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:22.858623 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:22.891511 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.906461 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:22.909723 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 77477.5us
I20260709 12:25:22.869412 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.871649 23286 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:22.858448 23255 consensus_queue.cc:1243] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:22.875702 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:22.876817 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:22.879172 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.880017 23250 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:22.839788 23209 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:22.856671 23235 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
W20260709 12:25:22.913884 13617 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.816s (SENT)
I20260709 12:25:22.872902 23252 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:22.897203 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:22.908816 23253 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:22.924813 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:22.926141 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49405us
I20260709 12:25:22.934623 23253 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:22.863322 23298 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:22.936067 23298 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:22.937315 23253 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:22.948099 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50991.9us
I20260709 12:25:22.951367 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.952872 13724 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:22.914503 23007 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:25:22.971576 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57728.9us
I20260709 12:25:22.932214 23255 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:22.903314 23249 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.977770 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:22.965291 22860 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:22.991163 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:22.987000 23324 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:22.987406 23292 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.979091 22670 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:22.924163 23286 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.981675 23007 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:23.003991 23237 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 12:25:23.004405 23255 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:23.007556 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115390us
I20260709 12:25:22.974144 13724 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:22.983203 23209 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:23.013104 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:22.964298 23252 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:22.993002 22860 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.025789 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:23.019546 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.027980 14418 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:23.031174 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:23.032459 13726 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:23.031316 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:23.032893 13726 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:23.039604 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68241.4us
I20260709 12:25:23.040850 13724 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:23.040949 13652 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:23.011618 22670 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.050251 13799 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:23.025216 23297 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:23.049050 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:22.956873 22846 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.043756 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:23.014869 23007 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:23.045619 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:23.054517 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:23.059319 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.064002 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:23.054903 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:23.061664 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.054111 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:23.054900 13721 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:23.057914 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:23.057031 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.060173 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.057668 13726 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.032063 22860 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:23.057152 22809 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:25:23.048897 22670 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.050671 23161 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.078085 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.078428 23331 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:23.078681 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.079607 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.080124 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.070384 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.082312 13644 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:23.082444 23286 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:23.086423 23224 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.005s
I20260709 12:25:23.054189 22677 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:25:23.069276 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:23.069166 13797 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:23.070978 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.071333 23255 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.053589 23282 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:25:23.064949 22852 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:23.087108 13726 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:23.070361 23249 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:23.091487 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45952.4us
I20260709 12:25:23.094887 13643 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:23.097846 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.098762 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.079640 23252 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.093415 23332 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:23.102662 23300 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.087366 23224 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.110981 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.058923 23297 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.015810 23292 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:23.113149 23328 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:25:23.108498 13647 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 17.
I20260709 12:25:23.141827 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:23.148944 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88415.8us
W20260709 12:25:23.154434 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 43754.1us
I20260709 12:25:23.155320 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:23.026685 23237 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.158442 13646 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:23.162086 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.062664 23299 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.088s
I20260709 12:25:23.169477 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.169636 13644 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:23.171511 14418 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:25:23.172972 13719 raft_consensus.cc:2368] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25.
I20260709 12:25:23.178289 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:23.151629 23297 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:23.186477 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.188370 13643 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:23.192047 13647 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:23.193714 13726 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:25:23.194092 13726 pending_rounds.cc:85] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:25:23.194607 13798 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:23.195173 13726 pending_rounds.cc:107] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:25:23.195597 13726 raft_consensus.cc:2889] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:23.151971 22852 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:23.210274 13648 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:23.170246 23252 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:23.179755 23296 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:23.225651 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56294us
I20260709 12:25:23.226730 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.195387 23094 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:23.228068 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.174121 23249 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:25:23.255077 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:23.264606 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.279421 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 101251us
I20260709 12:25:23.232491 23286 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:25:23.256393 22809 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:23.269227 23094 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:23.282135 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.296998 23336 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:23.287050 23337 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:23.303251 23337 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:23.304891 22670 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:23.309278 23339 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:23.310395 23094 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:23.311691 23237 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:23.324991 23292 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:23.298679 23335 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.176s
W20260709 12:25:23.335696 23287 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:25:23.306896 22852 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.338040 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:23.339752 13727 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:23.341881 13720 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:25:23.343195 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.343933 23282 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.344449 23282 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.346764 23282 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.304733 23337 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.351492 23337 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.308125 23293 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.287916 22860 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.355854 13796 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:23.354384 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.368849 22860 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:23.354197 23300 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.372781 14424 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:23.374449 23209 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:23.375260 13652 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:23.375384 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:23.378491 13723 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:23.378046 23293 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.383133 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:23.383294 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3010) took 1108 ms. Trace:
I20260709 12:25:23.383833 13798 rpcz_store.cc:276] 0709 12:25:22.274648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.274805 (+   157us) service_pool.cc:224] Handling call
0709 12:25:22.298325 (+ 23520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.206467 (+908142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.383020 (+176553us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:23.383209 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060647296}
I20260709 12:25:23.382843 23235 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:23.375406 22670 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.355732 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:23.386193 23235 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:23.387202 23235 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.387710 13792 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:23.389030 14418 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.403618 13647 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:23.405423 13724 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:23.408399 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:23.409091 14425 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.430056 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:23.388520 23235 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:23.355690 23224 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:23.441821 23326 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:25:23.446523 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56923.9us
I20260709 12:25:23.446903 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:23.447806 13793 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:23.281988 23334 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:23.353514 23298 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.468976 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:23.475852 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 158041us
I20260709 12:25:23.477600 14418 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:23.487623 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:23.478710 23342 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:23.464560 23334 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:23.532418 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 52920.6us
W20260709 12:25:23.532567 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53309us
W20260709 12:25:23.551331 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65696.4us
I20260709 12:25:23.551630 13721 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:23.554579 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 299567us
I20260709 12:25:23.562059 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.535099 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:23.567667 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 368652us
I20260709 12:25:23.558153 14425 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:23.481434 23293 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:23.576253 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.452205 23336 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.100s
I20260709 12:25:23.527297 23286 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:23.615772 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.552271 23161 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.595995 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.482796 22920 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.102s
I20260709 12:25:23.480490 23209 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 12:25:23.578459 23293 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.595476 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.643055 13643 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:23.579957 23292 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:23.607285 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:23.620692 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53753.4us
I20260709 12:25:23.565232 23342 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.490465 23331 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.117s
I20260709 12:25:23.559868 23224 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:23.562819 23334 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:23.647450 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:23.568728 23249 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:25:23.620358 23255 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:23.678531 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 71404us
I20260709 12:25:23.678722 22677 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:23.679373 22677 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:23.680346 22677 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:23.684188 23007 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.000s
W20260709 12:25:23.684409 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122437us
I20260709 12:25:23.684700 23007 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.685295 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:23.653290 23342 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:23.687460 23250 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.004s
I20260709 12:25:23.688011 22809 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.688505 22809 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.689647 23252 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:23.650563 23293 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:23.641819 23161 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:23.691004 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43654.7us
W20260709 12:25:23.737236 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 161066us
I20260709 12:25:23.739967 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.737725 13796 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:23.697271 13726 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:23.744408 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3020) took 1078 ms. Trace:
I20260709 12:25:23.744966 13796 rpcz_store.cc:276] 0709 12:25:22.666220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.778399 (+112179us) service_pool.cc:224] Handling call
0709 12:25:23.355405 (+577006us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b58001e32d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b500aa 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf827c 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c
0709 12:25:23.744334 (+388929us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24218496}
W20260709 12:25:23.746520 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.746680 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
W20260709 12:25:23.749051 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52232.8us
W20260709 12:25:23.718312 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48584.6us
I20260709 12:25:23.667537 23233 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:25:23.760126 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.753279 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:23.753432 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 147243us
I20260709 12:25:23.757309 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:23.758157 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72950.4us
I20260709 12:25:23.764034 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.761742 13649 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:23.766808 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.767596 13649 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:23.717180 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:23.692849 23261 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
W20260709 12:25:23.748050 23295 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.966s	user 0.015s	sys 0.001s
I20260709 12:25:23.782131 23019 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:23.782437 23353 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:23.783138 23353 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.785034 23237 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:23.785548 23353 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.763146 23337 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:23.716464 23235 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.772861 23075 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:23.790177 23129 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:23.790953 23129 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:23.792042 23129 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.772387 23298 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.786687 23161 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.795445 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:23.796370 14425 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:23.774513 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:23.786679 23235 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.799158 23347 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:23.803200 23347 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.803653 23347 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:23.803900 23352 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:25:23.804345 23235 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.799345 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:23.730893 23239 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:23.779260 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:23.797452 13794 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:23.812722 13649 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:23.813628 14417 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:25:23.821270 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:23.695796 23224 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.754436 23255 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.827353 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:23.826176 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51769.7us
W20260709 12:25:23.824663 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226557us
I20260709 12:25:23.793860 23282 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:25:23.840055 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:23.790439 23346 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:23.845216 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:23.847417 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:23.848645 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:23.852002 13719 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:23.852890 13723 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:23.854084 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.759047 23296 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.838022 23298 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:23.837502 23239 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:23.865195 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.866187 13649 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32.
I20260709 12:25:23.840503 23209 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:23.840932 23286 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:23.802143 23293 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.868603 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.852741 23108 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:23.876679 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.872057 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:23.872853 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.873726 13643 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: [28.19-28.19]
W20260709 12:25:23.864264 23297 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.000s
I20260709 12:25:23.886196 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.886947 13643 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:23.880748 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.864616 23249 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:23.852447 23075 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.023s
I20260709 12:25:23.893414 14391 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32.
I20260709 12:25:23.894520 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:23.869657 23211 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:23.876046 23007 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.869813 23345 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:23.844189 23255 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.903455 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.861099 23296 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.869444 23332 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:25:23.932932 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:23.934849 13793 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:23.894567 23249 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:23.929689 23333 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:23.957218 23255 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.958036 23235 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:23.918319 23209 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:23.959586 23209 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.960790 23299 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:23.928633 23224 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:23.962644 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:23.962577 23342 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.963424 13645 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:23.964936 23342 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.970564 23369 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:23.971278 23369 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.971706 13798 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:25:23.972836 23293 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.886715 23337 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:23.973521 23369 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.890909 23297 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.957666 23239 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.971727 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.980888 13643 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 18.
I20260709 12:25:23.982302 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:23.983194 13800 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:23.984104 14418 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:23.986398 23129 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:23.973569 23345 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.954514 23286 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:23.989689 23345 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.984758 23297 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.953791 23334 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:23.991159 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:23.992897 23334 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:23.993472 23250 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:23.985355 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:24.001269 13646 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:23.997062 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:23.962021 23209 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.008984 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:24.009533 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.910384 23358 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:23.993631 23332 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:24.011922 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:23.973770 23337 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:23.978359 23249 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:24.019448 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 228475us
I20260709 12:25:23.993901 23334 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.014s
I20260709 12:25:23.980521 23235 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:23.919198 23354 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:25:24.005767 23224 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:24.005767 23211 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:24.032120 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:24.000694 23250 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:24.033689 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:24.037887 13649 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:24.043555 13649 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:24.043979 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:24.041730 13643 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:25:24.038280 23358 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.050895 23137 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:24.051179 23358 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.048303 23075 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:25:24.028852 23332 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:24.053901 23332 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.047983 23369 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.055454 23369 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.056813 22809 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:24.057662 23369 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:24.027923 23255 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:24.046125 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:24.058844 13652 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:25:24.061293 13643 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:23.927685 23361 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.166s
I20260709 12:25:24.061728 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:24.046504 23209 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:24.076161 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:24.078063 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:24.079425 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46434.9us
I20260709 12:25:24.080260 13645 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:24.081262 13646 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:24.085335 23250 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.082955 22809 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:24.062022 23369 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:24.061022 23358 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:24.090947 23298 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:24.062693 23255 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:24.122181 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44233.3us
I20260709 12:25:24.123553 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:24.111557 23363 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.126159 23363 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.127511 23347 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.128944 23363 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:24.131767 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:24.105584 23256 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:25:24.094367 23233 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:25:24.105763 23235 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.152916 23255 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:24.096318 23337 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.203894 13766 leader_election.cc:400] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25.
W20260709 12:25:24.210680 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100082us
W20260709 12:25:24.216073 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.216965 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.218101 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:24.219065 13724 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:24.181887 23358 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:24.233966 13649 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33. Ops: [32.17-32.17]
W20260709 12:25:24.230691 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 107294us
I20260709 12:25:24.231688 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:24.238631 13644 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:24.239063 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:24.231480 23369 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:24.243789 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:24.244135 13796 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:24.246206 13724 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:24.247215 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:24.249308 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:24.251508 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:24.252796 13652 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 33 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:25:24.252815 13725 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:25:24.253540 13725 pending_rounds.cc:85] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:25:24.253835 13725 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:25:24.254123 13725 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.255398 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:24.256508 13645 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
W20260709 12:25:24.241765 23300 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.016s	sys 0.018s
I20260709 12:25:24.263447 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:24.265648 23300 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.017s	sys 0.019s
I20260709 12:25:24.268865 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:24.269626 13719 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:24.269667 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:24.268893 13724 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:24.282058 13722 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:25:24.282202 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:24.287308 13722 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:25:24.287761 13722 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:25:24.288089 13722 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:24.285558 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:24.267709 23358 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.283581 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:24.284066 23347 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.293684 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:24.294610 23358 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:24.371546 23386 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:24.513739 23369 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.546806 23369 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.549585 23369 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:24.549670 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 226996us
W20260709 12:25:24.553377 23365 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.468s	user 0.000s	sys 0.007s
I20260709 12:25:24.518028 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:24.527201 23249 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.545138 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:24.554149 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.550374 23137 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:24.598971 23137 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:24.540238 23375 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.281s	user 0.004s	sys 0.007s
W20260709 12:25:24.578392 23384 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.294s	user 0.003s	sys 0.008s
W20260709 12:25:24.577986 23129 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.011s
I20260709 12:25:24.308178 23261 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:24.617331 23261 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.622705 23261 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.626240 14417 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:24.626852 14417 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:24.550071 23296 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.576653 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:24.580348 23286 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.581738 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:24.638756 13727 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:24.584295 13646 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:24.588112 23358 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.588347 23249 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.589346 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:24.616153 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.649287 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.564147 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:24.644199 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77904.8us
I20260709 12:25:24.650539 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:24.656466 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:24.660777 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:24.668382 23358 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.004s
I20260709 12:25:24.670379 23358 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:24.670713 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3021) took 1929 ms (client timeout 2201 ms). Trace:
I20260709 12:25:24.670908 23358 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:24.671476 23358 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:24.671200 13794 rpcz_store.cc:269] 0709 12:25:22.741584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.815345 (+ 73761us) service_pool.cc:224] Handling call
0709 12:25:24.670496 (+1855151us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:24.670618 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159652224}
I20260709 12:25:24.653165 23369 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:24.676661 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2010) took 1454 ms. Trace:
I20260709 12:25:24.648732 23286 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.685570 23286 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.686887 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:24.698663 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:24.677340 13726 rpcz_store.cc:276] 0709 12:25:23.222120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.222276 (+   156us) service_pool.cc:224] Handling call
0709 12:25:23.222607 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.744345 (+521738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.676276 (+931931us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54002f6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:24.676553 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1091043712}
I20260709 12:25:24.642119 23256 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.730767 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:24.726480 22869 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:24.742010 13801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.017s	sys 0.069s
I20260709 12:25:24.724638 13767 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:24.667323 23378 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:24.709082 23369 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:24.769313 13793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.035s	sys 0.075s
W20260709 12:25:24.773793 13793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.035s	sys 0.075s
I20260709 12:25:24.691430 22809 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 12:25:24.691094 23342 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:24.735517 23137 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:24.767782 13801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.018s	sys 0.074s
I20260709 12:25:24.722985 23129 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.053s
W20260709 12:25:24.781484 23385 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:25:24.734675 23256 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.778767 23378 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:24.795910 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74557.8us
I20260709 12:25:24.809484 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:24.810845 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:24.813027 23295 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:24.813601 23295 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:24.828043 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 236353us
I20260709 12:25:24.840925 23284 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:24.842396 23371 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:25:24.845187 23389 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:24.850929 23378 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.840688 23369 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.802052 23237 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:24.864434 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1545) took 1094 ms. Trace:
I20260709 12:25:24.873843 13796 rpcz_store.cc:276] 0709 12:25:23.769719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.770403 (+   684us) service_pool.cc:224] Handling call
0709 12:25:23.770818 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.244523 (+473705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.864122 (+619599us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:24.864361 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088015744}
I20260709 12:25:24.865639 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4867) took 1113 ms. Trace:
I20260709 12:25:24.797163 23342 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:24.876082 13651 rpcz_store.cc:276] 0709 12:25:23.752063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.754001 (+  1938us) service_pool.cc:224] Handling call
0709 12:25:23.807221 (+ 53220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.865231 (+ 58010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.865285 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:23.865351 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:23.865400 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.865384 (+999984us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:24.865435 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.865578 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132700416}
I20260709 12:25:24.885349 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2011) took 1662 ms. Trace:
W20260709 12:25:24.850934 23391 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:24.901914 13721 rpcz_store.cc:276] 0709 12:25:23.222512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.223451 (+   939us) service_pool.cc:224] Handling call
0709 12:25:23.239025 (+ 15574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.574364 (+335339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.884907 (+1310543us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:24.885250 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664286464}
W20260709 12:25:24.868306 23375 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:25:24.897936 23367 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.859s	user 0.007s	sys 0.007s
I20260709 12:25:24.909296 14417 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.868040 23237 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:24.919786 23237 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:24.918100 14417 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:24.906756 23332 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:24.919101 23298 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.924038 23298 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.927098 23298 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:24.927441 23332 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:24.937090 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4851) took 1346 ms. Trace:
I20260709 12:25:24.937354 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.939050 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:24.937568 13647 rpcz_store.cc:276] 0709 12:25:23.591003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.594644 (+  3641us) service_pool.cc:224] Handling call
0709 12:25:23.595117 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.638513 (+ 43396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.638553 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.639124 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.756281 (+117157us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:23.756324 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.756393 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:24.740923 (+984530us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:24.932034 (+191111us) raft_consensus.cc:1685] finished
0709 12:25:24.932054 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.937015 (+  4961us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37208,"raft.queue_time_us":102364,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":13430,"spinlock_wait_cycles":7439616,"thread_start_us":84634,"threads_started":1,"wal-append.queue_time_us":243608}
I20260709 12:25:24.929860 23332 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:24.941582 13796 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33. Ops: [29.16-32.17]
I20260709 12:25:24.942816 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.944839 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:24.948030 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 287321us
I20260709 12:25:24.949939 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.883370 23295 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:24.972867 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:24.984381 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:24.977550 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 138844us
I20260709 12:25:24.948835 23332 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:24.996407 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:25.000325 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:24.997047 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.997274 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:25.023352 14417 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:25.026353 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:25.027424 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.968701 23343 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:25.028066 13794 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:25.032150 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:25.034283 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.045460 23343 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:25.151978 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 116317us
I20260709 12:25:25.032881 13650 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:25.166563 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:24.974439 23298 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:25:25.198622 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.200549 13796 raft_consensus.cc:2410] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:25:25.208947 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:25.212373 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59523us
I20260709 12:25:25.218160 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:25.221607 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 248832us
I20260709 12:25:25.031705 23332 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.203915 23256 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:25.214051 23298 consensus_queue.cc:1243] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 33.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:25:25.147125 23266 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:25:25.030990 23342 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:25.268126 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53954.5us
I20260709 12:25:25.276036 23408 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:25.277948 23394 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 12:25:25.278745 23392 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
W20260709 12:25:25.279838 23401 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:25.279883 23420 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.292188 13643 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:25:25.301904 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:25.267446 23286 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.280939 23420 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.321794 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.323709 23420 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.281330 23237 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.327136 13795 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:25.300830 23397 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.327649 13795 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:25.333205 23405 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:25:25.333711 23343 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:25.335573 23400 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 12:25:25.274503 23406 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:25:25.321130 23286 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.333810 23332 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.341349 23286 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.326399 23137 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.342429 13795 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.356155 13795 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:25.357431 13794 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 33: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:25.342583 23358 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.670s	user 0.017s	sys 0.000s
I20260709 12:25:25.363443 13796 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:25.496878 13652 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:25:25.358494 23358 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.342619 23397 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.570477 23425 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.606542 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.607825 13796 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:25:25.613291 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:25.618816 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 263918us
I20260709 12:25:25.619204 13800 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:25.619201 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:25.626668 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.501215 23094 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.656993 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:25.571234 23425 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.364132 23137 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.614400 23286 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.619022 23261 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:25.688628 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:25.701429 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82956.5us
I20260709 12:25:25.698449 13795 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:25:25.709779 13795 pending_rounds.cc:85] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 15
I20260709 12:25:25.710498 13795 pending_rounds.cc:107] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:25:25.710973 13795 raft_consensus.cc:2889] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:25.661428 22809 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
W20260709 12:25:25.718891 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90604.4us
I20260709 12:25:25.730720 23417 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:25.724067 23299 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:25.737722 23358 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:25.737239 23417 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.738035 23425 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.738440 23358 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:25.718400 23209 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.739187 23358 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:25.739712 23209 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.742983 23424 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.743589 23424 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.746109 23424 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.748533 23137 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:25.748966 23424 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.752710 23424 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.753271 23209 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.754029 23209 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.739810 23417 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:25.746109 23237 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:25.727730 23423 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.701659 23261 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:25.665685 23094 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.755470 23417 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.764232 23420 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:25.784333 23418 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
W20260709 12:25:25.785135 23409 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 12:25:25.786486 23412 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:25:25.764971 23423 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.755105 23209 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.797797 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:25.798655 13652 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:25.766953 23261 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:25.762027 23425 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.715847 23286 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.826876 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.843669 23422 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.844799 23422 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.848027 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:25.848986 13649 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:25.792681 23408 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.853367 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.808324 23397 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.853812 13649 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:25.783594 23417 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.867604 13796 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:25:25.792363 23420 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.872381 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:25.855319 23332 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.850018 23425 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.881511 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:25.875232 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 159258us
I20260709 12:25:25.895527 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.897039 23355 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:25.898527 23429 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:25:25.899183 23300 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.864266 23408 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.893682 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:25.914389 14418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1765) took 2029 ms (client timeout 2087 ms). Trace:
I20260709 12:25:25.908411 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:25.915143 14418 rpcz_store.cc:269] 0709 12:25:23.880353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.880541 (+   188us) service_pool.cc:224] Handling call
0709 12:25:25.909954 (+2029413us) spinlock_profiling.cc:243] Waited 86.7 ms on lock 0x7b54006be4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:25.910153 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199453568}
I20260709 12:25:25.892511 23407 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:25.899963 23300 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.924386 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.884938 23397 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:25:25.926224 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
W20260709 12:25:25.927042 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 236167us
I20260709 12:25:25.934818 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:25.920367 22809 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.883038 23332 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.943856 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.946427 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:25.938167 23426 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:25:25.918133 23237 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:25:25.948743 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:25.928725 23407 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.951968 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.957767 23094 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:25.952468 23434 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:25:25.958479 23438 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:25.952634 23440 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.959580 23440 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:25.960575 23407 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:25.961882 23404 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.747s	user 0.009s	sys 0.003s
I20260709 12:25:25.952852 23300 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:25.962080 23451 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:25.962677 23300 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:25.962878 23451 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.963369 23300 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:25.963413 23451 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.959074 23286 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:25.960109 23433 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:25.941676 22809 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:25.956619 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48347.3us
I20260709 12:25:25.971916 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:25.975920 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:25.987398 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:25.987751 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:25.964371 23440 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:25.996244 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95067.4us
I20260709 12:25:25.963992 23300 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:25.999249 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:25.980188 23433 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.005607 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:26.007112 13652 raft_consensus.cc:2368] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32.
I20260709 12:25:25.980523 23447 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:26.015123 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66475.3us
I20260709 12:25:26.016121 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.017825 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.009852 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:25.994038 23094 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:26.005813 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.021148 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.021618 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:26.026315 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.003862 23417 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.010912 23420 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.020229 23286 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.032871 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.005185 23433 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:26.015426 23441 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:26.051424 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.037549 23447 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:26.037868 23425 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.060508 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.061417 13798 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:26.051223 23408 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.063599 23450 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 12:25:26.064047 23451 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.062117 23446 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:25:26.064565 23436 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:25:26.064968 23452 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:26.065335 22809 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.065207 23439 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:25:26.069916 23455 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:26.070538 23455 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.072319 14423 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:26.073213 23455 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.073767 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.075305 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.077669 23421 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:26.077893 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:26.078209 23421 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.078769 14416 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:26.080551 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.081413 14420 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:26.081974 23419 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.082312 23442 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:26.083045 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4350) took 1540 ms. Trace:
I20260709 12:25:26.083532 13722 rpcz_store.cc:276] 0709 12:25:24.542847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.543031 (+   184us) service_pool.cc:224] Handling call
0709 12:25:24.543556 (+   525us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:24.628051 (+ 84495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.628088 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:24.639513 (+ 11425us) log.cc:844] Serialized 52 byte log entry
0709 12:25:24.943798 (+304285us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:25.215432 (+271634us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.223450 (+  8018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.959091 (+735641us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b18002e98a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db
0709 12:25:25.961131 (+  2040us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:26.082720 (+121589us) spinlock_profiling.cc:243] Waited 84.1 ms on lock 0x7b54002f5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:26.082769 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.082814 (+    45us) raft_consensus.cc:1685] finished
0709 12:25:26.082827 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.082983 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41866,"prepare.queue_time_us":335317,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":57,"raft.queue_time_us":9210,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":39500,"spinlock_wait_cycles":267968384,"thread_start_us":427309,"threads_started":2,"wal-append.queue_time_us":365009}
I20260709 12:25:26.087080 23239 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.087688 23239 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.088447 23249 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:26.088986 23249 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:26.082705 23419 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.038400 23286 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.093549 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:26.099790 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.102412 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.105880 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.107301 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.083039 23442 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:26.108912 23442 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.110482 23419 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.092345 23337 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:25:26.103627 13798 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:25:26.124716 14420 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:26.131628 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.101989 23408 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:26.130198 23284 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.288s	user 0.003s	sys 0.004s
I20260709 12:25:26.080543 23421 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.139699 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3034) took 1159 ms. Trace:
I20260709 12:25:26.141352 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.134021 23237 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.009s
I20260709 12:25:26.124353 23419 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.089977 23249 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.127815 23439 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:26.136458 23284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.294s	user 0.004s	sys 0.004s
I20260709 12:25:26.097812 23461 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:26.132051 22537 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.141368 23408 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.153872 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.155043 23300 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.158520 23419 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.163134 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.144263 23355 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:26.167903 14420 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:25:26.171211 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 131952us
I20260709 12:25:26.172492 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.172611 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.174460 13722 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:25:26.174609 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.175347 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:26.176029 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.176868 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:26.178546 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.178635 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.179267 13726 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:26.179498 13721 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:26.180490 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.181129 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.183621 13726 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:26.183990 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.184547 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.181711 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.182204 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.185153 13718 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:26.185564 13720 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:26.185910 13727 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:26.182652 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.182531 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.186928 13723 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:26.186813 13722 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:26.190065 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.188949 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.191656 13650 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:25:26.191910 13722 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:26.190546 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.193543 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.194386 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:26.195251 13648 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:26.195847 13720 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:26.197631 23239 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:26.199059 23419 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.151494 23284 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:26.202922 23460 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:26.203797 23286 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.167467 22537 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.199572 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:26.206290 13617 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.052s (SENT)
I20260709 12:25:26.206799 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:26.199594 13650 raft_consensus.cc:1217] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Deduplicated request from leader. Original: 28.15->[29.16-32.17]   Dedup: 29.16->[32.17-32.17]
I20260709 12:25:26.211380 13648 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:26.222823 13650 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33. Ops: [29.16-32.17]
I20260709 12:25:26.224284 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.165933 23249 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:26.208343 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:26.227046 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.228771 13652 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:26.235364 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.237186 22537 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.239987 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:26.239380 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.241631 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.234436 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.243492 13648 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 19.
I20260709 12:25:26.266918 13649 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:26.268666 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.233983 23397 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:26.272958 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.160780 23439 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.278363 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:26.274830 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.299172 23344 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:26.299702 23344 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.300065 23344 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:26.274849 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.277469 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.300945 23439 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.245381 23300 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.301457 23440 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:25:26.202955 23284 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.301929 23300 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.302977 23466 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:26.303270 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.303553 23466 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.304488 23300 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.271013 23334 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:26.238755 23336 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:26.305310 23334 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.304808 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.311188 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.305548 23336 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.245146 23286 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:26.272578 23395 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.387s	user 0.012s	sys 0.007s
I20260709 12:25:26.315590 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.275936 23461 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:26.335729 23461 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.317174 23455 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:26.338615 23455 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.339026 23455 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:26.339334 23456 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:26.340059 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4349) took 1821 ms. Trace:
I20260709 12:25:26.340152 23456 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.340572 13724 rpcz_store.cc:276] 0709 12:25:24.518424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.541270 (+ 22846us) service_pool.cc:224] Handling call
0709 12:25:24.542813 (+  1543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.546562 (+  3749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.546882 (+   320us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.548591 (+  1709us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.635717 (+ 87126us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:24.635886 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.635999 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.237461 (+1601462us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:26.339694 (+102233us) raft_consensus.cc:1685] finished
0709 12:25:26.339720 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.339984 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6953,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":122,"spinlock_wait_cycles":40801024,"thread_start_us":85216,"threads_started":1,"wal-append.queue_time_us":245344}
I20260709 12:25:26.314994 23334 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:26.244936 23371 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 12:25:26.286667 23450 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.347927 23450 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.319360 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.320542 23466 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.327407 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.297871 23419 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:26.352283 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:26.281008 22537 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:26.280647 23239 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:26.317533 23397 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:26.359514 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.323632 23336 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:26.325747 23363 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:26.363783 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:26.354379 13764 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:26.330312 23355 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:26.329133 23439 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:26.355295 23419 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:26.367398 23261 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.516s	user 0.014s	sys 0.000s
I20260709 12:25:26.318638 23300 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:25:26.358776 22809 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:26.321448 23451 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:25:26.355865 22537 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.361699 23239 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:26.378422 23233 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:26.374706 23363 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.385829 23471 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:26.390960 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:26.391666 23450 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:26.352566 23286 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.396636 23430 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:26.400949 13646 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:25:26.362921 23397 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.427951 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.377310 23235 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:26.371213 23465 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:26.376932 23440 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:26.414741 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.384004 23419 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:26.421164 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.423033 23250 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:26.443656 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.444407 14418 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:25:26.445248 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:26.423206 23233 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:26.408078 22537 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:26.446731 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.447990 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.447631 23233 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 33, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.448594 14419 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:26.419970 23466 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.408746 23239 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:26.450567 23466 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.453418 23466 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.428798 14416 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 33: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:26.418115 23363 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:26.410842 22809 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:26.414466 23461 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.407071 23261 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.430836 23397 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:26.441380 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.442615 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.433574 23235 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.443416 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.456560 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.384490 23439 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.492734 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.430171 23430 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:26.426429 23459 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:26.462435 14391 leader_election.cc:400] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32.
I20260709 12:25:26.495018 14391 leader_election.cc:403] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.428488 23450 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.441570 23456 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.432396 23408 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.475610 14418 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:26.479233 23461 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.436465 23465 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.496330 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.425146 23471 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.459399 23300 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:25:26.490885 23235 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:25:26.518167 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 62850.8us
I20260709 12:25:26.434046 23433 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.515538 14421 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:26.496685 23459 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:26.539984 23459 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:26.512483 23358 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:26.517200 23461 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.547000 23471 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:26.546609 22809 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.546617 23358 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.548995 23461 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.531889 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:26.550424 23358 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.551234 23471 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:26.554376 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.555143 13792 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:26.536614 23286 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.538944 23478 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32.
I20260709 12:25:26.534090 23344 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.562963 13797 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:26.564915 13795 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:26.566028 23261 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:26.571821 23336 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:26.534917 23468 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:25:26.513914 23465 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:26.580783 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:26.582223 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:26.562634 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.584254 13646 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:26.584272 14425 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:26.559479 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:26.586766 23261 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:26.587170 23261 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.587530 23261 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:26.590561 13719 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
W20260709 12:25:26.584719 13690 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 2.111s (SENT) [suppressed 27 similar messages]
I20260709 12:25:26.595304 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.597826 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.514453 23450 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.605214 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.624431 23334 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.625514 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.626466 14423 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:26.630652 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.561576 23344 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.640012 14425 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:26.564389 23433 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:26.591297 23378 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.660127 13646 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:26.660394 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:26.660741 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 76060.3us
I20260709 12:25:26.665287 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:26.558367 23459 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.671080 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3036) took 1688 ms. Trace:
I20260709 12:25:26.672853 13646 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:26.675768 13644 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:26.671710 13643 rpcz_store.cc:276] 0709 12:25:24.982944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.983135 (+   191us) service_pool.cc:224] Handling call
0709 12:25:26.670758 (+1687623us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:26.670971 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599922432}
I20260709 12:25:26.564302 23430 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.664122 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:26.655372 23439 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:26.685251 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:26.689953 13727 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:25:26.665750 23378 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:26.691030 23473 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:25:26.690078 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.693439 13724 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:26.683316 23456 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.697777 23465 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.678527 22537 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.698290 23456 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.667846 23335 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:26.700263 23430 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:26.700465 22537 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.701613 23456 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:26.695010 23300 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.705029 23335 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:26.705332 23300 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.706173 23335 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.708508 23300 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.702405 23371 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:26.702813 23252 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:26.703604 23446 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:26.704355 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:26.714895 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.715205 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:26.716672 13722 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:26.717104 13723 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:26.674217 23344 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:26.723122 13690 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:26.721753 23433 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:26.731778 23433 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:26.732421 23300 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:26.732883 23300 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:26.733526 23300 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:26.733045 23433 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.736152 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.739631 23433 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:26.740558 23465 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.742117 23459 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.742365 23461 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:25:26.736897 23456 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:26.746291 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:26.748972 13650 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:26.762499 23456 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:26.741918 23287 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:26.722608 23446 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.769464 23286 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:26.769963 23446 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:26.770035 23286 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:26.770431 23256 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:26.770813 23256 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:26.772224 23287 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:26.786707 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.790432 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:26.790993 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4087) took 2227 ms. Trace:
I20260709 12:25:26.791409 13725 rpcz_store.cc:276] 0709 12:25:24.563985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.666657 (+102672us) service_pool.cc:224] Handling call
0709 12:25:26.114762 (+1448105us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:25:26.115015 (+   253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.115432 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.115464 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.116002 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.355406 (+239404us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:26.374948 (+ 19542us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.375252 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.375381 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.789481 (+414100us) raft_consensus.cc:1685] finished
0709 12:25:26.790756 (+  1275us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.790909 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16536,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":256,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":1885,"spinlock_wait_cycles":13163392,"thread_start_us":238925,"threads_started":1,"wal-append.queue_time_us":359924}
I20260709 12:25:26.795801 23235 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.140240 13647 rpcz_store.cc:276] 0709 12:25:24.980404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.980559 (+   155us) service_pool.cc:224] Handling call
0709 12:25:24.980901 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.981143 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:26.112337 (+1131194us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b18003f90a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:25:26.112514 (+   177us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.139291 (+ 26777us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.139355 (+    64us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:26.139409 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.139491 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.139637 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9407,"prepare.queue_time_us":166593,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":5995,"raft.queue_time_us":85,"spinlock_wait_cycles":2973952,"thread_start_us":242036,"threads_started":2}
I20260709 12:25:26.724390 23239 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.771806 23344 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:26.785904 23287 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:26.771180 23286 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 30.23, Last appended by leader: 23, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:26.820885 23332 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.140618 23287 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:27.139976 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:26.926638 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 480034us
W20260709 12:25:27.144078 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.829133 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3041) took 1834 ms. Trace:
W20260709 12:25:26.920342 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 509569us
I20260709 12:25:26.874622 23363 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.145613 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:26.870278 23342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.592s	user 0.040s	sys 0.027s
W20260709 12:25:27.148241 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 174684us
I20260709 12:25:27.154421 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.827896 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4883) took 1334 ms. Trace:
W20260709 12:25:27.148262 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.828485 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3044) took 1796 ms. Trace:
W20260709 12:25:26.920051 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 600759us
I20260709 12:25:26.873175 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.799782 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:27.176337 13794 rpcz_store.cc:276] 0709 12:25:25.036318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.037343 (+  1025us) service_pool.cc:224] Handling call
0709 12:25:26.828273 (+1790930us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:26.828419 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367096576}
I20260709 12:25:27.177731 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:27.186887 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:27.186688 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 387003us
W20260709 12:25:26.806619 23249 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.003s
I20260709 12:25:27.188066 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:27.188542 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.166613 13644 rpcz_store.cc:276] 0709 12:25:25.493828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.494063 (+   235us) service_pool.cc:224] Handling call
0709 12:25:26.673506 (+1179443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.679488 (+  5982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.827636 (+148148us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002e4fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:26.827815 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448724224}
I20260709 12:25:27.189005 14419 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:27.148152 13800 rpcz_store.cc:276] 0709 12:25:24.994972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.995240 (+   268us) service_pool.cc:224] Handling call
0709 12:25:24.995753 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.628293 (+632540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.828852 (+1200559us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b54001ca2d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:26.829065 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1433295232}
I20260709 12:25:27.195806 14419 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:27.196516 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.189602 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.191133 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.191887 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.199332 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.192922 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.189924 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:27.202337 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.203687 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:27.204685 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:27.151700 23455 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.812s	user 0.015s	sys 0.003s
I20260709 12:25:26.826094 23209 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.822216 23450 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.196861 14419 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:27.186385 23342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.908s	user 0.075s	sys 0.071s
I20260709 12:25:27.212791 13725 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:27.213908 23250 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:27.227458 14419 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:27.214644 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:26.822767 23336 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.210645 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:27.223398 23282 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:25:27.233425 23282 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:27.233788 23282 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:27.234647 23282 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.235822 23282 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:27.044162 23422 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.936127 23408 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:27.207947 23414 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:27.186477 23129 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.194581 23428 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:27.185628 23439 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.236233 23363 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:26.890615 23419 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.209107 23347 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:26.825551 23261 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.264163 14425 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:27.265601 23261 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.253190 13800 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:27.224391 23403 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:27.270844 23403 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.271318 23403 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:27.252719 23428 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.236863 23427 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:25:27.274514 23428 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:25:27.248243 23361 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 25 current term: 25 is higher.
I20260709 12:25:27.252332 23129 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.259214 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.279219 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:27.208000 23007 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:25:26.885936 23397 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.280637 23129 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:27.281877 23397 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:27.282861 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.284232 14419 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:25:27.248059 23414 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.285321 23414 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:27.287117 23378 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:27.291297 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.291846 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3048) took 1065 ms. Trace:
I20260709 12:25:27.293255 13648 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:27.295241 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.295955 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.296758 13723 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:27.297410 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.296072 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.298494 13721 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 20: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:27.295876 13800 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:27.292273 13650 rpcz_store.cc:276] 0709 12:25:26.226603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.226755 (+   152us) service_pool.cc:224] Handling call
0709 12:25:27.291600 (+1064845us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54002b48d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:27.291783 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180038400}
I20260709 12:25:27.238375 23235 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.248013 23336 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:27.347718 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.805640 23256 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.245232 23408 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.356438 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.222347 23455 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.359390 13721 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:27.360301 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.371449 13727 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:27.361130 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.238590 23209 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:25:27.372087 13723 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:27.362347 13725 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 17. (term mismatch)
I20260709 12:25:27.373143 13725 pending_rounds.cc:85] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 16
I20260709 12:25:27.373469 13725 pending_rounds.cc:107] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 32.17
I20260709 12:25:27.373762 13725 raft_consensus.cc:2889] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.362371 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4888) took 1566 ms. Trace:
I20260709 12:25:27.375025 13645 rpcz_store.cc:276] 0709 12:25:25.795777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.795957 (+   180us) service_pool.cc:224] Handling call
0709 12:25:25.796309 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.905269 (+108960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.905316 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:25.905376 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:25.905411 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.361985 (+1456574us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001b5b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:27.362068 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.362284 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250064128}
I20260709 12:25:27.376647 13800 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:27.258378 23347 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.361390 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:27.362859 23408 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:27.387030 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.387703 13725 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:27.390228 23300 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.654s	user 0.006s	sys 0.006s
I20260709 12:25:27.390646 23347 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:27.390722 23300 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.355543 23336 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.392207 23336 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:27.393939 23423 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.347895 23255 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:27.395140 23239 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.395889 23424 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.255267 23344 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:27.398962 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4107) took 1210 ms (client timeout 1605 ms). Trace:
W20260709 12:25:27.399400 13720 rpcz_store.cc:269] 0709 12:25:26.188535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.189686 (+  1151us) service_pool.cc:224] Handling call
0709 12:25:27.398761 (+1209075us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b5400173c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:27.398872 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10365824}
I20260709 12:25:27.400735 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.401324 13720 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:27.408455 23396 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:27.409044 23396 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:27.412469 13723 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:27.415659 23414 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.420125 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:27.420827 13719 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4079) took 3151 ms (client timeout 2052 ms). Trace:
W20260709 12:25:27.421252 13719 rpcz_store.cc:269] 0709 12:25:24.279079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.279244 (+   165us) service_pool.cc:224] Handling call
0709 12:25:27.420563 (+3141319us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:27.420705 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5289044864}
I20260709 12:25:27.421777 23459 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:27.422396 23459 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:27.423369 23459 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:27.426401 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.257848 23439 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.427609 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4453) took 1407 ms. Trace:
I20260709 12:25:27.428047 13795 rpcz_store.cc:276] 0709 12:25:26.020291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.020707 (+   416us) service_pool.cc:224] Handling call
0709 12:25:27.427391 (+1406684us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:27.427541 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218000640}
I20260709 12:25:27.429004 23439 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.429374 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1499) took 1008 ms. Trace:
I20260709 12:25:27.429708 23403 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.432293 23403 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.433473 23419 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.435536 23455 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.435820 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:27.436225 23455 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:27.437157 23455 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:27.437260 13721 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:27.437587 23455 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:27.440907 13718 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:27.442214 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4902) took 1018 ms. Trace:
W20260709 12:25:27.442853 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 240353us
I20260709 12:25:27.442641 13648 rpcz_store.cc:276] 0709 12:25:26.423970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.424521 (+   551us) service_pool.cc:224] Handling call
0709 12:25:27.293012 (+868491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.293791 (+   779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.426116 (+132325us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b54002b48d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:27.442141 (+ 16025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996164864}
I20260709 12:25:27.429755 14421 rpcz_store.cc:276] 0709 12:25:26.420726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.420869 (+   143us) service_pool.cc:224] Handling call
0709 12:25:27.429161 (+1008292us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b540028ffd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:27.429284 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173448576}
I20260709 12:25:27.446314 23414 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:27.456017 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 244771us
W20260709 12:25:27.456564 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 239036us
I20260709 12:25:27.463653 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.464641 13797 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:25:27.458050 14423 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:27.465790 14423 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:27.450403 22677 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.458863 13725 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:27.474356 22677 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.475059 23233 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:27.476361 23464 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:25:27.476984 23464 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:27.478183 23464 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.480028 23464 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:27.488205 23300 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.427650 23428 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.491176 23428 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.480697 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:27.495092 23431 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:27.495204 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:27.495602 23431 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:27.495999 23431 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:27.409969 23396 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.502914 23396 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:27.502527 23455 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.504011 23455 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:27.506062 23446 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.735s	user 0.012s	sys 0.000s
I20260709 12:25:27.506603 23446 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.469597 23419 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.508847 23431 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.509665 23419 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:27.510964 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47377us
I20260709 12:25:27.512234 23419 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.513116 13618 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.515367 23209 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:27.516155 13724 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:27.518461 22860 consensus_queue.cc:1175] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 23 status { last_received { term: 23 index: 14 } last_committed_idx: 13 last_received_current_leader { term: 23 index: 14 } }
I20260709 12:25:27.468636 23335 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.524880 14423 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.532657 14423 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:27.522579 23419 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:27.535004 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:27.535665 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:27.551988 13724 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:27.556650 13644 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:27.530069 23443 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.565127 13721 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:27.560568 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.530916 23428 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.561637 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.560508 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.498039 23408 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.566675 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.568281 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:27.573102 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.576108 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.567423 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.579309 13725 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:27.526213 23335 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:27.591130 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.563011 23300 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.564291 23237 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:27.611034 13724 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:27.614786 13650 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:27.616569 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:27.619298 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51976.2us
I20260709 12:25:27.566059 23446 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.620673 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.620800 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:27.625072 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.625921 13724 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:27.560117 23266 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:25:27.593616 23129 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:27.541669 23419 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.590628 23261 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.649503 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.589043 23428 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.628674 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:27.621819 23237 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:27.655730 13725 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 31 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:25:27.639019 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:27.572949 23443 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.660172 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.645691 23007 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.611651 23336 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:27.672222 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45900.5us
I20260709 12:25:27.637349 13724 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:25:27.667235 13800 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:25:27.669843 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.672343 13720 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:27.664088 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.675323 13799 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:27.678227 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.679723 14421 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:27.680778 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.653405 23284 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.687868 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.688170 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.632725 23347 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.627333 23300 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.684087 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.686136 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.686584 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.686947 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1552) took 1187 ms. Trace:
I20260709 12:25:27.693378 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:27.695511 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.696573 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.698364 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.658087 23237 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:27.701400 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.690903 13798 rpcz_store.cc:276] 0709 12:25:26.499449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.499823 (+   374us) service_pool.cc:224] Handling call
0709 12:25:26.500305 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.500605 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:25:26.509313 (+  8708us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.493356 (+984043us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b1800432700. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:25:27.686400 (+193044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.686610 (+   210us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:27.686664 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.686736 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.686883 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31079,"prepare.queue_time_us":7881,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"spinlock_wait_cycles":15191936,"thread_start_us":151270,"threads_started":1}
I20260709 12:25:27.703054 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.656579 23446 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.707068 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.711506 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.712545 13647 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:27.714289 13797 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:27.677098 23007 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.676817 23094 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.652799 23261 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.676762 23108 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.725919 13652 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:27.684706 23498 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:27.700690 23431 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.730388 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.703255 23403 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.736408 13799 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:25:27.733359 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.739377 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.741590 13720 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:27.745029 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.746536 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.748724 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.748977 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.750722 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:27.754169 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52839.3us
I20260709 12:25:27.755425 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:27.758394 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.759228 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:27.759933 13719 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:27.705235 23284 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.763453 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:27.716497 23446 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.772223 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.773927 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.774550 23423 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:25:27.775481 23491 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:25:27.775969 23007 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:27.782733 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.731767 23108 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.786770 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.730142 23094 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.795778 23499 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:27.796483 23300 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.797513 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.798631 23335 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:27.801119 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:27.804191 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4881) took 2311 ms. Trace:
W20260709 12:25:27.804682 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.806044 23428 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.807232 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:27.776824 23424 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:27.777539 23332 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:27.759954 23355 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:27.828608 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.787739 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.830248 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.735702 23403 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.839128 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:27.870077 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:27.837867 13651 rpcz_store.cc:276] 0709 12:25:25.493154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.495532 (+  2378us) service_pool.cc:224] Handling call
0709 12:25:25.574933 (+ 79401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.660752 (+ 85819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.660793 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.687979 (+ 27186us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.424324 (+1736345us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b1800154360. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:25:27.628421 (+204097us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:27.628465 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.628553 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.803804 (+175251us) raft_consensus.cc:1685] finished
0709 12:25:27.803825 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.804064 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111297,"raft.queue_time_us":91855,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":12256,"spinlock_wait_cycles":3700992,"thread_start_us":203886,"threads_started":1,"wal-append.queue_time_us":2013242}
W20260709 12:25:27.842705 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.846813 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49402.1us
W20260709 12:25:27.851733 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.844985 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:27.855033 13795 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:27.854249 13719 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:27.857398 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:27.873768 13795 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:27.874114 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:27.874745 13795 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:27.841091 22920 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:25:27.899902 23423 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:27.837044 23007 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:25:27.899935 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.901595 23235 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:27.902093 23235 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.902498 23235 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:27.903106 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.904433 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.906368 23450 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:25:27.906841 23450 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.859651 23344 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:27.860261 23424 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:27.860525 23428 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:27.861923 23332 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.911752 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:27.863287 23347 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.863893 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.858533 23300 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.875128 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:27.858512 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:27.821918 23459 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.897765 23403 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.898669 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:27.907258 23450 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:25:27.839219 23408 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.877981 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:27.847074 22809 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:27.935412 13801 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:27.914422 13651 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:27.838438 23499 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:27.865331 23094 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.876142 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:27.927851 14423 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:27.908533 23355 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:27.907773 23344 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:27.996343 23344 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:27.914201 23332 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.003235 23446 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.009147 23209 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:27.940416 23336 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.015583 23261 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:27.913884 23424 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.024914 13798 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
W20260709 12:25:27.940035 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70031.3us
I20260709 12:25:28.028002 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:27.978212 23347 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:27.978973 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102939us
I20260709 12:25:27.979007 23129 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.031665 23129 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:27.978865 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 60149.8us
I20260709 12:25:27.983119 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:27.984706 23284 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:27.990965 23505 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.253s	user 0.001s	sys 0.004s
I20260709 12:25:27.935595 23419 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.028935 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.036310 14416 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:28.034286 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:27.939499 23403 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:28.037972 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.041131 13651 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:28.041352 23336 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:28.043118 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:28.043794 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.044579 13648 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 19: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:28.043236 23336 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.045346 23336 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:28.045500 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.046576 23239 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:28.046842 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.047816 13648 raft_consensus.cc:2410] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 30 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:25:28.048656 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:28.049715 23455 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.545s	user 0.014s	sys 0.002s
I20260709 12:25:28.049991 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:28.052168 13723 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:25:28.044405 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.052805 13651 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:28.053418 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.054097 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.041131 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.055784 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.057952 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:28.022583 23499 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:28.060348 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 120483us
I20260709 12:25:27.939881 23300 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.063302 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.069002 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:28.041870 23464 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.005s
I20260709 12:25:27.934849 23459 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.059938 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:28.075641 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.057446 23455 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:28.082415 13794 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:28.068159 23419 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:28.079209 23284 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:28.070852 23446 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:25:28.083875 23284 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.084303 23284 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.084357 22677 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.058736 23509 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:28.085045 23439 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.085340 23459 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.085660 23439 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.086040 23419 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.087146 23419 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.079310 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:28.092849 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.074416 23355 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:28.093703 23428 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:28.093858 23355 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.072618 23347 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:28.096325 23284 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.096889 23347 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.097913 23347 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.099812 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.100442 14416 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:28.100529 23129 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.061182 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.088068 23408 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:28.087967 23439 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.102802 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.070176 23403 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:28.073285 23282 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:28.110000 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.111173 23336 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.111866 13652 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:28.113448 23336 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.114215 13652 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:28.123063 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.123900 13652 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:25:28.121294 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1390) took 1676 ms. Trace:
I20260709 12:25:28.127713 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.128661 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.125382 14422 rpcz_store.cc:276] 0709 12:25:26.442026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.442817 (+   791us) service_pool.cc:224] Handling call
0709 12:25:27.434379 (+991562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.434629 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.434659 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:27.434707 (+    48us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:27.434738 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.434798 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.118096 (+683298us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b540028ffd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:28.118211 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279670528}
I20260709 12:25:28.129204 23261 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:25:28.129441 23455 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:28.129011 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.131210 23237 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.131995 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.150883 23336 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:28.151949 13797 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:28.135761 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.095260 23355 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.133275 13652 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:28.117524 23396 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:28.105836 23446 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:28.133337 23509 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.159821 13652 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:25:28.143608 23284 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.142601 22809 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:28.166706 22809 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:28.152307 23450 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.172106 23256 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:28.173570 23450 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.162899 13801 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:28.161995 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:28.157420 23455 consensus_queue.cc:1243] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 29.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:28.159103 22860 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.175038 22860 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.173731 23261 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.160180 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.180341 14424 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:25:28.171629 23300 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.189208 13647 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:25:28.190333 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.203495 14418 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:25:28.204208 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.159137 23094 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.157442 23396 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:28.217518 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.210810 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.212182 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.217353 13722 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:28.220152 14422 raft_consensus.cc:2393] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:25:28.213165 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77756.1us
I20260709 12:25:28.213656 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:28.213676 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.214143 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:28.223073 14420 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:28.214869 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.226332 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.226409 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:28.204748 23433 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:28.183764 22860 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.229532 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.222560 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:28.231268 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.231823 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.255594 13724 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:28.231817 13722 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
I20260709 12:25:28.230578 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.256891 13722 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:25:28.256767 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:28.251523 23235 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.249145 13727 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
W20260709 12:25:28.260902 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1547) took 2175 ms (client timeout 1940 ms). Trace:
I20260709 12:25:28.224852 23450 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:28.257318 13722 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 12:25:28.265563 13722 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:28.261356 13796 rpcz_store.cc:269] 0709 12:25:26.085617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.093264 (+  7647us) service_pool.cc:224] Handling call
0709 12:25:28.254581 (+2161317us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:28.260823 (+  6242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565170560}
I20260709 12:25:28.210115 23422 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.267616 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.213958 23509 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.264187 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.211267 23419 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:28.218315 23446 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.271260 13723 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:25:28.260253 23369 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:25:28.277918 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.264662 23361 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.282869 23408 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:28.283468 23408 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.283720 23108 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.265201 23450 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:28.284714 23450 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:28.284552 23408 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.283074 23361 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.288552 13727 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:28.272259 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:28.229916 23511 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.257771 23459 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:25:28.202881 23261 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.236375 13719 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:28.293862 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.348295 13794 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:25:28.255199 23433 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.333669 14420 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:28.270465 23300 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:28.269457 23129 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:28.216872 23486 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.060s
I20260709 12:25:28.218012 23499 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.222044 23094 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.385939 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:28.387178 13725 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:28.400103 13644 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:28.401135 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133753us
I20260709 12:25:28.403683 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.277379 23422 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.250073 23332 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.352315 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:28.353077 23369 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.356667 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:28.355974 23455 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:28.353719 23300 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.343973 23519 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.392850 23397 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.362937 23361 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.349386 23459 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:28.353609 23433 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:28.449903 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.350587 23396 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.435961 23300 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:28.286546 23446 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.233069 23439 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:28.432876 13647 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:28.420143 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.421994 23423 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.464722 23423 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.453629 23523 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:28.466691 23523 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.467054 23523 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:28.467290 23423 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.438181 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:28.443028 13652 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:28.472497 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 402821us
I20260709 12:25:28.475297 23446 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:28.475488 23518 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:25:28.446964 23397 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.478674 23397 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.483292 23459 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:28.484753 23520 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:25:28.486850 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.488291 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.489454 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.345700 23511 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.453347 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:28.462751 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.432298 13648 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:28.463418 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.451206 23137 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:25:28.500504 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:28.444396 13794 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:25:28.501156 13727 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:28.437582 23369 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:28.494975 23516 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:25:28.279000 23509 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.508016 23443 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.508774 23509 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.509806 23369 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.510398 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.512809 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:28.513830 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.515424 13646 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:28.515955 23511 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.517323 23499 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.518877 23332 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.519119 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.520313 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.443395 23519 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.520828 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:28.521433 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.521780 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:28.514055 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:28.524232 13725 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:28.524678 13727 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:28.526693 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.528306 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.526019 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.533687 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.534786 13794 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:28.536751 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:28.438133 23455 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.538542 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.539484 13798 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:28.471494 22860 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:28.548899 13797 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: []
I20260709 12:25:28.551465 13797 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:28.546398 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:28.544521 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124446us
I20260709 12:25:28.555408 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:28.557926 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 76032.3us
I20260709 12:25:28.559698 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.563474 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.564734 13719 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:28.556170 23235 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.569600 13725 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
W20260709 12:25:28.575582 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112237us
I20260709 12:25:28.556846 23332 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:28.584183 13725 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:28.587088 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.566318 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:28.568110 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.572549 23261 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.566442 23526 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:28.570521 23397 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.545994 23523 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.561975 23499 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.024s
I20260709 12:25:28.583513 23161 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:28.599123 23422 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.583689 23511 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:25:28.601596 23513 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:25:28.626943 13727 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:28.627408 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.603974 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:28.628877 13797 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:25:28.575798 23519 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.582589 23332 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.598338 23423 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:28.610842 14425 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: []
I20260709 12:25:28.586560 22677 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:25:28.604394 23284 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:25:28.595466 23094 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.678213 13792 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:28.587972 23464 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:25:28.618443 14419 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:28.621676 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.612970 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.585355 23459 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:28.685895 23256 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:28.686401 23256 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:28.685892 23371 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:25:28.627463 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.627004 23526 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:28.631759 13725 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:25:28.633663 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.578634 23347 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:25:28.635914 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:28.658857 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:28.639616 23129 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:28.578505 23137 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:25:28.643079 23235 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:28.680153 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:28.587863 23239 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 12:25:28.595111 23266 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:25:28.680583 23161 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:28.684448 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.701009 23108 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:28.675379 23419 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.726047 23419 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.702114 23509 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:28.727164 23509 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:28.727612 23509 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.642891 23332 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:28.637053 23369 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.076s
I20260709 12:25:28.638185 23498 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:28.713173 13800 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:25:28.713380 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.713780 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 126527us
I20260709 12:25:28.684689 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.726431 23523 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:28.693997 14424 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
W20260709 12:25:28.733567 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.734642 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 101112us
I20260709 12:25:28.637192 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.701166 23459 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.736157 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.732653 14419 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:28.735716 14425 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:25:28.734411 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 121566us
I20260709 12:25:28.738404 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.739424 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.717846 23423 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:28.566159 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:28.750798 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.755263 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:28.614765 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.764904 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.731935 23161 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:28.741408 23498 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:28.738124 23108 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.792963 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3066) took 1136 ms. Trace:
I20260709 12:25:28.796307 13644 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:28.753746 23446 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:28.742782 23519 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:28.797550 23129 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:28.763695 23336 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:28.755702 23423 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:28.781554 23094 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:28.740520 23347 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:25:28.765028 23455 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.804382 13647 rpcz_store.cc:276] 0709 12:25:27.656208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.420495 (+764287us) service_pool.cc:224] Handling call
0709 12:25:28.421042 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.463288 (+ 42246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.792512 (+329224us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5400198554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:28.792840 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26322176}
W20260709 12:25:28.821087 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 230701us
I20260709 12:25:28.821659 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.809851 23519 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.822564 23519 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:28.785535 23255 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:28.822942 23108 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.802695 23531 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:25:28.818671 23419 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.802757 23233 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:28.802982 23397 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:28.742926 23459 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:28.832377 23529 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:25:28.832728 23459 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:28.768538 23137 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:25:28.822944 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.834937 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.840267 14422 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:28.838594 13647 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:28.842836 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.816679 23423 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:25:28.852113 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 191855us
I20260709 12:25:28.856484 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.858937 13649 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:28.860756 14421 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:25:28.861300 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:28.862488 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 107506us
I20260709 12:25:28.863284 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.834021 23094 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.866549 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.868039 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:28.869218 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.817243 23129 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.880767 23129 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:28.873322 23255 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.010s
I20260709 12:25:28.871702 23419 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:28.866406 23336 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.017s
I20260709 12:25:28.884382 23336 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:28.884769 23336 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:28.871035 23369 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.017s
I20260709 12:25:28.888886 23094 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.876150 23397 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:28.834057 23161 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.895738 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:28.895958 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.873878 23495 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:28.870818 23450 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.580s	user 0.019s	sys 0.000s
I20260709 12:25:28.858497 23427 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:28.885619 23336 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:28.903700 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.902411 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.894718 23161 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:28.904955 23531 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:28.908555 23235 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:28.904278 23300 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:28.915220 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46082.6us
I20260709 12:25:28.931252 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:28.874065 23408 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:28.921326 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.923017 23336 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:28.927644 13722 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:28.926981 13718 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4114) took 1630 ms (client timeout 1868 ms). Trace:
I20260709 12:25:28.865576 23355 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 36.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.052s
W20260709 12:25:28.940749 13718 rpcz_store.cc:269] 0709 12:25:27.297052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.297178 (+   126us) service_pool.cc:224] Handling call
0709 12:25:28.926732 (+1629554us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54003fedd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:28.926913 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325312896}
W20260709 12:25:28.941555 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.944114 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.938181 14421 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:28.945811 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:28.947217 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:28.919211 23233 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:28.929246 22920 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:28.944183 23495 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:28.929736 23498 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.934903 23408 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.934634 23235 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.006999 23408 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:28.957544 23433 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.009039 23235 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.986939 23355 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:28.987619 23509 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.016810 23498 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:29.019114 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:29.019367 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:29.021291 23439 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.002s
I20260709 12:25:29.021934 23439 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:28.964968 22920 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:29.028229 22920 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.029374 22920 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:29.033878 23423 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.958448 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4920) took 1345 ms. Trace:
I20260709 12:25:29.036598 23423 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:29.048362 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:29.049620 23397 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:29.050802 23534 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:25:29.051575 23534 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:29.052820 23534 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:28.987552 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1406) took 1718 ms. Trace:
I20260709 12:25:28.971232 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.903075 23464 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:25:28.918337 23450 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.009545 23433 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:29.060849 14417 rpcz_store.cc:276] 0709 12:25:27.269201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.269369 (+   168us) service_pool.cc:224] Handling call
0709 12:25:27.269747 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.292582 (+ 22835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.292630 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.293300 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.703941 (+410641us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:27.703996 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.959105 (+1255109us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b54006f72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:28.959160 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.959204 (+    44us) raft_consensus.cc:1685] finished
0709 12:25:28.959216 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.987468 (+ 28252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51852032,"thread_start_us":319628,"threads_started":1,"wal-append.queue_time_us":419345,"wal-append.run_cpu_time_us":2054,"wal-append.run_wall_time_us":1016378}
I20260709 12:25:29.013866 23509 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:28.962179 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.948911 23286 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:29.036242 13652 rpcz_store.cc:276] 0709 12:25:27.613267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.165767 (+552500us) service_pool.cc:224] Handling call
0709 12:25:28.166068 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.496018 (+329950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.958131 (+462113us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400115154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:28.958379 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636471552}
I20260709 12:25:29.080914 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:29.083021 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:29.087469 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:29.090131 23450 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:29.091425 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.091826 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:29.092734 13652 raft_consensus.cc:2410] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:29.093587 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:29.093675 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.094602 13801 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:29.094173 23237 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:29.095386 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:29.096912 14417 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:25:29.097182 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.098806 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:29.017788 23332 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.100019 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:28.943145 23446 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.137s
I20260709 12:25:29.103602 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4915) took 1559 ms. Trace:
I20260709 12:25:29.108229 13650 rpcz_store.cc:276] 0709 12:25:27.544131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.544260 (+   129us) service_pool.cc:224] Handling call
0709 12:25:27.544554 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.620267 (+ 75713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.103258 (+1482991us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b54001e6754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:29.103513 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160829312}
I20260709 12:25:28.985991 23300 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:29.105234 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.105300 13652 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:29.100332 14425 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:29.118688 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.119532 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:29.119911 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.109661 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:29.121757 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1791) took 1179 ms. Trace:
I20260709 12:25:29.122151 14418 rpcz_store.cc:276] 0709 12:25:27.943425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.944082 (+   657us) service_pool.cc:224] Handling call
0709 12:25:28.184267 (+240185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.207398 (+ 23131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.121562 (+914164us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54006a72d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:29.121692 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551352576}
I20260709 12:25:29.123600 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.126881 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:29.127594 13652 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:29.129326 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.131906 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:29.133155 13652 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:29.134388 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:29.135871 23332 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:29.138685 23530 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:29.074383 23498 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.045s
I20260709 12:25:29.142204 13801 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:25:29.142886 23397 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:29.143394 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.144641 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.145629 23500 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:25:29.147493 23521 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.791s	user 0.005s	sys 0.000s
I20260709 12:25:29.150745 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:29.151834 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4477) took 1381 ms. Trace:
I20260709 12:25:29.152281 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:29.152969 13797 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:29.153563 13643 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:28.956000 23510 consensus_queue.cc:1175] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b4f6bfcda63148e4965ca02c639336f1" responder_term: 21 status { last_received { term: 20 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:29.163772 13647 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:29.164247 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:29.164875 13798 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:29.146025 23500 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:29.161217 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4926) took 1435 ms. Trace:
I20260709 12:25:29.178697 13643 rpcz_store.cc:276] 0709 12:25:27.726078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.152939 (+1426861us) service_pool.cc:224] Handling call
0709 12:25:29.153358 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.154026 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.154241 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.116541 23509 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:29.120543 23496 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:29.162671 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.170362 13650 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:29.171725 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:29.184226 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.177551 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:29.152283 13799 rpcz_store.cc:276] 0709 12:25:27.779447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.905310 (+125863us) service_pool.cc:224] Handling call
0709 12:25:27.905644 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:27.937582 (+ 31938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.937619 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:27.938335 (+   716us) log.cc:844] Serialized 52 byte log entry
0709 12:25:28.177282 (+238947us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b180031eee0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:25:28.212375 (+ 35093us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:28.212422 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.212518 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.146728 (+934210us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:29.151562 (+  4834us) raft_consensus.cc:1685] finished
0709 12:25:29.151585 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.151773 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31474,"raft.queue_time_us":32407,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":34886,"spinlock_wait_cycles":105953920,"thread_start_us":34851,"threads_started":1,"wal-append.queue_time_us":290042}
I20260709 12:25:29.180032 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:29.180871 23284 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 12:25:29.187110 13643 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:25:29.187719 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4927) took 1442 ms. Trace:
I20260709 12:25:29.143266 23397 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:29.189158 23493 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.454s	user 0.000s	sys 0.007s
I20260709 12:25:29.171722 13800 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 18.
I20260709 12:25:29.161983 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1561) took 1439 ms. Trace:
I20260709 12:25:29.196027 13652 rpcz_store.cc:276] 0709 12:25:27.722190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.132543 (+1410353us) service_pool.cc:224] Handling call
0709 12:25:29.132919 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.161742 (+ 28823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.161929 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.188834 13643 rpcz_store.cc:276] 0709 12:25:27.745047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.186493 (+1441446us) service_pool.cc:224] Handling call
0709 12:25:29.186804 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.187476 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.187662 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.194514 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:29.195519 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.201730 13798 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:29.207785 13650 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:29.205113 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:29.209369 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4932) took 1158 ms. Trace:
I20260709 12:25:29.209913 13650 rpcz_store.cc:276] 0709 12:25:28.051180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.207138 (+1155958us) service_pool.cc:224] Handling call
0709 12:25:29.207515 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.208917 (+  1402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.209302 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.212128 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:29.210210 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:29.210052 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.215672 13650 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:29.216928 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.212278 13797 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:29.175561 23492 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:25:29.219189 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3055) took 1685 ms. Trace:
I20260709 12:25:29.220264 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:29.204005 23414 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:29.222075 13645 rpcz_store.cc:276] 0709 12:25:27.533857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.533982 (+   125us) service_pool.cc:224] Handling call
0709 12:25:27.534304 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.657372 (+123068us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:25:27.753126 (+ 95754us) log.cc:844] Serialized 14 byte log entry
0709 12:25:28.069864 (+316738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:28.069927 (+    63us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:28.069975 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.218909 (+1148934us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54001b6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:29.218979 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.219123 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25581,"prepare.queue_time_us":204856,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":3546,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3715,"spinlock_wait_cycles":282435328,"thread_start_us":355127,"threads_started":2,"wal-append.queue_time_us":361231,"wal-append.run_cpu_time_us":1193,"wal-append.run_wall_time_us":1002010}
I20260709 12:25:29.190838 23540 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:29.233608 13645 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:25:29.262693 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4935) took 1211 ms. Trace:
I20260709 12:25:29.211409 23358 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:29.225502 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:29.181160 23300 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:25:29.269416 13650 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:29.208318 23334 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:29.213750 23530 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.189718 23408 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.263262 13645 rpcz_store.cc:276] 0709 12:25:28.051701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.230024 (+1178323us) service_pool.cc:224] Handling call
0709 12:25:29.233215 (+  3191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.262225 (+ 29010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.262617 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.213521 23538 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:29.218984 23255 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:29.193737 23509 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.307308 13797 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:29.319816 13797 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:29.228874 23422 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:29.219437 23496 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:29.235910 23455 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:29.318867 13650 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:29.320950 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4438) took 4483 ms. Trace:
I20260709 12:25:29.531376 13643 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:29.322511 13795 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:29.324153 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1551) took 1650 ms. Trace:
I20260709 12:25:29.533558 13644 rpcz_store.cc:276] 0709 12:25:27.673549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.433265 (+759716us) service_pool.cc:224] Handling call
0709 12:25:28.795880 (+362615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.804797 (+  8917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.323863 (+519066us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b5400198550. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:29.324050 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814595328}
I20260709 12:25:29.313735 23538 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.556313 23538 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:29.557205 23492 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:29.293730 23355 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:25:29.351409 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.422801 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.559474 23492 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:29.560106 23492 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:29.475127 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 258387us
I20260709 12:25:29.562887 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:29.437296 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.565194 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:29.320634 13797 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:29.327584 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3059) took 1659 ms (client timeout 2200 ms). Trace:
I20260709 12:25:29.566550 23255 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:29.566923 13797 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:29.318722 23519 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.567423 23446 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:29.568822 23255 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:29.569583 23519 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:29.569653 23255 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:29.560369 23513 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:29.570712 23513 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:29.570751 23255 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.571612 23513 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.572734 23513 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:29.570783 23496 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:29.574627 23495 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.004s
I20260709 12:25:29.575104 23495 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:29.304476 23397 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.575979 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4921) took 1959 ms. Trace:
I20260709 12:25:29.576613 13649 rpcz_store.cc:276] 0709 12:25:27.616411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.186664 (+570253us) service_pool.cc:224] Handling call
0709 12:25:28.187024 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.866821 (+679797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.575624 (+708803us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54001b6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:29.575893 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537674624}
I20260709 12:25:29.265007 23500 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.580899 23500 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:29.568588 23446 consensus_queue.cc:1243] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 29.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:29.536906 13650 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:29.538763 23550 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:25:29.538604 13644 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:29.531878 13793 rpcz_store.cc:276] 0709 12:25:24.829902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.830054 (+   152us) service_pool.cc:224] Handling call
0709 12:25:24.839978 (+  9924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.980841 (+140863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.980876 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.198041 (+1217165us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b48001b0258. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:27.049540 (+851499us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:25:27.112147 (+ 62607us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.446847 (+334700us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:27.446889 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.447042 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.187761 (+1740719us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:29.312837 (+125076us) raft_consensus.cc:1685] finished
0709 12:25:29.312863 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.313009 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28460,"raft.queue_time_us":3944,"raft.run_cpu_time_us":21786,"raft.run_wall_time_us":1353875,"spinlock_wait_cycles":74936704,"thread_start_us":426931,"threads_started":2,"wal-append.queue_time_us":523460}
I20260709 12:25:29.546193 23403 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:29.547664 23256 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.550777 23455 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:29.551230 23511 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:29.553030 23509 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:29.313653 23530 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:29.560101 23129 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:29.534942 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3061) took 1937 ms. Trace:
I20260709 12:25:29.649897 13648 rpcz_store.cc:276] 0709 12:25:27.597519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.092538 (+495019us) service_pool.cc:224] Handling call
0709 12:25:29.534694 (+1442156us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400096d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:29.534879 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776035072}
I20260709 12:25:29.535559 13646 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:29.585628 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4941) took 1111 ms. Trace:
I20260709 12:25:29.651526 13650 rpcz_store.cc:276] 0709 12:25:28.474584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.535862 (+1061278us) service_pool.cc:224] Handling call
0709 12:25:29.536697 (+   835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.584408 (+ 47711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.585512 (+  1104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.653036 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:29.653870 13650 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
W20260709 12:25:29.566632 13792 rpcz_store.cc:269] 0709 12:25:27.667955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.676418 (+  8463us) service_pool.cc:224] Handling call
0709 12:25:29.327362 (+1650944us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54000d20d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:29.327504 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271761152}
I20260709 12:25:29.589051 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:29.588887 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:29.656332 13721 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:29.590446 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:29.657649 13719 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:29.580158 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4942) took 1105 ms. Trace:
I20260709 12:25:29.659307 13649 rpcz_store.cc:276] 0709 12:25:28.474851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.577598 (+1102747us) service_pool.cc:224] Handling call
0709 12:25:29.578249 (+   651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.579769 (+  1520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.579797 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:29.579839 (+    42us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:29.579869 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.579920 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.580098 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.661360 13649 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:29.662153 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3077) took 1056 ms. Trace:
I20260709 12:25:29.663245 13649 rpcz_store.cc:276] 0709 12:25:28.605853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.660640 (+1054787us) service_pool.cc:224] Handling call
0709 12:25:29.661089 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.661825 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.662077 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.590831 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.591113 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:29.262877 23284 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:29.591559 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:29.684602 13724 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 18: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:29.595542 13795 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:29.614986 13797 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:25:29.622205 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.626400 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4937) took 1567 ms. Trace:
I20260709 12:25:29.476401 23300 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:29.630690 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.633843 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:29.685339 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:29.686182 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4472) took 1917 ms. Trace:
I20260709 12:25:29.689633 13795 rpcz_store.cc:276] 0709 12:25:27.768488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.778978 (+ 10490us) service_pool.cc:224] Handling call
0709 12:25:27.779314 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.952496 (+1173182us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b18000038a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393c2f646 00007f5393b6958e 00007f5393beaf9e 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4
0709 12:25:29.322238 (+369742us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b58008417d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b500aa 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4
0709 12:25:29.685817 (+363579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.686110 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303692288}
I20260709 12:25:29.686429 13644 rpcz_store.cc:276] 0709 12:25:28.058893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.537811 (+1478918us) service_pool.cc:224] Handling call
0709 12:25:29.538400 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.621374 (+ 82974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.626326 (+  4952us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.784066 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:29.790900 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 200154us
I20260709 12:25:29.792297 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.793115 13651 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:29.795480 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:29.798770 13724 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:25:29.799052 13721 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:29.800320 13721 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:25:29.800611 13725 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:29.801561 13721 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:29.801826 13725 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:29.835873 13719 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:29.621104 23334 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.847647 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 225539us
I20260709 12:25:29.848685 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:29.622453 23403 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:29.856913 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:29.860592 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173415us
I20260709 12:25:30.042181 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4384) took 1750 ms. Trace:
I20260709 12:25:29.782725 23129 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:29.579975 23492 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:29.901238 23256 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:29.630187 23511 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:29.809320 23450 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:30.045212 13722 rpcz_store.cc:276] 0709 12:25:28.291276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.296695 (+  5419us) service_pool.cc:224] Handling call
0709 12:25:28.927415 (+630720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.940395 (+ 12980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.025917 (+1085522us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54003fedd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.042111 (+ 16194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449064320}
W20260709 12:25:30.046368 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.049016 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4944) took 1264 ms. Trace:
I20260709 12:25:29.797456 23284 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.088793 23284 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:30.090891 23414 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.091620 23414 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.091028 23511 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.095117 23424 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:30.094897 23129 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.095805 23424 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.096477 23129 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:30.098677 23424 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:29.809863 23300 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.056334 13648 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:25:30.100265 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3082) took 1146 ms. Trace:
I20260709 12:25:30.100678 13648 rpcz_store.cc:276] 0709 12:25:28.953751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.851271 (+897520us) service_pool.cc:224] Handling call
0709 12:25:30.050292 (+199021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.099912 (+ 49620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.100189 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:30.101429 23300 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.053797 23530 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:30.094051 23414 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.102625 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1801) took 1053 ms. Trace:
I20260709 12:25:30.105450 23509 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:30.105304 14420 rpcz_store.cc:276] 0709 12:25:29.049418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.056372 (+  6954us) service_pool.cc:224] Handling call
0709 12:25:29.084268 (+ 27896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.084798 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.084837 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.105423 (+ 20586us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.182091 (+ 76668us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:25:29.182133 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.182207 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.102340 (+920133us) raft_consensus.cc:1685] finished
0709 12:25:30.102367 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.102545 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":76338,"threads_started":1,"wal-append.queue_time_us":106101}
I20260709 12:25:30.041706 23486 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:30.084064 13644 rpcz_store.cc:276] 0709 12:25:28.784745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.796121 (+1011376us) service_pool.cc:224] Handling call
0709 12:25:30.048210 (+252089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.048533 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.048570 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.048633 (+    63us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:30.048681 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.048760 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.048935 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:29.901747 23334 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.077210 23450 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:30.123672 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.109400 23286 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.021s	user 0.010s	sys 0.002s
I20260709 12:25:30.113648 23300 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:30.127211 23286 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.127686 22920 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.098s	user 0.014s	sys 0.001s
I20260709 12:25:30.128626 13796 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:30.130543 23334 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.132019 23408 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:30.132756 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.127715 23286 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.136533 23286 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.110663 23235 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:30.109253 23486 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:30.111652 23519 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.118459 23509 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:30.122181 23282 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.141263 23450 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:30.142355 23539 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.079s	sys 0.178s
I20260709 12:25:30.143105 23235 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:30.157599 23282 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:30.159602 23539 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.079s	sys 0.178s
I20260709 12:25:30.115420 23403 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.149732 23519 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.192030 23519 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.173969 23235 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.183612 23403 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:30.206573 23239 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:30.207155 23239 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:30.208267 23239 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:30.204067 23552 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 12:25:30.193701 23129 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:30.178992 23486 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.215075 23239 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:30.237944 23396 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:30.242192 23540 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:30.242654 23540 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:30.243527 23540 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.244890 23540 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:30.242074 23500 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.002s
I20260709 12:25:30.249166 23443 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:25:30.249630 23443 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:30.234040 23486 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:30.251518 23562 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.001s
W20260709 12:25:30.252542 23553 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:25:30.264290 23443 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.265796 23443 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:25:30.272102 23513 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.003s
W20260709 12:25:30.253293 23538 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.696s	user 0.014s	sys 0.000s
I20260709 12:25:30.283396 23538 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:28.459244 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:30.325807 13644 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:30.327692 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3084) took 1373 ms. Trace:
W20260709 12:25:30.328137 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 471122us
I20260709 12:25:30.328251 13644 rpcz_store.cc:276] 0709 12:25:28.958635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.112589 (+1153954us) service_pool.cc:224] Handling call
0709 12:25:30.324958 (+212369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.327051 (+  2093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.327505 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:30.330210 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:30.333323 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:30.335179 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:30.337981 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:30.339679 13644 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:25:30.345350 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:30.348793 13726 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:30.349407 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3085) took 1390 ms. Trace:
W20260709 12:25:30.354544 13799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.003s	sys 0.015s
W20260709 12:25:30.355228 13799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.015s
I20260709 12:25:30.356076 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1811) took 1185 ms. Trace:
I20260709 12:25:30.357234 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1574) took 1188 ms. Trace:
I20260709 12:25:30.358100 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1812) took 1151 ms. Trace:
I20260709 12:25:30.358703 14418 rpcz_store.cc:276] 0709 12:25:29.206184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.206496 (+   312us) service_pool.cc:224] Handling call
0709 12:25:29.206777 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.213400 (+  6623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.213458 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.221231 (+  7773us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.626668 (+405437us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:29.626715 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.626786 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.357546 (+730760us) raft_consensus.cc:1685] finished
0709 12:25:30.357593 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.357882 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7418,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":6588,"thread_start_us":402459,"threads_started":1,"wal-append.queue_time_us":916457}
I20260709 12:25:30.362525 14424 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: []
W20260709 12:25:30.157415 14390 connection.cc:256] RPC call timeout handler was delayed by -1.0900649600002907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 12:25:30.353284 13644 rpcz_store.cc:276] 0709 12:25:28.959127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.330000 (+1370873us) service_pool.cc:224] Handling call
0709 12:25:30.339185 (+  9185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.349010 (+  9825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.349321 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:30.364480 22809 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.356621 14416 rpcz_store.cc:276] 0709 12:25:29.169879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.170070 (+   191us) service_pool.cc:224] Handling call
0709 12:25:29.170568 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.180020 (+  9452us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:29.208728 (+ 28708us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.312562 (+103834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.314071 (+  1509us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.314216 (+   145us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:29.314250 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.314334 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.203439 (+889105us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:30.355093 (+151654us) raft_consensus.cc:1685] finished
0709 12:25:30.355309 (+   216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.355688 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23134,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":5658,"raft.queue_time_us":1639,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":29223,"spinlock_wait_cycles":81635072,"thread_start_us":88404,"threads_started":1,"wal-append.queue_time_us":567972}
I20260709 12:25:30.357872 13799 rpcz_store.cc:276] 0709 12:25:29.168382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.562360 (+393978us) service_pool.cc:224] Handling call
0709 12:25:29.562685 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.562974 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:29.788694 (+225720us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.356230 (+567536us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.356380 (+   150us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:30.356525 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.356679 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.357003 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":541124,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"thread_start_us":791976,"threads_started":2,"wal-append.queue_time_us":396362}
I20260709 12:25:30.372355 22809 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.400305 23282 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:30.483366 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112551us
I20260709 12:25:30.492822 23347 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.494032 23347 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.498835 23446 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:30.499553 23446 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.500022 23446 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:30.500502 23519 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:30.506662 23403 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:30.508082 23403 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.510423 23539 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:30.510970 23541 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:30.511658 23541 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.512147 23541 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:30.510468 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.005s	sys 0.012s
I20260709 12:25:30.518630 23455 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.519636 23455 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.521088 23538 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.522671 23551 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.523743 23551 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.491958 23358 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:30.518652 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.012s
W20260709 12:25:30.528844 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.532415 13723 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:30.538206 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.539178 23547 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.556471 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3083) took 1602 ms. Trace:
W20260709 12:25:30.557641 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 765253us
I20260709 12:25:30.557168 13648 rpcz_store.cc:276] 0709 12:25:28.953998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.101653 (+1147655us) service_pool.cc:224] Handling call
0709 12:25:30.102110 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.110274 (+  8164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.110309 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.113564 (+  3255us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.482844 (+369280us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:30.482947 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.485028 (+  2081us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.555862 (+ 70834us) raft_consensus.cc:1685] finished
0709 12:25:30.555887 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.556197 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76544,"thread_start_us":158466,"threads_started":1,"wal-append.queue_time_us":219213}
I20260709 12:25:30.563501 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4928) took 2781 ms. Trace:
I20260709 12:25:30.564127 13652 rpcz_store.cc:276] 0709 12:25:27.781576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.197028 (+1415452us) service_pool.cc:224] Handling call
0709 12:25:29.197359 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.296642 (+ 99283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.296678 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:29.296726 (+    48us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:29.296758 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.562739 (+1265981us) spinlock_profiling.cc:243] Waited 99.1 ms on lock 0x7b54001b5654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:30.562896 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.563223 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":227846400}
W20260709 12:25:30.566217 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.498104 23347 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.575030 13645 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:30.564846 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:30.570993 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3068) took 1970 ms (client timeout 2227 ms). Trace:
I20260709 12:25:30.583688 23571 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:30.584391 23378 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.585184 23378 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.586743 23566 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.584903 23571 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.580824 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4930) took 2649 ms. Trace:
I20260709 12:25:30.586297 13727 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:30.597960 23256 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.014s
I20260709 12:25:30.594854 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:30.596242 23378 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.597303 23455 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.589918 23566 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:30.599503 23573 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.078s	user 0.000s	sys 0.002s
W20260709 12:25:30.585686 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.577847 13651 rpcz_store.cc:269] 0709 12:25:28.600788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.087209 (+486421us) service_pool.cc:224] Handling call
0709 12:25:30.570407 (+1483198us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b5400097c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.570699 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1089292032}
I20260709 12:25:30.587261 23554 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.606313 23539 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:30.606781 23538 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.542122 23547 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.609843 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1513) took 1989 ms. Trace:
I20260709 12:25:30.610347 23551 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:30.614296 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.615365 14419 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:25:30.611451 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1509) took 1999 ms. Trace:
W20260709 12:25:30.615235 22677 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 29 current term: 29 is higher.
I20260709 12:25:30.616557 23570 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:30.611157 23561 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:30.605700 13643 rpcz_store.cc:276] 0709 12:25:27.908480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.206311 (+1297831us) service_pool.cc:224] Handling call
0709 12:25:29.206657 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.532174 (+325517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.557626 (+1025452us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:30.558091 (+   465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":737069312}
I20260709 12:25:30.623749 14422 rpcz_store.cc:276] 0709 12:25:28.620213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.620379 (+   166us) service_pool.cc:224] Handling call
0709 12:25:30.609345 (+1988966us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5400555fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.609594 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27357056}
I20260709 12:25:30.624145 14423 rpcz_store.cc:276] 0709 12:25:28.612258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.612460 (+   202us) service_pool.cc:224] Handling call
0709 12:25:28.659503 (+ 47043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.708501 (+ 48998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.708540 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:28.708586 (+    46us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:28.708616 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.610803 (+1902187us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:30.610931 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.611229 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112093952}
I20260709 12:25:30.617767 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:30.642002 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4137) took 1053 ms. Trace:
I20260709 12:25:30.642329 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:25:30.631841 23403 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.647333 13727 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:25:30.642827 13723 rpcz_store.cc:276] 0709 12:25:29.588028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.589460 (+  1432us) service_pool.cc:224] Handling call
0709 12:25:30.638779 (+1049319us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54001563d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.641809 (+  3030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815616896}
I20260709 12:25:30.654732 23554 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:30.656170 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2030) took 3463 ms (client timeout 2228 ms). Trace:
I20260709 12:25:30.656359 23570 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:30.656818 13726 rpcz_store.cc:269] 0709 12:25:27.192540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.192683 (+   143us) service_pool.cc:224] Handling call
0709 12:25:30.655734 (+3463051us) spinlock_profiling.cc:243] Waited 2.93 s on lock 0x7b5400174654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.655973 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6731633408}
I20260709 12:25:30.657696 23554 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.651135 13727 pending_rounds.cc:85] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:25:30.643440 23561 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:30.659749 13727 pending_rounds.cc:107] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:25:30.649607 23538 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.667318 13727 raft_consensus.cc:2889] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:30.665664 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112607us
I20260709 12:25:30.650743 23255 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.663266 23539 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.679531 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:30.682957 23397 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:30.687978 23336 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.688024 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:30.688856 23336 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.689808 13723 raft_consensus.cc:2410] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 12:25:30.694403 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4500) took 1569 ms. Trace:
I20260709 12:25:30.681720 23566 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.685967 23570 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.702826 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:30.695951 13801 rpcz_store.cc:276] 0709 12:25:29.121665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.209247 (+ 87582us) service_pool.cc:224] Handling call
0709 12:25:29.222766 (+ 13519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.071195 (+848429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.071246 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.071915 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.492861 (+420946us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:30.492957 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.690989 (+198032us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b5400288cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:30.691041 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.691111 (+    70us) raft_consensus.cc:1685] finished
0709 12:25:30.691127 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.691470 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30995,"raft.queue_time_us":30083,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":23993,"spinlock_wait_cycles":1464068992,"thread_start_us":368307,"threads_started":1,"wal-append.queue_time_us":452827}
I20260709 12:25:30.696774 23499 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:30.687166 23255 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.709918 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:30.711790 13723 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:30.711673 23419 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:30.717018 23235 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:30.713914 13766 connection.cc:256] RPC call timeout handler was delayed by -1.1773983480002244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
I20260709 12:25:30.706032 23261 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:30.721390 23336 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.725263 23261 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:30.728107 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.712920 23547 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.733316 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:30.737283 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:30.739398 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.751859 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.754608 23561 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.759307 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4936) took 2702 ms. Trace:
I20260709 12:25:30.759699 23538 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.760485 23500 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:30.761224 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 4925) took 3063 ms. Trace:
I20260709 12:25:30.755518 22809 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.752149 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:30.761796 13646 rpcz_store.cc:276] 0709 12:25:27.697268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.517384 (+820116us) service_pool.cc:224] Handling call
0709 12:25:29.535361 (+1017977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.783508 (+248147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.760617 (+977109us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400096d50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.760873 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340464896}
I20260709 12:25:30.741546 23571 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:30.759779 13645 rpcz_store.cc:276] 0709 12:25:28.057181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.309028 (+1251847us) service_pool.cc:224] Handling call
0709 12:25:30.574496 (+1265468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.575939 (+  1443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.759041 (+183102us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400097c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.759235 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2909286784}
I20260709 12:25:30.763170 13723 raft_consensus.cc:2376] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31. Re-sending same reply.
I20260709 12:25:30.761274 23500 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:30.776481 23450 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.007s
I20260709 12:25:30.777307 23450 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.759361 23539 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.779294 23539 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.780224 23492 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:30.780047 23539 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.780791 23450 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:30.785291 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.794701 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1412) took 2032 ms. Trace:
I20260709 12:25:30.795521 14419 rpcz_store.cc:276] 0709 12:25:28.762087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.762309 (+   222us) service_pool.cc:224] Handling call
0709 12:25:30.614534 (+1852225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.630366 (+ 15832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.789041 (+158675us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b5400555fd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.794552 (+  5511us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4252386048}
I20260709 12:25:30.789453 23500 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.802567 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:30.800513 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.800747 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:30.804373 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.804687 14416 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:30.804920 23419 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.805197 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:30.805714 23419 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.806838 14418 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:30.808743 23419 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.800385 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.782861 23255 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.813225 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:30.815200 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119978us
W20260709 12:25:30.816923 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.818428 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.818964 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.819365 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.819690 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.817515 23396 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.564s	user 0.025s	sys 0.020s
I20260709 12:25:30.820767 23335 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.820837 13794 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
W20260709 12:25:30.821987 23577 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:25:30.822966 22860 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.824060 22860 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:30.825433 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.826891 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.830720 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:30.830871 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.831130 23261 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.833228 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1522) took 1632 ms. Trace:
I20260709 12:25:30.836994 23547 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.837900 23547 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:30.838645 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.838667 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:30.840590 23547 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.821662 23335 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.843190 23129 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:30.844754 22860 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.833846 14424 rpcz_store.cc:276] 0709 12:25:29.201120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.201398 (+   278us) service_pool.cc:224] Handling call
0709 12:25:29.201741 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.363375 (+1161634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.832885 (+469510us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:30.833159 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2667445632}
W20260709 12:25:30.849530 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3073) took 2247 ms (client timeout 2104 ms). Trace:
W20260709 12:25:30.850332 13647 rpcz_store.cc:269] 0709 12:25:28.604645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.970830 (+366185us) service_pool.cc:224] Handling call
0709 12:25:30.848833 (+1878003us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b54001e5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.849221 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137148160}
I20260709 12:25:30.850720 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.853103 23511 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:30.855785 13801 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:25:30.855903 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4506) took 1694 ms. Trace:
I20260709 12:25:30.856163 23256 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.856400 13796 rpcz_store.cc:276] 0709 12:25:29.161030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.550325 (+389295us) service_pool.cc:224] Handling call
0709 12:25:29.550619 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.128971 (+578352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.855359 (+726388us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54001004d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:30.855688 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328439168}
I20260709 12:25:30.856779 23256 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.862990 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:30.863449 23256 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:30.864693 13542 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 14 to 16, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 16 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:30.866690 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4136) took 1278 ms. Trace:
I20260709 12:25:30.867997 22860 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.867112 13720 rpcz_store.cc:276] 0709 12:25:29.587723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.588690 (+   967us) service_pool.cc:224] Handling call
0709 12:25:29.588996 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.717309 (+1128313us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b18004ca080. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393becbfb 00007f5393bf764b
0709 12:25:30.717710 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.717740 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.724121 (+  6381us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.739544 (+ 15423us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:30.739611 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.739759 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.866080 (+126321us) raft_consensus.cc:1685] finished
0709 12:25:30.866114 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.866455 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30764,"raft.queue_time_us":257934,"raft.run_cpu_time_us":11651,"raft.run_wall_time_us":201252,"spinlock_wait_cycles":29482240,"thread_start_us":481485,"threads_started":2,"wal-append.queue_time_us":44389}
I20260709 12:25:30.868682 22860 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:30.870559 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 65520.8us
I20260709 12:25:30.871961 23568 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:30.873759 23443 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.607s	user 0.018s	sys 0.000s
I20260709 12:25:30.874500 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:30.874953 23547 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:30.872539 23568 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:30.876430 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:30.877537 23568 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.878990 23335 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:30.874646 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3089) took 1755 ms. Trace:
W20260709 12:25:30.886993 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.884850 13794 rpcz_store.cc:276] 0709 12:25:29.119481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.119683 (+   202us) service_pool.cc:224] Handling call
0709 12:25:30.874234 (+1754551us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54000e5bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:30.874459 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3909583232}
W20260709 12:25:30.887986 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 89471.7us
W20260709 12:25:30.893734 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58701.5us
I20260709 12:25:30.894968 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:30.895759 23419 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.896524 23419 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.900946 14416 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:30.899943 23347 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:25:30.902732 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.906041 13647 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:30.878806 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:30.910641 23499 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:30.910501 13542 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:30.911360 22860 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:30.912192 13643 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:25:30.911366 23499 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.912480 13542 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 14 to 21, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 21 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:30.911398 23336 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.915019 13646 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:30.936894 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.900213 23547 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.939584 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.935681 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:30.908156 23539 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.927457 23443 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:25:30.949164 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.953400 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:30.961462 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.915165 23336 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.968794 14424 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
W20260709 12:25:30.970479 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.969981 14420 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:30.966535 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:30.976888 23539 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.977869 23539 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:30.980144 23235 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:30.981331 23539 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.981808 23235 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:30.992471 23235 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:30.993300 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:30.994418 14417 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:25:30.982719 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 104106us
I20260709 12:25:30.998523 22677 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:30.998848 23396 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:30.998586 23335 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:25:30.984406 13647 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:30.986442 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.001881 13649 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:30.985517 23419 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.998570 23239 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:30.994158 23235 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.007859 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:30.973706 23547 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.016296 13650 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:30.952400 22860 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.019007 22860 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:31.021986 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.023307 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.024355 23358 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:31.025089 23439 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.026026 23439 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:30.958541 23256 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:31.030732 23439 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:30.991341 23446 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.035570 13542 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 14 to 15, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 15 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:31.031755 23256 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:31.059293 23587 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.964429 23499 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.032538 23519 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:31.063174 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.067903 13727 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:31.057080 23235 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:31.053257 23419 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.062165 23256 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:31.088958 13719 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:31.084838 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.087090 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.071846 23568 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.095849 23588 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.096312 23568 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.096797 23588 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.098232 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3089) took 1511 ms. Trace:
I20260709 12:25:31.098697 13644 rpcz_store.cc:276] 0709 12:25:29.586592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.378515 (+791923us) service_pool.cc:224] Handling call
0709 12:25:30.379172 (+   657us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:30.380112 (+   940us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:30.483661 (+103549us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.653121 (+169460us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.653398 (+   277us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:30.675749 (+ 22351us) log.cc:844] Serialized 52 byte log entry
0709 12:25:30.675890 (+   141us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:30.676054 (+   164us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.680680 (+  4626us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.692500 (+ 11820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.692600 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.829486 (+136886us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b18004c3f60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:30.842556 (+ 13070us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b18004c3f60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:30.845562 (+  3006us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b18004c3f60. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:31.097925 (+252363us) raft_consensus.cc:1685] finished
0709 12:25:31.097949 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.098133 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17951,"prepare.queue_time_us":554751,"prepare.run_cpu_time_us":215,"prepare.run_wall_time_us":4207,"raft.queue_time_us":112804,"raft.run_cpu_time_us":1111,"raft.run_wall_time_us":38389,"spinlock_wait_cycles":19781504,"thread_start_us":195930,"threads_started":2,"wal-append.queue_time_us":192285}
I20260709 12:25:31.100476 23587 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.078053 23336 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:31.090276 23541 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.010s
I20260709 12:25:31.105441 13722 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:31.107681 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.109133 23588 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.103799 23419 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.111485 22677 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.107285 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 212489us
I20260709 12:25:31.128036 22677 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.130704 13718 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:31.133163 23587 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.133855 23587 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.134913 23587 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.102505 23396 consensus_queue.cc:1243] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:25:31.124684 23588 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:31.125788 23261 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.138656 23261 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.140625 23335 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:31.141132 23335 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:31.142808 23397 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:31.143976 23587 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.144136 23397 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:31.146198 23500 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:31.146525 23499 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:31.147050 23499 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:31.145747 23397 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:25:31.148304 23499 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.150571 13542 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:31.154454 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:31.155269 23239 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:31.155722 23239 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:31.156744 23239 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.157493 23129 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:31.126806 23419 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.157006 23397 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:31.115873 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:31.142185 23335 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.135725 22860 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.158756 23439 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:31.169658 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:31.169971 23439 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:31.171233 23439 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.172773 23439 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.173121 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:31.149930 23499 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:31.176007 23446 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.158249 23239 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:31.161852 23261 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.162025 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.187861 13720 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:31.162132 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:31.192109 23129 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:31.192781 23129 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.193207 23129 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:31.194591 23492 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:31.164183 23568 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.165732 23539 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:31.203737 23539 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:31.205068 23539 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.165414 23541 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.194957 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:31.166178 23335 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:31.215462 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:31.217060 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:31.198588 23336 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:31.217888 23261 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:31.218765 23336 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:31.220327 23336 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.162950 23256 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.222450 23256 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.174556 23397 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:31.226788 23397 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:31.228123 23397 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.163125 23547 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.163303 22677 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.153157 13542 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 18 to 19, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 19 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:31.158447 23584 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:25:31.165218 23255 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.232539 23255 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.232235 13542 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 18 to 21, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 21 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:31.163772 23519 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.234642 23519 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.235203 23255 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:31.165833 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101007us
I20260709 12:25:31.236691 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.236974 23519 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.165104 23540 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.238018 23540 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.238834 23519 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:31.239234 23519 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.239591 23519 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:31.240674 23540 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.242940 13542 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 23 to 24, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:31.244863 13542 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 13 to 18, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 18 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:31.246933 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 74019.6us
I20260709 12:25:31.246793 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.247761 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.164638 23358 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.249639 23358 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.251206 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.252072 23358 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.252409 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:31.169095 13541 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 242) took 44770 ms (client timeout 15000 ms). Trace:
W20260709 12:25:31.238461 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.210000 23541 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:31.275513 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58562.1us
I20260709 12:25:31.264308 23492 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:31.278353 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.314035 13542 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 13 to 17, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 17 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:31.320775 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.314149 23584 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:25:31.326385 13726 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:25:31.326527 23584 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.327134 23584 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:31.327433 13542 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 17 to 25. New cstate: current_term: 25 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:31.330991 13720 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:31.329334 13542 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 19 to 22, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 22 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:31.325706 14424 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:31.345677 23419 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.346705 23419 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.322297 23547 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:31.350634 23547 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.350378 23541 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.351292 23547 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.351673 23541 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.356066 23541 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.357230 23419 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.359905 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:31.361085 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.362722 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.364617 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.365541 13647 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:31.362861 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.323770 23129 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.454411 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:31.455639 13643 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:31.456580 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 403708us
I20260709 12:25:31.454409 13542 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:25:31.463650 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 430942us
I20260709 12:25:31.477685 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.477803 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.487385 13726 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 21. Ops: []
I20260709 12:25:31.496048 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.497954 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.499032 13648 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:31.500094 23239 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.500856 23239 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.502418 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:31.503253 13645 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:31.503721 23239 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.504065 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.505352 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:31.505909 13645 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:31.515463 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.519129 13726 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:31.520855 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.523756 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.480738 23261 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:31.478157 23511 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.484548 23561 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:31.527112 13801 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:25:31.527848 13542 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 20 to 25, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 25 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:31.534883 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.535931 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.580258 13647 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 20.
I20260709 12:25:31.537388 13719 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:31.538275 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.473433 23335 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:31.517632 23450 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.545143 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:31.545109 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70720.5us
I20260709 12:25:31.546945 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.550596 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:31.550864 13718 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:31.558368 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.558830 13645 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:31.559406 13723 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:25:31.586364 13723 pending_rounds.cc:85] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 17
I20260709 12:25:31.586666 13723 pending_rounds.cc:107] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:25:31.586910 13723 raft_consensus.cc:2889] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.582388 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:31.561981 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:31.562319 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.585355 14418 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:31.598814 13651 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:31.593055 13648 raft_consensus.cc:2376] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22. Re-sending same reply.
W20260709 12:25:31.265563 13541 rpcz_store.cc:269] 0709 12:24:46.398659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.374810 (+12976151us) service_pool.cc:224] Handling call
0709 12:24:59.932148 (+557338us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390ad0640 00007f53999a7846 00007f5399ab190e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:31.168776 (+31236628us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:21.554355 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:21.555381 (+  1026us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:21.555471 (+    90us) write_op.cc:435] Acquired schema lock
0709 12:25:21.555499 (+    28us) tablet.cc:662] Decoding operations
0709 12:25:21.656942 (+101443us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:21.657061 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:21.657089 (+    28us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:25:21.658852 (+  1763us) tablet.cc:701] Row locks acquired
0709 12:25:21.658897 (+    45us) write_op.cc:260] PREPARE: finished
0709 12:25:21.659103 (+   206us) write_op.cc:270] Start()
0709 12:25:21.659265 (+   162us) write_op.cc:276] Timestamp: P: 1783599921659013 usec, L: 0
0709 12:25:21.659298 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:25:21.660136 (+   838us) log.cc:844] Serialized 7486 byte log entry
0709 12:25:22.313244 (+653108us) op_driver.cc:464] REPLICATION: finished
0709 12:25:22.520083 (+206839us) write_op.cc:301] APPLY: starting
0709 12:25:22.522784 (+  2701us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:22.766461 (+243677us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:22.774578 (+  8117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:22.876859 (+102281us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:22.877288 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:25:22.877349 (+    61us) write_op.cc:312] APPLY: finished
0709 12:25:22.877835 (+   486us) log.cc:844] Serialized 172 byte log entry
0709 12:25:22.878873 (+  1038us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:22.879153 (+   280us) write_op.cc:454] Released schema lock
0709 12:25:22.879241 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:22.880244 (+  1003us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:22.880296 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":472730,"reported_tablets":50,"spinlock_wait_cycles":132409088,"child_traces":[["op",{"apply.queue_time_us":185858,"apply.run_cpu_time_us":32749,"apply.run_wall_time_us":367180,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83699,"num_ops":19,"prepare.queue_time_us":326731,"prepare.run_cpu_time_us":61682,"prepare.run_wall_time_us":396442,"raft.queue_time_us":247297,"raft.run_cpu_time_us":6839,"raft.run_wall_time_us":83443,"replication_time_us":653730,"thread_start_us":641842,"threads_started":4,"wal-append.queue_time_us":362392,"wal-append.run_cpu_time_us":12120,"wal-append.run_wall_time_us":1858468}]]}
I20260709 12:25:31.619833 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:31.620882 13649 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:31.621395 13799 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:25:31.625707 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97315.6us
I20260709 12:25:31.624078 14417 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:31.627063 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.619561 13646 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:31.636714 13652 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:25:31.638381 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 315151us
I20260709 12:25:31.640553 13646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:31.641407 13646 raft_consensus.cc:606] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:31.641804 13646 raft_consensus.cc:614] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:31.642931 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.643726 23568 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:31.634045 13643 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
W20260709 12:25:31.632745 13794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.004s	sys 0.011s
W20260709 12:25:31.647329 13794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.004s	sys 0.011s
I20260709 12:25:31.627880 23358 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.647624 23588 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:31.648351 23358 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.649561 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.639551 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.652289 23129 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:31.632308 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.654614 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.654968 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.620697 23539 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.661540 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61044us
I20260709 12:25:31.644127 23568 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.589927 23335 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.662024 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:31.660223 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.662366 23511 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.597311 23450 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.654356 23235 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:25:31.679215 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 168795us
I20260709 12:25:31.680212 23539 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.681061 13690 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:31.661442 23129 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:31.686972 13721 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:31.687875 23255 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.688330 23511 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.688524 23255 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.689388 23446 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:31.689847 23446 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.690121 23335 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.690284 23446 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.690760 23129 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.690848 23255 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.691454 23129 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:31.691713 23450 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:31.692651 23255 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:31.693475 23255 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.688470 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.695649 13719 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:31.695984 23255 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:31.696926 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.697175 13795 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
W20260709 12:25:31.697546 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.702566 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.702612 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.704077 23419 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.698835 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.704792 23419 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.705292 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:31.706990 23358 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.707544 23419 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.704087 14418 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:25:31.707653 23358 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:31.708293 23358 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.708724 23358 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.705401 13796 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:25:31.671025 23540 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.712538 23540 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.679859 13542 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:31.681583 23235 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.717443 23235 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.718835 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:31.719326 23419 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:31.672277 23568 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:31.715423 23540 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.734879 13542 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 20 to 27, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 27 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:31.725807 23588 consensus_queue.cc:1243] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:25:31.660275 23584 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:31.747146 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95930.1us
W20260709 12:25:31.749058 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90490.3us
I20260709 12:25:31.757489 13650 raft_consensus.cc:606] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:31.757951 13650 raft_consensus.cc:614] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:31.749202 23519 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.509s	user 0.015s	sys 0.000s
I20260709 12:25:31.743542 23261 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:31.741294 23450 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:31.757095 13542 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:31.770521 23446 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.780206 23601 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.780879 23601 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.780534 13542 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 18 to 24, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 24 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:31.760027 23540 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 12:25:31.783239 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108696us
I20260709 12:25:31.784585 23601 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.784759 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:31.783119 23593 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:25:31.772687 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.785223 23129 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.792009 23601 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.792689 23601 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.797752 23588 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:31.798413 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:31.799753 23588 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:31.801038 23588 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:31.754835 22860 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:31.793984 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.790057 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.794831 23397 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:31.795295 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:31.808140 13794 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:31.807662 13726 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:31.741757 23547 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.789243 13719 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:31.778039 23261 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:31.839004 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.842206 13727 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:25:31.839382 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 132379us
W20260709 12:25:31.847229 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 145981us
I20260709 12:25:31.806890 22860 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:31.844767 13618 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.731s (SENT) [suppressed 12 similar messages]
I20260709 12:25:31.778326 23519 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.836633 23335 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:31.844177 23261 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:31.857023 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.862951 13794 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:31.841022 23500 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:25:31.864096 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.880059 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:31.849916 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48355.2us
I20260709 12:25:31.855268 23450 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:31.881170 13795 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:31.839067 23358 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:31.898468 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 226363us
I20260709 12:25:31.899482 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:31.889989 23492 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.023s	sys 0.031s
I20260709 12:25:31.876827 23593 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:25:31.912029 23492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.024s	sys 0.032s
I20260709 12:25:31.865751 23499 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:25:31.927112 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.910812 23500 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.897547 23335 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:25:31.940218 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:31.943960 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99243.1us
I20260709 12:25:31.928436 23608 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:31.948137 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.949023 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.899894 23450 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.924732 23593 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:31.946244 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55739.3us
I20260709 12:25:31.929441 23492 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:31.950213 13795 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:31.950594 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:31.959450 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.961050 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:31.932960 23605 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:31.945173 23600 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:25:31.977789 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:31.975356 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:31.981086 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:31.981951 14417 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:31.983915 13725 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 19. (term mismatch)
I20260709 12:25:31.984404 13725 pending_rounds.cc:85] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 18
I20260709 12:25:31.984665 13725 pending_rounds.cc:107] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 24.19
I20260709 12:25:31.984906 13725 raft_consensus.cc:2889] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:31.986738 23616 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.987509 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.987444 23616 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:31.989452 23547 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.989806 23616 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.990319 23547 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:31.990777 23547 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.990880 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.991297 23547 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:31.992395 23358 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.992810 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.993683 13800 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:31.994964 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.995709 13793 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:31.996640 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:31.997864 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:31.986739 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:31.989471 14421 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:31.999508 13724 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:32.000146 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:32.001070 23593 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.002592 23593 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:32.003218 23256 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:31.961148 23500 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:31.929668 22860 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.007687 22860 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:32.008778 23450 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:31.948307 23608 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:32.007826 23519 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:31.979549 14419 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:25:32.016330 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.990808 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:32.017136 14423 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.006439 13718 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
W20260709 12:25:32.013069 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.962383 23584 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.014616 13795 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
W20260709 12:25:32.018348 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53319.6us
I20260709 12:25:32.022183 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:32.026000 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:32.030694 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92111.9us
I20260709 12:25:31.979628 23605 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.042279 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.043396 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.057837 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.058838 13795 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
W20260709 12:25:32.061753 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93773.2us
I20260709 12:25:32.023774 23446 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.064725 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.038784 23619 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.069392 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.043761 23450 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:32.078289 13800 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:25:32.057677 23519 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.088589 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.089994 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.033509 23611 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:25:32.098376 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.109737 14423 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:32.110014 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:32.111096 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 192818us
I20260709 12:25:32.083842 23588 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.109400 13792 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:32.047006 23618 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.059s
I20260709 12:25:32.089767 23256 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:32.089573 23450 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:32.030774 23492 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.23, Last appended by leader: 23, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.125679 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:32.126760 13798 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:32.090723 23500 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.102123 23623 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:32.089499 23605 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:32.154505 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 357641us
I20260709 12:25:32.162911 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.190158 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.091899 23235 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.168015 23623 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.101555 23608 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.124342 23256 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.173907 23601 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.114038 23519 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.113483 23584 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:25:32.090461 23619 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.216392 23619 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.196522 23617 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:32.218477 23617 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.221565 23617 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:32.223109 23625 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 12:25:32.224257 23606 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 12:25:32.239604 23623 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.242862 23450 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.198555 13792 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:32.245579 13792 raft_consensus.cc:606] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:32.246001 13792 raft_consensus.cc:614] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:32.198905 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:32.202451 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:32.204242 23603 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
I20260709 12:25:32.198789 23499 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:32.209368 23397 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:32.203263 23256 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.122442 23588 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.248651 14423 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:32.251354 13798 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
W20260709 12:25:32.251770 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51371.7us
W20260709 12:25:32.252266 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49887.8us
I20260709 12:25:32.252883 13799 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:25:32.212741 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 148319us
I20260709 12:25:32.261520 14417 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:25:32.265583 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:32.274374 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.203146 23618 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.068s
I20260709 12:25:32.283820 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:32.263800 23446 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:32.272143 23608 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:32.295207 13800 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:25:32.297976 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 254669us
W20260709 12:25:32.280840 23568 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.538s	user 0.015s	sys 0.000s
I20260709 12:25:32.272418 23499 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:32.278393 23584 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:25:32.297942 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.311004 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:32.313055 13650 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:32.315871 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:32.317683 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.318590 23631 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.319195 23631 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.321609 23631 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:32.322643 13652 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:32.323421 23519 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.324738 23633 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:32.325251 23633 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.325677 23633 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:32.327634 23492 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:32.328317 23492 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.329370 23235 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.300487 23619 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:32.330281 23619 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.330921 23619 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:32.331236 23492 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.331758 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:32.333371 23607 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:32.340015 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.293208 23446 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:32.340411 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:32.272715 23547 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.330183 23129 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.273427 23335 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
W20260709 12:25:32.350795 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.350590 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:32.329391 23539 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.349754 23446 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.348899 23607 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:32.366637 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 69013.8us
I20260709 12:25:32.354374 23611 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:32.359918 23235 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:32.379943 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80eb1d112d05410a917177e781e466ed"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:32.381078 13650 raft_consensus.cc:606] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:32.313788 23568 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.381482 13650 raft_consensus.cc:614] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:32.380576 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.372429 23584 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:32.388489 23499 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.389381 23584 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 34.25, Last appended by leader: 25, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.390722 23511 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:32.391199 23511 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:32.391589 23511 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:32.392562 23547 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.394230 23615 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:32.394742 23615 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:32.399201 13796 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:32.371162 23623 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:25:32.401093 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86367.6us
I20260709 12:25:32.403007 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4400) took 1059 ms. Trace:
W20260709 12:25:32.408748 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68439.6us
I20260709 12:25:32.412175 23626 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:32.423130 23626 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:32.423153 23511 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.424532 23626 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.426898 23511 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:32.427038 23500 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.431793 23500 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.460150 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:32.405110 23623 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.463919 23500 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.464704 23588 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:32.403760 13720 rpcz_store.cc:276] 0709 12:25:31.344256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.426561 (+ 82305us) service_pool.cc:224] Handling call
0709 12:25:31.459421 (+ 32860us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.507168 (+ 47747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.507230 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.555832 (+ 48602us) log.cc:844] Serialized 52 byte log entry
0709 12:25:31.753009 (+197177us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:31.772088 (+ 19079us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.776647 (+  4559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.776751 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.402746 (+625995us) raft_consensus.cc:1685] finished
0709 12:25:32.402774 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.402938 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":730,"prepare.queue_time_us":25511,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":4484,"raft.queue_time_us":111093,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":27227,"thread_start_us":196763,"threads_started":1,"wal-append.queue_time_us":322816}
I20260709 12:25:32.409348 23541 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:32.477144 13650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:32.478794 13650 raft_consensus.cc:606] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:32.479207 13650 raft_consensus.cc:614] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:32.474715 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 59335us
I20260709 12:25:32.480510 23609 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:32.487318 23607 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.490408 23500 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:32.474790 23623 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.400614 23638 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.122s
I20260709 12:25:32.395757 23615 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:32.490971 23595 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:25:32.509663 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.512111 13643 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:32.502730 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.514838 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.515589 13644 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:32.509137 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.517122 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.519655 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.522128 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.522534 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.523506 13646 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:32.524252 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.517812 13650 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:32.524892 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.525205 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:32.525787 14423 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:32.521564 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.526180 13650 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:32.522017 13651 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:25:32.524428 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.531149 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:32.541014 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 129330us
I20260709 12:25:32.545065 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.503877 23568 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.553208 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.561362 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:32.566113 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.574141 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.568522 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.575095 13650 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:32.535537 23500 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:25:32.580118 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49061.6us
I20260709 12:25:32.582461 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.577785 23492 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:32.586459 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.580536 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.584684 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.589442 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.590166 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.554042 23588 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.560329 23608 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:32.583504 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:32.571362 23561 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:32.623939 23561 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.624097 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:32.624819 23561 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.599740 13800 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:32.612305 23547 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:32.628713 23547 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.629293 23547 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:32.621939 23619 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.632304 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.634500 23642 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:32.635092 23642 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.635473 23642 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:32.635869 23261 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.756s	user 0.007s	sys 0.011s
I20260709 12:25:32.626132 13649 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: [32.23-32.23]
I20260709 12:25:32.599695 13618 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.607491 13652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:32.612893 14423 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:32.600196 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.614598 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.615214 23615 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:32.620144 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:32.628489 23235 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:32.634049 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.658061 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:32.641156 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.591914 23541 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.600816 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.577348 23450 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.038s
I20260709 12:25:32.656330 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.656183 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.638480 13652 raft_consensus.cc:606] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Received request to transfer leadership to TS b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.597142 23492 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:32.674079 23608 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:32.673673 23601 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:32.654484 23638 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.674413 23650 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:25:32.677086 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.672915 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.654551 23605 raft_consensus.cc:2764] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:25:32.659412 23500 raft_consensus.cc:2764] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:25:32.654160 23336 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:32.662110 22677 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:32.665205 23239 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:32.689492 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:32.669981 23609 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:32.636552 23261 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:32.694456 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 93698.8us
I20260709 12:25:32.670432 23568 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:32.681048 23492 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:32.677922 23397 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:32.715072 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.719854 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.727576 22860 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:32.728106 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.728549 23648 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:25:32.729079 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.762789 13720 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:32.730438 23634 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.659335 23588 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.663703 23419 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:25:32.678345 23541 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.748396 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.757449 23589 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:32.758152 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.758934 23591 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:32.759147 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.777052 13727 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 18.
I20260709 12:25:32.759537 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.759485 23633 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.760322 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.706822 22677 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:32.707064 23239 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.787104 23239 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:32.762202 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.790958 13725 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:32.791952 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.792245 23588 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.766026 14422 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:32.752521 23397 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.692461 23638 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.741503 23611 raft_consensus.cc:2764] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:25:32.749521 23608 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.748957 23499 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:32.775924 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.773180 13724 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:32.772390 23609 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:32.740332 23619 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.778486 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:32.784458 23633 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.785784 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.793241 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.729346 23624 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.045s
I20260709 12:25:32.796553 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:32.821429 14424 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:32.802445 23397 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:32.749730 23336 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.762300 23568 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:32.804899 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43747.6us
I20260709 12:25:32.772352 23589 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.741307 23129 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:32.776808 23591 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.776485 23626 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.767045 23607 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:32.816910 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 144118us
I20260709 12:25:32.817989 13719 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:32.844699 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:32.822746 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.823139 13727 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:32.770355 13723 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:32.824859 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:32.825871 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.824519 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:32.811975 23499 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.775296 23646 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 20, Time since last communication: 0.047s
I20260709 12:25:32.847537 13722 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:32.780602 23634 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.852453 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:32.855024 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 254868us
I20260709 12:25:32.810925 23608 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.833371 23589 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:32.860041 23618 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.875595 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:32.845108 23633 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:32.860824 23541 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.862434 23658 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:32.863716 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:32.865794 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.847114 23591 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:32.833475 23129 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:32.833295 23623 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.870888 23593 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:32.873446 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:32.881342 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.860239 23261 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.885113 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.856163 23499 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:32.888098 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:32.889492 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:32.884405 23605 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.007s
I20260709 12:25:32.891121 23633 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.891877 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.898327 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.849179 23626 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.904098 13721 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 32 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:25:32.907908 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:32.899300 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:32.883302 23584 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:32.926380 13725 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:32.904983 13723 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:25:32.940651 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52657.2us
I20260709 12:25:32.941717 13727 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:32.903105 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.945400 13800 raft_consensus.cc:2410] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:25:32.945194 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:32.915144 23446 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.001s
I20260709 12:25:32.947229 13797 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:32.892491 23658 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:32.941605 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:32.948377 23665 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:32.948832 23627 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:32.949288 23633 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.949470 23626 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:32.949770 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:32.942412 23335 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.008s
I20260709 12:25:32.950280 13794 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:32.905283 23129 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:32.891515 23618 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.914324 23593 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.039912 23593 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:32.978750 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121879us
W20260709 12:25:32.981627 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138928us
I20260709 12:25:32.950583 23659 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:32.988617 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.049805 13800 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:32.990293 13721 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:32.992319 23665 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.994971 23638 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:32.975507 23446 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.910676 23607 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:32.997426 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:32.998672 23256 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.011s
I20260709 12:25:33.000483 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.001380 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.050081 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:33.006976 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 132203us
I20260709 12:25:32.906145 23568 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 29.23, Last appended by leader: 23, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:32.975363 23656 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
W20260709 12:25:33.036017 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56621.2us
I20260709 12:25:33.079674 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:33.014595 23601 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:33.015362 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:32.914841 23561 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.061934 13793 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:33.071496 13795 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:33.009378 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 110881us
I20260709 12:25:33.081961 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:33.093281 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:33.061797 23664 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:33.099030 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.106504 13720 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:25:33.108955 13795 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:33.062167 23665 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:33.082309 23601 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.050778 23547 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.089452 23638 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:33.149421 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.157116 13721 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:33.170825 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.157279 14422 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:33.179366 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.070652 23256 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.180217 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:33.189143 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:33.191157 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.118777 23659 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.195539 14422 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:33.153414 23601 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:33.197229 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.198537 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.203140 23256 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.124133 23541 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:25:33.205112 23541 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:25:33.148823 23605 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:33.219419 13652 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:25:33.212069 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89980us
I20260709 12:25:33.221127 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.205482 23397 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:33.227491 23677 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:25:33.239274 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:33.229527 23657 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.333s	user 0.000s	sys 0.002s
W20260709 12:25:33.230458 23667 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:25:33.231462 23589 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.192749 23608 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.153756 23675 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.109s
I20260709 12:25:33.219440 23336 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.235041 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.273814 13650 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:25:33.155750 23679 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:33.237201 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:33.237583 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 113732us
W20260709 12:25:33.243211 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 155304us
I20260709 12:25:33.250312 13725 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
W20260709 12:25:33.262602 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90756us
I20260709 12:25:33.248914 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.282608 13652 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:33.265192 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.283077 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.184580 23638 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:33.149848 23664 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.287726 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:33.288569 13725 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:33.285022 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:33.289589 14422 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:33.283933 13792 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:33.264614 23626 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:33.300061 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.250223 23634 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.300902 13650 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:33.333321 14422 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:33.347617 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.352698 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.207756 23659 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.283926 23615 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.370244 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 279316us
I20260709 12:25:33.378965 13651 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: [32.23-32.23]
I20260709 12:25:33.370728 13725 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:33.369342 13648 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:33.244992 23605 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.374476 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.375113 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:33.377732 13721 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:33.381891 13645 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
W20260709 12:25:33.382081 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 154817us
I20260709 12:25:33.379464 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.383246 13644 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: [30.24-30.24]
I20260709 12:25:33.383652 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:33.368949 23642 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.003s
I20260709 12:25:33.390365 23679 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:33.390846 13801 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:33.391287 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.391711 23547 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.391948 13725 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:33.392875 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.393447 13725 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:33.364571 23626 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.396796 23626 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.397965 23129 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.398844 23397 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.399503 23642 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.400466 23129 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.400568 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:33.401161 23689 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:33.291656 23646 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.086s
I20260709 12:25:33.401865 23642 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.403409 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.402412 23618 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.402587 13795 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.401373 13727 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:33.404601 13725 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:33.405874 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:33.412670 14419 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:33.419067 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:33.419924 14417 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:33.419199 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:33.419122 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.424008 13652 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:25:33.420097 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:33.425721 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:33.417763 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:33.401899 23689 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.436674 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.380064 23684 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:33.437669 13643 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:33.438036 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.441162 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.441857 23686 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:33.442565 23686 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.443490 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.444100 13801 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:33.445075 23686 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.384768 23615 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:33.447271 23492 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.008s
I20260709 12:25:33.447844 23492 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.450618 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.454357 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.455538 23664 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.455772 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.456228 23638 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.457324 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.458530 23561 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.459126 14425 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:25:33.460306 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.461516 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.462056 13798 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:33.462906 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.463485 13798 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:33.464221 23446 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.464563 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.464819 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.465204 13798 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:33.465538 13648 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:33.465659 23589 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:33.391990 23660 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.315s	user 0.000s	sys 0.002s
W20260709 12:25:33.466725 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 96298.7us
I20260709 12:25:33.468562 23609 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.469048 14423 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:33.469365 23588 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:33.428364 23568 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:25:33.470067 23671 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.139s
I20260709 12:25:33.442073 13797 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.442859 23607 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.436573 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:33.457983 13800 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:25:33.466096 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.501026 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.499598 13792 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:33.502942 13650 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:25:33.501714 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 67404.4us
I20260709 12:25:33.503450 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.504091 13792 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:33.505079 13798 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.471755 23642 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:33.516633 14417 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:33.463632 23605 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.471634 23646 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
W20260709 12:25:33.469061 23239 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.681s	user 0.014s	sys 0.000s
I20260709 12:25:33.524828 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.447860 23615 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:33.528476 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 96943.1us
I20260709 12:25:33.524828 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.494619 23618 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:33.469751 23588 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:33.535434 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.537168 13643 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:33.539875 14421 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:25:33.540881 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.542178 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.542578 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.543370 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.543591 13650 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:33.544984 13798 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:25:33.535260 23686 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.015s
I20260709 12:25:33.546723 23686 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:33.547109 23686 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:33.548846 13725 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:33.546073 13650 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:33.542680 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:33.543690 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:33.550590 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:33.552131 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:33.553439 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:33.554917 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:33.554920 13725 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:33.555873 14423 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:33.518772 23642 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:33.556965 13727 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
W20260709 12:25:33.530968 23591 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.004s
I20260709 12:25:33.561297 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:33.572482 13726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.028s	sys 0.018s
W20260709 12:25:33.572937 13726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.028s	sys 0.018s
I20260709 12:25:33.469059 23609 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.579879 13797 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:25:33.583889 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.581949 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:33.584832 13792 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:33.522354 23500 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:25:33.585459 13650 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:25:33.571457 23446 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:33.571666 23608 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:33.571862 23255 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:33.536011 23239 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.620898 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.652067 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.548158 23686 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.655251 23686 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:33.657577 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:33.622789 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.565285 23607 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.053s
I20260709 12:25:33.571983 23591 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.589954 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:33.659868 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:33.637336 23499 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.003s
I20260709 12:25:33.647828 23608 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.673008 23608 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:33.673928 23239 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.674446 23261 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:33.674875 23261 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:33.627550 23588 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 30.24, Last appended by leader: 24, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.676294 23588 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:33.675897 23261 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.663221 23627 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:33.678961 23627 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.679381 23627 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:33.662748 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:33.674983 13793 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.687052 14423 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:33.706530 14417 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
W20260709 12:25:33.708004 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45326.5us
I20260709 12:25:33.709582 13792 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.683475 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.687879 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:33.652129 13643 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:33.712667 13801 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:25:33.716168 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.716665 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:33.718756 13643 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:33.648906 23446 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:33.720242 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 130377us
I20260709 12:25:33.720702 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:33.721365 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:33.723032 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.719805 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.728900 13801 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.730726 13722 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:33.683935 23690 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:25:33.738154 13796 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:33.743319 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.712245 23568 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:33.748827 23568 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.749228 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:33.749384 23568 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:33.750485 23618 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:33.751163 23618 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.751502 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.752372 13647 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:33.752741 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4418) took 1009 ms. Trace:
I20260709 12:25:33.753152 13720 rpcz_store.cc:276] 0709 12:25:32.743510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.763527 (+ 20017us) service_pool.cc:224] Handling call
0709 12:25:33.752481 (+988954us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b5400155ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:33.752655 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662807424}
I20260709 12:25:33.754410 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:33.717854 23633 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:33.629556 23642 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.723701 23261 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:33.731974 23446 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:33.791513 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.783079 23255 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:33.734699 23684 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:33.793573 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.734324 23239 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:25:33.687268 23499 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.794308 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.798025 13647 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:33.799036 23704 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:33.800088 23708 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:33.801381 23492 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.826645 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4168) took 1103 ms. Trace:
W20260709 12:25:33.803053 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48780.6us
I20260709 12:25:33.806490 23589 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:33.811072 23705 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:33.664450 23668 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.115s
I20260709 12:25:33.807629 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:33.819764 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 237890us
I20260709 12:25:33.802044 23631 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:33.855762 23631 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:33.856714 23631 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.857724 23631 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:33.796839 23677 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:25:33.858422 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:33.859077 23697 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:25:33.859843 23593 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.819s	user 0.014s	sys 0.000s
W20260709 12:25:33.860265 23701 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:25:33.862043 23665 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.003s
I20260709 12:25:33.862429 13724 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:33.862489 23665 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.863620 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.870541 13724 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:33.873075 13724 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:25:33.850004 13690 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:33.875149 23705 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:33.875730 23705 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:33.875886 23358 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:33.876434 23682 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.852725 23662 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.961s	user 0.006s	sys 0.006s
I20260709 12:25:33.853498 23623 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.853665 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:33.877135 23682 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.878216 13725 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:33.828259 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:33.880029 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51832.5us
I20260709 12:25:33.880831 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:33.855090 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.855042 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:33.883132 23591 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.883551 13724 pending_rounds.cc:85] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:25:33.884066 13794 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.885275 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.789738 23615 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:33.886370 23615 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:33.860142 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:33.889210 13721 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:33.889391 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.890071 13647 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:33.868515 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.814011 23239 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.890774 13793 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:33.890924 23239 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:33.891811 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.892958 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.893625 13793 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.869585 22860 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:33.816761 13797 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:33.872972 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.894838 22860 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:33.895529 13798 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:25:33.896217 22860 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.877039 23705 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.897639 22860 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:33.884037 13724 pending_rounds.cc:107] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:25:33.900882 13724 raft_consensus.cc:2889] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:33.827159 13718 rpcz_store.cc:276] 0709 12:25:32.723453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.724045 (+   592us) service_pool.cc:224] Handling call
0709 12:25:32.724573 (+   528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.765311 (+ 40738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.765346 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.806374 (+ 41028us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.984991 (+178617us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:32.985042 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.985114 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.808856 (+823742us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b2400166988. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f539098d9c4 00007f539098bacb 00007f5390b46dc9 00007f5390b46a55 00007f5390b45fb0 00007f53937bc488 00007f53937bcbe8 00007f5393794398 00007f539378f76a 00007f539378ebb6
0709 12:25:33.816496 (+  7640us) raft_consensus.cc:1685] finished
0709 12:25:33.816519 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.826567 (+ 10048us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2373,"raft.queue_time_us":14083,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":10326,"spinlock_wait_cycles":21418752,"thread_start_us":126034,"threads_started":1,"wal-append.queue_time_us":229967}
I20260709 12:25:33.895658 13795 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.903210 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.903879 13795 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:33.848727 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.906440 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:33.906708 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.907511 13651 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:33.908401 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.909046 13651 raft_consensus.cc:2368] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:33.909250 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.896732 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:33.910566 13797 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:33.883610 13723 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:33.915588 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:33.916917 13792 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:33.917352 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:33.917805 23631 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:33.855196 23618 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.918192 23615 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:33.918699 23615 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.918787 23659 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.919137 13792 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:33.915630 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:33.920516 14425 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:33.849646 23642 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:33.854560 23708 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:33.875629 23609 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.926757 23618 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:33.927592 23618 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.929481 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:33.876364 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3101) took 1006 ms. Trace:
I20260709 12:25:33.931002 13796 rpcz_store.cc:276] 0709 12:25:32.869700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.879380 (+  9680us) service_pool.cc:224] Handling call
0709 12:25:33.876148 (+996768us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b5400129b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:33.876302 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807067008}
I20260709 12:25:33.830469 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:33.867982 23499 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.846959 23589 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:33.825948 23601 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.562s	user 0.012s	sys 0.001s
I20260709 12:25:33.883814 23591 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:33.940600 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2061) took 1130 ms. Trace:
I20260709 12:25:33.935354 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:33.946758 13798 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:33.949537 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 228295us
I20260709 12:25:33.824466 23624 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:33.952875 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:33.948242 13726 rpcz_store.cc:276] 0709 12:25:32.810340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.946033 (+135693us) service_pool.cc:224] Handling call
0709 12:25:32.978034 (+ 32001us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.982486 (+  4452us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:32.984572 (+  2086us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.822969 (+838397us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.828983 (+  6014us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.829279 (+   296us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:33.829527 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.829632 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.940266 (+110634us) raft_consensus.cc:1685] finished
0709 12:25:33.940297 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.940513 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3482,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":114457,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":6212,"spinlock_wait_cycles":1683840,"thread_start_us":772559,"threads_started":2,"wal-append.queue_time_us":726214}
I20260709 12:25:33.955734 13796 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:25:33.950634 23591 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:33.956521 23591 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:33.826610 23608 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:33.830591 23626 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:33.958920 14416 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:25:33.962280 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:33.950804 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:33.819448 23704 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.919153 23615 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:33.964433 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:33.968067 23256 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:33.970191 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:33.957684 23668 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 12:25:33.972591 13720 raft_consensus.cc:2368] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:25:33.974964 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:33.947144 23589 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:33.860301 23593 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.943743 23665 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.954708 23511 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:33.994549 23511 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:33.995512 23511 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.996421 23593 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.996639 23511 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:33.997334 13799 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:33.953248 23601 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.000191 23601 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.000800 23541 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:34.001541 23541 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:33.964030 23689 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.003481 14418 raft_consensus.cc:2368] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24.
I20260709 12:25:34.008774 23618 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:33.996657 23682 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:33.989897 13643 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:33.926846 23708 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.015650 23568 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.016724 23708 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:34.002063 23541 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:33.972251 23634 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:34.018736 23634 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:34.019161 23634 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:33.996461 22677 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:33.990686 23608 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:33.974633 23358 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:34.032378 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68144us
W20260709 12:25:34.035528 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47585.7us
I20260709 12:25:34.037855 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:34.037889 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:33.975570 23704 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:33.989413 23255 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 30.23, Last appended by leader: 23, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.047600 23685 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:25:34.048537 23685 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:34.049678 23685 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:34.050055 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:34.044548 23540 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:34.052008 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:34.052475 23684 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:34.052892 14424 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:34.054145 14425 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:33.974563 23648 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.069s
I20260709 12:25:34.055246 23618 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:34.055346 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:34.031965 23691 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.024s
I20260709 12:25:34.056488 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:34.056771 23499 raft_consensus.cc:993] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:25:34.057400 23499 raft_consensus.cc:1081] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:25:34.055577 13690 leader_election.cc:400] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24.
I20260709 12:25:34.059585 13690 leader_election.cc:403] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:34.043454 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:34.060515 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90393.2us
W20260709 12:25:34.049678 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.061843 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:34.062309 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:34.064591 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:34.050034 23685 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:34.070652 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:34.018153 23568 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.036965 23358 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:34.058089 23684 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:34.127628 23358 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:34.089529 13723 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:34.090543 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.094120 13794 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:34.084437 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.082458 23609 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.130520 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:34.130784 23609 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.131466 13648 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 21: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:34.132558 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.133241 23609 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:34.133459 13648 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:34.069483 23627 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.136546 23627 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.070649 23648 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:34.137691 23648 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.138186 23648 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:34.143839 23686 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.144659 23686 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.089722 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.148669 13722 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:34.101948 14422 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 23. Preceding OpId from leader: term: 31 index: 24. (index mismatch)
I20260709 12:25:34.084892 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.083927 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.106657 13721 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:34.090584 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:34.153455 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62938.9us
I20260709 12:25:34.108126 23658 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.109217 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.070293 23336 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:34.109874 23711 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:25:34.110172 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:34.110306 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 75736.3us
I20260709 12:25:34.087761 23626 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:34.096732 14425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48736
I20260709 12:25:34.111457 23709 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:25:34.112380 23691 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:25:34.112416 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47936.7us
I20260709 12:25:34.193863 13652 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:25:34.195719 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:34.196662 23633 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:34.069617 23541 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.070906 23631 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:34.114188 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:34.115684 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.121461 23588 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:34.121094 23446 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.066460 23540 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:34.071074 23608 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:34.055740 23618 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:34.203429 23691 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:34.208973 23691 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:34.209419 23691 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:34.123737 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.123371 23685 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.078727 23593 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 20, Time since last communication: 0.057s
I20260709 12:25:34.127707 23684 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:34.235985 13650 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:34.090111 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.082185 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:34.092136 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.158457 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:34.237262 13647 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:34.154851 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:34.163209 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4177) took 1238 ms. Trace:
W20260709 12:25:34.238513 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83797us
I20260709 12:25:34.071138 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:34.171471 13646 raft_consensus.cc:2368] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:25:34.172655 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:34.241129 13643 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:34.238328 13727 rpcz_store.cc:276] 0709 12:25:32.925092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.559380 (+634288us) service_pool.cc:224] Handling call
0709 12:25:33.755075 (+195695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.755399 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:33.800616 (+ 45217us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.962732 (+162116us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.962788 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:33.962849 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.962914 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.099049 (+136135us) spinlock_profiling.cc:243] Waited 9.91 ms on lock 0x7b18004ca140. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4
0709 12:25:34.163032 (+ 63983us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b5400155ed0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:34.163147 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42708,"prepare.queue_time_us":46155,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":187,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":5300,"spinlock_wait_cycles":471864960,"thread_start_us":161559,"threads_started":1,"wal-append.queue_time_us":397232}
I20260709 12:25:34.175372 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.185763 23686 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:34.186450 23707 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:34.206561 23540 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.250931 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:34.251827 13648 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:34.089973 23547 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:25:34.188908 23450 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:34.189659 23626 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:34.191409 23712 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:25:34.199005 13801 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:34.201575 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87475.8us
I20260709 12:25:34.243842 23684 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:34.171800 23568 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:34.201150 23588 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:34.202754 23615 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.210801 13796 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:25:34.131270 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:34.248189 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 177138us
I20260709 12:25:34.287118 13794 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:34.206774 23608 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:34.174149 23584 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.069s
I20260709 12:25:34.263979 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:34.265591 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:34.200537 23631 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:34.204504 23633 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:25:34.268397 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.279699 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:34.249061 23627 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:34.209734 23618 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.287905 13766 leader_election.cc:400] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:25:34.259335 23658 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:34.254805 23540 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:34.311033 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45559.1us
I20260709 12:25:34.310119 13766 leader_election.cc:403] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:34.239667 23591 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.256850 22860 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:34.320566 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:34.334904 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47149.2us
I20260709 12:25:34.337663 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:34.280433 23705 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.336021 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:34.343904 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.287483 23568 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:34.347642 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.255940 23336 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:25:34.309233 23608 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:34.350832 13721 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:34.309636 23627 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.350013 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.350389 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.309283 23631 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:25:34.370091 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 192801us
I20260709 12:25:34.284034 23623 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:25:34.319155 23708 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:34.374164 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:34.381858 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.383697 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:34.427450 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43887.1us
I20260709 12:25:34.365816 23686 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:25:34.431627 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57573.3us
I20260709 12:25:34.428438 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:34.338163 22860 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:34.365868 23712 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:25:34.432525 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.310065 23668 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.109s
I20260709 12:25:34.352325 23235 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:34.351737 23607 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:34.382880 13718 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:34.308696 23704 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.436648 13721 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:34.471158 23714 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:25:34.471982 13726 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:34.351400 23705 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.478436 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.479384 13727 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:34.387578 13801 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:34.455053 23541 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.435420 23609 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:34.379243 23627 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:34.435976 23397 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:34.430457 23588 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.424862 23589 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.459916 23615 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:34.449682 23607 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.469245 23591 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:34.470111 23646 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:34.370714 23568 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.439726 23664 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:34.526762 13795 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:34.441380 22860 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.501797 13722 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:34.509646 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.466660 23358 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.418589 23239 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.498577 23709 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24.
I20260709 12:25:34.515401 23607 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:34.544878 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 203065us
I20260709 12:25:34.517699 23646 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.522933 23588 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:34.517956 23397 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:34.566423 13798 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 12:25:34.560945 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 280860us
I20260709 12:25:34.513195 23511 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:25:34.563733 23646 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:34.562314 23256 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:34.559158 23693 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:34.574963 23634 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.555s	user 0.013s	sys 0.000s
I20260709 12:25:34.610195 14417 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:34.579015 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:34.622844 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43996.6us
W20260709 12:25:34.622857 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 171966us
I20260709 12:25:34.617573 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4553) took 1056 ms. Trace:
I20260709 12:25:34.614092 13801 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:34.625882 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:34.620750 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.624228 13792 rpcz_store.cc:276] 0709 12:25:33.560881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.920845 (+359964us) service_pool.cc:224] Handling call
0709 12:25:33.921145 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.921353 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:34.280697 (+359344us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.617081 (+336384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:34.617159 (+    78us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:34.617224 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.617308 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.617497 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35805,"prepare.queue_time_us":525484,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":35987,"thread_start_us":639665,"threads_started":2}
I20260709 12:25:34.630522 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:34.636004 23719 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:25:34.636477 23589 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:34.637280 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.637682 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.631914 23499 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:34.638365 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.638689 23499 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:34.639674 23499 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.639773 23397 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:34.647254 23721 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:25:34.655296 23727 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:25:34.580235 23608 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.629431 23693 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.569258 23710 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:25:34.621495 23668 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:25:34.627723 23591 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.787640 23693 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.786764 23591 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.789789 23608 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:34.790580 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 362241us
W20260709 12:25:34.683007 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52554.7us
W20260709 12:25:34.791983 13794 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.501s	user 0.004s	sys 0.008s
I20260709 12:25:34.791823 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:34.790756 13801 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:34.677096 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4555) took 1103 ms. Trace:
I20260709 12:25:34.731750 13796 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:34.746603 13649 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:34.750490 23659 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:34.751539 23705 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:34.752992 23648 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.009s
I20260709 12:25:34.753393 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.754130 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.807099 13799 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:34.634373 23634 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.807150 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:34.755471 23726 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:25:34.755618 23722 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:25:34.679900 23511 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:34.766239 23665 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:34.824572 23659 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:34.825088 23665 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:34.770164 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:34.773912 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.826745 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.829608 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:34.830385 14416 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.830112 23659 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.830729 23665 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.832062 23665 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:34.770711 23631 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.775290 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4547) took 1236 ms. Trace:
I20260709 12:25:34.775954 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4561) took 1124 ms. Trace:
W20260709 12:25:34.833194 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63019.3us
I20260709 12:25:34.804399 23591 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:34.776979 23684 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.781872 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.783303 23239 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:34.790882 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 160637us
I20260709 12:25:34.570336 23235 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:25:34.798060 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:25:34.798791 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 234282us
I20260709 12:25:34.799386 13794 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:34.803798 14418 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:34.793660 13798 rpcz_store.cc:276] 0709 12:25:33.573795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.953826 (+380031us) service_pool.cc:224] Handling call
0709 12:25:33.954143 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.567135 (+612992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.676543 (+109408us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b540006ac54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:34.676983 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407302912}
I20260709 12:25:34.804332 14423 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:34.814286 13796 raft_consensus.cc:2368] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26.
I20260709 12:25:34.804064 23593 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:25:34.862226 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.863518 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.864292 13794 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:34.834676 13800 rpcz_store.cc:276] 0709 12:25:33.651920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.185864 (+533944us) service_pool.cc:224] Handling call
0709 12:25:34.186387 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.186871 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.186916 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.187623 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.514931 (+327308us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:34.514998 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.515102 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.775694 (+260592us) raft_consensus.cc:1685] finished
0709 12:25:34.775725 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.775883 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5720,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":326745,"threads_started":1,"wal-append.queue_time_us":583741}
I20260709 12:25:34.833995 23704 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:34.835134 23511 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:34.852221 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.853093 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.834362 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:34.836655 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.834411 13795 rpcz_store.cc:276] 0709 12:25:33.538544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.904513 (+365969us) service_pool.cc:224] Handling call
0709 12:25:33.904813 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.530731 (+625918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.774921 (+244190us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54001ca7d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:34.775205 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420179712}
I20260709 12:25:34.873324 13721 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:34.873443 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.857733 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:34.858762 14423 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:34.874302 13794 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 21: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:34.875782 14425 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.815802 23648 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.879043 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.776691 23646 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.862089 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.856789 22860 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.880204 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:34.858096 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:34.882030 13693 leader_election.cc:400] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:34.882407 13693 leader_election.cc:403] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:34.850557 13644 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:34.806579 23608 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 12:25:34.871387 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 29 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.876405 14425 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:34.886180 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:34.886765 13800 raft_consensus.cc:2410] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 32 because replica has last-logged OpId of term: 30 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 23.
W20260709 12:25:34.887748 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53509.3us
I20260709 12:25:34.854843 23663 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.892403 22860 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:34.895457 23631 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:34.896241 23717 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.434s	user 0.001s	sys 0.002s
I20260709 12:25:34.874801 14423 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.24, Last appended by leader: 24, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.900200 14423 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:34.905776 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.901804 14425 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.908231 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:34.908952 13792 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:34.888933 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:34.860992 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:34.875734 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:34.871526 13799 raft_consensus.cc:2376] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28. Re-sending same reply.
I20260709 12:25:34.883569 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4179) took 1903 ms. Trace:
I20260709 12:25:34.884253 13798 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:34.884554 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:34.933836 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:34.935511 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74615.3us
I20260709 12:25:34.937827 13799 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:25:34.942420 23693 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.943332 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.943121 23693 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:34.945303 23685 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:34.946449 23239 raft_consensus.cc:2764] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 12:25:34.948529 23744 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:25:34.949748 23724 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:25:34.950909 23729 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:25:34.951853 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:34.952733 23731 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:34.953405 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1594) took 1380 ms. Trace:
I20260709 12:25:34.953645 23684 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.954113 23693 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.956768 23568 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.902194 23358 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.909541 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4548) took 1355 ms. Trace:
I20260709 12:25:34.959456 13800 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:25:34.899505 23691 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.690s	user 0.006s	sys 0.009s
I20260709 12:25:34.960299 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3130) took 1298 ms. Trace:
I20260709 12:25:34.960847 13800 rpcz_store.cc:276] 0709 12:25:33.662319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.901647 (+1239328us) service_pool.cc:224] Handling call
0709 12:25:34.959083 (+ 57436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.960018 (+   935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.960236 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:34.960920 23691 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.961796 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.902523 23704 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:25:34.880640 23668 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:34.952857 13718 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:34.976040 13798 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:34.969102 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4433) took 1265 ms. Trace:
W20260709 12:25:34.979079 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 90268.4us
I20260709 12:25:34.980121 13767 leader_election.cc:400] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23.
I20260709 12:25:34.980188 13723 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:34.962224 13800 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:34.973572 13722 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 21: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:34.981606 13799 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:25:34.899688 23713 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:34.959136 23500 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:34.953845 13793 rpcz_store.cc:276] 0709 12:25:33.572677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.933067 (+360390us) service_pool.cc:224] Handling call
0709 12:25:33.946955 (+ 13888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.953852 (+  6897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.953888 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.962176 (+  8288us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.491321 (+529145us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:25:34.491387 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.491485 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.953144 (+461659us) raft_consensus.cc:1685] finished
0709 12:25:34.953168 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.953342 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13726,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10185,"thread_start_us":450972,"threads_started":1,"wal-append.queue_time_us":600760}
I20260709 12:25:34.988322 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 19.
I20260709 12:25:34.981770 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.997120 23638 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:25:34.998343 23716 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.546s	user 0.000s	sys 0.003s
W20260709 12:25:34.999998 23626 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.001s
I20260709 12:25:35.002489 23618 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:34.967572 23721 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:34.869621 23740 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 12:25:34.902668 23624 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:34.968645 23634 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:34.967547 23648 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:35.027304 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.027760 13722 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:25:35.030067 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:35.035620 13650 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:35.009155 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.966667 23668 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.040210 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:35.057176 13799 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:35.036727 13800 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:25:35.035523 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.058844 13718 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:34.982442 23239 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:34.935405 13719 rpcz_store.cc:276] 0709 12:25:32.980364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.671216 (+690852us) service_pool.cc:224] Handling call
0709 12:25:33.671678 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.952290 (+280612us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:33.952328 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.962481 (+ 10153us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.421236 (+458755us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:34.421304 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.421563 (+   259us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.883323 (+461760us) raft_consensus.cc:1685] finished
0709 12:25:34.883348 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.883493 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127590,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"thread_start_us":646867,"threads_started":2,"wal-append.queue_time_us":496872}
I20260709 12:25:35.027169 14423 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 31 index: 24. Preceding OpId from leader: term: 32 index: 24. (term mismatch)
I20260709 12:25:35.068164 14423 pending_rounds.cc:85] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 23
I20260709 12:25:35.022177 13792 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:35.025002 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:35.012434 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:35.069682 13647 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:35.061043 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4567) took 1035 ms. Trace:
I20260709 12:25:35.070819 13800 rpcz_store.cc:276] 0709 12:25:34.025390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.990797 (+965407us) service_pool.cc:224] Handling call
0709 12:25:35.036405 (+ 45608us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.060768 (+ 24363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.060985 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:35.063484 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.045123 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.065845 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:35.068483 14423 pending_rounds.cc:107] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 31.24
I20260709 12:25:35.073735 13718 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:35.076085 23705 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:35.055680 23626 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.079191 13724 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:35.074186 14423 raft_consensus.cc:2889] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:34.977489 13725 rpcz_store.cc:276] 0709 12:25:33.703705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.878914 (+175209us) service_pool.cc:224] Handling call
0709 12:25:33.879242 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.026643 (+147401us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:25:34.026959 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.480904 (+453945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:34.480989 (+    85us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:34.481064 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.968026 (+486962us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b180034d7a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:25:34.968845 (+   819us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54002f6554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:34.968902 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.969040 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4979,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":122,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7594,"spinlock_wait_cycles":349352576,"thread_start_us":453465,"threads_started":1,"wal-append.queue_time_us":808208}
I20260709 12:25:35.082749 13723 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:35.084416 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.084527 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:34.963869 23691 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:35.051416 23540 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.003s
I20260709 12:25:34.959407 13797 rpcz_store.cc:276] 0709 12:25:33.554087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.911180 (+357093us) service_pool.cc:224] Handling call
0709 12:25:33.911496 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.911872 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.911899 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.912428 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.265008 (+352580us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:34.265059 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.265139 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.895286 (+630147us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:34.900515 (+  5229us) raft_consensus.cc:1685] finished
0709 12:25:34.900539 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.909465 (+  8926us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4043,"raft.queue_time_us":2159,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":29174,"spinlock_wait_cycles":31401856,"thread_start_us":346234,"threads_started":1,"wal-append.queue_time_us":549416}
I20260709 12:25:35.076870 23705 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:34.964066 23646 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:35.104285 13799 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:35.025904 23500 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.109987 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3135) took 1203 ms. Trace:
I20260709 12:25:35.031114 23713 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:35.059453 23669 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:35.112759 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.116464 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:35.121958 13725 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:35.041119 23631 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 12:25:35.080411 23624 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.122921 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:34.910320 23663 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.094945 23746 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:35.030565 23638 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.104911 23648 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:25:34.965940 23627 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.129347 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:35.116393 13651 rpcz_store.cc:276] 0709 12:25:33.906559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.909599 (+  3040us) service_pool.cc:224] Handling call
0709 12:25:33.909919 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:33.939081 (+ 29162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.939129 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:33.958720 (+ 19591us) log.cc:844] Serialized 52 byte log entry
0709 12:25:34.311345 (+352625us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:34.372572 (+ 61227us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.568428 (+195856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.570938 (+  2510us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.109610 (+538672us) raft_consensus.cc:1685] finished
0709 12:25:35.109649 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.109885 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14175,"prepare.queue_time_us":53352,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":95784,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":12897,"thread_start_us":492991,"threads_started":2,"wal-append.queue_time_us":530757}
W20260709 12:25:35.152669 23725 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.499s	user 0.000s	sys 0.002s
W20260709 12:25:35.156941 23748 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.251s	user 0.002s	sys 0.000s
I20260709 12:25:35.159021 23500 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.160871 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2064) took 1072 ms. Trace:
I20260709 12:25:35.160992 23500 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:35.164053 23705 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.166734 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:35.168052 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:35.161581 13726 rpcz_store.cc:276] 0709 12:25:34.088077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.089866 (+  1789us) service_pool.cc:224] Handling call
0709 12:25:35.160580 (+1070714us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54003bea54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:35.160787 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540304256}
I20260709 12:25:35.173049 13718 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:35.178664 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:35.181237 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.107375 23540 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.191639 13792 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:35.192236 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.138988 23626 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:35.144230 23607 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.000s
I20260709 12:25:35.201809 22677 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:35.190543 23631 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.203281 23631 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.205648 23739 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:35.187752 23686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.037s	sys 0.023s
I20260709 12:25:35.178493 23648 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:35.207907 23648 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:35.206863 23686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.037s	sys 0.023s
I20260709 12:25:35.209592 13723 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 18. Current term is 19. Ops: [17.11-18.12]
I20260709 12:25:35.209898 23686 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:35.195261 23740 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:35.211146 23686 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.216313 13766 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:35.190824 23623 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:35.223444 13795 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:35.224254 23704 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:35.213755 23740 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.221714 23623 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.227285 23627 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.229288 13721 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:35.237823 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.241672 23593 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:35.245131 23593 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:35.246357 23741 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:25:35.246174 23593 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.247738 23740 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.250878 23623 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.253244 23609 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.257537 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4562) took 1233 ms. Trace:
I20260709 12:25:35.206933 23739 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:25:35.195582 23705 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:35.258920 13794 rpcz_store.cc:276] 0709 12:25:34.024070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.903009 (+878939us) service_pool.cc:224] Handling call
0709 12:25:34.903482 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:34.904556 (+  1074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.904606 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:34.908927 (+  4321us) log.cc:844] Serialized 52 byte log entry
0709 12:25:35.137616 (+228689us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:35.255755 (+118139us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.257184 (+  1429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.257270 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.257316 (+    46us) raft_consensus.cc:1685] finished
0709 12:25:35.257326 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.257480 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":594,"raft.queue_time_us":4839,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":45673,"thread_start_us":203299,"threads_started":2,"wal-append.queue_time_us":215863}
I20260709 12:25:35.133544 23624 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.133832 23669 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.254534 23684 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.256861 23540 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.295964 13793 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 30 index: 24. Preceding OpId from leader: term: 31 index: 24. (term mismatch)
I20260709 12:25:35.296458 13793 pending_rounds.cc:85] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 23
I20260709 12:25:35.296696 13793 pending_rounds.cc:107] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 30.24
I20260709 12:25:35.296937 13793 raft_consensus.cc:2889] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:35.242518 23665 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:25:35.235680 23607 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.267544 23739 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.227589 23713 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.302304 23627 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:35.253891 23609 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.316318 13725 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:35.297096 23684 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.307742 23627 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.292299 23705 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.275271 23634 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:25:35.297709 23631 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.326193 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:35.343937 13727 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
W20260709 12:25:35.319976 23588 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.003s
I20260709 12:25:35.318895 23665 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:35.297003 23397 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.049s
I20260709 12:25:35.321753 23705 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.340837 23693 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.319120 23684 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.318977 23669 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.346899 23668 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.307094 23741 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 33.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.055s
I20260709 12:25:35.360752 23665 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.319417 23739 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.369060 23693 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:35.375492 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 80524.4us
I20260709 12:25:35.380776 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:35.318225 23627 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.392963 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:35.377213 23663 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:35.383180 23499 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:35.405336 23739 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:25:35.379136 23669 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:25:35.410948 23638 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.417333 23704 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:35.420114 23627 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:35.418159 23499 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.444139 23663 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:35.468750 23758 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:25:35.469470 23693 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.470103 23704 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.475262 23763 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.454146 23665 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:35.476166 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.466573 23746 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:35.480299 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.482636 13799 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:35.442428 23740 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:35.465597 23753 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.102s	user 0.001s	sys 0.001s
I20260709 12:25:35.453353 23235 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.481124 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.501468 13794 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:35.496462 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.496177 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.504242 13795 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:35.505599 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.508306 13799 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:35.508766 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 114360us
I20260709 12:25:35.512059 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1599) took 1251 ms. Trace:
I20260709 12:25:35.512684 14423 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 32. Current term is 34. Ops: []
I20260709 12:25:35.512773 13801 rpcz_store.cc:276] 0709 12:25:34.261009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.631147 (+370138us) service_pool.cc:224] Handling call
0709 12:25:35.511893 (+880746us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54002cb5d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:35.511997 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348774272}
I20260709 12:25:35.518838 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:35.497257 23624 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.529852 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.529999 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.531026 14416 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:35.514158 23607 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.493726 23663 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.547806 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.550239 14416 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:35.546763 13794 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:35.499374 23763 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.520936 23740 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.552006 14422 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:25:35.531965 23665 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:35.557770 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.518004 23235 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.500689 23746 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:35.555888 23759 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
W20260709 12:25:35.558709 23757 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:25:35.572471 23623 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:35.573026 23623 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:35.573410 23623 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:35.580245 23763 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.587592 23704 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:35.567551 23685 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.008s
I20260709 12:25:35.556388 23693 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:35.531208 23499 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.17, Last appended by leader: 17, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.606735 23235 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.608139 14425 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.608776 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:48736 (request call id 1553) took 1544 ms. Trace:
I20260709 12:25:35.609575 14425 rpcz_store.cc:276] 0709 12:25:34.068141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.068585 (+   444us) service_pool.cc:224] Handling call
0709 12:25:35.606813 (+1538228us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b5400516b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393beba3d 00007f5398bbbcd0 00007f5392140254 00007f53921401e2 00007f5392140111 00007f539214008c 00007f539213f342 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:35.608663 (+  1850us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1595448064}
I20260709 12:25:35.611888 23746 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.547499 23768 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:25:35.648496 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.615512 23740 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.618413 23631 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:35.618963 23767 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 12:25:35.686758 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.561796 23446 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.621688 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:35.689006 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67602.9us
I20260709 12:25:35.622704 23648 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.718538 23648 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.687762 23450 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.720109 23450 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.721076 23648 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:35.623261 23588 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:35.721817 23588 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.688072 23684 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:35.624174 23663 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:35.626032 23762 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:25:35.628105 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:35.628940 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:35.629339 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:35.631569 13796 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 12:25:35.609143 23764 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:25:35.645051 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 126252us
I20260709 12:25:35.648288 13797 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 12:25:35.648577 13794 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:35.649623 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.658610 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:35.668617 13799 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:35.667583 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:35.636826 23609 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:35.700498 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.702329 23631 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.666127 23693 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.726056 13647 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:35.725801 13651 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
W20260709 12:25:35.741801 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51903.3us
I20260709 12:25:35.742341 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.757376 13794 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:35.743271 13801 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 34: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:35.753914 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:35.749899 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.751318 13795 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:35.686803 23685 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.752547 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.765219 13792 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:35.749900 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.743448 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:35.730930 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5000) took 1564 ms (client timeout 2018 ms). Trace:
I20260709 12:25:35.754805 13647 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:35.763561 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.766942 13794 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:25:35.765846 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107350us
I20260709 12:25:35.722223 23450 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.768153 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:35.724068 23588 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:35.766804 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.766767 13649 rpcz_store.cc:269] 0709 12:25:34.166908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.171091 (+  4183us) service_pool.cc:224] Handling call
0709 12:25:35.730659 (+1559568us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:35.730836 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153965312}
I20260709 12:25:35.776620 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.783546 14423 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:35.784981 23450 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:35.785089 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.785786 23450 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:35.786043 13792 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 34
I20260709 12:25:35.786945 23450 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.776182 23739 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:35.722759 23684 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.751392 23511 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:35.796793 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.800669 14425 raft_consensus.cc:2368] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34.
W20260709 12:25:35.766710 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 99257.8us
I20260709 12:25:35.776713 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:35.691550 23738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.008s	sys 0.001s
I20260709 12:25:35.780788 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.780894 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:35.762497 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:35.815567 13792 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:25:35.816155 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:35.818653 13795 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:35.819090 13801 raft_consensus.cc:2368] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34.
I20260709 12:25:35.818125 13794 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:25:35.797277 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:35.765270 23547 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:35.759581 23689 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:35.815994 13647 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:35.818696 23738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.008s	sys 0.004s
I20260709 12:25:35.829706 13764 leader_election.cc:400] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34.
I20260709 12:25:35.834582 13764 leader_election.cc:403] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:35.780164 23704 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:25:35.829954 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2069) took 1740 ms (client timeout 1673 ms). Trace:
W20260709 12:25:35.835911 13720 rpcz_store.cc:269] 0709 12:25:34.092798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.108853 (+ 16055us) service_pool.cc:224] Handling call
0709 12:25:35.829759 (+1720906us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54000276d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:35.829880 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921262976}
I20260709 12:25:35.796504 23763 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:35.795854 23693 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.807042 23511 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.816883 23766 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:35.830613 23588 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:35.827364 23686 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.780640 23685 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.863415 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:35.869187 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:35.873085 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105028us
I20260709 12:25:35.823261 23713 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.022s
I20260709 12:25:35.844990 23511 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:35.895196 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.897282 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 97250.2us
W20260709 12:25:35.934752 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60840.6us
I20260709 12:25:35.846622 23704 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:35.855928 23588 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.961021 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4573) took 1644 ms. Trace:
I20260709 12:25:35.834699 23547 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.840822 23689 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:35.955920 13799 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:35.855280 23746 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 33. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34.
I20260709 12:25:35.872382 23609 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.874150 23624 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:35.874651 23780 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:35.978440 13793 rpcz_store.cc:276] 0709 12:25:34.316503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.060390 (+743887us) service_pool.cc:224] Handling call
0709 12:25:35.088812 (+ 28422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.303559 (+214747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.948321 (+644762us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:35.960934 (+ 12613us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475412864}
W20260709 12:25:36.005030 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49561.5us
W20260709 12:25:35.982888 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 239524us
I20260709 12:25:35.936808 23593 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:35.854293 23627 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.113s
I20260709 12:25:35.863808 23648 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.121s
W20260709 12:25:36.006605 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50220.7us
I20260709 12:25:35.981957 23588 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:36.012056 23780 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:36.012421 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.985589 23446 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:36.013661 23766 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 12:25:35.993566 23779 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:25:35.962303 23709 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:35.864147 23623 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.876286 23686 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.013198 23704 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:35.964280 23684 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:35.993386 23624 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:25:36.041528 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:36.050432 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:36.053714 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 296376us
I20260709 12:25:36.088730 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.064611 13643 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 30 index: 23. Preceding OpId from leader: term: 32 index: 24. (index mismatch)
I20260709 12:25:36.066854 13645 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:25:36.080454 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.094496 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.095966 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.096712 13651 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:36.101099 13649 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:36.102648 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.109344 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:35.994640 23609 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.133072 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:36.255721 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 228646us
I20260709 12:25:36.099960 23648 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:25:36.261001 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 128064us
I20260709 12:25:36.265115 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2070) took 2106 ms. Trace:
I20260709 12:25:36.104610 23631 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:36.268155 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.296121 13649 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:36.270671 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:36.271381 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:36.270394 13724 rpcz_store.cc:276] 0709 12:25:34.150712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.150869 (+   157us) service_pool.cc:224] Handling call
0709 12:25:34.180193 (+ 29324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.079721 (+899528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.256721 (+1177000us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54002f5154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:36.257009 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066945280}
I20260709 12:25:36.239198 23713 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:36.301450 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:36.301844 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.302785 13643 raft_consensus.cc:2410] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:25:36.090384 23709 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.047780 23446 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.303512 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:36.303864 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 426112us
I20260709 12:25:36.161607 23738 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:36.305099 23627 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.271514 23739 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:36.276527 23235 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:36.336031 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75257.4us
I20260709 12:25:36.334156 23664 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:36.331269 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:36.353971 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:36.268739 23648 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:36.354736 14417 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:36.336866 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.315953 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.345755 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:36.335675 23734 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.700s	user 0.005s	sys 0.016s
I20260709 12:25:36.355760 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:36.335821 23781 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:25:36.359539 14425 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:36.374585 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:36.419129 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59458.5us
W20260709 12:25:36.362139 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46359.9us
W20260709 12:25:36.421614 14390 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.007s	sys 0.005s
W20260709 12:25:36.406620 13764 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.002s
W20260709 12:25:36.422168 14390 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.008s	sys 0.005s
W20260709 12:25:36.422437 13764 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.009s	sys 0.002s
W20260709 12:25:36.439783 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.422627 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 620419us
I20260709 12:25:36.419250 13652 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
W20260709 12:25:36.423684 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68063.3us
I20260709 12:25:36.449533 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:36.447266 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:36.450202 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.459904 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.504762 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.373909 23446 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:36.470871 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.450203 14416 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:36.512305 13649 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:36.513152 13651 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:36.353667 23235 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:36.376137 23686 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:36.536613 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.537427 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.360275 23713 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:25:36.522890 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72776.3us
I20260709 12:25:36.539925 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.540428 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.539230 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.541412 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.541894 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.542096 13652 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:36.542662 13651 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:36.542932 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:36.362295 23747 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.520s	user 0.002s	sys 0.010s
I20260709 12:25:36.375172 23780 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:36.373798 23709 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:36.376300 23777 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:36.379405 23547 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.591452 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1556) took 1081 ms. Trace:
I20260709 12:25:36.592059 14419 rpcz_store.cc:276] 0709 12:25:35.510455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.520659 (+ 10204us) service_pool.cc:224] Handling call
0709 12:25:35.526324 (+  5665us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.526718 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.526749 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.534402 (+  7653us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.831246 (+296844us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:35.831328 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.831434 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:36.591141 (+759707us) raft_consensus.cc:1685] finished
0709 12:25:36.591173 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:36.591366 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2449,"raft.queue_time_us":505,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6489,"thread_start_us":264414,"threads_started":1,"wal-append.queue_time_us":539015}
I20260709 12:25:36.555593 23693 raft_consensus.cc:2764] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:25:36.377574 23689 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:36.378940 23609 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:36.522073 23760 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.125s	user 0.004s	sys 0.000s
I20260709 12:25:36.460273 23788 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.597539 23788 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.599051 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2072) took 2154 ms. Trace:
I20260709 12:25:36.599507 13727 rpcz_store.cc:276] 0709 12:25:34.444927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.486901 (+ 41974us) service_pool.cc:224] Handling call
0709 12:25:34.487773 (+   872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:35.356622 (+868849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.598701 (+1242079us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:36.598958 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968335232}
I20260709 12:25:36.601526 23788 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:36.545794 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 98621.4us
I20260709 12:25:36.451056 23739 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.350864 23627 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.375110 23631 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:36.451679 23623 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.445601 23450 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:36.580101 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:36.580299 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.580745 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.641837 13643 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:36.643541 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.644896 13651 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.559053 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:36.546902 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.460143 23256 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:36.653363 14417 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:36.562892 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.547083 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:36.655230 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108297us
I20260709 12:25:36.656313 14424 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:36.359297 23664 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:36.530112 23684 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:36.538127 23446 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.601699 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.539450 23235 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:36.543677 23789 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.648577 13649 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:36.650439 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.683048 13644 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:36.683508 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.683943 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.576370 23713 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:36.576562 23397 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:36.684190 13649 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:36.686120 23713 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.686930 13649 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:36.685062 23397 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.692404 23397 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.684576 13644 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:36.694432 13649 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:25:36.668910 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.651767 23807 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:36.672408 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:36.607787 23794 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:36.672137 23780 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.056s
I20260709 12:25:36.673480 23788 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.695164 13644 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:36.556094 23686 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.696400 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:36.679183 23631 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.740056 23631 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.672967 23450 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:36.698400 23642 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:36.700671 23638 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:36.702505 23789 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.715938 13649 pending_rounds.cc:85] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 11
I20260709 12:25:36.541169 23648 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.738199 13651 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
W20260709 12:25:36.738305 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 467678us
I20260709 12:25:36.675617 23782 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:36.667428 23256 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.747793 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.751535 13652 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:36.751732 13649 pending_rounds.cc:107] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:25:36.751957 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:36.752170 13649 raft_consensus.cc:2889] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:36.753010 23709 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:36.720036 23815 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.726671 23807 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.755116 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.679656 23446 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:36.756310 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119223us
I20260709 12:25:36.759171 13693 leader_election.cc:400] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26.
I20260709 12:25:36.760599 13693 leader_election.cc:403] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:36.727501 23609 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:36.736538 23713 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:36.745196 23638 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:36.764458 23689 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:36.765302 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:36.747226 23642 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.738229 23788 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.770102 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:36.750030 23738 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:25:36.778555 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 120843us
I20260709 12:25:36.760460 23709 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:36.779567 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.783989 23642 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.767455 23704 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26.
I20260709 12:25:36.784655 23642 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:36.785243 23642 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.726557 23794 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.787810 23631 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:36.788460 23794 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.789166 13645 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:25:36.791465 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.726751 23664 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.746744 23686 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.793272 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:36.794097 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.765900 23807 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.783917 23256 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:36.794826 23684 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:36.801199 23446 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.801729 23499 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:36.792672 23738 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.802448 23638 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.803468 23664 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.796105 23704 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:36.771859 23713 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.808312 23627 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:36.781634 23800 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.819921 23800 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.820999 23713 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:36.821885 23638 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:36.806327 23750 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.556s	user 0.004s	sys 0.004s
I20260709 12:25:36.809741 23789 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:36.781908 23709 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:36.781584 23689 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:36.825652 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.771103 23809 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:36.827337 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:36.824127 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:36.832545 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:36.833213 13720 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:25:36.833364 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:36.770730 23609 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.842954 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:36.837301 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:36.831748 13727 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:36.849773 13799 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:36.851120 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:36.853386 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:36.837809 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:36.859369 13800 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:36.833029 23709 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:36.765329 23815 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.863910 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:36.861933 13720 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.832743 23713 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:36.864609 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:36.865896 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.867758 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:36.862958 23511 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.881s	user 0.009s	sys 0.005s
I20260709 12:25:36.848723 23746 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:36.869534 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4459) took 1856 ms (client timeout 1609 ms). Trace:
I20260709 12:25:36.869731 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:36.870811 13727 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:36.872865 13726 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:36.870148 13721 rpcz_store.cc:269] 0709 12:25:35.012883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.026931 (+ 14048us) service_pool.cc:224] Handling call
0709 12:25:36.869225 (+1842294us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002f7454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:36.869438 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403445632}
I20260709 12:25:36.875458 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:36.879591 13800 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:36.881376 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:36.881814 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:36.850445 23809 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.860677 23663 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:36.882406 13725 raft_consensus.cc:2376] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24. Re-sending same reply.
I20260709 12:25:36.857283 23816 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:25:36.886945 13799 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:36.879293 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:36.848802 23446 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:25:36.882068 13797 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:36.866660 23689 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:36.857030 23789 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:25:36.873073 23684 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:36.880789 23631 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:36.889325 13720 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:36.835541 23704 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:25:36.877496 23256 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:36.888424 23813 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:25:36.902412 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111988us
I20260709 12:25:36.884110 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:36.886636 23800 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:36.910856 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.885846 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:36.884568 23809 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.878599 23511 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:36.899130 23684 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.900637 23631 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.914988 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.917100 14391 leader_election.cc:400] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34.
I20260709 12:25:36.914294 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:36.918495 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:36.940066 14391 leader_election.cc:403] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Cancelling election due to peer responding with higher term
I20260709 12:25:36.940539 13648 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:36.941898 13652 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:25:36.944096 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.907011 23446 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:36.904292 23813 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:36.940831 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 75029.8us
I20260709 12:25:36.913679 23663 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:25:36.951339 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72099.6us
I20260709 12:25:36.904778 23808 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.042s
I20260709 12:25:36.948904 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:36.913255 23789 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:36.918287 23684 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:36.954622 13690 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:36.959215 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.810s (SENT) [suppressed 20 similar messages]
I20260709 12:25:36.944825 23780 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:36.943568 23736 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:36.946122 23631 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:36.964593 23256 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:36.964856 23824 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:36.966234 23824 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.966747 23824 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:36.947947 23642 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:36.970775 23642 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:36.984562 23693 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election lost for term 33. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34.
I20260709 12:25:36.985641 23693 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:36.914614 23818 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:25:36.988117 23256 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:36.988448 23816 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:36.949283 23813 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:36.987497 23593 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:36.996629 23593 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:36.997758 23648 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:37.001523 23235 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.006560 23568 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:37.008085 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48916.4us
W20260709 12:25:37.021936 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.947750 23446 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:36.997613 23593 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:36.953656 23800 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:36.951221 23663 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.986078 23693 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:37.024622 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 148706us
I20260709 12:25:36.958714 23788 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.008165 23780 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.007706 23736 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:37.023816 23820 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:37.018975 23704 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:36.961680 23789 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:37.055930 23780 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.059155 23736 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.024462 23713 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:37.050755 23686 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.071792 23593 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:37.081910 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.090193 23782 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:37.092450 23818 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.092991 13796 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3141) took 2721 ms (client timeout 2104 ms). Trace:
I20260709 12:25:37.096436 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1597) took 2879 ms. Trace:
W20260709 12:25:37.140203 13796 rpcz_store.cc:269] 0709 12:25:34.371647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.864550 (+492903us) service_pool.cc:224] Handling call
0709 12:25:37.092584 (+2228034us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b540012b454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:37.092874 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1728285568}
I20260709 12:25:37.140278 13798 rpcz_store.cc:276] 0709 12:25:34.217141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.039943 (+822802us) service_pool.cc:224] Handling call
0709 12:25:37.094597 (+2054654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.094805 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.094830 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.094874 (+    44us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:37.094921 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.094974 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.096227 (+  1253us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540012b450. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:37.096367 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4709454720}
I20260709 12:25:37.097571 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4196) took 2210 ms. Trace:
I20260709 12:25:37.097723 23618 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:37.098600 13792 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 32. Current term is 34. Ops: []
I20260709 12:25:37.099239 23511 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.100181 13801 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:25:37.106451 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56898.3us
W20260709 12:25:37.108000 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 141184us
I20260709 12:25:37.101495 23820 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:37.097959 23738 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:37.048980 23663 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:37.114797 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 160750us
I20260709 12:25:37.093581 23807 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.051460 23693 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:37.176296 23807 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.176786 13650 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:25:37.070906 23815 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:37.116770 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 172765us
I20260709 12:25:37.179200 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:37.115952 23773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.843s	user 0.005s	sys 0.000s
W20260709 12:25:37.180133 23773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.006s	sys 0.000s
I20260709 12:25:37.180411 13798 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:37.141716 13726 rpcz_store.cc:276] 0709 12:25:34.887102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.171435 (+284333us) service_pool.cc:224] Handling call
0709 12:25:36.871264 (+1699829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.878662 (+  7398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.097326 (+218664us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54002f7450. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:37.097490 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3895341440}
I20260709 12:25:37.045053 23800 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.181389 23800 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:37.181878 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.181420 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:37.184871 13798 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31.
I20260709 12:25:37.185755 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:37.137776 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.188617 13798 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:37.132777 23609 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.155812 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.190124 13801 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:37.137178 23822 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
W20260709 12:25:37.157588 23588 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.111s	user 0.012s	sys 0.001s
I20260709 12:25:37.191839 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.191877 23588 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.192677 13797 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:37.158360 23789 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.151093 23788 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:37.161370 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.141644 23782 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:37.162807 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.172747 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.173393 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.174113 13647 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 34
I20260709 12:25:37.179378 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:37.129432 23713 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.197342 23684 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.116725 23704 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:37.212601 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 251050us
I20260709 12:25:37.279526 13800 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
W20260709 12:25:37.284164 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 104875us
I20260709 12:25:37.284745 13792 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:37.209967 23713 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.118453 23446 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.146167 23818 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.285041 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:37.161065 23736 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.147373 23618 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.209267 23807 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.320323 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:37.271023 23740 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.205709 23499 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.206503 23588 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.209371 23746 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:37.317164 13793 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:25:37.329279 13796 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:37.329695 23818 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.335096 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.336412 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.337132 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.297847 23780 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.337596 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.337761 23829 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:25:37.338296 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.338776 13652 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:37.339679 23794 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.339709 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.342439 23634 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:37.342514 23500 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.343564 23822 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:37.343917 23618 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.344266 23568 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:25:37.344553 23740 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.344290 23807 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.346378 23684 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.347024 13618 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31.
I20260709 12:25:37.423785 13618 leader_election.cc:403] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.347448 23763 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:37.347994 23704 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.353400 13647 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
W20260709 12:25:37.358668 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.680s (SENT)
I20260709 12:25:37.380544 13643 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:37.327448 23709 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.396263 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.305064 23450 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:37.397984 13793 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:25:37.412077 23830 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:25:37.304141 23825 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:37.426330 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 79452.5us
I20260709 12:25:37.368430 23499 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.369474 23824 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.441257 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:37.360574 23813 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.332242 23446 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:37.480543 13801 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:37.481681 23813 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.422966 23822 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:37.396806 23746 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.407914 23829 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:37.440510 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1599) took 1812 ms (client timeout 1879 ms). Trace:
W20260709 12:25:37.477012 23827 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
W20260709 12:25:37.477809 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 197093us
I20260709 12:25:37.477677 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:37.456670 13793 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:37.452447 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.440635 13646 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:25:37.409233 23638 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.000s
I20260709 12:25:37.486768 13643 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
W20260709 12:25:37.478223 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119631us
I20260709 12:25:37.490387 13800 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:25:37.450312 23664 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.632s	user 0.007s	sys 0.006s
W20260709 12:25:37.520480 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.520584 23822 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.522330 23822 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:37.425236 23500 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:37.477658 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 192706us
I20260709 12:25:37.523862 23499 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.525352 23500 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.525722 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:37.449826 23704 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.526636 23704 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.527395 23785 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:37.527906 23709 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.528834 23704 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.482466 23824 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.484694 23825 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.525568 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.530886 13643 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:37.532092 23825 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.534641 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:37.535475 14423 raft_consensus.cc:2410] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:25:37.449038 23684 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:37.536374 23684 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:37.536783 23684 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.539110 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.526634 23773 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:25:37.540988 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.541603 23740 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.539701 23663 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.541887 13652 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:37.539979 23627 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31.
I20260709 12:25:37.543448 23807 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.546000 23663 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.368682 23820 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.546862 13643 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:25:37.547346 23820 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:37.548794 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.533877 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:37.511344 13647 rpcz_store.cc:269] 0709 12:25:35.628482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.629104 (+   622us) service_pool.cc:224] Handling call
0709 12:25:37.173611 (+1544507us) spinlock_profiling.cc:243] Waited 66.3 ms on lock 0x7b58005579d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b500aa 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf827c 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c
0709 12:25:37.440409 (+266798us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152403712}
I20260709 12:25:37.555464 23763 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:37.556097 23763 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.556546 23763 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:37.526126 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.484488 23813 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.527055 23834 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:25:37.521397 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.788s (SENT)
I20260709 12:25:37.573933 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:37.575481 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.576584 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.427070 23634 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.564891 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.581300 13647 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:37.581781 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60420.7us
I20260709 12:25:37.577400 13797 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:37.561040 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:37.457855 23450 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:37.570427 23704 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.572775 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:37.422807 23794 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.579748 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.566053 13644 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.512349 23839 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:37.515334 23829 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:25:37.455016 23631 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.593129 13643 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:37.604586 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.605614 13643 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 34: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:37.587005 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:37.607293 13767 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:37.595368 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:37.600692 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:37.580504 23825 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:37.589897 23775 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:25:37.590387 23663 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.618762 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:37.612627 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1230149109997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260709 12:25:37.624639 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1074359379999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:25:37.625447 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0555149110000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 12:25:37.612799 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.604406 23841 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:37.631417 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1565) took 1088 ms. Trace:
W20260709 12:25:37.629294 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 141382us
I20260709 12:25:37.628091 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:37.631997 14425 rpcz_store.cc:276] 0709 12:25:36.543248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.543397 (+   149us) service_pool.cc:224] Handling call
0709 12:25:36.577651 (+ 34254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.904747 (+327096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:36.904799 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:36.905477 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.114376 (+208899us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:37.114426 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.274169 (+159743us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:37.274232 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.623375 (+349143us) raft_consensus.cc:1685] finished
0709 12:25:37.623400 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.631253 (+  7853us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16424,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"spinlock_wait_cycles":309748096,"thread_start_us":355221,"threads_started":2,"wal-append.queue_time_us":245475}
I20260709 12:25:37.633607 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:37.635565 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.636445 13720 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:25:37.620723 13647 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:37.575451 23813 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.638128 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:37.667486 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.614161 23807 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:37.670934 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:37.610349 23844 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:37.671887 13720 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:25:37.613968 23638 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:37.674706 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:37.599529 23704 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.626185 23823 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.603612 23773 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 12:25:37.666500 23809 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.003s
I20260709 12:25:37.694327 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:37.638845 23593 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:37.699008 13720 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:37.610695 23839 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.677386 23663 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:37.669659 23807 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.824031 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.728806 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.636662 23840 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.632377 23713 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.687940 23780 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.622071 23825 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.846561 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:37.847265 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 216913us
I20260709 12:25:37.840938 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.849893 13721 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
W20260709 12:25:37.850086 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 254804us
W20260709 12:25:37.725476 23842 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:25:37.691264 23847 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.692375 23638 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.697829 23841 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:37.674907 23618 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:37.857654 13724 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:37.858354 13720 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:37.863828 13721 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:37.847988 23593 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:37.879640 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.880290 13721 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:37.697877 23826 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:37.823235 23740 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.902987 23740 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.723661 23685 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:37.906494 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.907424 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.908082 23785 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:37.908846 23785 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.857889 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:37.907454 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.910854 23854 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:25:37.911049 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53271.6us
I20260709 12:25:37.912122 23773 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:37.912536 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:37.912881 23773 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.913429 23854 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:37.872853 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.848742 23807 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:37.914117 13719 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:37.914798 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:37.915659 23854 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:37.917706 23704 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:37.921878 23800 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.000s
I20260709 12:25:37.922412 23800 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.451630 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.928606 13547 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.697759 23450 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.872979 23638 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:37.937518 23235 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.889106 23847 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.942770 13722 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:25:37.723978 23813 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.731478 23634 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.856611 23663 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:37.892876 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.894531 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:37.856398 23848 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.058s
I20260709 12:25:37.894953 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:37.877279 23841 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:37.911131 13724 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:37.961077 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.888732 23826 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.967767 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:37.943950 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.856163 23840 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.845145 23809 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.952385 13647 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:37.896265 14425 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:37.689495 23844 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.958508 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:37.974421 13651 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:37.961943 13720 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:37.846473 23843 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:25:37.724254 23823 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:37.963130 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.003026 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:25:37.972720 23693 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.793s	user 0.010s	sys 0.003s
I20260709 12:25:37.944682 23824 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:25:37.969262 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:37.892886 23618 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 34 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:38.039353 23832 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.661s	user 0.003s	sys 0.000s
I20260709 12:25:38.039494 23618 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.041299 23839 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.971903 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.980993 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:37.984000 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3142) took 1934 ms (client timeout 2113 ms). Trace:
I20260709 12:25:37.987567 23663 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:37.988842 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:38.043529 13650 rpcz_store.cc:269] 0709 12:25:36.049593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.052286 (+  2693us) service_pool.cc:224] Handling call
0709 12:25:37.983746 (+1931460us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:37.983915 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2394561920}
I20260709 12:25:37.990657 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:37.963567 23841 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:37.990087 13799 raft_consensus.cc:2368] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29.
I20260709 12:25:37.994037 23844 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.995532 23847 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:37.996927 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.056413 13727 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:38.056658 13722 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:38.058071 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.059085 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:38.060377 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.061162 13722 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:37.998628 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.063359 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:37.996743 23845 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.472s	user 0.002s	sys 0.000s
W20260709 12:25:37.968591 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53879.1us
W20260709 12:25:37.930032 23736 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.001s
I20260709 12:25:38.075311 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:38.011969 13644 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:37.725172 23746 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:37.941260 23593 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.013010 23631 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:38.026501 23822 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 12:25:38.026767 23693 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.013298 13719 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:38.015478 23794 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:38.017055 23808 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.012s
W20260709 12:25:38.029095 23446 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.004s
I20260709 12:25:38.035239 23825 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.035869 23713 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:37.972122 23826 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:38.037724 23840 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.025482 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.114543 13726 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:38.048048 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.046746 13724 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:38.048709 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:38.117923 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:38.121374 13720 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:38.111896 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.123631 13719 raft_consensus.cc:2410] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:25:38.052804 13647 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:38.052858 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:38.045058 23782 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.766s	user 0.002s	sys 0.012s
I20260709 12:25:38.083345 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.155025 13721 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:38.090286 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:38.090988 23823 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:38.093653 23740 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.082863 23816 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:38.022995 23800 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:38.099696 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:38.130409 23813 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.049127 23820 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:38.107167 14418 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:38.104736 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:38.129213 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:38.188314 13646 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:38.188369 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83731.7us
I20260709 12:25:38.058975 23783 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.052s
I20260709 12:25:38.097941 23638 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:38.139217 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.127727 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:38.026158 23684 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.194828 14420 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:38.196053 23684 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.196841 23684 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.197402 23684 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.158327 13615 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:38.160382 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.199043 13726 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:38.119196 23822 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:38.169229 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:38.200024 23822 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:38.200464 23822 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:38.200935 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:38.174839 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.179388 13792 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:38.204358 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.205156 13792 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.020416 23795 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.082s
I20260709 12:25:38.130558 23824 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:38.184885 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3164) took 1032 ms. Trace:
I20260709 12:25:38.209344 13798 rpcz_store.cc:276] 0709 12:25:37.152301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.189335 (+ 37034us) service_pool.cc:224] Handling call
0709 12:25:37.189685 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.281834 (+ 92149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.281875 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.289869 (+  7994us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.487547 (+197678us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:37.487595 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.487665 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:37.996025 (+508360us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:38.177575 (+181550us) raft_consensus.cc:1685] finished
0709 12:25:38.177613 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.184771 (+  7158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82966,"raft.queue_time_us":85703,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":17679,"spinlock_wait_cycles":73930112,"thread_start_us":151654,"threads_started":1,"wal-append.queue_time_us":206389}
W20260709 12:25:38.210702 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83061.3us
I20260709 12:25:38.211628 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:38.212062 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:38.186820 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.190613 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:38.189998 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:38.178313 23829 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.006s
W20260709 12:25:38.206976 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 48987us
I20260709 12:25:38.130149 23825 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
W20260709 12:25:38.192353 23763 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
I20260709 12:25:38.181701 23816 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.212935 13798 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:38.257081 13649 raft_consensus.cc:2368] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:38.151293 23780 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:25:38.207230 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:38.255885 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.160244 23782 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.211628 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:38.264232 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52688.9us
I20260709 12:25:38.253921 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.265352 13800 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:38.265537 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:38.260011 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52867.8us
I20260709 12:25:38.260076 13726 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:38.258589 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:38.208300 23709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.017s	sys 0.006s
I20260709 12:25:38.237494 13722 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:38.228492 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:38.210238 23823 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:38.273237 23823 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:38.218405 23850 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.210249 23740 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:25:38.221253 23631 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.183216 23807 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.257453 23808 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.191624 23813 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.268976 23500 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:38.226468 23788 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:38.276697 23593 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:25:38.285130 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:38.269029 23709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.017s	sys 0.015s
I20260709 12:25:38.251447 23829 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.280018 23818 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:38.225605 23642 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.226449 23800 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:38.206159 23820 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.292066 14419 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:38.276310 23685 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:38.222152 23824 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:38.255535 23763 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.313220 13792 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
W20260709 12:25:38.316711 13690 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.581s (SENT)
W20260709 12:25:38.295040 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66639.2us
I20260709 12:25:38.295888 23500 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.302990 23839 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:38.331986 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:38.314002 13721 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:25:38.347486 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 218973us
I20260709 12:25:38.283869 23850 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.287329 23631 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.325418 23693 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.328477 23824 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:38.353821 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:38.317188 23818 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:38.358156 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46238.5us
I20260709 12:25:38.336519 23788 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:38.377097 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:38.325816 23642 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.344890 23500 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:38.343225 23800 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:38.373059 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.373711 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:38.378192 13726 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:38.355372 23840 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:38.359238 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:38.384279 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5036) took 1048 ms. Trace:
I20260709 12:25:38.348703 23618 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:38.425360 13800 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:38.357779 23823 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.395959 23813 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:38.397105 23782 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:38.374904 23685 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:38.399075 23631 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:38.401172 23763 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.401847 23638 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.401983 23850 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:38.375907 23839 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:38.446017 23839 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.449096 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4590) took 1354 ms. Trace:
I20260709 12:25:38.402964 23807 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:38.403280 23820 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:38.403935 23800 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:38.405241 23593 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:38.461182 23865 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:38.462332 23773 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.408910 23829 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:38.463661 23785 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:38.411136 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58513.3us
I20260709 12:25:38.393939 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:38.417138 13652 rpcz_store.cc:276] 0709 12:25:37.336324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.542607 (+206283us) service_pool.cc:224] Handling call
0709 12:25:37.543263 (+   656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.543872 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:37.698608 (+154736us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.084676 (+386068us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.106231 (+ 21555us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.111096 (+  4865us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:38.111164 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.111261 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.384039 (+272778us) raft_consensus.cc:1685] finished
0709 12:25:38.384065 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.384210 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":411360,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":124,"raft.queue_time_us":5378,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":14054,"spinlock_wait_cycles":5504,"thread_start_us":375874,"threads_started":2,"wal-append.queue_time_us":374413}
I20260709 12:25:38.465410 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:38.412588 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.467108 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.387571 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.432593 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.433621 13793 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:25:38.468102 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:38.468925 13797 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:38.469234 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:38.469826 13793 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:38.469271 14416 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:25:38.377681 23818 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:38.442981 23859 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:25:38.423904 23868 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:38.449656 13794 rpcz_store.cc:276] 0709 12:25:37.095025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.098983 (+  3958us) service_pool.cc:224] Handling call
0709 12:25:37.099291 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.352380 (+253089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.352424 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.352475 (+    51us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:37.352521 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.448773 (+1096252us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54001004d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:38.448839 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.449014 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581175680}
I20260709 12:25:38.454618 13646 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:38.461107 23593 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:38.429939 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.472513 13792 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:38.476346 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.505106 13721 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:38.476862 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.477185 13726 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 21. Ops: []
I20260709 12:25:38.455806 23684 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:38.443251 23685 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:38.481700 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87818us
I20260709 12:25:38.505710 13722 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:38.506619 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:38.471012 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 94054us
I20260709 12:25:38.494598 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:25:38.485252 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104888us
I20260709 12:25:38.509385 13794 raft_consensus.cc:606] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:38.510376 13794 raft_consensus.cc:614] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:38.507647 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:38.524190 13792 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:38.484464 23689 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:38.536252 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.528013 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:38.539963 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67117.5us
I20260709 12:25:38.540508 13726 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:38.504419 23593 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:38.546900 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.550998 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.571688 13726 raft_consensus.cc:2368] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22.
I20260709 12:25:38.572633 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:38.551905 23862 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:25:38.552279 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.580575 13720 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:38.553983 23840 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:38.554953 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4596) took 1390 ms (client timeout 1853 ms). Trace:
W20260709 12:25:38.584863 13799 rpcz_store.cc:269] 0709 12:25:37.164821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.172457 (+  7636us) service_pool.cc:224] Handling call
0709 12:25:38.554678 (+1382221us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54000d2fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:38.554870 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800373120}
W20260709 12:25:38.559461 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2077) took 1722 ms (client timeout 1850 ms). Trace:
I20260709 12:25:38.563680 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.587023 13727 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:38.588529 13727 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:38.564589 23500 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.539075 23741 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.007s
W20260709 12:25:38.594960 23877 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:25:38.595934 23807 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:38.599619 23500 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.569126 13722 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:38.601122 13722 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:38.602627 13722 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:25:38.580324 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73766.6us
W20260709 12:25:38.592447 13723 rpcz_store.cc:269] 0709 12:25:36.836797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.836951 (+   154us) service_pool.cc:224] Handling call
0709 12:25:38.559099 (+1722148us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:38.559362 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651177728}
I20260709 12:25:38.603881 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:38.581509 13721 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:38.567317 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:38.604251 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:38.552717 23873 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:25:38.608954 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:38.609133 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:38.609815 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4607) took 1041 ms. Trace:
I20260709 12:25:38.590215 13727 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:38.610286 13796 rpcz_store.cc:276] 0709 12:25:37.568637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.896753 (+328116us) service_pool.cc:224] Handling call
0709 12:25:37.897224 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.897759 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.897802 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.900596 (+  2794us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.151111 (+250515us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:38.158753 (+  7642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.159503 (+   750us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.609597 (+450094us) raft_consensus.cc:1685] finished
0709 12:25:38.609619 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.609755 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4770,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"thread_start_us":249965,"threads_started":1,"wal-append.queue_time_us":355692}
I20260709 12:25:38.613236 13721 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:38.613121 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:38.639546 13726 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:38.584991 23689 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:38.618381 23813 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:38.612665 23883 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:38.678718 23816 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:38.652302 23807 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:38.685595 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.691668 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.647040 23663 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.704413 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.670617 23808 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:38.707687 13720 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:38.667500 23813 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:38.708819 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.709995 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:38.710688 13644 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:38.710611 13647 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:25:38.712245 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.712581 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:38.713011 13721 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22. Re-sending same reply.
I20260709 12:25:38.717172 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:38.717919 13646 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:25:38.723328 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.688685 23807 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:38.727475 13720 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:25:38.728955 13720 pending_rounds.cc:85] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:25:38.729322 13720 pending_rounds.cc:107] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:25:38.729635 13720 raft_consensus.cc:2889] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:38.726557 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:38.707154 23789 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:38.701306 23883 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:38.754211 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.755903 13646 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:38.707391 23800 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:38.763155 13720 raft_consensus.cc:2376] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30. Re-sending same reply.
I20260709 12:25:38.760478 13722 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:38.762310 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.762686 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.764448 13643 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:25:38.715830 23813 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:38.771611 13644 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:38.722460 23642 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:38.776859 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 163811us
I20260709 12:25:38.752565 23807 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:38.783866 13793 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:38.707945 23663 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.784142 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:38.796804 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.800499 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4613215991134c66b270c6024bec9d73"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:38.801735 13793 raft_consensus.cc:606] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:38.803380 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.803529 13793 raft_consensus.cc:614] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:38.783746 23822 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.006s
I20260709 12:25:38.791294 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.812674 13646 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:38.814656 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.815364 13644 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:38.804209 13720 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:38.816176 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.815367 13725 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:38.820204 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:38.831934 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.783299 23809 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:38.859201 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.818830 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.861676 13646 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:38.863420 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.862905 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.864838 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.865697 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.866006 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 19.
I20260709 12:25:38.858714 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.858985 13793 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:38.788826 23642 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:38.807875 23841 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.004s
W20260709 12:25:38.867573 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 141098us
I20260709 12:25:38.805123 23808 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.867739 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:38.825037 23867 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.313s	user 0.001s	sys 0.001s
I20260709 12:25:38.870344 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:38.897228 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.898178 13721 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:38.765482 23689 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.17, Last appended by leader: 17, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:38.886991 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102938us
I20260709 12:25:38.892535 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:38.927538 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:38.824714 23895 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:25:38.890219 23837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.367s	user 0.003s	sys 0.017s
I20260709 12:25:38.936192 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:38.937132 13727 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:38.825639 23822 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.949432 13797 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:38.900374 23809 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.906770 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:38.928067 23879 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:25:38.985843 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 114473us
I20260709 12:25:38.941531 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:38.944205 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.004299 13727 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:39.014925 13693 leader_election.cc:336] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.104s (SENT)
I20260709 12:25:39.013902 13650 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:39.016976 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.018333 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:38.939805 23837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.417s	user 0.004s	sys 0.017s
I20260709 12:25:39.022989 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:39.025367 13797 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:39.025753 13797 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:38.931348 23895 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.028815 23895 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:38.886299 23642 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:39.020232 23882 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 12:25:38.981019 23627 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.013458 23829 raft_consensus.cc:2764] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:25:38.994915 23900 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:39.058842 23900 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:39.060902 23788 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.677s	user 0.004s	sys 0.008s
I20260709 12:25:39.061429 23788 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:39.043671 13719 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:39.062431 13800 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
W20260709 12:25:39.067512 23889 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.258s	user 0.000s	sys 0.000s
I20260709 12:25:39.039436 13721 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:25:39.048867 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4612) took 1464 ms. Trace:
I20260709 12:25:39.041448 23838 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:39.040997 23892 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:25:38.892951 23841 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.070075 13795 rpcz_store.cc:276] 0709 12:25:37.584710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.999341 (+414631us) service_pool.cc:224] Handling call
0709 12:25:37.999703 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.000130 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.000158 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.000644 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.328888 (+328244us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:38.328939 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.329020 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.048506 (+719486us) raft_consensus.cc:1685] finished
0709 12:25:39.048541 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.048773 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4766,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":10955,"thread_start_us":327854,"threads_started":1,"wal-append.queue_time_us":388828}
I20260709 12:25:39.017640 23809 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:38.901352 23808 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:39.142846 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 256666us
W20260709 12:25:39.155377 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 227914us
I20260709 12:25:39.158414 23850 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:39.158566 23838 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.159351 23850 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.159574 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:39.159504 23838 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.161131 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.162075 23838 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.162487 23831 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:39.162806 23838 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:39.163288 23831 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:39.164258 23831 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:39.085796 23901 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:39.147086 23822 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.166837 14425 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 19. Current term is 20. Ops: []
W20260709 12:25:39.170615 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.171187 13617 connection.cc:256] RPC call timeout handler was delayed by -1.3815662790002534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29940000000000005s for a response
I20260709 12:25:39.162501 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.069767 23627 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:39.173599 23887 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.359s	user 0.002s	sys 0.000s
I20260709 12:25:39.174901 23896 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.174913 23822 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:39.175706 23896 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.176343 23663 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.175207 23627 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.180894 23809 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.059855 23900 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.183771 23642 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:39.189065 23822 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:39.190627 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:39.203788 23838 raft_consensus.cc:2764] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
I20260709 12:25:39.204061 13693 leader_election.cc:400] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:39.204406 13693 leader_election.cc:403] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.204694 23807 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.156092 23788 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.196616 23872 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.400s	user 0.001s	sys 0.001s
I20260709 12:25:39.198961 23746 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.219985 23746 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.199869 23808 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.222648 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4614) took 1325 ms. Trace:
I20260709 12:25:39.222934 23808 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:38.935933 23883 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.223181 13801 rpcz_store.cc:276] 0709 12:25:37.896768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.028950 (+132182us) service_pool.cc:224] Handling call
0709 12:25:38.029424 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.029870 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.029913 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.085366 (+ 55453us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.277516 (+192150us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:38.277559 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.277630 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.222375 (+944745us) raft_consensus.cc:1685] finished
0709 12:25:39.222402 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.222578 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16036,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":10569,"thread_start_us":152170,"threads_started":1,"wal-append.queue_time_us":619143}
I20260709 12:25:39.216823 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:39.230288 13650 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:25:39.216729 23838 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:39.217778 23901 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:39.218461 23898 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
W20260709 12:25:39.244179 23886 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.380s	user 0.001s	sys 0.001s
I20260709 12:25:39.212131 23822 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:25:39.240693 23838 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:39.241807 23918 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:39.241623 23901 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:39.229007 23891 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.345s	user 0.001s	sys 0.001s
W20260709 12:25:39.229242 23906 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
W20260709 12:25:39.228704 23905 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:39.206768 23663 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.254009 13794 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.014s	sys 0.041s
W20260709 12:25:39.254875 13792 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.011s	sys 0.025s
W20260709 12:25:39.275970 13794 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.014s	sys 0.041s
W20260709 12:25:39.276031 13792 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.011s	sys 0.026s
I20260709 12:25:39.216094 23850 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:39.250559 23918 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:39.251236 23838 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:39.229279 23788 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:39.278822 23824 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.890s	user 0.007s	sys 0.007s
I20260709 12:25:39.274480 23896 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:39.303045 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1568) took 1262 ms. Trace:
I20260709 12:25:39.304301 14425 rpcz_store.cc:276] 0709 12:25:38.040842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.046438 (+  5596us) service_pool.cc:224] Handling call
0709 12:25:38.082882 (+ 36444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.168855 (+1085973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.302701 (+133846us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400555fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:39.302958 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492321536}
I20260709 12:25:39.307701 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4486) took 1308 ms. Trace:
I20260709 12:25:39.308583 13718 rpcz_store.cc:276] 0709 12:25:37.999586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.014138 (+ 14552us) service_pool.cc:224] Handling call
0709 12:25:38.014553 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:38.041948 (+ 27395us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:38.334402 (+292454us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.535854 (+201452us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.537631 (+  1777us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:38.540841 (+  3210us) log.cc:844] Serialized 52 byte log entry
0709 12:25:38.541022 (+   181us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:38.568801 (+ 27779us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.569047 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.569137 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.307351 (+738214us) raft_consensus.cc:1685] finished
0709 12:25:39.307386 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.307614 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16568,"prepare.queue_time_us":778608,"prepare.run_cpu_time_us":147,"prepare.run_wall_time_us":152,"raft.queue_time_us":19427,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":48753,"spinlock_wait_cycles":46061056,"thread_start_us":405834,"threads_started":2,"wal-append.queue_time_us":412981}
I20260709 12:25:39.290714 23746 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.279660 23663 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.309558 23824 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.333217 23838 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:39.342449 23904 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
W20260709 12:25:39.349578 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.351373 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 145340us
I20260709 12:25:39.355239 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:39.356199 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0249755209997602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 12:25:39.356768 13795 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:39.347500 23813 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.002s
I20260709 12:25:39.361321 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:39.364624 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:39.365733 13797 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:39.368296 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:39.369946 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.373081 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:39.359905 23593 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.754s	user 0.010s	sys 0.004s
I20260709 12:25:39.381009 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.385716 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:39.357983 23837 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:39.387760 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.361505 23841 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.336526 23918 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.443192 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.484198 23900 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.504611 23824 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:39.504881 23919 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:25:39.379927 23568 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.125s
I20260709 12:25:39.505895 23568 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:39.507261 23914 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:25:39.379652 23807 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.515213 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4621) took 1204 ms. Trace:
I20260709 12:25:39.517050 13792 rpcz_store.cc:276] 0709 12:25:38.310868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.529775 (+218907us) service_pool.cc:224] Handling call
0709 12:25:38.540672 (+ 10897us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.610094 (+ 69422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.610125 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.636652 (+ 26527us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.061936 (+425284us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b48001b0258. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:39.277138 (+215202us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:39.277195 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.277289 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.514504 (+237215us) raft_consensus.cc:1685] finished
0709 12:25:39.514558 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.514959 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68172,"raft.queue_time_us":68935,"raft.run_cpu_time_us":2518,"raft.run_wall_time_us":448452,"spinlock_wait_cycles":44243328,"thread_start_us":609150,"threads_started":1,"wal-append.queue_time_us":678458}
I20260709 12:25:39.520458 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.521189 23822 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:39.521767 13792 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:25:39.488135 13797 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:39.490840 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4211) took 1014 ms. Trace:
I20260709 12:25:39.522783 13719 rpcz_store.cc:276] 0709 12:25:38.476636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.476909 (+   273us) service_pool.cc:224] Handling call
0709 12:25:38.502974 (+ 26065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.062726 (+559752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.490522 (+427796us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b54002f7954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:39.490764 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1233990912}
W20260709 12:25:39.495376 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109759us
I20260709 12:25:39.380095 23746 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:39.533002 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.533787 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:39.523696 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.524283 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:39.502777 13795 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:39.538128 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.539110 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:39.540292 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.361358 23813 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.548372 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.554317 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.555141 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.558588 13792 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:39.559420 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.559813 13724 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:39.559916 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.560672 13719 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:39.563743 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.564401 13721 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
W20260709 12:25:39.545595 23826 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.415s	user 0.013s	sys 0.005s
I20260709 12:25:39.547837 23500 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:25:39.569473 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45271.8us
I20260709 12:25:39.570644 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.573944 13693 leader_election.cc:400] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:39.574803 13722 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:39.574908 13693 leader_election.cc:403] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.386562 23593 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.581300 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.582289 23928 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23.
I20260709 12:25:39.389183 23627 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.163s
I20260709 12:25:39.592116 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.564966 23918 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:39.549135 23912 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:39.596792 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.601392 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.605316 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:39.606516 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.577696 23788 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.570035 23920 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:39.593670 23713 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:39.527980 23921 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:25:39.579100 23896 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:39.579241 23500 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:39.600353 23837 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:25:39.615747 23915 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:25:39.616662 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1571) took 1545 ms. Trace:
I20260709 12:25:39.618867 23499 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:39.586884 23826 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.578598 23746 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:39.610018 23818 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.131s	user 0.017s	sys 0.000s
I20260709 12:25:39.610945 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.630072 23831 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.630724 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.638890 23920 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.641203 23813 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.643221 23921 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.694196 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 307042us
I20260709 12:25:39.647954 23500 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.670502 23928 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:39.651427 14420 rpcz_store.cc:276] 0709 12:25:38.071652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.071965 (+   313us) service_pool.cc:224] Handling call
0709 12:25:38.091243 (+ 19278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.195483 (+104240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.616271 (+1420788us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54006e0fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:39.616582 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237169536}
I20260709 12:25:39.652526 23499 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:39.709638 13795 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:39.672879 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71634.8us
I20260709 12:25:39.652380 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1621) took 1274 ms. Trace:
I20260709 12:25:39.711879 13794 rpcz_store.cc:276] 0709 12:25:38.377579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.606817 (+229238us) service_pool.cc:224] Handling call
0709 12:25:38.607136 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.607653 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.607733 (+    80us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.609108 (+  1375us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.279165 (+670057us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:39.279613 (+   448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.279696 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.652105 (+372409us) raft_consensus.cc:1685] finished
0709 12:25:39.652132 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.652309 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":194,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":11071,"thread_start_us":667357,"threads_started":1,"wal-append.queue_time_us":758174}
I20260709 12:25:39.655287 23826 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.664752 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.694602 13801 raft_consensus.cc:2368] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27.
I20260709 12:25:39.716795 13724 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:39.643576 23663 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.699146 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.703003 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.705204 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.701105 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:39.656777 23789 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.892s	user 0.015s	sys 0.000s
W20260709 12:25:39.630157 23895 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.601s	user 0.016s	sys 0.000s
I20260709 12:25:39.726861 23895 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.729334 23895 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:39.663313 23879 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.005s
I20260709 12:25:39.680661 13725 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:25:39.632016 23593 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:39.719233 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.720556 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.731824 13724 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:39.732798 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.721487 13618 leader_election.cc:400] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29.
I20260709 12:25:39.723528 23831 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.723834 23336 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:39.713995 13722 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:39.659469 23818 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.723847 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.724959 23499 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.730122 23879 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:39.719575 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:39.734957 13618 leader_election.cc:403] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.738775 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.718103 23500 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:39.741395 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.745844 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:39.707785 23785 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:25:39.745106 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:39.747404 13722 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:39.745713 13724 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:25:39.734431 23928 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.751798 13793 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:39.752544 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.755681 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.757066 13722 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:39.759690 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.760084 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.760414 13724 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:39.656147 23746 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.761672 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.762321 13722 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:39.761547 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.764539 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.768301 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.770551 13693 leader_election.cc:400] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27.
I20260709 12:25:39.770913 13693 leader_election.cc:403] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.781453 14424 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:39.709120 23921 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.751302 23499 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:39.750432 23831 raft_consensus.cc:2764] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:25:39.791604 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70269us
I20260709 12:25:39.769874 23775 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:39.801182 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.767108 23918 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:39.769565 23837 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.765058 23934 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:39.770238 23689 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:39.782244 23917 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:39.792454 23713 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:39.806178 23840 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27.
I20260709 12:25:39.799345 23879 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:39.840492 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:39.822680 23918 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:39.829742 23933 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 12:25:39.803167 23937 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29.
I20260709 12:25:39.754710 23928 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.811690 23935 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.837425 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:39.830498 23336 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:39.846398 23713 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.855129 23920 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.830137 23627 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:25:39.865402 13722 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:25:39.847105 23918 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:39.855672 23936 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:25:39.871608 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70599.9us
I20260709 12:25:39.873483 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.857048 23785 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:25:39.877333 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.847618 23879 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.871757 23944 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:39.891136 23941 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:39.899518 23928 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.904127 23921 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:39.879537 23937 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:39.913280 23901 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:39.915791 23901 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.919414 23879 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:39.919503 23844 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:39.920291 23844 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:39.921061 23818 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:39.921239 23938 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:39.922457 23780 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:39.923872 23336 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.925202 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1573) took 1825 ms. Trace:
I20260709 12:25:39.869102 23713 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:39.877364 23785 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:39.934460 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.42677e+06us
I20260709 12:25:39.932009 13724 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:39.935359 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:39.936426 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:39.937516 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.925650 14422 rpcz_store.cc:276] 0709 12:25:38.099398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.201874 (+102476us) service_pool.cc:224] Handling call
0709 12:25:38.202210 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.202472 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 24
0709 12:25:38.351612 (+149140us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.924844 (+1573232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.924896 (+    52us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:25:39.924953 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.925017 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.925143 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1474268,"prepare.queue_time_us":463299,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":3026,"thread_start_us":246150,"threads_started":2,"wal-append.queue_time_us":291097}
W20260709 12:25:39.940073 23899 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.908s	user 0.009s	sys 0.005s
I20260709 12:25:39.940523 23901 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:39.932914 23937 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:39.941175 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:39.941754 23693 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:39.938021 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:39.942219 23693 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:39.942886 23693 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:39.942234 23713 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:39.913589 23944 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:39.945742 23945 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:39.940369 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:39.944918 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.956132 13796 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:39.952905 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:39.972580 13794 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:39.972820 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:39.973495 13796 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:39.978080 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:39.979522 13796 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:39.989967 23945 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:39.993496 23945 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.993875 23945 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:39.985052 23841 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:39.968657 23824 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.998994 23841 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:39.946171 23885 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:39.946614 23816 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:40.017520 13796 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:39.999840 23824 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.026057 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:40.028761 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.049077 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.050308 14422 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:40.054013 13794 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:39.921350 23844 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.019658 23450 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.019172 23736 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:25:40.057114 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:40.067055 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.020697 23775 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:40.065125 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:40.083877 14418 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 20. (term mismatch)
I20260709 12:25:40.089355 14418 pending_rounds.cc:85] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 19
I20260709 12:25:40.089798 14418 pending_rounds.cc:107] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 29.20
I20260709 12:25:40.090147 14418 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:40.080996 14423 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:25:40.080437 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:40.101028 14423 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:40.104686 13795 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 37. Current term is 38. Ops: []
W20260709 12:25:40.105789 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91949.9us
W20260709 12:25:40.097548 23897 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.066s	user 0.008s	sys 0.013s
I20260709 12:25:39.941982 23937 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.051939 23826 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:40.221565 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:39.940541 23785 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.056044 23684 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:39.946431 23944 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:40.239161 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 134514us
W20260709 12:25:40.107788 23897 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.009s	sys 0.013s
I20260709 12:25:40.237622 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:40.084570 23736 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:40.053206 23825 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.168s
I20260709 12:25:40.078990 23885 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:40.261155 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:40.264700 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.115621 23934 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:40.269414 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.100942 23816 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:40.279824 13794 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
W20260709 12:25:40.290645 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53119us
I20260709 12:25:40.291932 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:40.078861 23946 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.212s
I20260709 12:25:40.251885 23935 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.097832 23450 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:40.330047 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.294577 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.341391 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:40.254071 23841 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:40.309924 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:40.267619 23736 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:40.267277 23820 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:40.284576 23785 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:40.351902 23948 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:25:40.352648 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.358983 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.399925 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4620) took 2089 ms. Trace:
I20260709 12:25:40.407428 13800 rpcz_store.cc:276] 0709 12:25:38.310589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.441527 (+130938us) service_pool.cc:224] Handling call
0709 12:25:38.503242 (+ 61715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.062812 (+559570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.399595 (+1336783us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54000e4cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:40.399851 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285219712}
I20260709 12:25:40.225474 23713 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:40.266399 23840 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.077s
I20260709 12:25:40.261511 23826 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.419497 14418 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:40.266569 23689 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.133s
W20260709 12:25:40.340734 23903 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:25:40.339110 23746 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.425683 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:40.275725 23684 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.440867 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:40.469159 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.333417 23935 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.461555 13719 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:40.497098 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:40.498129 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 393372us
I20260709 12:25:40.447281 23839 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.454578 23942 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:40.505241 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4213) took 2028 ms. Trace:
I20260709 12:25:40.505760 13726 rpcz_store.cc:276] 0709 12:25:38.477218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.572332 (+ 95114us) service_pool.cc:224] Handling call
0709 12:25:40.505028 (+1932696us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b54001f0c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:40.505174 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55075200}
I20260709 12:25:40.418689 23820 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.509518 23627 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.510942 23935 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:40.510787 23789 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 37.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:25:40.511792 23838 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:40.511884 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:40.437649 23713 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:40.513674 23949 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:25:40.513895 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.514492 23824 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:40.493237 23795 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:40.449745 23885 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.517639 13723 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:25:40.518813 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:40.523751 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.029s	sys 0.055s
I20260709 12:25:40.372649 23775 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:40.504801 23588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.008s	sys 0.004s
I20260709 12:25:40.503090 23921 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.516042 23897 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.459890 23746 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.447038 23816 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:40.527666 23839 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.548074 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:40.575690 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:40.741721 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1613) took 2623 ms (client timeout 2560 ms). Trace:
I20260709 12:25:40.596640 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:40.576050 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.029s	sys 0.055s
I20260709 12:25:40.576123 13726 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:40.626299 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:40.621683 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.742221 13798 rpcz_store.cc:269] 0709 12:25:38.118153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.258182 (+140029us) service_pool.cc:224] Handling call
0709 12:25:40.724876 (+2466694us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54003c0354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:40.741609 (+ 16733us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":980767488}
I20260709 12:25:40.747383 13652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:40.748067 13652 raft_consensus.cc:606] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:40.748411 13652 raft_consensus.cc:614] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:40.618361 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 284442us
W20260709 12:25:40.593250 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 283416us
I20260709 12:25:40.610064 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:40.744977 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 732929us
I20260709 12:25:40.590771 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:40.754376 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.761767 13724 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:40.764688 13726 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:40.765832 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:40.766467 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:40.767697 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:40.620826 23736 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.768476 13646 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:25:40.771124 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:40.771912 13647 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:25:40.772091 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:40.753250 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:40.578506 23588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.008s	sys 0.004s
I20260709 12:25:40.773916 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:40.772766 13726 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:40.774785 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:40.591423 23945 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.003s
I20260709 12:25:40.776000 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:40.778010 13652 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:40.830353 13724 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:25:40.765832 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 202466us
W20260709 12:25:40.626230 23499 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.808s	user 0.008s	sys 0.008s
I20260709 12:25:40.582602 23970 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.578404 23951 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 12:25:40.590066 23818 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:40.945062 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 193455us
I20260709 12:25:40.582809 23942 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:40.586130 23820 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:40.790026 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:40.584210 23826 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:40.787495 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:40.871995 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.590808 23888 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.591845 23500 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.841s	user 0.011s	sys 0.003s
I20260709 12:25:40.566867 23825 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:40.871443 13724 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:40.869747 13646 raft_consensus.cc:2410] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:25:40.832351 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:40.994853 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:25:40.910799 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 136959us
I20260709 12:25:40.997918 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:40.917236 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:41.000852 13797 raft_consensus.cc:606] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:41.001441 13797 raft_consensus.cc:614] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:40.951445 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:40.955857 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:40.781718 23957 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:40.837334 23795 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:25:40.938042 23918 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.068s	user 0.009s	sys 0.006s
W20260709 12:25:40.970556 14418 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.005s
I20260709 12:25:41.013651 14418 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:40.599294 23897 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.595703 23921 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.988905 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4625) took 2457 ms. Trace:
I20260709 12:25:40.781317 23935 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.179s
I20260709 12:25:40.867870 23823 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:40.907068 23928 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:40.865880 23736 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.003295 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.012534 14419 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:40.550644 23627 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:40.837220 23972 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.210s
I20260709 12:25:41.017091 13793 rpcz_store.cc:276] 0709 12:25:38.512134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.868174 (+356040us) service_pool.cc:224] Handling call
0709 12:25:38.868498 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.752414 (+883916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.969600 (+1217186us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54002409d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:40.969793 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030927232}
I20260709 12:25:40.973196 23746 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:40.898741 23945 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.040352 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.040581 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:40.874766 23959 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.173s
I20260709 12:25:41.055804 13652 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:41.053606 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:41.014495 23795 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.078168 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4630) took 1128 ms. Trace:
I20260709 12:25:40.958704 23818 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.018077 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:41.052593 23693 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.109s	user 0.011s	sys 0.004s
I20260709 12:25:41.041306 23500 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:41.099359 23839 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.080807 13801 rpcz_store.cc:276] 0709 12:25:39.949477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.950679 (+  1202us) service_pool.cc:224] Handling call
0709 12:25:39.951067 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.951597 (+   530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.951631 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.952142 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.025286 (+ 73144us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:40.025352 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.025463 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.077692 (+1052229us) raft_consensus.cc:1685] finished
0709 12:25:41.077720 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.078075 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3438,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"thread_start_us":72753,"threads_started":1,"wal-append.queue_time_us":273998}
I20260709 12:25:41.090966 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:41.101068 23839 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.101449 14419 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:41.103149 23980 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:41.103027 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:41.103523 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:41.103749 23980 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:41.104176 23980 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:41.104755 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:41.104879 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:41.105718 14419 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:40.989506 23499 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.076s
I20260709 12:25:40.946473 23970 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.106931 23967 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:41.107789 23967 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.109548 23970 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.081866 23976 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.017s
I20260709 12:25:41.017190 23825 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:41.085450 23795 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:41.111665 23825 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.098307 23693 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.115340 14420 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:41.107183 23969 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:41.060688 23928 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.117789 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:41.118016 23693 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.118067 23928 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.008255 23942 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:41.123422 23942 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:41.124276 23942 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:41.053983 23823 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.128144 23823 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.083609 23746 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.129521 23746 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.130295 23967 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.132339 23746 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.133889 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.134511 23897 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:41.134982 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:41.136257 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.137542 13726 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:41.140128 23627 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.040910 23918 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.144585 23918 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.145426 13720 raft_consensus.cc:2368] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29.
I20260709 12:25:41.149683 23795 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.150234 23945 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.152274 23547 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:41.153697 23547 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:41.154951 23921 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.154798 23547 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.155081 23689 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:41.092033 23736 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:41.159713 13727 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:25:41.162184 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:41.162655 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4249) took 1624 ms. Trace:
I20260709 12:25:41.163072 13727 rpcz_store.cc:276] 0709 12:25:39.538128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.541527 (+  3399us) service_pool.cc:224] Handling call
0709 12:25:41.159163 (+1617636us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:25:41.159370 (+   207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.162287 (+  2917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.162588 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19477120}
I20260709 12:25:41.165957 23713 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.171006 13800 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:41.084071 23957 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.001782 23888 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.101635 23839 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.175382 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:41.175910 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 270000us
W20260709 12:25:41.179585 13764 leader_election.cc:336] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.781s (SENT)
I20260709 12:25:41.179801 14423 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:41.151991 23942 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.180290 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:41.168709 23736 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:41.161429 23689 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:41.171649 23547 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:41.139693 23980 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.188094 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:41.189807 23689 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.190968 23736 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.191633 23689 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:41.192297 23736 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:41.173589 23684 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:41.193987 23950 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.758s	user 0.003s	sys 0.000s
I20260709 12:25:41.185988 23897 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:41.179205 23957 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:41.268122 23960 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.772s	user 0.000s	sys 0.002s
I20260709 12:25:41.280004 23954 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:41.280797 23954 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:41.301368 23839 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.302963 23942 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.315306 23980 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.315915 23957 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:41.282078 23954 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.359645 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:25:41.365091 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.367417 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.372025 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 584441us
I20260709 12:25:41.373934 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:41.386392 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.048540 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.052232 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.019405 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.389773 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.391062 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.392724 13648 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:41.389678 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.395530 13647 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:41.389618 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:41.403808 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.426230 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.427649 13801 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:41.427692 13797 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:41.428481 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:41.429877 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1866) took 1068 ms. Trace:
I20260709 12:25:41.434363 14419 rpcz_store.cc:276] 0709 12:25:40.361366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.104525 (+743159us) service_pool.cc:224] Handling call
0709 12:25:41.429738 (+325213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.437367 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:41.438757 23336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.021s	user 0.000s	sys 0.011s
W20260709 12:25:41.439455 23336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.040s	user 0.000s	sys 0.012s
I20260709 12:25:41.440529 23336 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:41.441150 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1862) took 1080 ms. Trace:
I20260709 12:25:41.441242 23336 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:41.444633 23499 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:41.431735 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:41.448400 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.432416 13801 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:25:41.448812 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.449472 13727 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:41.444415 23336 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.458904 23500 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:41.459811 23500 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:41.431108 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 475208us
I20260709 12:25:41.202497 14423 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:41.441709 14419 rpcz_store.cc:276] 0709 12:25:40.360207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.439492 (+1079285us) service_pool.cc:224] Handling call
0709 12:25:41.440098 (+   606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.440639 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.440669 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.440735 (+    66us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:41.440777 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.440893 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.441032 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.461916 23500 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.465847 14419 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:25:41.466678 23500 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:41.467706 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.004s	sys 0.021s
I20260709 12:25:41.467639 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:41.487447 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1867) took 1123 ms. Trace:
I20260709 12:25:41.488067 14419 rpcz_store.cc:276] 0709 12:25:40.361672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.464627 (+1102955us) service_pool.cc:224] Handling call
0709 12:25:41.465202 (+   575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.483462 (+ 18260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.485111 (+  1649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.482949 23746 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:41.496177 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.495893 23746 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.499349 23795 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.500478 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.500694 23983 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:41.501570 13801 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:41.501653 23983 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:41.503506 23944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.990s	user 0.004s	sys 0.004s
W20260709 12:25:41.503993 23944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.004s	sys 0.004s
I20260709 12:25:41.504648 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:41.503984 14419 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:41.505524 13801 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:41.506476 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1874) took 1005 ms. Trace:
I20260709 12:25:41.507189 14419 rpcz_store.cc:276] 0709 12:25:40.500737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.494836 (+994099us) service_pool.cc:224] Handling call
0709 12:25:41.503336 (+  8500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.505533 (+  2197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.506373 (+   840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:41.510324 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.511606 23839 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:41.511600 13797 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:41.512964 23839 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:41.515684 23888 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:41.516480 23986 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.516916 23588 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.517130 23980 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.517341 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.509809 14419 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:41.518639 23808 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:41.504881 23983 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:41.553989 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:41.507333 23954 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:41.561641 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.562656 23986 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.524937 23808 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.567288 23944 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.568236 23944 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.569049 23808 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.575078 23980 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.577435 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:41.578913 13797 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:41.582861 23588 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.584981 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:41.585736 23588 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:41.562198 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.587730 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.566176 23986 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.582024 23746 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:41.557951 23983 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:41.637369 13801 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:41.640416 23982 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.641260 23945 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.642407 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.642194 23942 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:41.644116 13725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4252) took 2104 ms (client timeout 1896 ms). Trace:
I20260709 12:25:41.646850 13797 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:41.649842 13649 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:41.650820 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:41.651199 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4250) took 2112 ms. Trace:
W20260709 12:25:41.651407 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.010s	sys 0.018s
I20260709 12:25:41.656191 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1473) took 1607 ms. Trace:
I20260709 12:25:41.657521 23935 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.642242 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:41.621567 23808 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:41.675145 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.010s	sys 0.019s
I20260709 12:25:41.681020 13794 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:41.683472 13644 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:25:41.683969 14424 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1460) took 4158 ms (client timeout 1581 ms). Trace:
I20260709 12:25:41.684290 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:41.684444 14424 rpcz_store.cc:269] 0709 12:25:37.525264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.525427 (+   163us) service_pool.cc:224] Handling call
0709 12:25:41.683565 (+4158138us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.683763 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5059552640}
I20260709 12:25:41.685724 13727 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:41.686213 23588 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.686739 23959 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.687500 23957 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:41.688649 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 178488us
I20260709 12:25:41.688817 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.689765 13651 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:25:41.691291 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2101) took 1751 ms (client timeout 1609 ms). Trace:
I20260709 12:25:41.691088 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:41.668699 13725 rpcz_store.cc:269] 0709 12:25:39.539256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.539974 (+   718us) service_pool.cc:224] Handling call
0709 12:25:41.643092 (+2103118us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54000f5d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.643534 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319978368}
I20260709 12:25:41.670188 14420 rpcz_store.cc:276] 0709 12:25:40.048883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.049032 (+   149us) service_pool.cc:224] Handling call
0709 12:25:40.086779 (+ 37747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.115790 (+1029011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.655595 (+539805us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:41.655955 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364971264}
I20260709 12:25:41.669734 13723 rpcz_store.cc:276] 0709 12:25:39.538543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.541340 (+  2797us) service_pool.cc:224] Handling call
0709 12:25:40.505213 (+963873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.575125 (+ 69912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.650704 (+1075579us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54001f0c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.650969 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2214801152}
I20260709 12:25:41.701063 14423 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:41.680562 23983 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:41.705729 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63685.5us
I20260709 12:25:41.701437 23450 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.701843 23945 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.702018 23957 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.703291 13726 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:41.699426 13720 rpcz_store.cc:269] 0709 12:25:39.939554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.940905 (+  1351us) service_pool.cc:224] Handling call
0709 12:25:41.669864 (+1728959us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000285d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.691006 (+ 21142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2767759232}
W20260709 12:25:41.733330 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 132461us
I20260709 12:25:41.736232 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.711844 23944 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.784976 23917 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.743065 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:41.740530 23982 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.785673 23917 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:41.786361 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43539.6us
I20260709 12:25:41.787326 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:41.787746 23982 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:41.751971 23689 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.559s	user 0.015s	sys 0.011s
I20260709 12:25:41.788498 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:41.788673 23917 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.752439 13649 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:25:41.789661 23942 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:41.739657 13652 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:41.790939 13651 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:25:41.788442 23982 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:41.756872 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.759624 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:41.765352 23945 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.711570 23935 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.727474 23980 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.707811 23588 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:41.744166 23838 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.177s	user 0.015s	sys 0.010s
I20260709 12:25:41.775755 23987 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.744166 23736 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.551s	user 0.019s	sys 0.008s
I20260709 12:25:41.780004 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.780339 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.745750 23965 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:41.745337 23568 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:25:41.746239 23959 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:41.745458 14418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1856) took 1379 ms (client timeout 1555 ms). Trace:
I20260709 12:25:41.782822 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:41.749974 13650 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.656s	user 0.013s	sys 0.011s
I20260709 12:25:41.752779 23500 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.790112 23942 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.782812 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:41.794162 23953 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.376s	user 0.001s	sys 0.002s
I20260709 12:25:41.783489 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.756211 23983 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.828846 23588 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:41.829373 13727 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:41.830755 13650 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:41.830492 23500 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:41.832350 13719 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 16. (term mismatch)
I20260709 12:25:41.832396 23627 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:41.832778 13719 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 15
I20260709 12:25:41.814658 23965 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.833153 13719 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 28.16
I20260709 12:25:41.833340 23965 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:41.833571 13719 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:41.816471 14418 rpcz_store.cc:269] 0709 12:25:40.325451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.341013 (+ 15562us) service_pool.cc:224] Handling call
0709 12:25:41.667311 (+1326298us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b54005302d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.705124 (+ 37813us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177972096}
I20260709 12:25:41.835188 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:41.825004 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65850.3us
I20260709 12:25:41.805714 13794 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:25:41.826539 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:41.826324 23987 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.828035 23958 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:41.840219 13649 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:41.839190 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:41.833253 23500 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.848809 23972 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:25:41.850767 23547 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.009s
I20260709 12:25:41.851351 13652 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:41.834619 23942 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:41.857650 23972 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:41.859490 23959 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:41.851718 23935 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:41.838361 23983 raft_consensus.cc:2764] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election decision vote started in defunct term 34: won
I20260709 12:25:41.866068 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1577) took 1536 ms. Trace:
I20260709 12:25:41.838759 23958 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:41.878177 14417 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1873) took 1390 ms (client timeout 1611 ms). Trace:
I20260709 12:25:41.881170 13644 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:25:41.887157 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 117526us
I20260709 12:25:41.887562 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:41.883378 14416 rpcz_store.cc:276] 0709 12:25:40.329156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.329986 (+   830us) service_pool.cc:224] Handling call
0709 12:25:40.330370 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.508645 (+178275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.508684 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.508740 (+    56us) raft_consensus.cc:1645] Marking committed up to 26
0709 12:25:40.508786 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.865454 (+1356668us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54006c1e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:41.865554 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.865835 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409323776}
W20260709 12:25:41.890043 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78988.1us
I20260709 12:25:41.889284 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2103) took 1262 ms. Trace:
I20260709 12:25:41.866384 23987 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:41.885119 14417 rpcz_store.cc:269] 0709 12:25:40.487422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.550817 (+ 63395us) service_pool.cc:224] Handling call
0709 12:25:41.877473 (+1326656us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54005546d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.877849 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1816357248}
W20260709 12:25:41.901844 13800 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1629) took 1341 ms (client timeout 1543 ms). Trace:
I20260709 12:25:41.901412 23958 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:41.902333 13800 rpcz_store.cc:269] 0709 12:25:40.560524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.561114 (+   590us) service_pool.cc:224] Handling call
0709 12:25:41.901407 (+1340293us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:41.901632 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935606016}
I20260709 12:25:41.903810 23958 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:41.904829 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1475) took 1846 ms. Trace:
I20260709 12:25:41.906960 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:41.907455 13651 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:25:41.905341 14422 rpcz_store.cc:276] 0709 12:25:40.058740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.058923 (+   183us) service_pool.cc:224] Handling call
0709 12:25:40.059335 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.486278 (+426943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.486322 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.486375 (+    53us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:25:40.486426 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.904261 (+1417835us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54007125d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:41.904333 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.904609 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981146496}
I20260709 12:25:41.913193 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:41.920385 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:41.932359 13725 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 12:25:41.935714 13690 leader_election.cc:336] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.848s (SENT)
I20260709 12:25:41.936712 23685 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:41.938793 23972 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.940464 23972 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:41.894243 23935 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:25:41.892510 13724 rpcz_store.cc:276] 0709 12:25:40.581391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.007411 (+426020us) service_pool.cc:224] Handling call
0709 12:25:41.007867 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.413819 (+405952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:41.413898 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.415478 (+  1580us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.598759 (+183281us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:41.801790 (+203031us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.808057 (+  6267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.819331 (+ 11274us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.836461 (+ 17130us) raft_consensus.cc:1685] finished
0709 12:25:41.836777 (+   316us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.843233 (+  6456us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2285,"prepare.queue_time_us":183935,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":58,"raft.queue_time_us":99011,"raft.run_cpu_time_us":590,"raft.run_wall_time_us":8556,"thread_start_us":719036,"threads_started":3,"wal-append.queue_time_us":242648}
I20260709 12:25:41.957484 23818 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:41.964373 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50044.7us
I20260709 12:25:41.969086 23500 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:41.970171 23500 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:41.972357 23500 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.17, Last appended by leader: 17, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:41.975121 13727 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:25:41.975737 13718 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
W20260709 12:25:41.965258 13767 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.916s (SENT) [suppressed 39 similar messages]
I20260709 12:25:41.969123 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1578) took 1639 ms. Trace:
I20260709 12:25:41.929463 23959 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:41.988456 13794 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:41.979123 14421 rpcz_store.cc:276] 0709 12:25:40.329631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.330101 (+   470us) service_pool.cc:224] Handling call
0709 12:25:40.353753 (+ 23652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.566606 (+212853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.566657 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.566724 (+    67us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:40.566772 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.967067 (+1400295us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:41.967134 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.968902 (+  1768us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488582528}
I20260709 12:25:41.991086 13651 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:25:41.991895 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.992213 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.995502 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.996145 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:41.997426 13795 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:25:42.003157 13720 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:42.005743 23888 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.006565 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:42.007398 23968 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.108s	user 0.003s	sys 0.003s
I20260709 12:25:42.010784 23824 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.011732 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.013818 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:42.033494 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.033855 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68623.8us
I20260709 12:25:42.038673 23888 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.039341 23959 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:42.045573 23959 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.045967 23888 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:42.049398 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.037494 14418 raft_consensus.cc:2376] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29. Re-sending same reply.
I20260709 12:25:42.066581 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.068138 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:42.068171 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 161706us
W20260709 12:25:42.069734 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.070333 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.081316 23885 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.017s	sys 0.016s
W20260709 12:25:42.083323 23885 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.017s	sys 0.017s
I20260709 12:25:42.090778 23685 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.091269 13764 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.999s (SENT)
I20260709 12:25:42.092193 23685 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.101577 13647 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:25:42.094229 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:42.111552 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.111485 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.112799 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:42.111774 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70241.6us
I20260709 12:25:42.113415 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.114092 13726 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:42.114552 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.096221 13645 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:42.111562 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.106640 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.106802 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.112285 13718 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:42.104677 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.107553 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.104298 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.120052 13651 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:25:42.120026 13720 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:42.125018 13721 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:42.107636 13724 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:42.124927 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:42.057150 23824 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.129805 13652 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:25:42.138041 14422 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:25:42.148494 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54543.2us
I20260709 12:25:42.150095 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:42.154511 23901 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:42.155148 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:42.136672 24001 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:25:42.166893 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.168236 13720 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:25:42.168592 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.169631 13726 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:42.169513 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.170244 13720 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:42.171842 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.172376 23818 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:42.170558 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.173911 13726 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:42.174952 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.175233 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:42.176086 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.176434 23901 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:42.176787 13726 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:42.177150 23901 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:42.177775 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.134128 23547 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
W20260709 12:25:42.178670 23997 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 12:25:42.179288 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:42.179441 14425 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1478) took 2093 ms (client timeout 1848 ms). Trace:
I20260709 12:25:42.180372 23888 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:42.185379 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.186456 13727 raft_consensus.cc:2376] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25. Re-sending same reply.
I20260709 12:25:42.186878 23901 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.187711 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:42.188453 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3178) took 1406 ms. Trace:
I20260709 12:25:42.188624 13727 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 30 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:42.188812 13643 rpcz_store.cc:276] 0709 12:25:40.781768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.783267 (+  1499us) service_pool.cc:224] Handling call
0709 12:25:40.783593 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.992900 (+209307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.992934 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.992978 (+    44us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:40.993018 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.187898 (+1194880us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54001e7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:42.187966 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.188228 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473990528}
I20260709 12:25:42.182188 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4515) took 1666 ms. Trace:
W20260709 12:25:42.191401 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3173) took 1424 ms (client timeout 1644 ms). Trace:
I20260709 12:25:42.191500 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:42.192162 13650 rpcz_store.cc:269] 0709 12:25:40.767309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.767436 (+   127us) service_pool.cc:224] Handling call
0709 12:25:42.191070 (+1423634us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54001e7b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:42.191221 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712634752}
I20260709 12:25:42.192785 13645 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
W20260709 12:25:42.179936 14425 rpcz_store.cc:269] 0709 12:25:40.085993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.088661 (+  2668us) service_pool.cc:224] Handling call
0709 12:25:42.179019 (+2090358us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:42.179227 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2934171904}
W20260709 12:25:42.194720 23785 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.780s	user 0.009s	sys 0.020s
I20260709 12:25:42.195703 23685 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:42.161892 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.187405 23627 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.198326 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:42.201047 13646 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:42.191223 13719 rpcz_store.cc:276] 0709 12:25:40.515717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.517617 (+  1900us) service_pool.cc:224] Handling call
0709 12:25:40.518148 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.834854 (+1316706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.181866 (+347012us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540021fcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:42.182107 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2987127040}
I20260709 12:25:42.205235 13725 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
W20260709 12:25:42.207046 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49136.4us
I20260709 12:25:42.209487 13793 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:42.203610 23627 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:42.144840 23935 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.059s
I20260709 12:25:42.196125 23942 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.214308 13723 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 12:25:42.215899 13723 pending_rounds.cc:85] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 12
I20260709 12:25:42.216223 13723 pending_rounds.cc:107] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:25:42.216496 13723 raft_consensus.cc:2889] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.213577 23618 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.010s
I20260709 12:25:42.224822 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.236528 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 20.
I20260709 12:25:42.236958 13725 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
W20260709 12:25:42.239698 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.236224 23685 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
I20260709 12:25:42.245285 23897 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:42.230101 23959 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.253636 23780 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:25:42.256129 23959 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.254992 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.259801 13547 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 23 to 32. New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:42.261437 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:42.265491 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.266307 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.267109 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.251899 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1865) took 1890 ms. Trace:
I20260709 12:25:42.276168 13646 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:42.269857 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.274482 14423 rpcz_store.cc:276] 0709 12:25:40.361086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.463563 (+1102477us) service_pool.cc:224] Handling call
0709 12:25:41.680461 (+216898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.704801 (+ 24340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.251556 (+546755us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54005302d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:42.251796 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488108800}
I20260709 12:25:42.276472 13645 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:42.276242 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.271369 23780 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:42.281301 13727 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:42.291453 13651 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:42.298710 23780 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 37.25, Last appended by leader: 25, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.300163 23780 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:25:42.307605 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:42.244554 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:42.283676 23970 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:42.317632 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.284142 23824 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:42.322569 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80236us
W20260709 12:25:42.324849 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.326910 13645 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:42.331959 23685 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:42.343218 23685 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:42.343447 23957 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:42.344175 13796 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:25:42.344551 23685 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.345588 23983 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.346388 23983 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.345072 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:42.348570 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.348006 13547 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 19 to 22, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 22 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:25:42.349644 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.349812 13547 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 22 to 25, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 25 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:42.350960 13727 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:42.351789 13547 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:42.356487 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.357362 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.353751 13547 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 21 to 25, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 25 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:42.345640 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:42.365188 13727 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:25:42.338738 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:42.319391 23970 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:42.376852 13793 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:42.386894 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.391573 23942 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:42.395179 24008 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:25:42.335412 23965 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.399426 13648 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:42.400524 23965 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:42.374620 23983 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:42.406060 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:42.406730 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.407789 13652 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:25:42.409529 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.410218 13652 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:42.410494 23935 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:42.411130 23935 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.413561 23935 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:42.416162 23958 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.003s
I20260709 12:25:42.425613 23831 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.426522 23987 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:42.427065 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.427799 23971 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 12:25:42.428087 23983 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:42.426543 23831 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.430613 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:42.431416 23831 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:42.432457 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.433226 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.433979 23970 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.434791 23957 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:42.435259 23957 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:42.435387 23970 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:42.435767 23957 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.436833 23884 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.054s
W20260709 12:25:42.441284 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60942us
I20260709 12:25:42.443075 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:42.443323 13794 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 37. Current term is 38. Ops: []
I20260709 12:25:42.377101 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.445251 14423 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:42.450634 13649 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:42.455993 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.456450 13796 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:25:42.461609 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:42.466888 13723 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:42.474668 23642 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:25:42.475286 13645 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:25:42.477339 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:42.503934 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:42.479161 23888 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:42.521319 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:42.528868 13794 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:25:42.528854 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54814.6us
I20260709 12:25:42.535772 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.509528 23642 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:25:42.544411 13793 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:42.546574 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:42.549980 24001 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:42.551115 23642 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:42.551565 23642 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.551808 23959 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:42.551937 23642 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:42.559676 24010 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:25:42.574126 13547 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:42.367210 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:42.577181 13547 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 25 to 26, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 26 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:42.532892 23884 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.058s
W20260709 12:25:42.599327 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 52928.7us
I20260709 12:25:42.600603 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:42.596225 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92052.9us
W20260709 12:25:42.606967 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.612308 23959 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:42.616879 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.609380 23971 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:42.619015 23971 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.619503 23971 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:42.616452 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:42.622046 13647 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:42.589433 23998 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.032s
W20260709 12:25:42.639856 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.646085 14423 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:42.658623 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:42.660058 13794 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:42.674845 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:42.676103 13793 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:42.677934 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.669426 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:42.625582 23901 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 12:25:42.691148 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 203739us
I20260709 12:25:42.681571 23831 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.711073 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:42.721314 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:42.723485 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.725677 23746 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:42.726447 23547 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.727185 13792 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:25:42.729478 13797 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:42.664582 23685 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:25:42.743561 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.747045 13794 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:42.669672 23897 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 20, Time since last communication: 0.049s
W20260709 12:25:42.738794 24012 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:25:42.747642 13648 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:42.755209 23998 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:42.738768 23746 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:42.756448 24014 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:25:42.764782 23831 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:42.788407 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46398.1us
I20260709 12:25:42.739181 23942 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:42.827137 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 157093us
W20260709 12:25:42.836479 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.835774 13547 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:42.837231 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.726656 23642 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.844137 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:42.840171 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:42.799918 24023 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.002s
I20260709 12:25:42.849393 13649 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:42.843205 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:42.842423 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.789781 23998 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.847136 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:42.847891 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:42.874142 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 204854us
I20260709 12:25:42.798616 23795 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.828984 23942 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:42.832598 23965 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:42.769780 23547 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.805732 23944 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:25:42.870587 23998 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:42.903478 13651 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:42.915194 13793 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:42.919560 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.882797 23942 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:42.936622 23627 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.709s	user 0.012s	sys 0.000s
I20260709 12:25:42.937162 23627 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:42.934640 24028 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:42.942272 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58417.8us
I20260709 12:25:42.955755 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:42.953764 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:42.950490 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:42.960104 24026 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:25:42.962944 23627 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:42.962491 24025 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.009s
I20260709 12:25:42.918372 23987 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:42.971436 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1628) took 2629 ms. Trace:
I20260709 12:25:42.891146 23965 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:42.883145 23795 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:42.977977 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 144970us
W20260709 12:25:42.988786 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83573.9us
I20260709 12:25:42.989822 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:42.992014 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.993052 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.991003 13650 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:42.973839 24020 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.975521 13799 rpcz_store.cc:276] 0709 12:25:40.104002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.104339 (+   337us) service_pool.cc:224] Handling call
0709 12:25:40.104588 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.104800 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:42.355937 (+2251137us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b1800220600. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:25:42.356295 (+   358us) log.cc:844] Serialized 14 byte log entry
0709 12:25:42.676930 (+320635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:42.687960 (+ 11030us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:42.705923 (+ 17963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.725585 (+ 19662us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:42.733370 (+  7785us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":182266,"prepare.queue_time_us":201323,"prepare.run_cpu_time_us":1946,"prepare.run_wall_time_us":29502,"raft.queue_time_us":65167,"raft.run_wall_time_us":17192,"spinlock_wait_cycles":34452352,"thread_start_us":299362,"threads_started":2,"wal-append.queue_time_us":319646}
I20260709 12:25:43.014891 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.015713 13648 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 21.
I20260709 12:25:42.997303 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:43.022159 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.023272 13651 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:43.030344 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.031579 13650 raft_consensus.cc:2368] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:42.999620 23983 raft_consensus.cc:2764] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
W20260709 12:25:43.050379 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60639.7us
I20260709 12:25:42.991268 23987 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.023298 24026 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:43.058351 23642 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.044359 24036 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:25:43.042193 23901 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:43.035080 24029 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:43.076718 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:43.069919 24018 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:25:43.061828 23820 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:43.075865 24021 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.271s	user 0.001s	sys 0.001s
I20260709 12:25:43.075525 23831 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.064339 24026 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:43.102445 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111258us
I20260709 12:25:43.106693 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.097460 23971 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.110030 23998 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.026127 24020 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.092005 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:43.123013 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46418.1us
I20260709 12:25:43.124359 24047 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:43.127584 13649 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:25:43.127590 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:43.127678 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:43.142302 24033 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 12:25:43.143687 23800 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:43.144063 24033 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:43.144699 24033 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:43.146938 24030 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:25:43.144155 23800 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:43.134829 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:43.141594 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:43.157433 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:43.159370 13646 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:43.161820 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:43.125506 24028 raft_consensus.cc:2764] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:25:43.128172 24026 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:43.137156 23547 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.158281 24036 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:43.160356 23897 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:43.158473 23800 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:43.165251 23642 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:43.185222 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:43.172494 23818 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:43.200986 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:43.171658 24040 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:43.207387 23901 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:43.208031 23888 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:43.208597 23888 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:43.209676 23888 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.210805 23888 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:43.209810 23901 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:43.145829 24033 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.217724 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:43.183696 24046 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.188341 24036 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.193465 23897 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:43.227785 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 135908us
I20260709 12:25:43.194569 24028 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:43.234802 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.197647 23642 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:43.204200 23547 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:25:43.242049 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80593.1us
I20260709 12:25:43.205617 23818 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:43.219233 23901 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.171818 24047 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:43.235754 23957 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.799s	user 0.016s	sys 0.002s
I20260709 12:25:43.278888 23957 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.231863 24036 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:43.244423 23642 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:43.257892 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61963.5us
I20260709 12:25:43.240846 23897 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:43.241730 24029 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
W20260709 12:25:43.291093 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.297689 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:43.298651 13794 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:43.299506 13799 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:43.301452 13793 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 33 index: 17. Preceding OpId from leader: term: 35 index: 18. (index mismatch)
I20260709 12:25:43.259297 23983 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:43.302845 23822 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:43.303297 23822 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:43.261427 23831 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.303968 23831 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.304248 23822 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.264046 24033 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:43.305310 23822 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:43.265550 24020 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.306890 24020 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.307360 24020 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:43.308202 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.268031 23685 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:25:43.308923 23685 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:43.309870 23685 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.313498 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:43.319746 23957 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:43.269634 23987 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.328871 23987 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.255055 23901 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:43.229089 23785 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:43.271215 23998 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.248711 23746 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.248638 23818 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:43.246379 23547 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.249845 24046 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.254339 23795 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.277005 23971 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.254047 23965 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:43.341387 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.342666 13801 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:43.289460 13547 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 27 to 29, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 29 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:43.350116 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61141.8us
I20260709 12:25:43.351107 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:43.364897 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.368230 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:43.369069 14425 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:43.370771 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:43.371436 14422 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:43.334173 23547 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:43.377501 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:43.389819 13547 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 21 to 26, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 26 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.390584 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:43.395229 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:43.385373 23685 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:25:43.405346 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.408900 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.412626 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:25:43.413688 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 211194us
I20260709 12:25:43.347872 23987 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:43.396366 23970 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.960s	user 0.015s	sys 0.008s
I20260709 12:25:43.426697 13547 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 18 to 20, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 20 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.426694 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:43.420128 13799 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:43.412866 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:43.425786 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.426146 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.433506 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:43.439495 13644 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:43.433696 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:43.432193 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:43.433161 13723 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:43.406330 23971 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:43.405227 23795 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:43.433112 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:43.446717 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:43.447470 13547 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 26 to 28, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:43.453177 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:43.453445 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68377.6us
I20260709 12:25:43.454427 13794 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:43.454874 13799 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:43.421276 23957 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.457230 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3188) took 1024 ms. Trace:
I20260709 12:25:43.457414 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:43.458811 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.454584 13724 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:25:43.457629 13644 rpcz_store.cc:276] 0709 12:25:42.432845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.432975 (+   130us) service_pool.cc:224] Handling call
0709 12:25:43.457071 (+1024096us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400095954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:43.457170 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2312308992}
W20260709 12:25:43.467923 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 193510us
I20260709 12:25:43.437206 24037 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:43.432502 23831 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.466514 23946 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:43.450201 23971 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.452769 23627 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:43.492316 13723 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:43.415464 23958 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 12:25:43.493901 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 99813.8us
I20260709 12:25:43.495657 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.499605 13617 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:43.502789 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.503603 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.476589 13547 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.504904 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.505090 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.511178 13644 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:25:43.512236 13727 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:25:43.521453 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.510398 13724 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:43.530238 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.531064 13718 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:43.483124 24047 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.533808 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.544623 13652 raft_consensus.cc:2376] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32. Re-sending same reply.
I20260709 12:25:43.539042 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.538683 13547 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 28 to 30, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 30 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.539790 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.540408 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:43.546514 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.498767 23946 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:43.507886 23971 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:43.559223 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:43.534632 24027 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:43.506645 23935 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:43.560052 13727 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:43.526436 23831 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.561856 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46205.7us
W20260709 12:25:43.562322 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.535240 24038 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:43.517524 24020 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.566105 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:43.490808 23795 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.570150 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.569577 24055 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.494108 23959 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:25:43.580899 13547 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 29 to 31, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 31 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:43.572301 13720 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27. Re-sending same reply.
W20260709 12:25:43.540990 23780 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.240s	user 0.019s	sys 0.005s
I20260709 12:25:43.566133 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:43.508067 23972 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 12:25:43.596283 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:43.597332 13727 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:43.523452 23800 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.563844 23946 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:43.599143 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52854.3us
I20260709 12:25:43.494666 23957 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.622859 14422 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:43.600524 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.625948 13652 raft_consensus.cc:2376] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35. Re-sending same reply.
I20260709 12:25:43.603327 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:43.607605 24020 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:43.583540 23831 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:43.608330 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.630247 13724 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: [28.19-28.19]
W20260709 12:25:43.607650 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108192us
I20260709 12:25:43.630945 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.615082 13547 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.617995 13720 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
W20260709 12:25:43.632580 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 142637us
I20260709 12:25:43.601454 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:43.606010 23942 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.650s	user 0.017s	sys 0.000s
W20260709 12:25:43.615238 24059 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:43.548429 23500 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:25:43.651801 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.622566 23884 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:43.639967 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.663928 13645 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:43.645453 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.665304 13793 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:43.582902 24047 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:25:43.627962 24020 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.671747 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.629179 23627 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:43.667974 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 236) took 42035 ms (client timeout 15000 ms). Trace:
I20260709 12:25:43.634783 24037 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:43.667951 13547 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 20 to 23, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 23 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.645907 23958 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:43.678368 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:43.682035 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78834.9us
I20260709 12:25:43.665304 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.632376 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:43.681941 13547 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:43.683414 13618 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:43.666134 24046 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.669950 23780 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:25:43.675545 24020 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:43.690069 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:43.691152 23935 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:43.695056 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63810.8us
I20260709 12:25:43.619371 24055 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.650506 23942 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.706411 23942 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.708148 13643 raft_consensus.cc:2393] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:43.695794 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:43.708901 13793 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:43.698328 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:43.701965 23935 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.704371 24055 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.711723 24055 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:43.712252 24055 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.712597 23935 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:43.713039 23888 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.003s
I20260709 12:25:43.707937 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:43.713958 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 81670.2us
I20260709 12:25:43.714565 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:43.715664 23958 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:43.716228 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:43.716415 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:43.716945 14422 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:43.717073 23958 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.718227 23820 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:43.718972 23820 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:43.719470 23820 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:43.719542 23972 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:43.720108 23972 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:43.721165 23972 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.722045 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:43.722586 24066 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.717676 23824 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:43.718869 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:43.734071 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:43.735149 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53655.2us
I20260709 12:25:43.735713 14422 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:43.724677 13652 raft_consensus.cc:2376] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27. Re-sending same reply.
I20260709 12:25:43.736770 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:43.737725 23822 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:43.739423 13693 leader_election.cc:400] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:43.739775 13693 leader_election.cc:403] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:25:43.680338 24040 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.043s
I20260709 12:25:43.748661 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:43.754544 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71210.9us
I20260709 12:25:43.669364 23884 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:43.756413 14417 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
W20260709 12:25:43.759990 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.759188 13547 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:43.775380 24070 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:25:43.774925 24037 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.775521 23818 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.776908 24067 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:25:43.777558 24067 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:43.777873 23818 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:43.778499 24065 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:43.776901 24055 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.778751 24067 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.748217 23942 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:25:43.780267 13801 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:43.780911 24069 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:43.781041 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:43.781613 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.782459 23627 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:43.783174 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66831.8us
I20260709 12:25:43.756971 23888 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:43.786970 23800 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:43.787384 24026 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
W20260709 12:25:43.788029 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54028.9us
I20260709 12:25:43.792366 23795 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:43.794947 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61090.3us
I20260709 12:25:43.795148 23831 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:43.796186 14420 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:25:43.797792 24029 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:43.750130 24064 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:25:43.751039 23824 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:43.749984 24066 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:43.674968 13542 rpcz_store.cc:269] 0709 12:25:01.553836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.229279 (+14675443us) service_pool.cc:224] Handling call
0709 12:25:43.581450 (+27352171us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:32.207785 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:32.208468 (+   683us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:32.208531 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:25:32.208544 (+    13us) tablet.cc:662] Decoding operations
0709 12:25:32.372011 (+163467us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:32.386448 (+ 14437us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:32.400755 (+ 14307us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:25:32.418109 (+ 17354us) tablet.cc:701] Row locks acquired
0709 12:25:32.418183 (+    74us) write_op.cc:260] PREPARE: finished
0709 12:25:32.476540 (+ 58357us) write_op.cc:270] Start()
0709 12:25:32.540734 (+ 64194us) write_op.cc:276] Timestamp: P: 1783599932459732 usec, L: 0
0709 12:25:32.558597 (+ 17863us) op_driver.cc:348] REPLICATION: starting
0709 12:25:32.653453 (+ 94856us) log.cc:844] Serialized 7025 byte log entry
0709 12:25:33.268056 (+614603us) op_driver.cc:464] REPLICATION: finished
0709 12:25:33.683174 (+415118us) write_op.cc:301] APPLY: starting
0709 12:25:33.683292 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:34.080769 (+397477us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:34.080913 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:34.119277 (+ 38364us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b18001e2800. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:34.129701 (+ 10424us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:34.130124 (+   423us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:25:34.130184 (+    60us) write_op.cc:312] APPLY: finished
0709 12:25:34.130697 (+   513us) log.cc:844] Serialized 164 byte log entry
0709 12:25:34.520645 (+389948us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:34.544490 (+ 23845us) write_op.cc:454] Released schema lock
0709 12:25:34.568071 (+ 23581us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:34.632227 (+ 64156us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:34.656363 (+ 24136us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":193622,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":394362,"apply.run_cpu_time_us":64437,"apply.run_wall_time_us":922510,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79511,"num_ops":18,"prepare.queue_time_us":193087,"prepare.run_cpu_time_us":30560,"prepare.run_wall_time_us":487104,"raft.queue_time_us":308123,"raft.run_cpu_time_us":17798,"raft.run_wall_time_us":388696,"replication_time_us":648020,"spinlock_wait_cycles":38003200,"thread_start_us":833503,"threads_started":5,"wal-append.queue_time_us":593206,"wal-append.run_cpu_time_us":24711,"wal-append.run_wall_time_us":2402914}]]}
I20260709 12:25:43.796276 23957 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:43.805379 23547 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:43.806665 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:43.810303 13793 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
W20260709 12:25:43.823216 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67910.8us
I20260709 12:25:43.749531 23822 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:43.840742 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:43.842056 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61135.2us
I20260709 12:25:43.757718 23987 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:43.852689 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.041272 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:44.043022 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.043850 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 237258us
I20260709 12:25:44.001641 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:44.045053 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:44.045930 13797 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:25:43.958680 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.925762 13547 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 21 to 29, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 29 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:44.045110 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:44.050912 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:44.052595 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.053426 13794 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:44.053957 13645 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:44.055254 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:44.055826 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 259) took 22975 ms (client timeout 15000 ms). Trace:
W20260709 12:25:44.074928 13542 rpcz_store.cc:269] 0709 12:25:21.080373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.055592 (+22975219us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:44.055737 (+   145us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:25:44.076833 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 237) took 21535 ms (client timeout 15000 ms). Trace:
W20260709 12:25:44.055951 24043 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.613s	user 0.000s	sys 0.006s
I20260709 12:25:44.056398 23935 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:44.056329 23987 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:25:44.077373 13542 rpcz_store.cc:269] 0709 12:25:22.541033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.076514 (+21535481us) service_pool.cc:211] Skipping call since client already timed out
0709 12:25:44.076728 (+   214us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:25:44.059514 23971 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.080655 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:44.080441 23824 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.057289 24057 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:44.081423 23971 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.071027 13793 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:43.879372 24055 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:43.844265 24046 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:44.044018 23822 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:44.043560 23888 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:44.077322 13792 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:43.950073 23820 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.084497 13644 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 12:25:43.953797 23642 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.012s
W20260709 12:25:44.018567 23785 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.006s
I20260709 12:25:44.108196 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:44.091411 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:43.758337 24036 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.022364 23998 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:44.098485 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:44.098716 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58304.8us
I20260709 12:25:44.134366 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:44.103127 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:44.104830 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44088.5us
W20260709 12:25:44.081476 24074 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:44.051579 24026 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.087393 24055 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.111943 23642 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.112646 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.108975 23800 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.113849 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.115410 24062 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:44.117959 23785 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:44.108045 23965 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.119338 23901 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.789s	user 0.015s	sys 0.001s
I20260709 12:25:44.119560 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.170001 13726 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:44.124303 23820 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:44.124946 23987 consensus_queue.cc:1243] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 35.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:44.127825 24073 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.128522 23959 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:25:44.091961 23897 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.009s
I20260709 12:25:44.059793 23972 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:44.111763 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.188077 13720 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:44.085213 24057 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.147708 23935 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:25:44.108449 24029 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.004s
I20260709 12:25:44.140026 13724 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:44.138722 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58193.6us
W20260709 12:25:44.141378 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.108150 23888 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:44.097280 24065 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:44.149371 13646 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 12:25:44.159207 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.149932 24055 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:44.174018 13796 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:44.175176 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:44.176141 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:44.170965 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 129989us
I20260709 12:25:44.213749 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:44.156842 24062 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:44.090960 23957 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:44.194512 24057 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:44.181618 23946 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.004s
I20260709 12:25:44.209601 24050 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:44.195151 23972 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:44.162667 23800 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.218154 13722 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:44.140697 23998 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:44.199553 24033 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.894s	user 0.013s	sys 0.003s
I20260709 12:25:44.174893 23965 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:44.238155 23685 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.828s	user 0.006s	sys 0.006s
I20260709 12:25:44.239483 23998 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:44.241451 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.176456 23901 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.246126 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.247612 13647 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:44.256953 13720 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:44.338068 24082 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:44.404846 24084 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.427165 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 252035us
I20260709 12:25:44.189386 23897 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.462349 23800 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:44.465759 13646 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:44.329607 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:44.415423 13724 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:44.471073 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.280612 13721 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:44.187215 23959 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.399672 24091 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.227443 24020 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.289570 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.317163 24050 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:44.233327 23946 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:44.300594 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 86981.9us
I20260709 12:25:44.316327 13795 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:44.224941 24065 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:44.223923 24062 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:44.234333 23820 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:44.337131 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.323298 23547 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:44.379822 24078 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:44.425257 24036 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.425971 23965 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:44.324884 23957 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:44.187377 24073 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:44.429385 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.496253 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.432585 23987 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:44.433010 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.472256 24082 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:44.434140 23972 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.436774 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:44.438136 24089 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:25:44.406296 24054 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.438624 24072 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.439113 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.440064 24084 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:44.426519 24076 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.256s	user 0.000s	sys 0.002s
W20260709 12:25:44.441052 24058 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.822s	user 0.003s	sys 0.000s
I20260709 12:25:44.441366 24026 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:44.485219 23780 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.036s
I20260709 12:25:44.446576 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:44.452433 23971 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:44.453840 24087 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.467182 23901 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:44.470785 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:44.476373 24069 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.004s
I20260709 12:25:44.476976 24080 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.477779 24091 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.489342 13798 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:44.484481 13722 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:44.490414 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:44.434435 23642 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:44.514998 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:44.500209 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 170724us
I20260709 12:25:44.549207 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:44.518064 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.550618 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.551155 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.489418 23800 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:44.521167 14422 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:44.505996 23959 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.448534 13726 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:44.536741 13721 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
W20260709 12:25:44.520412 23970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.020s	sys 0.023s
W20260709 12:25:44.542198 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 89541.5us
I20260709 12:25:44.565267 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.559136 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.568430 24067 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.568975 24067 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.570000 23934 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:44.570425 23934 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:44.571411 24067 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:44.571362 23934 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.572501 23934 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:44.573267 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.573498 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:44.573909 13721 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30. Re-sending same reply.
I20260709 12:25:44.575117 13726 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:25:44.575377 23946 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:44.576467 13726 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
W20260709 12:25:44.577534 23970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.020s	sys 0.024s
I20260709 12:25:44.579885 13720 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:44.581350 13720 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:25:44.582811 13721 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:25:44.586912 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102553us
I20260709 12:25:44.587803 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:44.578042 13726 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:44.592803 13726 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:44.544531 24069 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:44.533874 24055 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.594831 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:44.598109 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 166440us
I20260709 12:25:44.598294 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:44.549479 23987 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:44.594134 23824 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:44.540369 23971 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.567147 23642 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:44.609344 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:44.608846 23946 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.623692 23946 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.627262 23888 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.630366 23901 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:44.630875 23901 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:44.631275 23901 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:44.635056 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5104) took 1124 ms. Trace:
I20260709 12:25:44.635644 24046 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:44.636240 24046 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:44.635842 13650 rpcz_store.cc:276] 0709 12:25:43.512148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.512261 (+   113us) service_pool.cc:224] Handling call
0709 12:25:43.541540 (+ 29279us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:43.634088 (+ 92548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.634125 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:43.669267 (+ 35142us) log.cc:844] Serialized 52 byte log entry
0709 12:25:44.627317 (+958050us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b180036c6a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1
0709 12:25:44.627590 (+   273us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:44.628238 (+   648us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.628667 (+   429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.630210 (+  1543us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b54002224d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:44.634682 (+  4472us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.634753 (+    71us) raft_consensus.cc:1685] finished
0709 12:25:44.634771 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.634975 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47310,"prepare.queue_time_us":5006,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":61486,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":13018,"spinlock_wait_cycles":225526912,"thread_start_us":323221,"threads_started":1,"wal-append.queue_time_us":327182}
I20260709 12:25:44.639118 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:44.640588 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:44.597908 23685 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: INVALID_TERM, Last received: 33.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.045s
I20260709 12:25:44.642480 24026 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:44.642329 23897 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:44.651434 23822 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.000s
I20260709 12:25:44.651906 23897 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:44.652551 23897 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.644623 13724 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:44.533648 24072 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.645903 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:44.607714 13726 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 34.18, Last appended by leader: 18, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.647683 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:44.534333 24054 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.647322 24026 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.555583 24080 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.598129 23780 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:25:44.667193 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.677407 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.678819 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:44.668520 13799 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 38: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:44.671716 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:44.680389 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71163.9us
I20260709 12:25:44.681566 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:44.686688 14423 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:25:44.646601 23642 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:44.648849 23888 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:44.669127 23800 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:44.697052 14416 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:44.542194 24087 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:44.701910 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61415.4us
I20260709 12:25:44.671245 24026 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:44.687747 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.705096 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:44.706782 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:44.707576 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5114) took 1083 ms. Trace:
I20260709 12:25:44.656421 24091 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:44.711122 13647 rpcz_store.cc:276] 0709 12:25:43.624596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.651468 (+ 26872us) service_pool.cc:224] Handling call
0709 12:25:44.707328 (+1055860us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b54002b48d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:44.707491 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1368293120}
I20260709 12:25:44.651929 23822 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.695569 23841 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.689631 24047 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:44.720806 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:44.613428 24057 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.696398 23946 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.554793 24065 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.697177 23642 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:44.696274 23970 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:44.732285 14422 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:44.606299 23957 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.566403 24050 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.738631 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5107) took 1210 ms. Trace:
I20260709 12:25:44.754403 13644 rpcz_store.cc:276] 0709 12:25:43.527695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.777604 (+249909us) service_pool.cc:224] Handling call
0709 12:25:43.777995 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.094613 (+316618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.737864 (+643251us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400113354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:44.738198 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":704273664}
I20260709 12:25:44.736883 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.752971 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4686) took 1351 ms. Trace:
I20260709 12:25:44.756760 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:44.756429 13801 rpcz_store.cc:276] 0709 12:25:43.391961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.392292 (+   331us) service_pool.cc:224] Handling call
0709 12:25:43.410930 (+ 18638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.836574 (+425644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.742715 (+906141us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54001ca7d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:44.743415 (+   700us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848725760}
I20260709 12:25:44.723141 24020 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.768647 13727 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: [29.16-29.16]
I20260709 12:25:44.771445 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:44.736510 23959 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.789842 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:44.790927 13644 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
W20260709 12:25:44.806527 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.806635 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.751488 24107 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:44.812250 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:44.812940 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:44.814571 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:44.818140 13644 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:25:44.816107 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:44.820573 13793 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:44.822559 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.823342 14418 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33. Ops: []
W20260709 12:25:44.826640 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69975us
I20260709 12:25:44.742621 23841 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.673640 24082 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:45.027210 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80760.7us
I20260709 12:25:44.835314 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:45.028208 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.029088 13650 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:25:44.806793 24067 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.843346 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:44.844439 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:44.762692 23685 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:25:44.847167 24072 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.832883 23800 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:45.058557 23800 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:45.059240 23800 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:44.848402 14419 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:45.063687 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:45.065400 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:44.854074 24098 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:25:44.855093 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3187) took 1377 ms. Trace:
I20260709 12:25:45.066807 13796 rpcz_store.cc:276] 0709 12:25:43.477469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.477624 (+   155us) service_pool.cc:224] Handling call
0709 12:25:44.854780 (+1377156us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54000e4cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:44.854994 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1493621504}
W20260709 12:25:44.858845 24095 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 12:25:45.069226 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:44.825693 24090 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:45.070745 24090 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.074716 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2124) took 1021 ms. Trace:
I20260709 12:25:45.075740 13718 rpcz_store.cc:276] 0709 12:25:44.053451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.053618 (+   167us) service_pool.cc:224] Handling call
0709 12:25:44.078432 (+ 24814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.196226 (+117794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.196256 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.224816 (+ 28560us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.482359 (+257543us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:44.482398 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.482478 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.074368 (+591890us) raft_consensus.cc:1685] finished
0709 12:25:45.074398 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.074624 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18945,"raft.queue_time_us":42679,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"thread_start_us":245185,"threads_started":2,"wal-append.queue_time_us":238841}
I20260709 12:25:44.861007 23822 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.078904 23822 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:45.079811 23822 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:44.862380 24073 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.769161 24093 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.239s
W20260709 12:25:44.875731 24102 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:25:44.815768 24107 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:45.085654 24107 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:44.893828 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 165588us
I20260709 12:25:45.087282 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:44.905207 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:44.906888 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.090132 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.905242 24020 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.917441 24057 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.773286 24108 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.114s
I20260709 12:25:44.919230 24054 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:44.811655 23959 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:44.781025 23970 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.006991 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:45.035225 24067 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.037109 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.038707 24111 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.044561 23841 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.044938 24046 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.052996 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:45.028231 24062 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.098381 14419 raft_consensus.cc:2410] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 30 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 12:25:44.925436 24109 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:45.145298 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.184171 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.162746 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:45.188567 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:25:45.189491 14418 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 20: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:25:45.166935 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.096493 24055 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.104936 23971 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:45.179601 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 110544us
I20260709 12:25:44.905135 23824 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:45.193162 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:45.209424 14416 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
W20260709 12:25:45.203397 13690 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 1.510s (SENT)
I20260709 12:25:45.088143 23901 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.199710 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.212581 13647 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:45.178889 23897 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.186926 24090 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.215968 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:45.210567 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:45.146214 24020 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.246202 14419 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:45.209750 24050 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:45.252411 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.180300 23642 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.180960 24111 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:45.263085 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 256218us
W20260709 12:25:45.282593 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 79261.2us
I20260709 12:25:45.240826 24047 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:45.302098 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.243546 23685 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:45.290416 23824 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.293916 23884 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.296212 24105 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:25:45.295367 13766 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:45.253993 23822 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.254419 24050 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:45.247670 24087 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.327524 13794 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:45.299343 24111 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:45.254582 24114 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:25:45.295332 23901 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:45.343925 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48793.5us
I20260709 12:25:45.352051 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:45.279465 23841 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:45.200448 24109 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.355234 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:45.345960 24106 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:25:45.348456 24050 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:45.358096 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 155353us
W20260709 12:25:45.360772 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 144949us
I20260709 12:25:45.362030 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.356994 23841 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:45.371249 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 284031us
I20260709 12:25:45.377179 23970 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.373674 23897 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:45.387220 24062 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.388417 23642 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:45.388662 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:45.389989 24113 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:25:45.374845 23841 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:45.364218 23901 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:25:45.416882 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57361.4us
I20260709 12:25:45.418296 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:45.421819 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4301) took 1024 ms. Trace:
W20260709 12:25:45.387288 24104 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:25:45.430998 14424 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:25:45.422447 13720 rpcz_store.cc:276] 0709 12:25:44.397670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.589704 (+192034us) service_pool.cc:224] Handling call
0709 12:25:44.589969 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.594407 (+  4438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.594442 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.594950 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.735342 (+140392us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:45.230624 (+495282us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.238835 (+  8211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.238969 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.421477 (+182508us) raft_consensus.cc:1685] finished
0709 12:25:45.421525 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.421725 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11367,"prepare.queue_time_us":544434,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":2903,"raft.queue_time_us":7317,"raft.run_cpu_time_us":554,"raft.run_wall_time_us":17569,"thread_start_us":345750,"threads_started":2,"wal-append.queue_time_us":445801}
I20260709 12:25:45.433689 24129 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:45.436285 24129 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.436936 24129 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:45.439352 24117 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:25:45.441902 24112 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:45.453678 23897 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:45.453806 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71293.3us
W20260709 12:25:45.454919 24116 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:25:45.454679 23642 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:45.455457 23897 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.461562 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4295) took 1145 ms. Trace:
I20260709 12:25:45.413355 24108 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:25:45.454550 24125 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:45.462178 13721 rpcz_store.cc:276] 0709 12:25:44.316816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.584213 (+267397us) service_pool.cc:224] Handling call
0709 12:25:44.584663 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.585828 (+  1165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.585858 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.587626 (+  1768us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.927614 (+339988us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:45.253623 (+326009us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.253921 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.254021 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.461226 (+207205us) raft_consensus.cc:1685] finished
0709 12:25:45.461256 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.461475 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123815,"prepare.queue_time_us":428746,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":1971,"raft.queue_time_us":454,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":21814,"thread_start_us":450563,"threads_started":2,"wal-append.queue_time_us":570297}
W20260709 12:25:45.639104 13767 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.640s (SENT)
I20260709 12:25:45.651152 13648 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 23. Current term is 24. Ops: []
W20260709 12:25:45.660988 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 242849us
I20260709 12:25:45.655396 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.668134 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:45.678643 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1624) took 2832 ms (client timeout 2204 ms). Trace:
W20260709 12:25:45.679476 13643 rpcz_store.cc:269] 0709 12:25:42.801918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.803966 (+  2048us) service_pool.cc:224] Handling call
0709 12:25:45.634098 (+2830132us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b5400198554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:45.634309 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982825216}
I20260709 12:25:45.684602 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:45.640571 24095 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:25:45.628774 24046 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.654187 24111 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:45.695535 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:45.695866 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 276503us
I20260709 12:25:45.480676 23959 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.685480 14421 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:45.700235 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:45.710417 13722 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:45.701040 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.695122 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:45.702539 13721 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:25:45.711192 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.706409 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:45.712224 13723 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:45.707563 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:45.712996 13719 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:45.636937 23901 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:45.720820 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:45.659780 24108 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:45.721834 13721 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:45.714484 13643 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:25:45.685262 24020 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.723197 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:45.726776 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:45.732856 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:45.731444 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101181us
W20260709 12:25:45.734643 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.735845 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:45.736626 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:45.732120 13719 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 20.
I20260709 12:25:45.744439 13720 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:45.745910 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:45.757845 24120 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:45.751338 24109 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.753044 24108 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:45.751026 24026 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.018s	user 0.023s	sys 0.001s
I20260709 12:25:45.763185 24026 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.764564 23841 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.766757 13797 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 12:25:45.767521 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.752413 23987 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.143s	user 0.014s	sys 0.014s
I20260709 12:25:45.767993 23841 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.768620 23987 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:45.768848 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4691) took 1716 ms. Trace:
I20260709 12:25:45.769294 13797 rpcz_store.cc:276] 0709 12:25:44.052903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.653492 (+600589us) service_pool.cc:224] Handling call
0709 12:25:44.656982 (+  3490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.767239 (+1110257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.768549 (+  1310us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400103454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:45.768779 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2551184768}
I20260709 12:25:45.770568 14419 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:45.774820 23901 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.778693 24093 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:25:45.779412 23800 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.720s	user 0.010s	sys 0.015s
I20260709 12:25:45.779686 24093 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:45.781812 24107 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.678s	user 0.018s	sys 0.000s
I20260709 12:25:45.783071 24129 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.784134 13720 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:45.785247 23934 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.212s	user 0.006s	sys 0.009s
I20260709 12:25:45.779866 23800 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.716820 24111 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.765793 24026 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:45.816944 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:45.746186 23959 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:45.818689 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5130) took 1001 ms. Trace:
I20260709 12:25:45.743723 23685 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:25:45.781443 24093 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.750360 24124 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.810701 13723 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:45.750584 24122 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.834478 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.801627 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:45.836560 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.837448 13800 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:45.838718 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:45.828867 24026 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:45.743573 24105 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.074s
I20260709 12:25:45.830860 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.831115 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.839673 13721 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:45.840332 13792 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:45.825703 13643 rpcz_store.cc:276] 0709 12:25:44.817406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.786768 (+969362us) service_pool.cc:224] Handling call
0709 12:25:45.818615 (+ 31847us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:45.841356 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.816440 23800 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.826681 23685 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.809718 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:45.825977 23959 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.838717 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.851734 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:45.832582 23935 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.848304 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:45.816574 23944 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.029s
W20260709 12:25:45.840541 13649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.020s	sys 0.065s
I20260709 12:25:45.844287 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:45.857623 13649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.020s	sys 0.065s
I20260709 12:25:45.852514 13800 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:45.856527 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:45.799269 24107 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.844503 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.858769 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:45.842730 23970 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:45.847518 24093 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:45.871889 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:45.872632 13723 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:45.854966 23685 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:25:45.855765 23959 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:45.874514 13721 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:45.860406 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5121) took 1051 ms. Trace:
I20260709 12:25:45.899715 13795 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:45.899956 13643 rpcz_store.cc:276] 0709 12:25:44.808843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.851413 (+1042570us) service_pool.cc:224] Handling call
0709 12:25:45.860333 (+  8920us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:45.879894 13690 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Timed out: RequestConsensusVote RPC to 127.12.45.68:42965 timed out after 2.074s (SENT)
I20260709 12:25:45.839246 24124 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.859201 24100 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:45.901194 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:45.871614 24133 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:25:45.843523 24122 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.909479 13726 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:45.854059 24026 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.859522 23935 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:45.920540 13723 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:45.926604 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 190877us
I20260709 12:25:45.926872 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:45.927736 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:45.873521 23970 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:45.930807 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:45.932674 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:45.931284 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3209) took 1095 ms. Trace:
I20260709 12:25:45.934245 13643 rpcz_store.cc:276] 0709 12:25:44.836386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.920956 (+1084570us) service_pool.cc:224] Handling call
0709 12:25:45.931209 (+ 10253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:45.920085 24072 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:45.937242 13726 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 18. Preceding OpId from leader: term: 35 index: 18. (term mismatch)
I20260709 12:25:45.938436 13726 pending_rounds.cc:85] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 17
I20260709 12:25:45.938833 13726 pending_rounds.cc:107] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 34.18
I20260709 12:25:45.939096 13726 raft_consensus.cc:2889] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:45.939891 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4288) took 1729 ms. Trace:
I20260709 12:25:45.940229 13726 rpcz_store.cc:276] 0709 12:25:44.211251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.577478 (+366227us) service_pool.cc:224] Handling call
0709 12:25:44.577908 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.870869 (+1292961us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b54003127a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b69a1d 00007f5393b501a4 00007f5393bd2447 00007f5393beafe7 00007f5393bec7de 00007f5393bf6b49 00007f5393bf7416 00007f5393bf3d8f 00007f5393bf383e 00007f5398bb8419
0709 12:25:45.939542 (+ 68673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.939763 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5739904}
I20260709 12:25:45.934693 24105 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:45.947849 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:45.948807 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3207) took 1134 ms. Trace:
I20260709 12:25:45.949136 13643 rpcz_store.cc:276] 0709 12:25:44.814565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.942624 (+1128059us) service_pool.cc:224] Handling call
0709 12:25:45.948736 (+  6112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:45.950229 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:45.947269 24141 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:45.953756 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4682) took 2657 ms. Trace:
I20260709 12:25:45.953099 13643 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:45.954180 13798 rpcz_store.cc:276] 0709 12:25:43.296683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.302679 (+  5996us) service_pool.cc:224] Handling call
0709 12:25:43.303104 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.548273 (+1245169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.953493 (+1405220us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540012aa54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:45.953693 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2727590272}
W20260709 12:25:45.951762 24135 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:25:45.959554 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:45.955541 24124 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:45.967754 24134 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:25:45.969045 24111 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:45.975418 23935 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:45.976611 13726 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 19.
I20260709 12:25:45.984157 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:45.953786 24141 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:46.009574 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:45.992854 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5103) took 2482 ms. Trace:
I20260709 12:25:46.005576 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.008934 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5106) took 2484 ms. Trace:
W20260709 12:25:45.993162 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4294) took 1685 ms (client timeout 1638 ms). Trace:
I20260709 12:25:45.982832 24111 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:45.992081 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.030566 13649 rpcz_store.cc:276] 0709 12:25:43.510242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.510384 (+   142us) service_pool.cc:224] Handling call
0709 12:25:43.533563 (+ 23179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.586881 (+ 53318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.586913 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.620455 (+ 33542us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.135778 (+1515323us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b18004ac740. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:25:45.858107 (+722329us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:45.858146 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.858219 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.992526 (+134307us) raft_consensus.cc:1685] finished
0709 12:25:45.992548 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.992775 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54147,"raft.queue_time_us":87387,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6259,"spinlock_wait_cycles":6566784,"thread_start_us":722099,"threads_started":1,"wal-append.queue_time_us":2235532}
I20260709 12:25:46.014108 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:46.034571 13722 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:46.003557 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.031077 14416 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:46.011656 24067 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.037051 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:46.037906 23971 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:46.038354 23971 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:46.038745 23971 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:46.040126 24080 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.990193 24122 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:46.031013 13652 rpcz_store.cc:276] 0709 12:25:43.527313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.838324 (+311011us) service_pool.cc:224] Handling call
0709 12:25:43.838988 (+   664us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:43.948746 (+109758us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:25:44.039598 (+ 90852us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.229682 (+190084us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.229906 (+   224us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:44.261094 (+ 31188us) log.cc:844] Serialized 52 byte log entry
0709 12:25:44.442572 (+181478us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:44.449561 (+  6989us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.453424 (+  3863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.455392 (+  1968us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b480033c558. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:46.008547 (+1553155us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:46.008609 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.008662 (+    53us) raft_consensus.cc:1685] finished
0709 12:25:46.008675 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.008872 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141006,"prepare.queue_time_us":94599,"prepare.run_cpu_time_us":148,"prepare.run_wall_time_us":156,"raft.queue_time_us":26793,"raft.run_cpu_time_us":3348,"raft.run_wall_time_us":258056,"spinlock_wait_cycles":166601216,"thread_start_us":128484,"threads_started":1,"wal-append.queue_time_us":245755,"wal-append.run_cpu_time_us":10955,"wal-append.run_wall_time_us":1130683}
I20260709 12:25:46.046500 23820 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:46.047016 23820 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:46.049232 13724 rpcz_store.cc:269] 0709 12:25:44.307690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.470806 (+163116us) service_pool.cc:224] Handling call
0709 12:25:45.992991 (+1522185us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54002f7954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:45.993095 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397083520}
W20260709 12:25:46.005374 24140 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
I20260709 12:25:45.989501 23935 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.055262 23935 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:45.993511 23831 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 35 current term: 35 is higher.
I20260709 12:25:46.056744 24084 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:46.012539 23998 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:46.057473 23998 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.058666 23998 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.059957 23998 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:25:46.061203 24050 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.694s	user 0.007s	sys 0.006s
I20260709 12:25:46.061725 24050 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.063966 24141 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.066161 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:46.067512 13724 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:46.004920 23970 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:45.990607 24107 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:45.993906 24026 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.081275 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:46.082168 13726 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:46.088656 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.075181 24050 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:45.987535 23824 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.088s
W20260709 12:25:46.090968 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.106015 23824 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:46.106688 23824 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.110994 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:46.110852 23935 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:46.112339 23824 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:46.110970 13724 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:25:46.051275 24111 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:46.118281 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.116128 13724 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:46.127660 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.443s	user 0.003s	sys 0.033s
I20260709 12:25:46.131381 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:46.137310 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.003s	sys 0.034s
W20260709 12:25:46.139226 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.45768e+06us
I20260709 12:25:46.140166 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.118553 23831 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.022s
W20260709 12:25:46.141757 14420 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1593) took 2426 ms (client timeout 1914 ms). Trace:
W20260709 12:25:46.143708 14420 rpcz_store.cc:269] 0709 12:25:43.715718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.718415 (+  2697us) service_pool.cc:224] Handling call
0709 12:25:46.141558 (+2423143us) spinlock_profiling.cc:243] Waited 61.4 ms on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.141679 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141204224}
W20260709 12:25:46.131193 13617 connection.cc:256] RPC call timeout handler was delayed by -1.1715922169996702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2504s for a response
W20260709 12:25:46.137989 14391 connection.cc:256] RPC call timeout handler was delayed by -1.8032748129999163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 12:25:46.145082 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0455646780001189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30920000000000003s for a response
I20260709 12:25:46.150406 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.151003 23935 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:46.151129 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.151312 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:46.151906 24072 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:46.151671 23935 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.152328 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.153887 23935 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.154784 13724 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:46.155962 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.122565 24084 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:25:46.156011 24047 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:46.159818 24047 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:46.160408 13647 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5123) took 1351 ms (client timeout 1600 ms). Trace:
I20260709 12:25:46.160681 24047 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:46.161814 13651 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5097) took 2673 ms (client timeout 1640 ms). Trace:
I20260709 12:25:46.114158 23971 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:46.162237 13651 rpcz_store.cc:269] 0709 12:25:43.488453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.488660 (+   207us) service_pool.cc:224] Handling call
0709 12:25:46.161621 (+2672961us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.161746 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2330305920}
I20260709 12:25:46.163601 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.164304 14416 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
W20260709 12:25:46.160843 13647 rpcz_store.cc:269] 0709 12:25:44.809412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.052576 (+243164us) service_pool.cc:224] Handling call
0709 12:25:46.160211 (+1107635us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b5400114254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.160350 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86063488}
I20260709 12:25:46.171094 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.175311 24111 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.178776 24066 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.178632 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.162032 24047 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:46.164844 23971 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.183223 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:46.183849 14416 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:46.184803 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.185391 13647 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:46.183272 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:46.185194 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:46.114113 23685 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.188053 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.196990 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:46.198180 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.199172 23685 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.199950 23998 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:46.166980 23959 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.201583 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:46.183740 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:46.202558 23888 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:46.202926 23888 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.203259 23888 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:25:46.192266 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.205152 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.194855 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.195230 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.207274 13650 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:25:46.195451 24066 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:46.196074 24111 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:46.198288 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:46.154836 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.135320 13726 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:46.220116 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:46.227442 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.213495 23831 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:46.209765 24080 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:46.212822 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:46.230947 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4548) took 2127 ms. Trace:
I20260709 12:25:46.167346 24093 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:46.238916 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:46.239194 13799 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:46.206450 24126 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
W20260709 12:25:46.224368 13725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4285) took 2021 ms (client timeout 1907 ms). Trace:
I20260709 12:25:46.240531 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:46.212263 24105 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:25:46.185979 23957 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:46.230373 23831 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:46.216756 13651 raft_consensus.cc:2368] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30.
I20260709 12:25:46.231616 13727 rpcz_store.cc:276] 0709 12:25:44.103021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.103693 (+   672us) service_pool.cc:224] Handling call
0709 12:25:44.119144 (+ 15451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.771621 (+652477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.230564 (+1458943us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b540021fcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:46.230861 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492864384}
I20260709 12:25:46.240703 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.241137 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5108) took 2713 ms. Trace:
W20260709 12:25:46.240937 13725 rpcz_store.cc:269] 0709 12:25:44.202939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.490357 (+287418us) service_pool.cc:224] Handling call
0709 12:25:46.224048 (+1733691us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002f7454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.224188 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2811920256}
I20260709 12:25:46.242616 13801 raft_consensus.cc:2410] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
W20260709 12:25:46.243263 13617 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.503s (SENT)
I20260709 12:25:46.205319 23959 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:46.239941 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88700us
I20260709 12:25:46.246722 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.254479 13800 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:46.249475 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.543s (SENT)
I20260709 12:25:46.250622 24105 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:46.246618 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.225471 23998 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:46.225873 24066 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:46.248052 13649 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:46.252872 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.259433 13650 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 20.
I20260709 12:25:46.247992 13645 rpcz_store.cc:276] 0709 12:25:43.527954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.091089 (+563135us) service_pool.cc:224] Handling call
0709 12:25:44.099621 (+  8532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.136441 (+ 36820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.136478 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.151490 (+ 15012us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.228049 (+ 76559us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:44.228102 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.224521 (+1996419us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:46.224574 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.224806 (+   232us) raft_consensus.cc:1685] finished
0709 12:25:46.224826 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.241068 (+ 16242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12616,"raft.queue_time_us":4240,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":17660,"spinlock_wait_cycles":74660224,"thread_start_us":36054,"threads_started":1,"wal-append.queue_time_us":113795,"wal-append.run_cpu_time_us":6264,"wal-append.run_wall_time_us":1322153}
I20260709 12:25:46.261343 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.261618 23831 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.263036 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:46.251734 24124 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:46.266569 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:46.267238 14423 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:46.268371 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.268567 13617 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.600s (SENT)
I20260709 12:25:46.265198 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.255192 24107 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:25:46.263489 23831 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:46.263588 24066 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:46.257112 24105 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:46.278126 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.263939 24126 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:46.283895 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.284638 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.244567 24093 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:46.290449 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.638s (SENT)
I20260709 12:25:46.266222 23934 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.285389 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.293176 13651 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:46.267614 24124 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:46.286850 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.290654 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.291229 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.296770 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:46.297688 14421 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:25:46.298125 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.303129 24111 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:46.303918 24111 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.304435 24111 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:46.270886 24092 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:25:46.286036 24029 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:46.311331 24029 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.312674 24029 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.299798 23934 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.280395 24107 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.285125 24040 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:46.280285 23970 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.317266 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 105842us
I20260709 12:25:46.315744 24107 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:46.321820 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:46.320394 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.322685 14391 leader_election.cc:400] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30.
I20260709 12:25:46.322970 14391 leader_election.cc:403] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Cancelling election due to peer responding with higher term
I20260709 12:25:46.323983 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:46.285933 23972 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.286823 23946 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:46.294040 23998 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:46.286520 24067 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:46.299618 23500 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:25:46.349285 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.335661 13647 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
W20260709 12:25:46.354707 14424 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1487) took 1667 ms (client timeout 2074 ms). Trace:
W20260709 12:25:46.355182 14424 rpcz_store.cc:269] 0709 12:25:44.687467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.687903 (+   436us) service_pool.cc:224] Handling call
0709 12:25:46.354489 (+1666586us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b5400644bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.354637 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793123584}
I20260709 12:25:46.349215 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.359071 14425 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:46.360045 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.326086 24050 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:46.361012 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74267.8us
W20260709 12:25:46.361891 13617 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.627s (SENT)
I20260709 12:25:46.360929 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.363984 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.365489 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:46.360989 13647 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:25:46.338586 24080 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:46.334776 24091 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:46.370491 13643 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:25:46.376657 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5122) took 1567 ms. Trace:
I20260709 12:25:46.377060 13643 rpcz_store.cc:276] 0709 12:25:44.809138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.949918 (+1140780us) service_pool.cc:224] Handling call
0709 12:25:46.376587 (+426669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.325603 23970 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.382936 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:46.385495 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.385773 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.386104 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.386276 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.387091 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.387322 13797 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:46.390756 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.327044 24040 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.391728 13643 raft_consensus.cc:2410] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:25:46.392508 14416 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:46.366922 24126 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:46.394567 14419 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
W20260709 12:25:46.392506 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 91246.6us
I20260709 12:25:46.372689 24084 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:46.340030 23972 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:46.402635 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 3.092s (SENT)
I20260709 12:25:46.371117 23998 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.405906 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:46.400935 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:46.404349 13718 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:25:46.373824 24072 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:46.372859 24139 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:46.405386 13645 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:46.361788 24067 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.419162 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.359570 23946 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.422109 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1482) took 1817 ms. Trace:
I20260709 12:25:46.423099 14417 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:46.393515 23642 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:46.419236 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.424697 14422 rpcz_store.cc:276] 0709 12:25:44.604301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.604519 (+   218us) service_pool.cc:224] Handling call
0709 12:25:44.648950 (+ 44431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.744832 (+ 95882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.421737 (+1676905us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:46.422027 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172938624}
W20260709 12:25:46.434917 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110975us
I20260709 12:25:46.374374 23901 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:25:46.402190 24084 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:46.400811 24050 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.451573 13645 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:46.451892 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.437083 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:46.410889 24139 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.396751 24081 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:25:46.373021 24105 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.405128 24064 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:25:46.424677 24091 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.369222 24124 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.470515 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.453145 24084 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:46.463402 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.482131 13796 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 39: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:46.478873 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.487582 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.498378 24124 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:46.458320 23642 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.458465 24139 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:46.460016 23934 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:46.459625 24069 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:46.516844 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.451072 24072 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:25:46.524058 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.530591 13797 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:25:46.532066 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.532932 13797 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:25:46.534596 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:46.515305 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 109509us
I20260709 12:25:46.526274 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:46.556309 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 148007us
I20260709 12:25:46.558137 24105 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:46.567095 14421 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 39.
I20260709 12:25:46.578860 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.568950 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:46.584803 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 147871us
I20260709 12:25:46.585855 14423 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:46.590736 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.597596 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4694) took 1952 ms. Trace:
W20260709 12:25:46.601512 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.570360 24047 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.600435 13794 rpcz_store.cc:276] 0709 12:25:44.645142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.645277 (+   135us) service_pool.cc:224] Handling call
0709 12:25:44.651174 (+  5897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.345147 (+693973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.597240 (+1252093us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b54000d16d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:46.597503 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554804608}
I20260709 12:25:46.569474 24141 raft_consensus.cc:2764] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election decision vote started in defunct term 38: won
I20260709 12:25:46.586808 24080 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:46.618935 13651 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:46.619889 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5124) took 1810 ms. Trace:
I20260709 12:25:46.620409 13651 rpcz_store.cc:276] 0709 12:25:44.809712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.270656 (+1460944us) service_pool.cc:224] Handling call
0709 12:25:46.619795 (+349139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.587384 23780 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30.
I20260709 12:25:46.626308 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.513693 23998 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:46.626883 13617 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.907s (SENT)
I20260709 12:25:46.628686 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.633349 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.635435 24159 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:46.635751 23820 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 34.25, Last appended by leader: 25, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.611924 24081 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:25:46.618778 23642 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.638888 23946 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:46.645473 23972 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:46.647851 24066 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.650780 24067 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:46.653064 13798 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:46.654788 24162 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:25:46.657330 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78728.5us
I20260709 12:25:46.707080 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:46.662609 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 100209us
I20260709 12:25:46.672073 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:46.663995 24156 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 12:25:46.688838 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3198) took 3658 ms. Trace:
I20260709 12:25:46.698308 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5127) took 1881 ms. Trace:
I20260709 12:25:46.700076 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1598) took 1881 ms. Trace:
I20260709 12:25:46.708796 13651 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 24 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 12:25:46.707551 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:46.662138 24069 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:46.713621 14417 rpcz_store.cc:276] 0709 12:25:44.818697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.818882 (+   185us) service_pool.cc:224] Handling call
0709 12:25:46.422632 (+1603750us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.424946 (+  2314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.699781 (+274835us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54005e4f50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.699994 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3687269504}
I20260709 12:25:46.726497 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:46.727952 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.713462 13645 rpcz_store.cc:276] 0709 12:25:44.816497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.267911 (+1451414us) service_pool.cc:224] Handling call
0709 12:25:46.698009 (+430098us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002b39d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.698184 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263998848}
W20260709 12:25:46.730577 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 292393us
I20260709 12:25:46.587447 24091 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.667273 24159 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:46.766155 13643 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 35: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:46.735787 24111 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.667388 24040 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:46.592051 24050 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.739305 13798 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
W20260709 12:25:46.739853 23888 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.001s
I20260709 12:25:46.741834 24084 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.627976 24047 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.716403 24072 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:25:46.743337 24166 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:46.743927 24080 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.744513 23500 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:25:46.713292 13648 rpcz_store.cc:276] 0709 12:25:43.030121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.036624 (+  6503us) service_pool.cc:224] Handling call
0709 12:25:45.637232 (+2600608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.651764 (+ 14532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.686850 (+1035086us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b54003186a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf7a02 00007f5393bf3dd6 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:25:46.688394 (+  1544us) spinlock_profiling.cc:243] Waited 2.58 s on lock 0x7b5400198550. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:46.688726 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5946102784}
I20260709 12:25:46.749145 13644 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
I20260709 12:25:46.750236 13794 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:46.813616 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5119) took 2005 ms. Trace:
I20260709 12:25:46.750597 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3210) took 1904 ms. Trace:
I20260709 12:25:46.814827 13651 rpcz_store.cc:276] 0709 12:25:44.845902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.659469 (+1813567us) service_pool.cc:224] Handling call
0709 12:25:46.750527 (+ 91058us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.816632 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.817726 13651 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:46.750957 13652 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:46.819212 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3208) took 1991 ms. Trace:
I20260709 12:25:46.752076 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5120) took 1921 ms. Trace:
I20260709 12:25:46.760125 13724 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:46.760305 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:46.819943 13794 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:46.753284 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:46.822690 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69485.7us
W20260709 12:25:46.824542 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.767793 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:46.787573 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:46.751613 24081 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:46.713299 24171 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.801671 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85742.4us
W20260709 12:25:46.801602 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79768.5us
I20260709 12:25:46.801440 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:46.805846 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47294.2us
I20260709 12:25:46.765383 24167 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.812189 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.897560 13726 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:46.800400 24072 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:46.814085 13644 rpcz_store.cc:276] 0709 12:25:44.808197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.748584 (+1940387us) service_pool.cc:224] Handling call
0709 12:25:46.748906 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.813225 (+ 64319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.813540 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.820474 13645 rpcz_store.cc:276] 0709 12:25:44.808544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.728924 (+1920380us) service_pool.cc:224] Handling call
0709 12:25:46.729263 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.729457 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.729482 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.729525 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:46.729560 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.729614 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.729726 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.902283 13798 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:46.800549 24162 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:46.819723 13652 rpcz_store.cc:276] 0709 12:25:44.827998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.726196 (+1898198us) service_pool.cc:224] Handling call
0709 12:25:46.819108 (+ 92912us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.928256 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:46.931418 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 107564us
W20260709 12:25:46.914947 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 113780us
W20260709 12:25:46.920907 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133429us
I20260709 12:25:46.934461 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:46.935742 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:46.929111 13652 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:46.941798 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.942149 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.942879 13652 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:46.932578 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:46.949069 13644 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:46.799723 23888 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:46.958336 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.959820 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.926339 24163 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:46.939909 23831 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:46.806779 23972 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.120s
I20260709 12:25:46.977594 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.981652 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:46.930683 24072 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:46.982054 13644 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:46.978524 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:46.979702 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 211976us
I20260709 12:25:46.927016 24162 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:46.997017 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:47.006178 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:47.007035 13651 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:47.007643 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.008687 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.896952 24169 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.176s
I20260709 12:25:46.960848 24069 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:46.976151 24029 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:46.982882 23824 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:46.948304 24067 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:25:47.029786 24069 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:47.054579 13652 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:46.953668 23627 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 12:25:46.975389 24087 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.072s
I20260709 12:25:47.086959 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:47.021623 23934 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:46.919853 24171 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:46.931082 24167 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.093684 13652 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:47.104800 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:47.105809 13652 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
W20260709 12:25:47.106927 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 172586us
I20260709 12:25:47.035207 23824 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.096352 13792 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:47.111415 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:46.983103 24172 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.149s
W20260709 12:25:47.163552 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47598.4us
I20260709 12:25:47.102160 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:47.165191 13645 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:47.091032 24164 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.114319 13651 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:25:47.083609 24069 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:47.006742 24176 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.112s
I20260709 12:25:47.179783 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:47.119604 13652 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:47.192691 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:47.129416 23824 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:47.111442 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.211088 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:46.974530 24159 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 32.24, Last appended by leader: 24, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.240816 13644 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:25:47.242208 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.097085 24186 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:47.246052 13645 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:25:47.276189 14417 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:47.283672 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5125) took 2473 ms. Trace:
I20260709 12:25:47.284198 13645 rpcz_store.cc:276] 0709 12:25:44.810001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.188630 (+2378629us) service_pool.cc:224] Handling call
0709 12:25:47.245691 (+ 57061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.283294 (+ 37603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.283598 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.220893 13792 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:47.242089 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:47.272807 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 161496us
I20260709 12:25:47.243484 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.275362 13797 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:47.287654 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:47.286985 13801 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:47.280763 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3211) took 2434 ms. Trace:
I20260709 12:25:47.289727 13644 rpcz_store.cc:276] 0709 12:25:44.846331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.239981 (+2393650us) service_pool.cc:224] Handling call
0709 12:25:47.240448 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.272868 (+ 32420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.280684 (+  7816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.288112 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:47.291492 13794 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
I20260709 12:25:47.244581 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:47.117427 24182 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.131s
I20260709 12:25:47.291961 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:47.294449 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 308542us
I20260709 12:25:47.295523 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.250344 14419 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:47.300938 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:47.090904 24184 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.183s
I20260709 12:25:47.163594 24165 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.145s
I20260709 12:25:47.307077 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:47.189657 24066 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:47.312956 13799 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 23: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:47.311486 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.250878 24169 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:47.244876 24026 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:25:47.315858 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.316812 13645 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:47.228235 24161 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:47.275234 24157 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:47.321894 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:47.311170 24174 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.202s	user 0.004s	sys 0.000s
I20260709 12:25:47.188285 24164 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.292869 24047 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.307596 23831 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:47.334081 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 401621us
I20260709 12:25:47.395525 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:47.323055 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 326202us
I20260709 12:25:47.399413 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:47.400276 24087 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:47.404907 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:47.406597 24087 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:47.308797 24111 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:47.342149 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5128) took 2525 ms. Trace:
I20260709 12:25:47.163743 24162 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.409161 24162 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:47.409873 24107 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.060s	user 0.013s	sys 0.001s
W20260709 12:25:47.329516 24170 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.412s	user 0.000s	sys 0.002s
I20260709 12:25:47.322012 24169 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:47.308969 24191 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:47.344237 23618 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.063s
I20260709 12:25:47.091760 24081 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 37.25, Last appended by leader: 25, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.345005 24157 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:47.414724 24087 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.418473 24026 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:47.419989 24066 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:47.409893 23888 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.424294 23831 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.506610 23831 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:47.449821 24047 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:25:47.449872 24105 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.873s	user 0.012s	sys 0.002s
I20260709 12:25:47.336411 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3213) took 2489 ms. Trace:
I20260709 12:25:47.518417 13645 rpcz_store.cc:276] 0709 12:25:44.846976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.285779 (+2438803us) service_pool.cc:224] Handling call
0709 12:25:47.316537 (+ 30758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.335960 (+ 19423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.336317 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:47.455085 24064 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.090s	sys 0.171s
I20260709 12:25:47.488257 13647 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:25:47.427881 24185 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:47.435384 24184 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.428036 24193 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:47.499204 24066 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:47.426822 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:47.497641 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77639.5us
I20260709 12:25:47.454734 24176 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:25:47.521351 13649 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:25:47.462939 24087 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.520931 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5136) took 1866 ms. Trace:
W20260709 12:25:47.522351 24139 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.990s	user 0.015s	sys 0.000s
I20260709 12:25:47.480962 24065 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:47.450349 24107 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.534039 13645 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:47.549865 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5137) took 1837 ms. Trace:
I20260709 12:25:47.550235 13645 rpcz_store.cc:276] 0709 12:25:45.712661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.519645 (+1806984us) service_pool.cc:224] Handling call
0709 12:25:47.533252 (+ 13607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.549546 (+ 16294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.549802 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.534711 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5135) took 1893 ms. Trace:
I20260709 12:25:47.535912 13647 rpcz_store.cc:276] 0709 12:25:45.654999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.487366 (+1832367us) service_pool.cc:224] Handling call
0709 12:25:47.487917 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.520550 (+ 32633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.520840 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.425102 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:47.551503 13649 rpcz_store.cc:276] 0709 12:25:45.641050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.481332 (+1840282us) service_pool.cc:224] Handling call
0709 12:25:47.521014 (+ 39682us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.534387 (+ 13373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.534655 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.552408 13647 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:47.553262 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5139) took 1802 ms. Trace:
I20260709 12:25:47.553478 13796 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:47.460578 24026 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:47.523355 24124 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.016s	user 0.014s	sys 0.000s
W20260709 12:25:47.523828 24064 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.090s	sys 0.172s
W20260709 12:25:47.539237 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 374297us
W20260709 12:25:47.540799 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 114080us
I20260709 12:25:47.543989 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.522859 24171 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.567334 13793 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:47.553570 13647 rpcz_store.cc:276] 0709 12:25:45.751476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.551729 (+1800253us) service_pool.cc:224] Handling call
0709 12:25:47.552034 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.552954 (+   920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.553207 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.546828 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.556957 13800 raft_consensus.cc:606] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:47.366230 24161 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.560829 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.532711 24167 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.528833 24066 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:47.562778 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4578) took 1175 ms. Trace:
W20260709 12:25:47.573922 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 263942us
W20260709 12:25:47.540169 24050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.019s	sys 0.018s
I20260709 12:25:47.563563 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:47.565140 13797 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:47.565217 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:47.564652 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 269244us
I20260709 12:25:47.565174 13645 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:47.552214 23998 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:47.570791 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:47.605602 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3212) took 2759 ms. Trace:
I20260709 12:25:47.540688 24065 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:47.573729 13725 rpcz_store.cc:276] 0709 12:25:46.387392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.388889 (+  1497us) service_pool.cc:224] Handling call
0709 12:25:46.389220 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:46.441846 (+ 52626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.441893 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:46.454916 (+ 13023us) log.cc:844] Serialized 52 byte log entry
0709 12:25:46.592119 (+137203us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:46.592174 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.592288 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.562479 (+970191us) raft_consensus.cc:1685] finished
0709 12:25:47.562512 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.562697 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4223,"raft.queue_time_us":25142,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":16629,"thread_start_us":147101,"threads_started":2,"wal-append.queue_time_us":275213}
I20260709 12:25:47.574483 14418 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:25:47.622148 14418 pending_rounds.cc:85] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 15
I20260709 12:25:47.621865 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.622383 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5138) took 1902 ms. Trace:
I20260709 12:25:47.567579 24176 consensus_queue.cc:1243] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:25:47.542121 23944 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:25:47.632638 14419 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:47.633070 14419 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:47.572417 13649 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
W20260709 12:25:47.578107 24050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.019s	sys 0.018s
I20260709 12:25:47.622800 14418 pending_rounds.cc:107] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:25:47.633914 14419 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.602694 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:47.635279 14419 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:47.635425 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5141) took 1835 ms. Trace:
I20260709 12:25:47.635839 13649 rpcz_store.cc:276] 0709 12:25:45.799930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.553148 (+1753218us) service_pool.cc:224] Handling call
0709 12:25:47.572226 (+ 19078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.635141 (+ 62915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.635362 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.637710 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5132) took 2180 ms. Trace:
I20260709 12:25:47.623363 13645 rpcz_store.cc:276] 0709 12:25:45.720455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.551099 (+1830644us) service_pool.cc:224] Handling call
0709 12:25:47.564996 (+ 13897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.605227 (+ 40231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.622322 (+ 17095us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.613045 13650 rpcz_store.cc:276] 0709 12:25:44.846660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.247035 (+2400375us) service_pool.cc:224] Handling call
0709 12:25:47.307114 (+ 60079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.307345 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:25:47.332155 (+ 24810us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.605186 (+273031us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.605245 (+    59us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:47.605294 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.605373 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.605523 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24675,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"thread_start_us":269263,"threads_started":1}
I20260709 12:25:47.530759 24184 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.626888 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:47.541662 24139 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.530463 24185 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.642684 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.646466 24066 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.647475 24066 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:47.532212 24169 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.631405 13801 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:47.634667 14418 raft_consensus.cc:2889] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.638059 13649 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 34. Current term is 35. Ops: [34.25-34.25]
I20260709 12:25:47.638203 13644 rpcz_store.cc:276] 0709 12:25:45.457284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.297315 (+1840031us) service_pool.cc:224] Handling call
0709 12:25:47.297815 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.304829 (+  7014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.304878 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.309194 (+  4316us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.538529 (+229335us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:47.538582 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.538698 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.637417 (+ 98719us) raft_consensus.cc:1685] finished
0709 12:25:47.637442 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.637621 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6425,"raft.queue_time_us":6552,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":11665,"thread_start_us":228929,"threads_started":1,"wal-append.queue_time_us":324784}
I20260709 12:25:47.627709 24092 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:25:47.625504 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5142) took 1711 ms. Trace:
I20260709 12:25:47.641491 24176 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.680183 24092 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:47.667047 24198 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:47.681401 24092 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.669898 24184 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.561748 24157 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.671809 24069 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.668373 24161 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.674091 24139 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.685002 24069 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.671841 24185 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.676252 13645 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:47.662178 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1634) took 1504 ms. Trace:
I20260709 12:25:47.687914 24065 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 31.24, Last appended by leader: 24, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.688607 13649 rpcz_store.cc:276] 0709 12:25:46.157852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.637016 (+1479164us) service_pool.cc:224] Handling call
0709 12:25:47.637826 (+   810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.661758 (+ 23932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.662108 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.689713 24065 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:47.651598 23998 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:47.663318 13644 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 35 index: 25. (index mismatch)
I20260709 12:25:47.662616 24210 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:47.679618 24169 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:47.407459 13652 rpcz_store.cc:276] 0709 12:25:44.816822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.190638 (+2373816us) service_pool.cc:224] Handling call
0709 12:25:47.190994 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.191209 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:47.194889 (+  3680us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.341577 (+146688us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.341661 (+    84us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:47.341738 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.341841 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.342052 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3078,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":82,"raft.queue_time_us":195,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":50197,"thread_start_us":146189,"threads_started":1}
I20260709 12:25:47.675657 13650 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:25:47.718531 24214 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:47.719333 24214 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.642300 24206 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:25:47.724740 24173 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:47.725410 24173 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:47.728015 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5110) took 4199 ms. Trace:
I20260709 12:25:47.732725 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4712) took 1453 ms. Trace:
W20260709 12:25:47.733781 23824 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.005s
I20260709 12:25:47.734794 24126 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:47.735499 24126 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:47.680236 24176 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:47.658982 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.676939 13647 rpcz_store.cc:276] 0709 12:25:45.913838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.574920 (+1661082us) service_pool.cc:224] Handling call
0709 12:25:47.603046 (+ 28126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.625170 (+ 22124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.625192 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.625231 (+    39us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:47.625254 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.625301 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.625409 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.692564 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.639675 24164 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.743733 24164 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:25:47.659993 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57380.8us
I20260709 12:25:47.748344 13644 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:47.749873 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:47.713456 23998 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.754007 13647 raft_consensus.cc:2376] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22. Re-sending same reply.
I20260709 12:25:47.757191 13798 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:47.759814 13796 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:47.728498 13646 rpcz_store.cc:276] 0709 12:25:43.528431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.466644 (+938213us) service_pool.cc:224] Handling call
0709 12:25:44.466991 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:44.482634 (+ 15643us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.482668 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:44.487609 (+  4941us) log.cc:844] Serialized 26 byte log entry
0709 12:25:44.695161 (+207552us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:44.718613 (+ 23452us) log.cc:844] Serialized 14 byte log entry
0709 12:25:44.733214 (+ 14601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.733328 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.443299 (+1709971us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b18004ac500. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1
0709 12:25:47.720116 (+1276817us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b18004ac500. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:25:47.727723 (+  7607us) raft_consensus.cc:1685] finished
0709 12:25:47.727758 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.727947 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34410,"prepare.queue_time_us":24075,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":2917,"raft.queue_time_us":18698,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":13083,"spinlock_wait_cycles":78398080,"thread_start_us":156149,"threads_started":1,"wal-append.queue_time_us":233034}
I20260709 12:25:47.733248 13793 rpcz_store.cc:276] 0709 12:25:46.279054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.749546 (+470492us) service_pool.cc:224] Handling call
0709 12:25:46.749862 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.567783 (+817921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.732407 (+164624us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b540006ac54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:47.732646 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1879531264}
I20260709 12:25:47.776448 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:47.777627 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85189.5us
I20260709 12:25:47.778735 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:47.779106 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:47.762581 24163 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:47.777220 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:47.781553 24163 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:47.782533 23970 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.782696 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:47.783313 24214 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.782899 24163 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.785833 24163 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:47.785051 23987 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:25:47.786412 23987 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:47.786712 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1629) took 1631 ms. Trace:
I20260709 12:25:47.787508 24182 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.787314 23987 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.787443 13643 rpcz_store.cc:276] 0709 12:25:46.155739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.487191 (+331452us) service_pool.cc:224] Handling call
0709 12:25:47.786512 (+1299321us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:47.786650 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633253760}
I20260709 12:25:47.788563 23987 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:47.788432 23831 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.789331 24210 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:25:47.793807 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.709167 24080 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:25:47.796095 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.600s (SENT) [suppressed 42 similar messages]
W20260709 12:25:47.777064 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.765345 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:47.780748 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:47.801246 13650 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:47.804273 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:47.814463 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.815881 13723 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:47.823051 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:47.844468 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:47.845861 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.671667 24107 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.851753 14419 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:47.855999 13725 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
I20260709 12:25:47.855749 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:47.857775 13719 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:47.858942 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.859714 13721 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:47.797850 23957 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.868719 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:47.864809 13720 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:25:47.866590 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:47.870093 13721 raft_consensus.cc:2376] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28. Re-sending same reply.
I20260709 12:25:47.817011 23970 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:47.869925 13718 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:47.845834 24080 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:47.847311 23618 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.878284 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102095us
I20260709 12:25:47.848896 24214 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:47.878690 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.879581 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.788368 24182 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.885147 13721 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:47.878580 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.883617 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.886904 13719 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:47.892167 13721 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:47.877839 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:47.894024 13725 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:47.894672 13725 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:47.876353 23970 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.803673 23824 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.909271 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:47.910033 13720 raft_consensus.cc:2376] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26. Re-sending same reply.
I20260709 12:25:47.908070 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:47.911689 13719 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:25:47.893054 13799 raft_consensus.cc:606] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:47.912233 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.913604 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.877357 24080 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:47.876209 24171 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:25:47.932622 13718 raft_consensus.cc:2410] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 31 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 18.
I20260709 12:25:47.918866 13725 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.927139 23780 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:47.894627 24176 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.914073 23970 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:47.943059 13725 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:47.947249 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.943787 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:47.944036 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.915023 24195 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:47.950344 13718 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:25:47.914239 23831 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:47.952054 13719 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 33 because replica has last-logged OpId of term: 32 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 18.
I20260709 12:25:47.891927 23618 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:47.973824 24072 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.955s	user 0.007s	sys 0.006s
I20260709 12:25:47.974661 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:47.881753 24173 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:47.917860 24159 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 32.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.040s
I20260709 12:25:47.877476 23957 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:47.989087 23618 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:47.996042 23824 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:47.940994 24214 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 12:25:47.998602 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:48.007413 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.010334 24105 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:25:48.010972 13721 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:25:48.012269 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.014926 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.015841 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:48.016912 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.017554 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:48.017766 24108 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:48.018280 24108 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:48.018492 23957 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:47.930632 24080 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:48.019062 24182 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.019176 24108 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.020421 24108 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:47.982529 24072 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.013607 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:48.026221 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.014154 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.026935 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 230867us
I20260709 12:25:48.029561 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.032809 24072 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.033015 24173 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.033566 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:48.034214 13651 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:47.943500 24158 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:25:48.042851 13725 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 20. (term mismatch)
W20260709 12:25:48.042972 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.043391 13725 pending_rounds.cc:85] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 19
I20260709 12:25:48.043646 13725 pending_rounds.cc:107] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 26.20
I20260709 12:25:48.043962 13723 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:25:48.044140 13725 raft_consensus.cc:2889] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:48.032853 13646 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
W20260709 12:25:48.045025 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.045775 13644 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:25:48.043576 13718 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:48.049865 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:48.050722 13644 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:25:48.053210 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 148392us
I20260709 12:25:48.054745 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.055621 13725 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:48.057001 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.058434 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.061527 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.064808 24084 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:48.065943 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.066809 13720 raft_consensus.cc:2410] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 31 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 18.
I20260709 12:25:48.068370 24214 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:48.072880 24214 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:48.074348 24214 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.076788 24157 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.077381 24214 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.078456 23970 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:48.080974 24087 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.001s
I20260709 12:25:48.081705 24087 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.084010 24093 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:48.084167 24087 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:48.087155 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.089200 13648 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:25:48.093035 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.093876 13645 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:48.094856 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:48.095753 13647 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:48.072290 13651 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:25:48.096866 24126 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.209328 14417 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:25:48.059927 24210 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:48.123199 24105 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.082s
I20260709 12:25:48.066579 24159 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.222693 13719 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:48.223114 24159 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.226426 24182 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:25:48.228546 24126 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:48.054440 23987 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.059604 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:48.211884 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 139612us
I20260709 12:25:48.237705 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:48.238593 13797 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:25:48.239878 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.237663 13721 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:48.242163 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:25:48.177430 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.242889 14416 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:48.216478 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.207497 24047 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:48.246814 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.246984 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.244104 14417 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:25:48.248245 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.243372 13720 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:48.249073 13721 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.226073 24159 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:48.244987 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 185478us
I20260709 12:25:48.284974 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:48.289036 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.281427 13799 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:25:48.282681 13801 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 12:25:48.283486 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 269313us
I20260709 12:25:48.284097 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:48.284842 13794 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 34. Current term is 35. Ops: [34.25-34.25]
I20260709 12:25:48.285784 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.286437 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:48.290930 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.292656 13648 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 39.
I20260709 12:25:48.293854 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:48.295534 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.297402 24164 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:48.297964 24164 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.298496 24164 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:48.299803 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:25:48.305884 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.306630 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.286234 24047 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:48.308804 13617 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.682s (SENT)
I20260709 12:25:48.308697 24092 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:48.290962 24084 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.309818 24092 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:48.311374 24092 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:48.311751 24092 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:48.312671 24092 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.313620 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.308188 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.313961 24092 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:48.314738 24047 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.319355 24159 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:48.315423 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:48.296777 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.315188 14391 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:48.309520 24212 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:25:48.345192 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.316031 24029 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:25:48.309901 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.764s (SENT)
I20260709 12:25:48.346832 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.290930 13800 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
W20260709 12:25:48.305955 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 61026.2us
I20260709 12:25:48.318799 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.349757 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.321627 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2142) took 1573 ms. Trace:
W20260709 12:25:48.316402 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.315285 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.349064 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.350463 13726 rpcz_store.cc:276] 0709 12:25:46.748154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.748272 (+   118us) service_pool.cc:224] Handling call
0709 12:25:46.776210 (+ 27938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.911678 (+135468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.321274 (+1409596us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54002f6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:48.321540 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278544768}
I20260709 12:25:48.293871 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:48.358220 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48274.8us
I20260709 12:25:48.358669 13720 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.359393 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:48.360097 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:48.360332 13652 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:48.359118 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51015us
I20260709 12:25:48.361372 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:48.362161 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 47254.6us
I20260709 12:25:48.363317 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:48.365195 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:48.367614 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:48.368531 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.278445 24105 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.372781 13722 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 22: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.346575 24029 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:48.377363 24159 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:25:48.377763 24228 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 12:25:48.293409 24210 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:48.402968 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.404361 13726 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.377897 23831 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.380728 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.378994 13645 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:48.245461 23987 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.386760 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.374323 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:48.390600 13792 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:48.391134 13652 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:48.395360 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:48.409526 13651 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:48.396293 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.291115 24072 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.310187 24084 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.416659 23618 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:25:48.417517 24191 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:25:48.371657 23970 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.401300 23959 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:25:48.418354 24175 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:48.411947 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:48.420272 13648 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5149) took 1690 ms (client timeout 1593 ms). Trace:
I20260709 12:25:48.420718 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.421366 13794 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:48.421468 13792 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:25:48.385949 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.411552 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.423287 23957 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:25:48.424451 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:48.424787 24203 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:48.426613 24175 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:25:48.420722 13648 rpcz_store.cc:269] 0709 12:25:46.730143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.811665 (+ 81522us) service_pool.cc:224] Handling call
0709 12:25:48.420112 (+1608447us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400112954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:48.420205 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2708355968}
I20260709 12:25:48.429507 24191 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:48.389122 24214 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.431095 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:48.431648 24159 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:48.433292 23959 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:25:48.423718 13721 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:25:48.434031 13651 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:48.415899 24072 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.434521 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.434928 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.435657 13721 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:48.436410 24072 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:48.436575 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.399010 13727 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:48.385813 24192 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:48.438717 24192 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:48.439826 23780 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:48.439944 24192 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.445587 24108 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:48.446349 23500 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:48.447160 23500 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:48.442888 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.403200 24047 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:48.449872 23785 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:48.450479 24047 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:48.451614 24047 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.461133 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.371335 24212 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:25:48.465657 13721 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:48.385061 24182 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.467679 24182 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.382490 24206 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:48.470376 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:48.386914 24169 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:25:48.441757 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51286.4us
I20260709 12:25:48.430698 24214 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.433046 24159 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.448501 23500 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.413636 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.414659 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:48.480387 13792 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:48.453871 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:48.491302 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.498101 13726 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.401547 24029 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.512409 24182 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:48.513067 24214 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.522051 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:48.506664 13727 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:48.524770 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.530649 13727 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:48.525691 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.528687 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.529628 24175 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:48.538614 13720 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:25:48.540159 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.554939 13722 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:48.552713 13797 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
W20260709 12:25:48.553212 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65012us
I20260709 12:25:48.508949 24169 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:48.473778 24210 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.516119 24206 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:25:48.563815 23959 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:48.551671 24040 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.569383 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.573921 13727 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:48.530406 23785 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:25:48.604403 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.588487 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:48.599741 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.565044 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.606835 13718 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:25:48.582602 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.605197 13727 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
W20260709 12:25:48.605151 24080 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.000s
W20260709 12:25:48.566541 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 183947us
W20260709 12:25:48.613401 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.590399 24173 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.553s	user 0.016s	sys 0.000s
I20260709 12:25:48.615056 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:48.617036 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.604313 23959 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.621704 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:48.584249 24163 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.005s
I20260709 12:25:48.622838 13727 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
W20260709 12:25:48.594635 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 227083us
I20260709 12:25:48.585522 23685 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.025s
I20260709 12:25:48.564132 13726 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:25:48.626569 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.600893 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21.
I20260709 12:25:48.630367 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:48.638880 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.639912 13718 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:48.640362 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.633162 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.591459 24168 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:25:48.647626 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.648689 13720 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 21.
I20260709 12:25:48.650007 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.650691 13727 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:48.652215 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:48.593379 23618 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.040s
I20260709 12:25:48.612371 24185 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:25:48.648882 24186 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.011s
I20260709 12:25:48.652979 24169 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.663646 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.664820 13648 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:48.665656 13726 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:48.668010 13647 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:48.706681 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:48.681013 23987 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:48.683501 24212 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.683964 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.719990 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.720304 13800 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:48.685055 24173 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.565315 23970 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:25:48.607903 24040 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.687227 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:48.689787 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.691066 13720 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:48.692786 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.619083 24084 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:48.627290 23959 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:48.725816 13724 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.714857 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:48.622192 24175 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.087s
I20260709 12:25:48.726534 13651 raft_consensus.cc:2376] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 39. Re-sending same reply.
I20260709 12:25:48.616467 24080 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.726796 13727 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:25:48.731070 13724 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: [29.16-29.16]
I20260709 12:25:48.731396 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:48.729804 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.732421 13727 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:48.728998 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:48.732932 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.724964 24212 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.735512 23618 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:48.736086 23618 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:48.736742 23618 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:48.736792 24212 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.738425 24084 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:48.738904 24084 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:48.739416 24084 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:48.740933 13725 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:48.741554 24080 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.708899 24163 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.034s
I20260709 12:25:48.743656 23780 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:48.744074 24158 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:48.744382 23780 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:48.744649 24158 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.736088 24040 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:48.745213 24158 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.727903 24173 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.748872 24173 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:48.749473 24080 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:48.749842 24210 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:48.752538 24164 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:48.755075 24231 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:48.755313 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:48.755648 24214 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:48.761106 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:48.761319 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:48.785769 13726 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.728308 23957 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.034s
I20260709 12:25:48.763793 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.805456 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.775671 13727 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:25:48.762025 23618 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.778662 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.726403 23987 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:48.721731 24185 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:48.783931 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55057.4us
W20260709 12:25:48.803002 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47782us
W20260709 12:25:48.754004 24157 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.000s
W20260709 12:25:48.808663 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.745683 23780 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:48.814919 13793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.017s	sys 0.034s
I20260709 12:25:48.816074 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1921) took 2442 ms. Trace:
I20260709 12:25:48.820592 13727 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:25:48.814579 24163 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:48.778025 24026 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.812960 24164 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.828419 13720 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:25:48.829275 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:48.839622 13793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.017s	sys 0.035s
I20260709 12:25:48.840126 14418 rpcz_store.cc:276] 0709 12:25:46.370175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.370951 (+   776us) service_pool.cc:224] Handling call
0709 12:25:46.422813 (+ 51862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.661285 (+1238472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.807954 (+1146669us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:48.812371 (+  4417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2634397952}
I20260709 12:25:48.810755 24210 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:48.813822 23685 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:48.807246 24214 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.843140 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:48.847714 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:48.849583 23957 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:48.841588 24173 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.882812 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:48.852516 24185 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.827172 23831 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:48.855985 24026 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:48.856158 23618 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:48.856485 24163 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:48.859004 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.834514 24212 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:48.832420 24126 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.865049 24093 raft_consensus.cc:993] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:25:48.849089 13726 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:48.873431 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:48.885417 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:48.897763 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:48.899874 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 347758us
I20260709 12:25:48.861822 24105 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:48.904579 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.904889 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.911680 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.914619 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:48.894815 23780 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:48.885754 24214 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:48.878334 23685 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:48.826211 24158 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.922385 14423 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:48.922612 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.913321 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:48.903751 24194 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:48.931234 24194 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.931430 23831 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:48.931913 24194 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:48.913157 24065 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:48.932124 23831 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:48.911815 24040 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:48.919180 24192 raft_consensus.cc:1081] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Signalling peer 4dafe08a4f404502822a0732fe570de4 to start an election
I20260709 12:25:48.877735 24080 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:25:48.959462 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:48.943585 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.933873 24186 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:48.909425 23800 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:25:48.916613 24182 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:48.919350 24157 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:48.933373 23685 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:48.958288 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4735) took 1233 ms. Trace:
I20260709 12:25:48.925601 23998 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:48.925202 24210 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:48.905169 24173 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.944883 24040 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:48.963404 24186 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:48.911111 24126 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:48.955442 24242 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:25:48.974143 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.977269 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:48.942148 23618 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:25:48.970184 13793 rpcz_store.cc:276] 0709 12:25:47.725313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.238964 (+513651us) service_pool.cc:224] Handling call
0709 12:25:48.239284 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:48.245347 (+  6063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.245386 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:48.250099 (+  4713us) log.cc:844] Serialized 52 byte log entry
0709 12:25:48.880950 (+630851us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:48.881018 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.881124 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.957972 (+ 76848us) raft_consensus.cc:1685] finished
0709 12:25:48.958000 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.958194 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24103,"raft.queue_time_us":1990,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":42232,"thread_start_us":583393,"threads_started":1,"wal-append.queue_time_us":657964}
I20260709 12:25:48.952240 24158 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.989076 24210 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:48.952991 24084 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.967217 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:49.041357 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75559.6us
I20260709 12:25:49.042519 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:48.987018 23970 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 12:25:48.986385 24206 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:25:49.017200 13796 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:48.968524 23800 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:48.959671 23957 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:48.986316 24040 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:48.990638 24186 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:49.030675 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5155) took 1283 ms. Trace:
W20260709 12:25:49.032838 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56751.4us
I20260709 12:25:48.954950 23831 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:48.985874 13722 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:48.966650 24065 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:25:49.052440 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:49.053856 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:49.055162 24238 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:49.031708 23959 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.104542 23800 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.105535 24084 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.105891 23959 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:48.986244 24163 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.18, Last appended by leader: 18, Current term: 35, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.063293 24210 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.070526 13796 raft_consensus.cc:606] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.112484 13796 raft_consensus.cc:614] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:49.073235 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:49.091123 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:49.090343 13650 rpcz_store.cc:276] 0709 12:25:47.747108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.752136 (+  5028us) service_pool.cc:224] Handling call
0709 12:25:47.797329 (+ 45193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.801851 (+  4522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.028988 (+1227137us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b54002e4ad0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.030588 (+  1600us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103085824}
I20260709 12:25:49.095460 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4308) took 1274 ms. Trace:
I20260709 12:25:49.031461 24236 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:49.090122 23500 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:49.108970 13648 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:25:49.114287 13723 rpcz_store.cc:276] 0709 12:25:47.821284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.821454 (+   170us) service_pool.cc:224] Handling call
0709 12:25:49.095071 (+1273617us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54002f6f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.095376 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452245248}
I20260709 12:25:49.120743 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:49.132262 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:49.057621 24194 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.134651 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:49.136686 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:49.137845 13723 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:49.140049 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:49.135172 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62116.4us
I20260709 12:25:49.142794 13796 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:49.142824 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.145661 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:49.146853 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:49.148017 13797 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:49.166669 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.169471 13797 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:49.173178 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:49.175454 23959 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:25:49.176340 23959 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:49.092362 24168 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:25:49.133225 24191 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:49.178496 13723 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:49.141875 23970 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:49.191982 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:49.137218 24084 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:49.194607 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 211412us
I20260709 12:25:49.174889 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:49.163596 24072 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:25:49.142112 24080 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.058s
I20260709 12:25:49.205852 24242 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:49.213804 24169 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:49.214690 24169 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:49.215418 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:49.226670 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 155927us
I20260709 12:25:49.191905 23959 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:25:49.202081 24084 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:49.215740 24169 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.216665 23780 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:49.219168 23820 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:49.211400 24191 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:49.243482 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4585) took 1194 ms. Trace:
I20260709 12:25:49.245208 13718 rpcz_store.cc:276] 0709 12:25:48.049147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.651784 (+602637us) service_pool.cc:224] Handling call
0709 12:25:48.715782 (+ 63998us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.721863 (+  6081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.721903 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.724815 (+  2912us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.967876 (+243061us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:48.967937 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.968023 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.243252 (+275229us) raft_consensus.cc:1685] finished
0709 12:25:49.243279 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.243419 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5629,"raft.queue_time_us":172,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":7345,"thread_start_us":225234,"threads_started":1,"wal-append.queue_time_us":392631}
I20260709 12:25:49.234422 24126 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.251721 13800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:25:49.252501 13800 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.254508 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:49.255375 13649 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:49.255729 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4345) took 1028 ms. Trace:
I20260709 12:25:49.238199 24084 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:49.260066 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.258435 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.262202 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4322) took 1296 ms. Trace:
I20260709 12:25:49.288015 13719 rpcz_store.cc:276] 0709 12:25:47.965899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.966061 (+   162us) service_pool.cc:224] Handling call
0709 12:25:49.261842 (+1295781us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54000f6254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.262113 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563161984}
I20260709 12:25:49.287981 13645 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
W20260709 12:25:49.290480 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86745.8us
I20260709 12:25:49.263020 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:49.263599 13646 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:49.265677 24235 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:49.257645 24236 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:49.254670 24191 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.267227 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:49.243278 23780 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.267733 24072 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.298777 24072 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.299367 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.269529 24173 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.271445 24214 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.272928 13721 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:25:49.302404 13721 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:25:49.302754 24214 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.303582 13721 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:25:49.278156 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.278944 13726 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 20. Current term is 21. Ops: []
I20260709 12:25:49.280395 24029 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.283788 24065 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:25:49.306214 13721 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:25:49.284528 24093 raft_consensus.cc:993] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:25:49.288574 13649 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:49.292959 13643 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:25:49.256177 13724 rpcz_store.cc:276] 0709 12:25:48.227355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.747632 (+520277us) service_pool.cc:224] Handling call
0709 12:25:48.747977 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.748407 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:48.748771 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.933696 (+184925us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.940962 (+  7266us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.945061 (+  4099us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.945229 (+   168us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:48.945461 (+   232us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.961735 (+ 16274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.961831 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.208889 (+247058us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:49.255440 (+ 46551us) raft_consensus.cc:1685] finished
0709 12:25:49.255466 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.255638 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20317,"prepare.queue_time_us":22065,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":2865,"raft.queue_time_us":27314,"raft.run_cpu_time_us":3589,"raft.run_wall_time_us":290940,"spinlock_wait_cycles":30226560,"thread_start_us":178963,"threads_started":1,"wal-append.queue_time_us":317737}
I20260709 12:25:49.309666 24236 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.311292 13724 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:49.310187 14421 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 34. Current term is 35. Ops: []
W20260709 12:25:49.256757 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52237.1us
I20260709 12:25:49.298065 13652 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: [24.20-24.20]
I20260709 12:25:49.313227 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:49.313874 13646 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:49.308135 24065 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:49.294206 24235 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:49.317015 24065 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.317317 24235 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.318202 14421 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 39 index: 26. (index mismatch)
I20260709 12:25:49.318606 24065 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:25:49.317071 14420 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:49.302404 23780 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.325673 13724 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:49.332491 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.333570 13652 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 39.
I20260709 12:25:49.336086 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.320724 23901 raft_consensus.cc:1081] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:25:49.312361 24214 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:49.344835 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:49.352795 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:49.361123 24163 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.004s
W20260709 12:25:49.375175 13617 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.968s (SENT)
I20260709 12:25:49.375630 13646 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:49.377852 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.368364 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.352465 24235 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:49.379591 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:49.403767 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4591) took 1112 ms. Trace:
I20260709 12:25:49.407305 13721 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:49.404537 13722 rpcz_store.cc:276] 0709 12:25:48.291669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.611743 (+320074us) service_pool.cc:224] Handling call
0709 12:25:49.403470 (+791727us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54003fe8d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.403663 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625419136}
I20260709 12:25:49.414757 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:49.419260 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:49.427304 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:49.428208 14419 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:49.427326 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:49.432062 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:49.434424 13797 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:49.347255 24259 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:25:49.441107 14391 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.444200 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:49.447176 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:49.447892 24169 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:49.448035 14416 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:25:49.431207 24262 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
W20260709 12:25:49.451783 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76648.7us
I20260709 12:25:49.453506 24169 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.453666 24158 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:49.454116 24169 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.457062 24169 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.457630 24169 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.458393 24169 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.459362 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1510) took 1220 ms. Trace:
I20260709 12:25:49.459820 14423 rpcz_store.cc:276] 0709 12:25:48.238861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.239308 (+   447us) service_pool.cc:224] Handling call
0709 12:25:48.285531 (+ 46223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.927269 (+641738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.459069 (+531800us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54006f72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:49.459298 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464025472}
I20260709 12:25:49.460542 24169 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.464537 23785 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:49.465070 23785 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:49.469714 13644 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 39 index: 26. (index mismatch)
I20260709 12:25:49.472754 24087 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:49.473256 24087 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:49.474120 24087 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.475600 24087 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:49.430801 24159 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.047s
I20260709 12:25:49.431192 24080 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:49.478313 24159 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:49.478948 24159 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:49.480317 24159 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.482758 24126 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:49.483532 24126 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.486341 23618 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:49.487641 23618 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:49.488128 23618 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:49.489238 23618 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.490700 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:49.514444 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 63946.4us
I20260709 12:25:49.372080 24236 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:49.422930 24026 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.376191 24260 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:49.378683 23685 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.446146 23957 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.050s
I20260709 12:25:49.496956 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4331) took 1448 ms. Trace:
I20260709 12:25:49.478768 24080 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.499404 13727 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:49.525866 13725 rpcz_store.cc:276] 0709 12:25:48.048330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.058107 (+  9777us) service_pool.cc:224] Handling call
0709 12:25:49.496656 (+1438549us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54000f6c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.496861 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550226176}
I20260709 12:25:49.358228 24214 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.507512 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.508070 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.517272 24168 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.547844 13644 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:25:49.515584 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:49.453506 24163 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:25:49.516994 13724 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:49.500993 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:49.482069 24159 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:49.522724 24050 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.441262 23642 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.108s
W20260709 12:25:49.552860 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51971.9us
I20260709 12:25:49.553109 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:49.550944 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:49.523013 24260 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.555387 24210 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:49.555758 23685 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:49.555810 23831 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:49.523705 24158 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:49.564235 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:49.464565 24040 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.552219 23618 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:49.575978 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:49.571393 24105 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:49.522538 23987 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.043s
I20260709 12:25:49.526952 24026 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.566115 24050 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.578440 24182 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:49.575616 24158 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:49.575457 24235 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:49.575773 23831 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:49.553562 24163 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.586184 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4347) took 1358 ms. Trace:
I20260709 12:25:49.585737 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
W20260709 12:25:49.631371 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.597883 14425 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 34 index: 25. Preceding OpId from leader: term: 35 index: 25. (term mismatch)
I20260709 12:25:49.636328 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.569361 24254 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:49.639671 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:49.616933 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4593) took 1205 ms. Trace:
I20260709 12:25:49.587987 24040 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:49.588709 24265 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
W20260709 12:25:49.646811 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 71838.9us
I20260709 12:25:49.635131 23831 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:49.729447 23998 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:49.731987 23998 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:49.567521 24260 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:49.732497 24175 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:49.733048 23998 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.733074 24175 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.734570 23987 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.735112 23987 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.735957 24175 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.737172 13645 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:49.737242 23987 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.610169 24084 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.643748 24163 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:25:49.655633 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.740538 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2155) took 1122 ms. Trace:
I20260709 12:25:49.622311 23957 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:25:49.644542 13719 rpcz_store.cc:276] 0709 12:25:48.411249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.289374 (+878125us) service_pool.cc:224] Handling call
0709 12:25:49.291390 (+  2016us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:49.292448 (+  1058us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.292485 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:49.312673 (+ 20188us) log.cc:844] Serialized 52 byte log entry
0709 12:25:49.498851 (+186178us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:49.527906 (+ 29055us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.528166 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.528251 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.616585 (+ 88334us) raft_consensus.cc:1685] finished
0709 12:25:49.616614 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.616840 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11391,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":1711,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11660,"spinlock_wait_cycles":25569536,"thread_start_us":149669,"threads_started":1,"wal-append.queue_time_us":282132}
W20260709 12:25:49.658600 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.621325 23780 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:49.659638 24186 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.000s
I20260709 12:25:49.660041 24262 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.660547 24214 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.753324 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:49.654948 24254 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:49.663759 14425 pending_rounds.cc:85] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 24
I20260709 12:25:49.665190 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2146) took 1235 ms. Trace:
I20260709 12:25:49.622084 23642 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:49.587936 24080 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.558967 24168 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.774960 14418 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:49.734200 23998 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:49.670624 24026 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:49.672605 24212 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:49.637063 24105 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.673317 24235 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:25:49.761119 24040 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.637233 24182 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:49.635095 13720 rpcz_store.cc:276] 0709 12:25:48.227897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.884770 (+656873us) service_pool.cc:224] Handling call
0709 12:25:48.914686 (+ 29916us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.915131 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.915172 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.915798 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.168545 (+252747us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:49.197678 (+ 29133us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.585532 (+387854us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b180008fc20. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:25:49.585701 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.585804 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.585849 (+    45us) raft_consensus.cc:1685] finished
0709 12:25:49.585971 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.586118 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7939,"prepare.queue_time_us":28784,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":4203,"raft.queue_time_us":47250,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":28714,"spinlock_wait_cycles":4750592,"thread_start_us":261964,"threads_started":2,"wal-append.queue_time_us":263040}
I20260709 12:25:49.676853 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.682897 24191 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:49.684268 24194 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:49.730136 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4590) took 1488 ms. Trace:
I20260709 12:25:49.730947 23785 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.742473 13644 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:25:49.855998 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.856838 13644 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:49.586388 24164 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:49.741209 13724 rpcz_store.cc:276] 0709 12:25:48.617723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.331470 (+713747us) service_pool.cc:224] Handling call
0709 12:25:49.337802 (+  6332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.550147 (+212345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.720983 (+170836us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:49.740475 (+ 19492us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411225344}
I20260709 12:25:49.747907 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:49.748754 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 407435us
I20260709 12:25:49.741555 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:49.761739 14425 pending_rounds.cc:107] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 34.25
I20260709 12:25:49.762287 13721 rpcz_store.cc:276] 0709 12:25:48.430124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.324063 (+893939us) service_pool.cc:224] Handling call
0709 12:25:49.406857 (+ 82794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.408030 (+  1173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.664813 (+256783us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b54003fe8d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.665091 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189339392}
I20260709 12:25:49.863118 14425 raft_consensus.cc:2889] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:49.863227 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 121758us
I20260709 12:25:49.865157 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:49.763494 13647 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:49.771314 13649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:54768
I20260709 12:25:49.866417 13649 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:49.866948 13649 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.771836 24093 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:25:49.772238 13643 pending_rounds.cc:85] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 12
I20260709 12:25:49.772801 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4737) took 1328 ms. Trace:
I20260709 12:25:49.875159 13643 pending_rounds.cc:107] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:25:49.875217 13799 rpcz_store.cc:276] 0709 12:25:48.444079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.753010 (+1308931us) service_pool.cc:224] Handling call
0709 12:25:49.772733 (+ 19723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.875692 13643 raft_consensus.cc:2889] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.751273 23957 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:49.773741 13650 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:49.764282 24186 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.774188 23780 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.773902 23642 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.764679 24262 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:49.778031 14420 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:25:49.773741 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.780352 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:49.818486 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 254657us
W20260709 12:25:49.786866 24256 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.538s	user 0.004s	sys 0.001s
I20260709 12:25:49.674103 24158 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.877012 24191 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:49.806854 24212 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:49.833086 24105 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:49.918246 14418 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:49.840988 13725 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 33: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:49.923234 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.923997 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:49.924904 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:49.849629 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.850033 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92215.4us
I20260709 12:25:49.914762 24214 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.926409 24214 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.926556 24254 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.889192 13649 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.929530 24214 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.933292 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.848233 13727 rpcz_store.cc:276] 0709 12:25:48.241812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.928071 (+686259us) service_pool.cc:224] Handling call
0709 12:25:49.499118 (+571047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.526570 (+ 27452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.703064 (+176494us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54000f6c50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:49.730070 (+ 27006us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1271250688}
I20260709 12:25:49.873400 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.951838 13725 raft_consensus.cc:2376] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30. Re-sending same reply.
I20260709 12:25:49.952406 13721 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:49.874204 13647 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:49.953079 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:49.953645 13725 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:49.954962 13647 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:49.853623 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:49.955915 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 102402us
I20260709 12:25:49.956162 13647 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:25:49.874380 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.862386 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:49.668844 24050 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:49.962006 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99761.6us
I20260709 12:25:49.957340 13724 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:49.965626 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.845465 24040 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.892904 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:49.891422 13719 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:49.892638 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:49.968587 13646 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:49.858950 24168 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:49.859187 24084 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.902484 23642 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:49.902984 13651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845"
dest_uuid: "4dafe08a4f404502822a0732fe570de4"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:32958
I20260709 12:25:49.982698 13651 raft_consensus.cc:606] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:49.983060 13651 raft_consensus.cc:614] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:49.907370 13650 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:49.833165 24026 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:25:49.845172 24188 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.049s
I20260709 12:25:49.909983 24158 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:49.915989 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:49.858004 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:49.878240 24235 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:50.035820 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.039295 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:50.039698 13727 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:49.939368 24126 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:49.939177 24212 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.039994 13650 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:49.940622 13649 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:49.875535 24160 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.913589 24262 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:49.948424 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.888453 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:49.973557 13725 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:49.974085 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:49.978829 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 148391us
I20260709 12:25:49.978619 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:49.980201 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87669.3us
I20260709 12:25:50.059034 13721 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:49.981463 13647 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:49.986025 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:49.988355 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:49.938014 24072 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:50.037537 13644 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:25:50.042400 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 202605us
I20260709 12:25:49.973084 24159 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:50.064035 24159 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:49.976292 24169 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:50.065001 24159 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.071995 24260 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:49.980995 24191 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.074834 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:50.076520 13722 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:50.090361 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.091169 13724 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:50.062569 24235 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:50.111049 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:50.113816 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:50.115151 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.113897 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:50.111474 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:50.116691 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.057837 24278 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:25:50.121680 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.121533 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:50.131386 13793 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:50.132658 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.133472 13797 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 35 index: 19. (index mismatch)
I20260709 12:25:50.133750 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:50.117772 13724 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:50.122337 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.126714 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:50.136272 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.137149 13724 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:50.130184 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.138144 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.080101 23642 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.154682 13721 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:50.156001 13646 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:50.158771 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:50.165740 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.166656 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:50.167493 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 181526us
W20260709 12:25:50.169914 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.174168 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.194032 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1924) took 2589 ms. Trace:
I20260709 12:25:50.168843 13795 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:50.141005 24212 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:50.173270 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4349) took 1944 ms. Trace:
I20260709 12:25:50.194199 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:50.194614 14425 rpcz_store.cc:276] 0709 12:25:47.604630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.630960 (+ 26330us) service_pool.cc:224] Handling call
0709 12:25:47.631243 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.875342 (+2244099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.174819 (+299477us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400516b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:50.193939 (+ 19120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4521615104}
I20260709 12:25:50.131072 24278 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:50.201155 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:50.201885 13649 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:50.108345 24262 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.083s
I20260709 12:25:50.204150 13726 rpcz_store.cc:276] 0709 12:25:48.228562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.904744 (+676182us) service_pool.cc:224] Handling call
0709 12:25:49.278719 (+373975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.305352 (+ 26633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.173035 (+867683us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54001a4850. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:50.173197 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859106048}
I20260709 12:25:50.129014 24026 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:50.226725 24278 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:50.226650 24026 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.160509 24072 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.235575 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.237635 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:50.238066 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.241515 13793 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:50.198189 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2157) took 1543 ms. Trace:
I20260709 12:25:50.208222 24188 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:50.209442 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.214018 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.214179 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.215103 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.221407 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:50.223227 13794 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:50.252426 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:50.253257 13796 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:25:50.108000 24191 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.231739 13651 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:25:50.253769 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:50.242733 13721 rpcz_store.cc:276] 0709 12:25:48.654709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.197000 (+1542291us) service_pool.cc:224] Handling call
0709 12:25:50.197451 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.197727 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.197771 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.197835 (+    64us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:50.197885 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.197960 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.198111 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.243777 14418 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:50.255892 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:50.226990 13801 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [22.11-23.12]
I20260709 12:25:50.236936 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:50.259428 24130 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:50.246281 24093 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 12:25:50.171800 23685 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.093s
I20260709 12:25:50.269291 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:50.275406 24040 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:50.199714 23970 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:50.276739 23970 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:50.277627 23970 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:50.282733 24029 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:50.283756 24029 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:50.284883 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45590.3us
I20260709 12:25:50.288080 13726 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:50.288573 23800 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:50.289080 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4607) took 1600 ms. Trace:
I20260709 12:25:50.141541 24253 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 12:25:50.289667 13726 rpcz_store.cc:276] 0709 12:25:48.688953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.218844 (+1529891us) service_pool.cc:224] Handling call
0709 12:25:50.287548 (+ 68704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.288705 (+  1157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.289001 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:50.317272 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.300508 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2160) took 1469 ms. Trace:
W20260709 12:25:50.319643 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 69791.7us
I20260709 12:25:50.319782 13725 rpcz_store.cc:276] 0709 12:25:48.831499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.299169 (+1467670us) service_pool.cc:224] Handling call
0709 12:25:50.299723 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.299995 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.300037 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.300106 (+    69us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:50.300156 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.300236 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.300403 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.320916 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:50.321727 13725 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:25:50.304318 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2156) took 1670 ms. Trace:
I20260709 12:25:50.322636 13722 rpcz_store.cc:276] 0709 12:25:48.634177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.430805 (+796628us) service_pool.cc:224] Handling call
0709 12:25:49.431190 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.084006 (+652816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.303896 (+219890us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400174654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:50.304207 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483204480}
I20260709 12:25:50.253655 13796 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:50.309381 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72538.1us
I20260709 12:25:50.157742 24171 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:50.312960 13718 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:25:50.331044 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4606) took 1642 ms. Trace:
I20260709 12:25:50.331553 13718 rpcz_store.cc:276] 0709 12:25:48.688672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.198423 (+1509751us) service_pool.cc:224] Handling call
0709 12:25:50.247624 (+ 49201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.330590 (+ 82966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.330953 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:50.333272 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.291188 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:50.335361 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:50.335932 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49634.3us
I20260709 12:25:50.345068 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:50.351438 13719 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:25:50.352568 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2162) took 1242 ms. Trace:
I20260709 12:25:50.353018 13719 rpcz_store.cc:276] 0709 12:25:49.110024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.350319 (+1240295us) service_pool.cc:224] Handling call
0709 12:25:50.350890 (+   571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.352151 (+  1261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.352467 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.355109 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:50.356220 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:50.359167 13651 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 35 index: 19. (index mismatch)
I20260709 12:25:50.362356 13651 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:25:50.366678 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.368276 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:50.368418 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:50.325037 13796 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.166659 24192 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 12:25:50.329632 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:50.398093 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53117.4us
I20260709 12:25:50.505050 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:50.401510 13726 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:50.506844 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4354) took 1084 ms. Trace:
I20260709 12:25:50.507351 13726 rpcz_store.cc:276] 0709 12:25:49.422404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.316629 (+894225us) service_pool.cc:224] Handling call
0709 12:25:50.401107 (+ 84478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.506452 (+105345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.506757 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.290091 23800 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:50.083443 24160 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.403337 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:50.248739 24288 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.340087 24169 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:50.449031 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 133297us
I20260709 12:25:50.167953 23642 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.285226 24029 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.197003 24285 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.243s
I20260709 12:25:50.465127 13800 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:50.467680 13792 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:50.546617 13800 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:50.469296 13718 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:25:50.476428 13801 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:50.402285 24050 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:25:50.431996 23998 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 12:25:50.526031 24159 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:50.414744 24242 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:25:50.430449 24283 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:25:50.413787 24291 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:50.511379 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 181908us
I20260709 12:25:50.572335 13799 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:25:50.550170 13800 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:50.574628 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.574954 13800 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:50.576166 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 382090us
I20260709 12:25:50.518426 13796 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:50.592082 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:50.291865 24072 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.577917 24291 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.595396 24291 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:50.596442 24072 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:50.597391 24072 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:50.319231 24188 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.531601 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:50.537078 24284 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.336s	user 0.000s	sys 0.001s
I20260709 12:25:50.536532 23957 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 34.24, Last appended by leader: 24, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:50.537789 24279 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.418s	user 0.001s	sys 0.000s
W20260709 12:25:50.538616 24282 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 12:25:50.539618 24287 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:25:50.395020 24235 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.396528 24171 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.360123 24047 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:25:50.548418 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4360) took 1124 ms. Trace:
I20260709 12:25:50.597952 24072 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.509686 24026 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:50.524204 24173 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:50.472923 24093 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.055s
I20260709 12:25:50.517947 23800 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:50.536590 24281 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.395s	user 0.001s	sys 0.001s
W20260709 12:25:50.481979 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 161205us
I20260709 12:25:50.529836 24260 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:50.514726 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 146448us
I20260709 12:25:50.565541 24050 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.690245 24262 consensus_queue.cc:1243] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 28.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:50.739660 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:50.547655 24130 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.573832 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:50.746948 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 173199us
I20260709 12:25:50.619848 13726 raft_consensus.cc:2376] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 21. Re-sending same reply.
I20260709 12:25:50.748900 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:50.749150 13796 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:50.749717 13726 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:50.571780 24242 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:50.544373 24214 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.067s
I20260709 12:25:50.568642 24185 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:25:50.652467 24290 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:25:50.658609 13718 rpcz_store.cc:276] 0709 12:25:49.424135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.332731 (+908596us) service_pool.cc:224] Handling call
0709 12:25:50.417809 (+ 85078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.548065 (+130256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.548298 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:50.646453 24254 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
W20260709 12:25:50.659902 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 121335us
I20260709 12:25:50.531112 24288 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:50.652482 23831 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.937s	user 0.014s	sys 0.000s
W20260709 12:25:50.729282 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.716521 13720 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:50.740407 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:50.747726 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:50.750510 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5176) took 1007 ms. Trace:
I20260709 12:25:50.780913 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:50.780264 13648 rpcz_store.cc:276] 0709 12:25:49.743195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.743432 (+   237us) service_pool.cc:224] Handling call
0709 12:25:49.743785 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.745734 (+  1949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.745766 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.761471 (+ 15705us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.958587 (+197116us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:49.958634 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.958718 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.750289 (+791571us) raft_consensus.cc:1685] finished
0709 12:25:50.750315 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.750447 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21622,"raft.queue_time_us":7302,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10395,"thread_start_us":196753,"threads_started":1,"wal-append.queue_time_us":438998}
W20260709 12:25:50.719364 24105 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.801s	user 0.007s	sys 0.004s
I20260709 12:25:50.649500 24192 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.087s
I20260709 12:25:50.709615 24173 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:50.711570 24126 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.024s	sys 0.006s
I20260709 12:25:50.493760 24286 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:50.756579 24157 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:50.780975 13726 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:25:50.719774 24212 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:50.783947 13718 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:50.802857 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5177) took 1058 ms. Trace:
I20260709 12:25:50.803130 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5174) took 1064 ms. Trace:
I20260709 12:25:50.758226 24026 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:25:50.829176 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.828488 13645 rpcz_store.cc:276] 0709 12:25:49.738985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.743332 (+  4347us) service_pool.cc:224] Handling call
0709 12:25:49.778211 (+ 34879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.778622 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.778656 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.779172 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.120388 (+341216us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:25:50.120434 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.120502 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.802877 (+682375us) raft_consensus.cc:1685] finished
0709 12:25:50.802902 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.803073 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4882,"raft.queue_time_us":625,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":20429,"thread_start_us":267808,"threads_started":1,"wal-append.queue_time_us":406718}
I20260709 12:25:50.811494 13795 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:50.831722 13652 rpcz_store.cc:276] 0709 12:25:49.744420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.744606 (+   186us) service_pool.cc:224] Handling call
0709 12:25:49.744928 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.751588 (+  6660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.751624 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.754314 (+  2690us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.885570 (+131256us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:49.885639 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.885733 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.802579 (+916846us) raft_consensus.cc:1685] finished
0709 12:25:50.802612 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.802783 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31801,"raft.queue_time_us":6395,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":25808,"thread_start_us":130839,"threads_started":1,"wal-append.queue_time_us":298674}
I20260709 12:25:50.749993 24050 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:50.756712 24191 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:50.778823 24040 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.800346 24253 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:50.757828 23998 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:25:50.805612 24262 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:50.824244 24286 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.824683 24157 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.760895 24285 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:25:50.804169 24214 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.846712 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 260523us
I20260709 12:25:50.828027 24026 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.841231 24301 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.807525 24173 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:50.834437 24293 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.754227 24130 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.859526 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:50.849442 24159 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:50.848340 24253 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:50.861901 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 333653us
I20260709 12:25:50.837419 24212 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.800773 23831 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.856406 13726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:25:50.858331 24157 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:50.887394 13726 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:50.887851 13726 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:50.804963 24105 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.883345 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:50.889493 14418 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:50.864281 24026 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:50.858703 24286 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:50.810629 23800 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.870703 23642 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:50.895047 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:50.908373 24294 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.183s	user 0.001s	sys 0.001s
I20260709 12:25:50.874883 24301 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.909058 23800 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:50.888571 24185 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:50.910480 24185 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:50.911707 23998 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:50.854242 24040 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.880062 24293 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.916564 24040 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:50.917881 24300 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:50.918566 24300 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.919021 24300 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:50.919329 24212 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:50.921360 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.921867 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.886032 24253 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:50.928874 13800 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:25:50.942989 13800 pending_rounds.cc:85] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 15
I20260709 12:25:50.943611 13800 pending_rounds.cc:107] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:25:50.944089 13800 raft_consensus.cc:2889] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:50.924618 24278 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.687s	user 0.003s	sys 0.007s
I20260709 12:25:50.873124 24214 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.910888 24301 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:50.916728 24293 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:50.924090 24309 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:50.940872 24305 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:50.976648 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.896929 24188 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:50.930197 24163 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.151s	user 0.013s	sys 0.001s
I20260709 12:25:50.848954 24242 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:50.944732 13726 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:50.984642 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 244325us
W20260709 12:25:50.984203 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.984350 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.005s	sys 0.009s
I20260709 12:25:51.001176 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:50.940630 24262 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:51.008605 13800 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:25:51.010679 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:51.012120 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:51.012430 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:51.012874 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:51.013808 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:51.015311 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.982007 24278 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:51.025424 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.006s	sys 0.009s
W20260709 12:25:51.034195 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 927092us
I20260709 12:25:51.035516 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:51.029066 13793 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:51.029996 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:51.028087 13792 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:25:50.979574 24072 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:50.981328 24171 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.040232 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:51.035605 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 254803us
I20260709 12:25:50.980742 24260 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.987977 24191 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.022826 24301 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.042872 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:51.055157 13726 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.056278 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.060746 14420 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:51.042410 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:51.028436 24163 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.075839 24214 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.077145 23831 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.039414 24188 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.077698 24163 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.035691 24047 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:51.078068 24188 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.080495 24188 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.044461 24254 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:51.055394 24242 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:51.053313 24260 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.088873 24260 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.053840 24304 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:25:51.057432 24191 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.090468 24171 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:51.043810 24326 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:51.043673 24321 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.092227 24171 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:25:51.092775 24191 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.022589 24291 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.094105 14421 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 39. Current term is 40. Ops: [39.26-39.26]
I20260709 12:25:51.075064 24105 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.096791 14416 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:25:51.097716 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4613) took 1239 ms. Trace:
W20260709 12:25:51.097911 24296 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 12:25:51.098770 24262 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.060287 24310 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:25:51.100096 24182 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.239s	user 0.003s	sys 0.011s
I20260709 12:25:51.025552 24305 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.121451 24182 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.122852 14423 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:51.123404 24305 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.125422 24182 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.134330 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3243) took 1138 ms. Trace:
W20260709 12:25:51.135754 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93480.7us
W20260709 12:25:51.104326 23970 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.002s
I20260709 12:25:51.108450 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4353) took 1686 ms. Trace:
I20260709 12:25:51.037250 24159 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.140322 24316 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:51.139365 13723 rpcz_store.cc:276] 0709 12:25:49.422031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.302959 (+880928us) service_pool.cc:224] Handling call
0709 12:25:50.322604 (+ 19645us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:50.742184 (+419580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.742227 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:50.743144 (+   917us) log.cc:844] Serialized 52 byte log entry
0709 12:25:50.975294 (+232150us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:50.975913 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.976003 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.108185 (+132182us) raft_consensus.cc:1685] finished
0709 12:25:51.108216 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.108377 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72844,"raft.queue_time_us":419333,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":11717,"thread_start_us":502803,"threads_started":2,"wal-append.queue_time_us":327733}
I20260709 12:25:51.140846 24316 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:51.141216 24316 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:51.141681 13723 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:51.061422 24301 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.145453 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4615) took 1285 ms. Trace:
I20260709 12:25:51.146067 13723 rpcz_store.cc:276] 0709 12:25:49.859938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.141125 (+1281187us) service_pool.cc:224] Handling call
0709 12:25:51.141504 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.142117 (+   613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.145387 (+  3270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.150451 23970 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.152256 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4618) took 1291 ms. Trace:
I20260709 12:25:51.152490 13644 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:51.152765 13727 rpcz_store.cc:276] 0709 12:25:49.860976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.074631 (+213655us) service_pool.cc:224] Handling call
0709 12:25:51.152033 (+1077402us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400027bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:51.152174 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663768960}
I20260709 12:25:51.092818 24321 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.112233 24050 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:51.108461 24158 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.070s	user 0.010s	sys 0.003s
I20260709 12:25:51.098124 13718 rpcz_store.cc:276] 0709 12:25:49.857942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.836014 (+978072us) service_pool.cc:224] Handling call
0709 12:25:50.851397 (+ 15383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:50.861082 (+  9685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.861128 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:50.862065 (+   937us) log.cc:844] Serialized 52 byte log entry
0709 12:25:50.987160 (+125095us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:50.987211 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.987286 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.097424 (+110138us) raft_consensus.cc:1685] finished
0709 12:25:51.097455 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.097645 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9052,"raft.queue_time_us":10436,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":9913,"thread_start_us":93149,"threads_started":1,"wal-append.queue_time_us":218227}
I20260709 12:25:51.094988 24291 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.134881 13647 rpcz_store.cc:276] 0709 12:25:49.995450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.060390 (+ 64940us) service_pool.cc:224] Handling call
0709 12:25:50.060740 (+   350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:50.127149 (+ 66409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.127189 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:50.160341 (+ 33152us) log.cc:844] Serialized 52 byte log entry
0709 12:25:50.531298 (+370957us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:50.531344 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.531417 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.097287 (+565870us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:51.133911 (+ 36624us) raft_consensus.cc:1685] finished
0709 12:25:51.133937 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.134213 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146868,"raft.queue_time_us":66789,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":50094,"spinlock_wait_cycles":6237568,"thread_start_us":282929,"threads_started":1,"wal-append.queue_time_us":440920}
I20260709 12:25:51.162590 13718 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:25:51.162573 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:51.163947 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 102022us
I20260709 12:25:51.166493 24318 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:51.166956 24318 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:51.167301 24318 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:51.168175 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:51.168934 13801 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:25:51.171160 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:51.171432 24321 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.172426 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:51.173136 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:51.173231 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.174052 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:51.174690 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:51.174731 13793 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:51.175429 13724 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 27
W20260709 12:25:51.107748 24169 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.000s
I20260709 12:25:51.144075 24301 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.162768 24314 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:51.187150 13798 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:51.168072 24315 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:51.155620 13727 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:51.177415 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4622) took 1204 ms. Trace:
I20260709 12:25:51.195065 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.180781 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:51.191443 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:34464 (request call id 4614) took 1197 ms. Trace:
I20260709 12:25:51.196964 13726 rpcz_store.cc:276] 0709 12:25:49.858233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.856201 (+997968us) service_pool.cc:224] Handling call
0709 12:25:51.055644 (+199443us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.136921 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:51.194989 13718 rpcz_store.cc:276] 0709 12:25:49.958966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.158241 (+1199275us) service_pool.cc:224] Handling call
0709 12:25:51.162195 (+  3954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.163375 (+  1180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.163613 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.200722 13801 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:51.188192 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:51.207626 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4621) took 1323 ms. Trace:
I20260709 12:25:51.208137 13727 rpcz_store.cc:276] 0709 12:25:49.884382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.153796 (+1269414us) service_pool.cc:224] Handling call
0709 12:25:51.154121 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.207185 (+ 53064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.207554 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.209275 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.154362 23970 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:51.210965 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74186.5us
I20260709 12:25:51.202436 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:51.216499 24065 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:51.214141 24314 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.219715 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4373) took 1110 ms. Trace:
I20260709 12:25:51.220074 24278 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.215740 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:51.250077 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:51.228483 13767 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:51.216105 13726 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:25:51.230108 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:51.253631 13792 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:51.236814 13718 rpcz_store.cc:276] 0709 12:25:50.109382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.214489 (+1105107us) service_pool.cc:224] Handling call
0709 12:25:51.219173 (+  4684us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.219380 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.219405 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.219454 (+    49us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:51.219487 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.219540 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.219652 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:51.238458 14388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.012s	sys 0.009s
W20260709 12:25:51.255782 14388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.012s	sys 0.009s
I20260709 12:25:51.245931 13724 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:51.162573 23800 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:51.257977 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4620) took 1374 ms. Trace:
I20260709 12:25:51.187923 24169 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:25:51.259714 13727 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:51.263829 13724 rpcz_store.cc:276] 0709 12:25:49.883915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.168652 (+284737us) service_pool.cc:224] Handling call
0709 12:25:51.169875 (+1001223us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b5800155798. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bba0a0 00007f5398bb9984 00007f53921346d4 00007f5392134662
0709 12:25:51.257900 (+ 88025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21870336}
I20260709 12:25:51.268157 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4375) took 1158 ms. Trace:
I20260709 12:25:51.268555 13727 rpcz_store.cc:276] 0709 12:25:50.109914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.214589 (+1104675us) service_pool.cc:224] Handling call
0709 12:25:51.259351 (+ 44762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.267860 (+  8509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.268096 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.226605 24158 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:51.274238 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:51.247185 24291 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:25:51.258718 24203 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:51.264643 24242 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:51.225625 24316 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.287222 13725 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 12:25:51.267144 23901 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:51.266873 24326 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:25:51.276001 24333 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:25:51.273128 24158 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.278980 23970 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.014s
I20260709 12:25:51.294435 24188 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:51.270407 23800 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.284228 24242 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:51.290194 24182 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.094s
I20260709 12:25:51.286304 24169 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.291970 24203 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:51.290446 24108 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.277621 24065 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
I20260709 12:25:51.386905 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4356) took 1964 ms. Trace:
I20260709 12:25:51.291262 24291 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:51.303311 24285 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.279318 24278 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:51.394531 24186 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.388617 23998 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:25:51.382957 23820 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.385706 24158 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:25:51.385166 24173 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.505s	user 0.016s	sys 0.032s
I20260709 12:25:51.388823 24168 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:51.277165 24093 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.114s
I20260709 12:25:51.397281 24169 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.412811 24168 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:51.418031 24168 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.419225 24168 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:51.401281 13722 rpcz_store.cc:276] 0709 12:25:49.422971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.323822 (+900851us) service_pool.cc:224] Handling call
0709 12:25:50.400099 (+ 76277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.665522 (+265423us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:50.851855 (+186333us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.189584 (+337729us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.190957 (+  1373us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.191202 (+   245us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:51.191242 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.191325 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.386660 (+195335us) raft_consensus.cc:1685] finished
0709 12:25:51.386684 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.386833 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16205,"prepare.queue_time_us":225221,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4036,"raft.queue_time_us":418610,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":32007,"thread_start_us":712273,"threads_started":4,"wal-append.queue_time_us":191934}
I20260709 12:25:51.393308 24242 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:51.316888 24300 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.388589 24157 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.409279 24050 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.324326 24286 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:51.399341 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.391983 24026 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:51.423138 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.407038 24326 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:51.427497 23800 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.393764 24253 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.435406 13722 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 35 index: 19. (index mismatch)
I20260709 12:25:51.392163 24188 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.443261 24169 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.441242 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2167) took 1031 ms. Trace:
I20260709 12:25:51.443993 24188 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.444355 24286 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.444747 13796 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:51.445675 23785 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:51.403100 24108 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.413024 23820 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.414391 24173 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.448066 24130 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.436251 24326 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:51.407428 24285 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.425163 24278 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:51.455317 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4367) took 2029 ms (client timeout 1943 ms). Trace:
I20260709 12:25:51.459626 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1622) took 1563 ms. Trace:
I20260709 12:25:51.490100 13650 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:51.479640 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.490319 14423 rpcz_store.cc:276] 0709 12:25:49.896402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.896577 (+   175us) service_pool.cc:224] Handling call
0709 12:25:49.914953 (+ 18376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.123386 (+1208433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.459327 (+335941us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:51.459554 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2777364352}
I20260709 12:25:51.458799 13722 rpcz_store.cc:276] 0709 12:25:50.409978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.423079 (+1013101us) service_pool.cc:224] Handling call
0709 12:25:51.433409 (+ 10330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.440917 (+  7508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.441175 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:51.470400 13649 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:25:51.471904 13647 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 17. (term mismatch)
I20260709 12:25:51.492810 13647 pending_rounds.cc:85] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 16
I20260709 12:25:51.493217 13647 pending_rounds.cc:107] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:25:51.493598 13647 raft_consensus.cc:2889] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.478366 13644 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:25:51.488065 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:51.490150 13720 rpcz_store.cc:269] 0709 12:25:49.426622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.880728 (+454106us) service_pool.cc:224] Handling call
0709 12:25:51.455099 (+1574371us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:51.455231 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563346176}
I20260709 12:25:51.465518 24108 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.502935 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.505556 13649 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:25:51.510747 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:51.511423 13649 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:51.456525 24026 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.455447 24316 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.456429 24300 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.477006 24291 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 31.24, Last appended by leader: 24, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.516079 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:51.518596 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.523841 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.487715 24169 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.503947 13647 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:25:51.456455 24160 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.526957 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.465307 24305 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:25:51.533598 13649 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
W20260709 12:25:51.528748 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 240114us
I20260709 12:25:51.534600 13645 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:51.497834 23970 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.509670 24314 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.450585 24203 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.534725 24169 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.535612 24093 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:51.519626 24278 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.542907 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.563339 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.587344 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4754) took 1263 ms. Trace:
I20260709 12:25:51.589521 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:51.538770 24286 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.042s
I20260709 12:25:51.570178 13643 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:51.593253 14425 raft_consensus.cc:2393] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:51.570729 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.600010 14417 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 21.
I20260709 12:25:51.574723 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:51.575035 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.600855 14424 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:51.559450 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.547518 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:51.587826 13795 rpcz_store.cc:276] 0709 12:25:50.324198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.326724 (+  2526us) service_pool.cc:224] Handling call
0709 12:25:50.327206 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.831713 (+504507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.586984 (+755271us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:51.587250 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113104128}
W20260709 12:25:51.607378 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59952.3us
I20260709 12:25:51.529762 24258 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.539847 24108 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.070s
I20260709 12:25:51.575778 24173 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.542797 23970 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.581243 23820 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.608129 23998 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:25:51.565884 24169 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.625105 24160 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.624858 24315 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:51.609961 24186 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:25:51.614748 24026 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:51.638738 14417 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:25:51.548172 24111 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 23, Time since last communication: 0.071s
I20260709 12:25:51.555523 24188 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:25:51.626788 24258 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.591868 24326 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.648139 24278 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.649127 24285 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.643273 24186 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:51.650239 24311 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.702s	user 0.001s	sys 0.001s
W20260709 12:25:51.663293 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.654615 23998 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:51.633930 24130 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.029s
I20260709 12:25:51.659539 24182 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.676638 24188 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:51.643385 24160 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.668926 24026 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:51.674829 24253 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.680127 24158 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.669301 24278 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:25:51.707386 24326 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:51.690743 24188 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:51.695748 24182 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.728085 24160 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.671661 24318 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.732292 24318 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:51.733136 24188 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.733428 24242 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.734302 24188 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:51.735773 24182 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.735957 24242 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:51.782559 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.009s	sys 0.010s
W20260709 12:25:51.783761 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.009s	sys 0.011s
I20260709 12:25:51.789503 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5198) took 1547 ms. Trace:
I20260709 12:25:51.790390 13651 rpcz_store.cc:276] 0709 12:25:50.241667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.521080 (+279413us) service_pool.cc:224] Handling call
0709 12:25:50.521794 (+   714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.549194 (+ 27400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.549234 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.549823 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.809210 (+259387us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:25:50.809259 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.809340 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.649619 (+840279us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:51.788672 (+139053us) raft_consensus.cc:1685] finished
0709 12:25:51.788734 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.789239 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3192,"raft.queue_time_us":26923,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":16641,"spinlock_wait_cycles":114207616,"thread_start_us":121833,"threads_started":1,"wal-append.queue_time_us":235592}
I20260709 12:25:51.804919 14422 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:51.807834 24026 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.811609 23998 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.817517 24158 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.828413 24100 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:51.829285 24100 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:51.836393 14417 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:25:51.838341 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:51.840056 14425 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:25:51.860275 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3244) took 1739 ms. Trace:
I20260709 12:25:51.861191 23987 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:25:51.862964 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 238439us
I20260709 12:25:51.881326 24242 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.861917 13799 rpcz_store.cc:276] 0709 12:25:50.120513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.123052 (+  2539us) service_pool.cc:224] Handling call
0709 12:25:50.123463 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.572939 (+449476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.859341 (+1286402us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b5400288cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:51.859858 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1028408704}
I20260709 12:25:51.865700 24100 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.863859 23987 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:51.897212 24100 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:51.897541 24186 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.901288 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.900074 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:51.907599 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.11505e+06us
I20260709 12:25:51.915630 23987 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.917485 23987 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:51.923985 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:51.925179 13646 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:51.926640 13644 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
W20260709 12:25:51.931836 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.932569 24297 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:51.933257 24297 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:51.934751 24169 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.935734 24169 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.937642 24288 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:51.938915 24173 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.908483 24326 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.929109 24186 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:51.944152 24253 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:25:51.943933 24173 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.930652 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:51.948470 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 40 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:51.950670 23618 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:25:51.953315 24332 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.543s	user 0.001s	sys 0.000s
I20260709 12:25:51.955955 24326 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:51.965462 24344 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:51.966217 24344 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:51.965791 23785 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:51.963114 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.985767 24344 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:51.987881 24344 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:51.990829 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:51.993120 13650 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:51.995769 24326 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:51.982961 24169 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.000299 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4630) took 1590 ms. Trace:
I20260709 12:25:52.000972 24173 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.006703 13646 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:51.960721 24253 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.044s
I20260709 12:25:52.000849 13727 rpcz_store.cc:276] 0709 12:25:50.409293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.269451 (+860158us) service_pool.cc:224] Handling call
0709 12:25:51.279883 (+ 10432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.280104 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:51.280384 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.984829 (+704445us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.984945 (+   116us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:51.999635 (+ 14690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.999843 (+   208us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.000029 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":444800,"prepare.queue_time_us":141,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"thread_start_us":258836,"threads_started":2,"wal-append.queue_time_us":252662}
I20260709 12:25:52.005209 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:52.015823 13644 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:25:52.017637 23618 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 12:25:52.021695 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:52.022740 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68376us
I20260709 12:25:52.024719 23831 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.025530 23831 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.026473 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:52.026731 24047 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.026932 24072 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:52.027525 24316 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:25:52.032181 24191 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.983783 24242 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.041764 24191 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.042884 13645 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:52.045581 24191 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.081683 23785 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:52.030498 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:52.031409 24235 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.032081 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.032316 24260 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.040752 24316 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.024976 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4617) took 2164 ms. Trace:
I20260709 12:25:52.027709 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:52.111732 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108617us
I20260709 12:25:52.082755 23957 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:52.028137 24173 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.079530 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:52.046969 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.067816 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.068627 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4608) took 3328 ms. Trace:
I20260709 12:25:52.045581 24242 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:52.040834 24047 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.114398 13651 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
W20260709 12:25:52.061859 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3248) took 1694 ms (client timeout 1618 ms). Trace:
I20260709 12:25:52.081861 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:52.097385 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1938) took 2347 ms. Trace:
I20260709 12:25:52.088424 23831 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:52.050649 13617 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.943s (SENT)
I20260709 12:25:52.111673 24316 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:52.111974 13723 rpcz_store.cc:276] 0709 12:25:49.860662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.147288 (+1286626us) service_pool.cc:224] Handling call
0709 12:25:51.147672 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.021663 (+873991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.021715 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:52.021793 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:52.021829 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.021941 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.022245 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":817895,"raft.queue_time_us":875912,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":37342,"threads_started":1}
I20260709 12:25:52.127288 24173 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.130761 23831 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:52.130614 24235 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.133180 13649 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:52.126693 13650 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 22.
W20260709 12:25:52.116798 13798 rpcz_store.cc:269] 0709 12:25:50.367750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.367957 (+   207us) service_pool.cc:224] Handling call
0709 12:25:52.061208 (+1693251us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54002891d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:52.061548 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1804114304}
I20260709 12:25:52.084940 24297 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 33.25, Last appended by leader: 25, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.122824 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:52.140213 24297 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:52.140715 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:52.124845 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 94725.4us
I20260709 12:25:52.142150 24188 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:52.142711 24188 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:52.143214 24188 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:52.120720 24260 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.114825 13721 rpcz_store.cc:276] 0709 12:25:48.689195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.255952 (+1566757us) service_pool.cc:224] Handling call
0709 12:25:50.256416 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.256828 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:25:50.706681 (+449853us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.040605 (+333924us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.959485 (+918880us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b180035bde0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:25:51.959887 (+   402us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.960030 (+   143us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:51.983299 (+ 23269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.004741 (+ 21442us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.017579 (+ 12838us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":943673,"prepare.queue_time_us":617888,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":6515,"raft.queue_time_us":989372,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":9522,"spinlock_wait_cycles":31576448,"thread_start_us":651638,"threads_started":2,"wal-append.queue_time_us":402443}
I20260709 12:25:52.158701 24302 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:52.141189 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:52.159950 24302 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.162150 24340 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:52.163913 24340 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:52.164898 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4372) took 2691 ms. Trace:
I20260709 12:25:52.165167 24340 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.117057 14421 rpcz_store.cc:276] 0709 12:25:49.749924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.751606 (+  1682us) service_pool.cc:224] Handling call
0709 12:25:49.805568 (+ 53962us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.095527 (+1289959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.096591 (+1001064us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400584b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:52.097121 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2962726784}
I20260709 12:25:52.171840 24235 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.172930 24260 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:52.174598 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.14448e+06us
I20260709 12:25:52.175848 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:52.176939 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.177532 24047 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:52.178548 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53130.7us
I20260709 12:25:52.178715 24253 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:25:52.181143 23785 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:25:52.182403 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4628) took 1773 ms. Trace:
I20260709 12:25:52.218701 13718 rpcz_store.cc:276] 0709 12:25:50.408605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.254946 (+846341us) service_pool.cc:224] Handling call
0709 12:25:51.273103 (+ 18157us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:51.983190 (+710087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.002342 (+ 19152us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.018949 (+ 16607us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.114230 (+ 95281us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:52.181425 (+ 67195us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.181756 (+   331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.181892 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.181943 (+    51us) raft_consensus.cc:1685] finished
0709 12:25:52.181955 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.182175 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":687386,"prepare.queue_time_us":65668,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":78,"raft.queue_time_us":750981,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":213,"thread_start_us":94409,"threads_started":1,"wal-append.queue_time_us":86587}
W20260709 12:25:52.229489 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.231776 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 139204us
W20260709 12:25:52.227319 24353 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
W20260709 12:25:52.234685 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.183442 13719 rpcz_store.cc:276] 0709 12:25:49.473721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.370017 (+896296us) service_pool.cc:224] Handling call
0709 12:25:50.370613 (+   596us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.672098 (+301485us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:51.468626 (+796528us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b18003d1da0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:25:51.468784 (+   158us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.984351 (+515567us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.051617 (+ 67266us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.051947 (+   330us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:52.110391 (+ 58444us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.112880 (+  2489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.164135 (+ 51255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.164405 (+   270us) raft_consensus.cc:1685] finished
0709 12:25:52.164438 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.164647 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":229168,"prepare.queue_time_us":303940,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":126,"raft.queue_time_us":625793,"raft.run_cpu_time_us":544,"raft.run_wall_time_us":103574,"spinlock_wait_cycles":8833920,"thread_start_us":713655,"threads_started":5,"wal-append.queue_time_us":215075}
I20260709 12:25:52.250799 24340 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:52.256508 13643 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:52.266510 13651 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:52.270586 23785 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:52.270924 24050 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:52.271829 23785 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.272264 24260 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:52.241436 24242 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:25:52.271860 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 221312us
I20260709 12:25:52.273957 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:52.295185 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:52.273890 24173 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:52.303156 23785 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:52.303000 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:52.304767 14421 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 31 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:25:52.306591 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:52.307963 13651 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:52.308138 24192 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:52.308768 24192 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:52.308174 13644 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:52.310128 24188 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.309985 24192 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.316427 24188 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:52.317327 24192 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:52.296370 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.319314 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:52.340811 13650 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:25:52.344385 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49431.9us
W20260709 12:25:52.346271 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51817.2us
I20260709 12:25:52.348500 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:52.350435 24173 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:25:52.358507 24260 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.400401 24362 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.401376 24362 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.403683 24173 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:52.404012 24362 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.405491 24173 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.389909 24047 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:52.409945 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4357) took 2965 ms. Trace:
I20260709 12:25:52.416391 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4629) took 1879 ms. Trace:
I20260709 12:25:52.375046 24235 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:52.425784 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: true dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:52.367097 24188 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.049s
I20260709 12:25:52.423784 13725 rpcz_store.cc:276] 0709 12:25:49.423249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.326924 (+903675us) service_pool.cc:224] Handling call
0709 12:25:50.327524 (+   600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.289711 (+962187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.348361 (+1058650us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b5400173754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:52.388145 (+ 39784us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206233472}
I20260709 12:25:52.438567 13643 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:25:52.441243 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 167631us
I20260709 12:25:52.442147 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:25:52.443213 24029 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.443761 24029 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.445732 24288 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.445884 24366 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:52.446619 24288 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.446780 24366 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.449383 24288 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:52.450438 13646 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:52.455027 14421 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:52.434686 24047 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:52.424618 24368 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.449311 24366 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.425097 13724 rpcz_store.cc:276] 0709 12:25:50.408967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.268420 (+859453us) service_pool.cc:224] Handling call
0709 12:25:51.269020 (+   600us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.996551 (+727531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.996603 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.997593 (+   990us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.219240 (+221647us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:52.219698 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.226833 (+  7135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.232130 (+  5297us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.281694 (+ 49564us) raft_consensus.cc:1685] finished
0709 12:25:52.281714 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.288109 (+  6395us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":703682,"prepare.queue_time_us":378,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":761486,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":2345,"thread_start_us":237293,"threads_started":2,"wal-append.queue_time_us":248092}
I20260709 12:25:52.439049 23957 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:52.464322 14420 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1624) took 2323 ms (client timeout 1810 ms). Trace:
I20260709 12:25:52.464736 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.457285 24362 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:52.477033 13647 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:52.454625 23831 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:52.475657 14420 rpcz_store.cc:269] 0709 12:25:50.112653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.112829 (+   176us) service_pool.cc:224] Handling call
0709 12:25:52.415752 (+2302923us) spinlock_profiling.cc:243] Waited 884 ms on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:52.436179 (+ 20427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2033021824}
I20260709 12:25:52.485391 23785 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.456163 24326 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:25:52.493109 24235 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.483819 24368 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.497669 24366 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:25:52.521986 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:52.527971 24364 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:52.528688 24364 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.529616 24364 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:52.500247 23957 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:52.518536 24371 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.150s	user 0.000s	sys 0.002s
W20260709 12:25:52.539319 24372 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:25:52.504485 23785 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.549520 13798 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:52.550829 23785 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:52.551514 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.551961 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.552157 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:52.554632 24369 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:25:52.556064 24235 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.557420 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.506963 24362 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.557421 13724 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:25:52.560213 13643 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:52.564548 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:52.582762 24260 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.583626 24373 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:52.571723 24047 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.584646 24373 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:52.536027 24191 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:25:52.530364 23831 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.561602 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:52.599779 13650 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:52.609654 13651 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:52.613555 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:52.618762 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.621414 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.622946 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.622680 23831 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.559799 24316 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:52.638232 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.641427 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:52.648036 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.649986 24173 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:52.652002 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:52.652629 24173 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.655063 13718 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: [33.25-33.25]
W20260709 12:25:52.656242 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.646071 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.681214 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.684589 13650 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:25:52.687240 13792 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:52.693637 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:52.694998 24368 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.690198 13719 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
W20260709 12:25:52.695776 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.696462 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:52.700289 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:52.703747 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:52.704427 13724 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:52.705163 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.705700 13649 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:52.704375 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.707608 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.707226 13793 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:52.708819 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.710690 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:52.710454 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.718770 13798 raft_consensus.cc:2410] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 31 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:25:52.720525 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4627) took 2324 ms. Trace:
W20260709 12:25:52.723270 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.724372 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.724712 13793 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:52.727264 23998 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.727960 23998 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.733630 13724 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:52.736728 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.723394 13792 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [22.11-23.12]
I20260709 12:25:52.739961 23901 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.743405 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.721101 13726 rpcz_store.cc:276] 0709 12:25:50.395766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.252167 (+856401us) service_pool.cc:224] Handling call
0709 12:25:51.252681 (+   514us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:52.231707 (+979026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.234738 (+  3031us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.308704 (+ 73966us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.502329 (+193625us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:52.502367 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.513008 (+ 10641us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.720042 (+207034us) raft_consensus.cc:1685] finished
0709 12:25:52.720065 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.720340 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":713822,"raft.queue_time_us":771248,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"thread_start_us":416143,"threads_started":2,"wal-append.queue_time_us":324732}
I20260709 12:25:52.720409 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.745388 13799 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:25:52.747888 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.755865 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:52.757475 23780 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:52.757985 23780 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.758399 23780 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:52.760723 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.762696 24373 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:52.762524 24171 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:52.763518 24171 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:52.763865 24171 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:52.763981 24316 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:52.764863 24171 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.770926 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:52.755486 24362 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:52.756940 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44159.9us
I20260709 12:25:52.784122 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:52.799253 13718 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:52.785063 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:52.805053 13643 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:52.742628 23957 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:52.843019 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:52.844906 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61369.8us
W20260709 12:25:52.796818 24297 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.656s	user 0.018s	sys 0.000s
I20260709 12:25:52.776085 24171 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:52.851337 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.858534 13643 raft_consensus.cc:2410] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:25:52.859547 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:52.859707 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.860354 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:52.860502 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:52.861117 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:52.861747 24373 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader election lost for term 40. Reason: could not achieve majority
I20260709 12:25:52.740671 23901 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.861837 13643 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:52.866755 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:52.867918 13792 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:52.869336 13724 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:52.878333 24173 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:52.885640 13721 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:25:52.894027 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:52.905406 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:52.912245 14422 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 1955) took 1353 ms (client timeout 1641 ms). Trace:
W20260709 12:25:52.912885 14422 rpcz_store.cc:269] 0709 12:25:51.558919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.559074 (+   155us) service_pool.cc:224] Handling call
0709 12:25:52.907184 (+1348110us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400651dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:52.912137 (+  4953us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462246272}
I20260709 12:25:52.915737 13649 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:52.896072 24047 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:52.903281 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:52.927956 14418 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:52.928508 24157 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:52.933274 24352 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.055s	user 0.001s	sys 0.005s
I20260709 12:25:52.935074 14422 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:52.936765 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:52.898910 23957 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 12:25:52.959012 23957 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:52.959477 23957 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:52.959762 24344 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:25:52.960220 24344 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:52.961252 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46008.6us
I20260709 12:25:52.965750 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1669) took 1007 ms. Trace:
I20260709 12:25:52.967702 13794 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
W20260709 12:25:52.969796 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47939.7us
W20260709 12:25:52.976595 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81175.4us
I20260709 12:25:52.966358 13645 rpcz_store.cc:276] 0709 12:25:51.963026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.005743 (+ 42717us) service_pool.cc:224] Handling call
0709 12:25:52.965452 (+959709us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:52.965661 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45884800}
I20260709 12:25:52.991334 24047 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:52.993474 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:52.904832 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 188364us
I20260709 12:25:52.998567 24065 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:52.999110 24065 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:52.999234 13795 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 24. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:25:52.999935 24047 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.003085 24302 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:53.007108 14422 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:53.008435 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:53.009145 14420 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:52.960523 23957 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.015241 23957 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:53.021961 24379 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:25:53.022356 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.023885 24168 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:25:53.024875 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47243.6us
I20260709 12:25:53.025197 24316 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:52.921967 24192 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
I20260709 12:25:53.025816 24316 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.027889 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:53.028237 24316 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.030625 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:53.030702 24344 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.031947 24344 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:53.035080 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1660) took 1249 ms. Trace:
I20260709 12:25:53.035739 23970 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.035514 13652 rpcz_store.cc:276] 0709 12:25:51.785946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.793401 (+  7455us) service_pool.cc:224] Handling call
0709 12:25:51.794231 (+   830us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.795001 (+   770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.795034 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.796149 (+  1115us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.864296 (+ 68147us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:51.864351 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.864451 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.932311 (+1067860us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:53.034534 (+102223us) raft_consensus.cc:1685] finished
0709 12:25:53.034582 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.034869 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33347328,"thread_start_us":67475,"threads_started":1,"wal-append.queue_time_us":81745}
I20260709 12:25:53.038039 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:53.039026 13794 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:25:53.025871 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.042146 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.042315 24188 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:53.043205 24188 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.043267 13650 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:25:53.043735 24188 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:25:53.045949 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62977.4us
I20260709 12:25:52.999241 14419 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:25:53.051720 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.056794 13649 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:25:53.062774 13801 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:53.071587 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:53.074761 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.077883 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.082490 14420 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:25:52.985191 24297 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.093s
I20260709 12:25:53.092034 13794 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:25:53.085378 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:53.103348 13766 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.779s (SENT) [suppressed 13 similar messages]
W20260709 12:25:53.131422 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.132411 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.135061 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49809.2us
I20260709 12:25:53.076098 24316 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:53.085500 24373 raft_consensus.cc:2764] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election decision vote started in defunct term 35: won
I20260709 12:25:53.087236 24192 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:53.073562 24157 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.056s
I20260709 12:25:53.137600 24169 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:25:53.155133 13726 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:25:53.156878 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 126375us
I20260709 12:25:53.167364 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.140873 24316 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.139780 23970 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.022s
W20260709 12:25:53.182677 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.144938 23780 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.184303 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:53.167201 24169 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:53.194207 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:53.195343 14419 raft_consensus.cc:2410] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 34 because replica has last-logged OpId of term: 33 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 24.
I20260709 12:25:53.196617 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.196370 24065 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.198557 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.199450 14421 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:53.200768 23901 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:53.201865 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:53.203127 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:53.203732 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.204217 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.204922 14416 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:53.203150 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.206099 14424 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:53.207324 23901 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:53.207764 23901 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.208132 23901 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:53.208819 24029 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:53.209965 24364 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 12:25:53.214150 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1682) took 2047 ms. Trace:
I20260709 12:25:53.214412 24065 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.235502 13797 rpcz_store.cc:276] 0709 12:25:51.167090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.170571 (+  3481us) service_pool.cc:224] Handling call
0709 12:25:53.213907 (+2043336us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:53.214070 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2756512000}
I20260709 12:25:53.235513 24364 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.241187 24364 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:53.215461 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.179157 23785 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:25:53.242763 14422 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:53.243075 23785 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:53.243525 23785 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:53.244464 23785 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.245951 23785 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.217518 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.278690 14419 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:53.277493 13718 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:25:53.279660 24368 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.280364 24368 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.217661 24288 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:25:53.282761 24288 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:53.284017 24288 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.280381 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.285497 24288 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:53.221529 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:53.224893 14425 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:25:53.286449 24368 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:53.228638 14417 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:25:53.231391 23901 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.289549 23901 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:53.232275 23780 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:53.234644 24316 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:53.216053 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.255136 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.293463 13723 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:53.295059 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.296190 13723 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 32: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:53.257682 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.297448 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:53.259029 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.286528 13795 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:53.261600 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.299664 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.300308 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.301393 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.262774 13801 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:25:53.263090 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.302881 13798 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:25:53.264614 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.266537 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.267670 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.267987 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.167686 24242 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 12:25:53.270763 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.216239 24373 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:53.285668 13718 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:53.308125 24373 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.308738 24373 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:53.226883 24297 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:53.309725 24297 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.310254 13801 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 16
I20260709 12:25:53.310702 13801 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 25.17
W20260709 12:25:53.310997 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0431873550001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
I20260709 12:25:53.203765 13794 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:53.311062 13801 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:53.312450 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.314733 24368 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:53.327090 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.334939 24297 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.362056 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:53.362421 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.277493 24168 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:53.367937 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.368432 24168 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:53.369858 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 196939us
I20260709 12:25:53.351703 24316 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:53.350080 24364 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:25:53.370232 24065 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.390384 24065 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.393353 24065 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.394469 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.394869 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.395640 13725 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:53.396420 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.396809 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.397400 13719 raft_consensus.cc:2368] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31.
I20260709 12:25:53.389837 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.398715 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.398170 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.399906 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.406534 13795 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:53.416023 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.341753 24188 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.420063 13725 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 21.
I20260709 12:25:53.399477 13720 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:53.423863 13722 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:53.424738 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:53.427475 24258 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:53.269934 24169 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.24, Last appended by leader: 24, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:53.449259 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81777us
I20260709 12:25:53.434968 13718 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:53.451117 13723 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:53.435516 13794 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:25:53.423664 24242 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:53.455919 24242 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.456279 24242 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:53.456923 24188 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:53.457620 24368 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:53.460461 13797 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:53.470851 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:53.406049 24171 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
W20260709 12:25:53.500810 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 303992us
I20260709 12:25:53.498485 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:53.517903 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47433.2us
I20260709 12:25:53.470683 24373 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.538718 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.541527 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:53.424119 24344 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:53.494076 24258 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:53.554298 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.552326 24302 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:53.565033 24302 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.517797 23785 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:53.518877 24188 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:53.570848 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.005s	sys 0.012s
W20260709 12:25:53.577292 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.005s	sys 0.012s
I20260709 12:25:53.539027 24192 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:53.577080 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.555557 24344 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.504038 24364 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:53.577884 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 535765us
I20260709 12:25:53.561036 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.591387 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:53.592375 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48019.3us
I20260709 12:25:53.593739 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.594782 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:53.602711 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 296468us
I20260709 12:25:53.597105 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:53.583750 24373 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:53.583968 24344 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:53.581254 24395 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:25:53.623346 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82890.7us
I20260709 12:25:53.629240 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.629868 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.634038 13718 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:53.602896 23785 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:25:53.634919 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 261307us
I20260709 12:25:53.608937 24258 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:25:53.622565 24065 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.627061 24340 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:25:53.647910 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.630942 24157 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:53.602815 24188 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.662890 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.667778 14420 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:53.669919 23970 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:53.674554 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:53.679919 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50137.7us
I20260709 12:25:53.645061 24258 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:53.646559 24397 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:53.686883 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1626) took 1852 ms. Trace:
I20260709 12:25:53.687397 24169 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.690572 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:53.693146 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:53.712476 13643 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:53.673509 24242 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.696023 13648 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:25:53.695637 24065 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.609964 24364 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.697118 24302 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.690106 24188 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:53.698381 24157 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.697118 13644 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:25:53.708222 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:53.694746 13646 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
W20260709 12:25:53.714505 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.721604 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:53.710858 24258 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:53.736542 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 152983us
I20260709 12:25:53.743330 24395 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.743788 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.743913 24395 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.744658 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.745208 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.745431 13724 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:53.724206 24169 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.746526 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.725956 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:53.726511 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:53.727133 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:53.727567 13765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.031s	sys 0.045s
W20260709 12:25:53.748224 13765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.032s	sys 0.045s
W20260709 12:25:53.749926 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.650491 23785 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:53.732466 13648 pending_rounds.cc:85] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 12
I20260709 12:25:53.753789 13648 pending_rounds.cc:107] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:25:53.754163 13652 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 25 because replica has last-logged OpId of term: 23 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:25:53.754184 13648 raft_consensus.cc:2889] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:53.755522 13643 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:25:53.735651 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75535.3us
I20260709 12:25:53.711261 14418 rpcz_store.cc:276] 0709 12:25:51.835092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.835563 (+   471us) service_pool.cc:224] Handling call
0709 12:25:52.927333 (+1091770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.928869 (+  1536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.686605 (+757736us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400651dd0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:53.686751 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479742720}
W20260709 12:25:53.742865 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.747452 13723 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:53.637521 24192 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:53.762207 13649 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:25:53.762899 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.763880 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.764940 13724 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 24: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:53.706729 23970 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:53.769999 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62198.3us
I20260709 12:25:53.759078 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.761919 13651 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:53.778215 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 249701us
I20260709 12:25:53.779858 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:53.783327 24393 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.784170 24393 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:53.786458 24297 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:53.787616 24393 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.796312 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:53.797185 13646 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:53.806308 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:53.799772 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:53.829916 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:53.756978 24316 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.043s
I20260709 12:25:53.792150 24397 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:53.858407 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:53.853446 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:53.860833 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99793.1us
W20260709 12:25:53.858332 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 49872.1us
I20260709 12:25:53.860404 24340 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:53.793530 24373 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.068s
I20260709 12:25:53.828718 24395 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:53.912492 24397 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.858584 23901 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:25:53.853909 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:25:53.918293 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60018.6us
I20260709 12:25:53.808822 24326 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.103s
W20260709 12:25:53.923019 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52516.8us
I20260709 12:25:53.923740 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.927131 13647 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:53.859356 24414 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:53.928215 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 142723us
I20260709 12:25:53.930207 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:53.909734 24188 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:25:53.944880 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 198431us
W20260709 12:25:53.980049 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55587.4us
W20260709 12:25:53.981842 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58202.4us
I20260709 12:25:53.924396 23901 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:53.999189 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:53.926669 24393 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.929739 24397 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:54.019447 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:54.019802 24408 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:25:53.931967 24414 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:54.020542 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 370415us
I20260709 12:25:54.013039 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:54.025077 13724 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:25:54.025310 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.931653 24412 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:54.012065 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:53.995146 24395 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:25:54.002789 24188 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:54.039201 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 282020us
I20260709 12:25:54.032248 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:54.040122 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:54.032332 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:54.043471 13647 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:54.002321 24169 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:54.050668 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:54.017789 23901 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:54.023839 24316 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:25:54.052163 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.011933 24393 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.026623 24414 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:54.036736 24428 raft_consensus.cc:2764] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:25:54.068533 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.049224 24188 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:54.065482 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:54.072822 13794 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 36 because replica has last-logged OpId of term: 35 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 18.
I20260709 12:25:54.009682 24401 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.050s
I20260709 12:25:54.056391 24169 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.075958 13793 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:54.053752 23998 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:54.053534 24373 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:25:54.079859 13724 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:25:54.079042 24422 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:25:54.082229 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 152250us
W20260709 12:25:54.084251 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 65025.2us
I20260709 12:25:54.082489 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.058804 24242 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.086297 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.046726 23957 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:25:54.055274 24423 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:54.090507 13800 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:54.022691 23785 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.069144 24316 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:54.014830 23970 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.114454 13646 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:25:54.126300 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 159) took 44427 ms (client timeout 15000 ms). Trace:
I20260709 12:25:54.137584 13644 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 30 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:25:54.114454 24364 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:54.169464 24410 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 12:25:54.169884 24418 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:25:54.127065 24428 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:54.171350 24428 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.076690 24344 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.101912 24065 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:54.169727 24419 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:25:54.196128 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 130340us
W20260709 12:25:54.179453 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 147203us
I20260709 12:25:54.181087 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.063100 24340 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:54.178699 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.115618 24422 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:54.167119 24399 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:25:54.216787 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 150951us
W20260709 12:25:54.218907 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43957.9us
I20260709 12:25:54.218791 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:54.191056 24409 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.209s	user 0.001s	sys 0.001s
I20260709 12:25:54.176450 24157 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:54.234515 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58717.9us
I20260709 12:25:54.240177 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4632) took 1063 ms. Trace:
W20260709 12:25:54.242764 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 116268us
W20260709 12:25:54.213912 24416 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:25:54.128803 24169 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:54.249112 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50703.9us
I20260709 12:25:54.254442 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:54.249650 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.243055 13726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.004s	sys 0.031s
I20260709 12:25:54.245039 13721 rpcz_store.cc:276] 0709 12:25:53.185313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.190963 (+  5650us) service_pool.cc:224] Handling call
0709 12:25:53.202951 (+ 11988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.203439 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.203467 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.204279 (+   812us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.609414 (+405135us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:53.794556 (+185142us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.239473 (+444917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.239780 (+   307us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.239852 (+    72us) raft_consensus.cc:1685] finished
0709 12:25:54.239875 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.240077 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":317484,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":387609,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":20015,"thread_start_us":1003830,"threads_started":3,"wal-append.queue_time_us":490016}
I20260709 12:25:54.181017 24373 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.067s
I20260709 12:25:54.256512 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:54.219513 24422 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:54.365178 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:54.373471 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 119184us
I20260709 12:25:54.265368 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:54.377796 14417 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:54.360972 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:54.362188 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:54.366381 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 112722us
I20260709 12:25:54.366972 13645 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:54.373373 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:54.383075 13646 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:54.388010 13647 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:25:54.390703 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:54.391474 13644 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:54.255625 23970 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:54.399238 24430 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
W20260709 12:25:54.399365 24403 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 12:25:54.400128 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:54.401232 24344 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.402530 24364 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:54.406862 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:54.407742 24427 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.316s	user 0.000s	sys 0.000s
I20260709 12:25:54.407435 24401 consensus_queue.cc:1059] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 37.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:25:54.445736 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4634) took 1198 ms. Trace:
I20260709 12:25:54.409106 24173 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:54.409502 24426 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.409982 24443 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:54.487617 24446 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:54.488564 23957 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.489189 23957 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.490195 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4636) took 1242 ms. Trace:
I20260709 12:25:54.490648 24452 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.491255 24452 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.490613 13727 rpcz_store.cc:276] 0709 12:25:53.248118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.255985 (+  7867us) service_pool.cc:224] Handling call
0709 12:25:53.256505 (+   520us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.280459 (+ 23954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.280510 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:53.281481 (+   971us) log.cc:844] Serialized 52 byte log entry
0709 12:25:53.586405 (+304924us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:53.586479 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.586591 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.489911 (+903320us) raft_consensus.cc:1685] finished
0709 12:25:54.489941 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.490123 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23197,"raft.queue_time_us":25191,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"thread_start_us":304291,"threads_started":1,"wal-append.queue_time_us":480659}
I20260709 12:25:54.415791 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:54.492835 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77161.6us
I20260709 12:25:54.493290 24425 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:54.494074 24425 consensus_queue.cc:1243] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 24.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:25:54.433583 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:54.440018 24364 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.495471 24050 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:54.495880 24364 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:54.496313 24387 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.309s	user 0.007s	sys 0.000s
W20260709 12:25:54.495275 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 77711us
I20260709 12:25:54.498099 24452 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.498485 24168 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:25:54.499377 24168 consensus_queue.cc:1243] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:54.464078 24439 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:54.399276 24433 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:54.470657 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.471030 24393 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:54.471877 24169 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.445712 24316 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.479297 24402 raft_consensus.cc:3055] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Stepping down as leader of term 39
W20260709 12:25:54.481585 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.482049 23998 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:54.483371 24401 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.484484 24258 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.714s	user 0.004s	sys 0.008s
I20260709 12:25:54.485939 13643 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 24. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:25:54.487505 13766 leader_election.cc:400] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31.
I20260709 12:25:54.578392 13766 leader_election.cc:403] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:25:54.488272 24443 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.495256 24438 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.581506 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:54.381651 24428 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.498776 24452 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:54.463857 24297 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:54.495358 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:54.600396 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 105149us
I20260709 12:25:54.446488 13726 rpcz_store.cc:276] 0709 12:25:53.247317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.348232 (+100915us) service_pool.cc:224] Handling call
0709 12:25:53.373358 (+ 25126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.390596 (+ 17238us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:53.645729 (+255133us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.265307 (+619578us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.408186 (+142879us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.408335 (+   149us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:54.408368 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.408447 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.445469 (+ 37022us) raft_consensus.cc:1685] finished
0709 12:25:54.445497 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.445658 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10842,"prepare.queue_time_us":419360,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":11570,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"thread_start_us":776307,"threads_started":2,"wal-append.queue_time_us":708282}
I20260709 12:25:54.521486 13648 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:25:54.500056 24288 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:54.523312 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4651) took 1129 ms. Trace:
I20260709 12:25:54.533658 23957 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:54.550416 24433 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.551838 24393 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:54.607898 13650 raft_consensus.cc:2410] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:25:54.568256 13794 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:25:54.581216 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:54.582129 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:54.577442 24373 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.589066 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 149610us
I20260709 12:25:54.560003 24402 raft_consensus.cc:740] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:54.610154 13792 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:25:54.537740 24414 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.603879 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.615520 13643 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:54.583250 24344 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.616938 13797 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:54.604477 24288 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:54.137491 13547 rpcz_store.cc:269] 0709 12:25:09.717969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.856289 (+11138320us) service_pool.cc:224] Handling call
0709 12:25:20.934415 (+ 78126us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:25:54.056232 (+33121817us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:44.782244 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:44.863649 (+ 81405us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:44.863725 (+    76us) write_op.cc:435] Acquired schema lock
0709 12:25:44.863753 (+    28us) tablet.cc:662] Decoding operations
0709 12:25:45.171058 (+307305us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:45.208544 (+ 37486us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:45.214589 (+  6045us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:25:45.750474 (+535885us) tablet.cc:701] Row locks acquired
0709 12:25:45.750534 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:25:45.750730 (+   196us) write_op.cc:270] Start()
0709 12:25:45.750906 (+   176us) write_op.cc:276] Timestamp: P: 1783599945750651 usec, L: 0
0709 12:25:45.750944 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:25:45.752211 (+  1267us) log.cc:844] Serialized 8134 byte log entry
0709 12:25:46.206014 (+453803us) op_driver.cc:464] REPLICATION: finished
0709 12:25:46.292840 (+ 86826us) write_op.cc:301] APPLY: starting
0709 12:25:46.296202 (+  3362us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:46.778861 (+482659us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:46.779022 (+   161us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:47.994179 (+1215157us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:48.015221 (+ 21042us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 12:25:48.015351 (+   130us) write_op.cc:312] APPLY: finished
0709 12:25:48.016149 (+   798us) log.cc:844] Serialized 188 byte log entry
0709 12:25:48.204361 (+188212us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:48.214729 (+ 10368us) write_op.cc:454] Released schema lock
0709 12:25:48.250652 (+ 35923us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:48.379102 (+128450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:48.379236 (+   134us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5194116,"reported_tablets":28,"spinlock_wait_cycles":77194112,"child_traces":[["op",{"apply.queue_time_us":63235,"apply.run_cpu_time_us":93318,"apply.run_wall_time_us":2100932,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92075,"num_ops":21,"prepare.queue_time_us":210767,"prepare.run_cpu_time_us":32804,"prepare.run_wall_time_us":1142371,"raft.queue_time_us":105430,"raft.run_cpu_time_us":22541,"raft.run_wall_time_us":43365,"replication_time_us":454865,"thread_start_us":539591,"threads_started":5,"wal-append.queue_time_us":521097,"wal-append.run_cpu_time_us":36817,"wal-append.run_wall_time_us":2320906}]]}
W20260709 12:25:54.611964 24397 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.000s
I20260709 12:25:54.598086 24168 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:54.602180 24297 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:54.604586 13720 rpcz_store.cc:276] 0709 12:25:53.414247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.452768 (+ 38521us) service_pool.cc:224] Handling call
0709 12:25:53.519465 (+ 66697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.562930 (+ 43465us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:53.786309 (+223379us) log.cc:844] Serialized 14 byte log entry
0709 12:25:54.492445 (+706136us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b1800199f00. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:25:54.522853 (+ 30408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:54.522938 (+    85us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:54.522990 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.523076 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.523240 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":342188,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":335097,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11931,"spinlock_wait_cycles":11244288,"thread_start_us":359659,"threads_started":3,"wal-append.queue_time_us":675120}
I20260709 12:25:54.612473 24393 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.589161 24258 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.588873 23998 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.654747 13648 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:54.663554 13722 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:54.555099 24188 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:54.664814 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 452255us
I20260709 12:25:54.618454 24373 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.671142 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:54.669571 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:54.626959 24452 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:54.664086 24393 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:54.658964 24297 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:54.625353 24344 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.692158 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:54.695343 24373 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.685035 24302 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:25:54.558576 24426 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.583770 24401 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.569317 24439 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.605500 24438 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.661628 24397 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.712667 24423 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:54.725279 24392 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:25:54.740289 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 379488us
W20260709 12:25:54.725553 24412 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.678s	user 0.009s	sys 0.004s
I20260709 12:25:54.727646 24443 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:54.728695 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61651.3us
I20260709 12:25:54.502524 24446 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.743747 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:54.716825 23901 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.648s	user 0.014s	sys 0.000s
I20260709 12:25:54.663309 24340 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 12:25:54.739315 13720 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:54.745186 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3267) took 1020 ms. Trace:
I20260709 12:25:54.713033 24457 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:54.752676 13647 rpcz_store.cc:276] 0709 12:25:53.725063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.437136 (+712073us) service_pool.cc:224] Handling call
0709 12:25:54.745116 (+307980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:54.790881 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:54.791204 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:54.791519 24414 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:54.791913 13647 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:54.792479 24397 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:54.781515 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:54.794032 13726 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:54.784710 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:54.784963 13794 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
I20260709 12:25:54.792393 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:54.785095 13690 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.025s	sys 0.029s
W20260709 12:25:54.809602 13690 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.025s	sys 0.029s
W20260709 12:25:54.810168 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 566887us
I20260709 12:25:54.809710 13719 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:25:54.819248 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:54.840389 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:54.842185 13720 raft_consensus.cc:2376] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31. Re-sending same reply.
I20260709 12:25:54.751199 24412 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:54.751462 24302 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:54.856003 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 71374.2us
I20260709 12:25:54.847100 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:54.862810 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:54.869453 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:54.869900 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:54.870666 14419 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:54.789775 24443 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.866854 13727 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:54.869139 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:54.881925 14416 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:54.788087 24457 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:54.795058 24050 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:54.897305 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:54.881356 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:54.880446 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:54.900882 14425 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:54.869369 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:54.880023 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:54.889133 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:54.794598 24436 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:54.816509 24458 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.781383 23901 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.908300 14422 raft_consensus.cc:2410] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 34 because replica has last-logged OpId of term: 33 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 24.
I20260709 12:25:54.852954 24414 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.846717 24392 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.915031 14418 raft_consensus.cc:2368] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:25:54.915602 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:54.723114 24402 consensus_queue.cc:260] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 39.26, Last appended by leader: 26, Current term: 39, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:54.929486 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:54.892292 24340 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:54.839818 23780 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:25:54.872855 24451 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:54.934410 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68221.1us
I20260709 12:25:54.906869 24457 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:54.947335 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:54.948680 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:54.920610 24458 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:54.927477 24368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.315s	user 0.001s	sys 0.027s
I20260709 12:25:54.924476 24253 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:54.951965 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:54.846534 24373 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.073s
I20260709 12:25:54.964546 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:54.930177 24192 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:54.958645 13720 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:54.965457 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:54.963796 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 453932us
W20260709 12:25:54.965420 13644 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.006s
I20260709 12:25:54.994743 13644 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:25:54.995541 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1679) took 1149 ms. Trace:
I20260709 12:25:54.965512 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:54.930955 24436 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:54.995968 13644 rpcz_store.cc:276] 0709 12:25:53.845990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.264256 (+418266us) service_pool.cc:224] Handling call
0709 12:25:54.995436 (+731180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:54.895855 24443 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.006701 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:54.960675 24368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.352s	user 0.001s	sys 0.027s
I20260709 12:25:54.917171 24364 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.019379 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:54.960218 24462 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:55.011868 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:54.959797 24455 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:25:54.932549 24414 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:54.961750 24458 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:54.961728 24397 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:25:55.085124 24465 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:55.026724 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:55.024859 14417 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.000s
I20260709 12:25:54.960902 24253 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:54.959488 24442 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:55.111315 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:55.033720 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.115710 24468 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:54.961738 24344 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.034935 23998 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:54.938002 24392 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.035447 13719 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:55.126106 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.036559 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:55.037628 24459 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:25:54.916589 24452 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.093451 24462 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:55.059536 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.009229 24433 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.067867 14425 raft_consensus.cc:1240] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 39. Current term is 40. Ops: [39.26-39.26]
I20260709 12:25:55.023900 24340 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:55.013417 24188 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:55.078377 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 147328us
W20260709 12:25:55.013447 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53322.5us
I20260709 12:25:55.118503 24446 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.051779 24192 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:55.093019 13727 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:55.086791 13721 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:55.025255 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.109817 14417 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:54.935542 24423 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.115064 24436 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.113488 14421 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:25:54.955061 24460 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:55.118829 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 154363us
I20260709 12:25:55.091595 24401 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:55.122737 14418 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 36: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:55.128153 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.101565 24426 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.010190 24258 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.116164 24364 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.139086 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.060456 24302 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.000072 24451 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.146582 24253 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:55.149207 24471 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:25:55.151984 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.153802 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.154179 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.154590 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:55.237108 13648 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:55.155030 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:55.238050 13649 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:55.136148 24439 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.132587 24438 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.120829 24442 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.167387 24443 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.168166 23901 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:55.166175 24168 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.000s
W20260709 12:25:55.176374 24466 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:55.179098 24470 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.179710 24452 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:55.181715 24392 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:55.135711 24461 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.262s	user 0.001s	sys 0.001s
I20260709 12:25:55.126463 24397 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:55.186475 24414 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.187656 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.282598 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.283313 13644 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:55.189286 24393 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:55.191205 24467 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:25:55.200784 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.201045 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1678) took 1365 ms. Trace:
I20260709 12:25:55.137315 24462 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:55.209421 13726 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:25:55.141280 24390 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.503s	user 0.011s	sys 0.007s
I20260709 12:25:55.214133 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:55.219630 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.293596 14421 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 40: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:25:55.293488 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 79465.9us
I20260709 12:25:55.223482 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.218103 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:55.225374 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.226557 13719 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:55.230614 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.206025 24192 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:55.233553 24253 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:55.221982 24344 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:55.310572 24344 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.313191 24344 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:55.247355 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.247855 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.250280 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.230649 24446 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:55.193360 24457 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:55.316411 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.260031 13652 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:55.265739 24340 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.276010 13724 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:55.318967 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:55.278113 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58077.8us
I20260709 12:25:55.285055 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.286675 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.286404 13651 rpcz_store.cc:276] 0709 12:25:53.835607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.476897 (+641290us) service_pool.cc:224] Handling call
0709 12:25:54.477666 (+   769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.782166 (+304500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.782356 (+   190us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.876881 (+ 94525us) log.cc:844] Serialized 26 byte log entry
0709 12:25:55.082818 (+205937us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:55.082889 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.195344 (+112455us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:55.200479 (+  5135us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400147dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:55.200541 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.200594 (+    53us) raft_consensus.cc:1685] finished
0709 12:25:55.200611 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.200907 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34506,"raft.queue_time_us":37415,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3063,"spinlock_wait_cycles":645778688,"thread_start_us":150845,"threads_started":1,"wal-append.queue_time_us":268190}
I20260709 12:25:55.288641 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:55.300521 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82490.9us
I20260709 12:25:55.322634 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.295634 13721 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:25:55.324664 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.325850 24397 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.326402 24446 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:55.330999 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.280066 24168 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31.
I20260709 12:25:55.305931 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.306061 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.335539 13723 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:55.321107 13796 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:25:55.313524 24402 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:25:55.306790 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.307180 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:55.304279 24192 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:55.279039 24297 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.005s
I20260709 12:25:55.306517 14418 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
I20260709 12:25:55.348219 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.318964 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.342443 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.344470 13792 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:55.343775 13799 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:25:55.345387 13643 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:25:55.334889 13645 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:55.315532 24316 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:25:55.358827 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.351771 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.353075 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.356935 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:55.360378 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.361080 14418 raft_consensus.cc:2368] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31.
I20260709 12:25:55.363864 13797 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 31 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:25:55.277496 24470 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.360042 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:55.360620 13718 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:25:55.371403 13720 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:25:55.319239 24483 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:55.372180 13793 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:55.372591 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.372715 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:55.386745 13799 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:25:55.373914 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55070.1us
W20260709 12:25:55.374701 24405 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.430s	user 0.007s	sys 0.018s
I20260709 12:25:55.375016 24442 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.375137 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.406893 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.366307 24168 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:25:55.379245 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.380283 13652 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:25:55.336946 24443 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:55.381712 23970 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.382411 24426 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.383098 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.388072 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:55.413954 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44557.8us
I20260709 12:25:55.389681 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 31 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.392391 14422 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:25:55.392930 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3264) took 1682 ms. Trace:
I20260709 12:25:55.353577 24169 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.379145 24446 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.410630 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2183) took 1137 ms. Trace:
I20260709 12:25:55.361546 24316 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:55.418123 13645 raft_consensus.cc:2393] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 25: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:55.421447 14391 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:55.433911 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.445503 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.445739 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:55.446511 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.446955 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:55.402809 13720 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:55.345114 23957 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:25:55.362831 24326 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.455399 13727 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:55.415628 13650 rpcz_store.cc:276] 0709 12:25:53.710187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.406518 (+696331us) service_pool.cc:224] Handling call
0709 12:25:55.392728 (+986210us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002e54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:55.392867 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403357952}
I20260709 12:25:55.430094 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.457322 13719 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:55.431294 24169 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.416951 13690 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.432708 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:55.437597 24485 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:25:55.437748 23780 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:25:55.439599 13724 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:25:55.365669 24173 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:55.384816 24392 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:55.442625 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.478629 13723 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
W20260709 12:25:55.479156 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124943us
I20260709 12:25:55.355918 24297 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.437479 24462 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.443971 24443 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.427376 13722 rpcz_store.cc:276] 0709 12:25:54.257928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.361810 (+103882us) service_pool.cc:224] Handling call
0709 12:25:55.394565 (+1032755us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:55.394731 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626855168}
I20260709 12:25:55.444821 24316 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:55.447115 13645 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:25:55.339218 24340 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:55.425537 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.447116 13718 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:55.449456 13800 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:25:55.448180 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.449105 13792 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:55.459950 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.464412 13799 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:55.496348 13720 raft_consensus.cc:2376] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27. Re-sending same reply.
W20260709 12:25:55.473763 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56959.5us
I20260709 12:25:55.481545 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:55.447301 24478 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:25:55.484928 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 190614us
I20260709 12:25:55.499653 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:55.490235 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 111633us
I20260709 12:25:55.457579 24456 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:55.456470 24253 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:55.502624 13645 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:55.494504 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.475806 24414 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:55.506640 13693 connection.cc:256] RPC call timeout handler was delayed by -1.2174183519996404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16240000000000002s for a response
W20260709 12:25:55.507134 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1382183519995124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 12:25:55.507448 13693 connection.cc:256] RPC call timeout handler was delayed by -1.031118351999794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:25:55.507732 13693 connection.cc:256] RPC call timeout handler was delayed by -1.0077183519997561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 12:25:55.508715 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:55.496382 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.511402 13718 raft_consensus.cc:2376] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32. Re-sending same reply.
I20260709 12:25:55.426857 23970 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.500497 13722 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:55.426643 24426 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.503674 13723 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:55.505194 24482 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:25:55.512984 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.519481 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.528470 13720 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:55.529733 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97112.1us
I20260709 12:25:55.532864 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.533443 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.533648 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:55.533761 13718 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:55.534078 13720 raft_consensus.cc:2376] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24. Re-sending same reply.
I20260709 12:25:55.532917 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.530612 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:55.536593 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.498842 24050 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:25:55.503726 24478 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.462467 24326 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.531785 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:55.521028 24412 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.550475 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:55.527837 24442 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:55.555310 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.564834 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:25:55.530287 23957 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.560721 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:55.562784 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.542747 24438 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:55.570353 13723 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:55.591666 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.498548 24173 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.548355 24392 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:25:55.559617 24422 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.143s	user 0.008s	sys 0.005s
I20260709 12:25:55.548754 24253 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:55.573495 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.570868 24194 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:55.562110 24451 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:55.611404 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61030.4us
I20260709 12:25:55.613758 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:55.616744 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.573395 13764 leader_election.cc:400] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31.
I20260709 12:25:55.559242 24478 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:55.602555 13643 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 12:25:55.617794 13764 leader_election.cc:403] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.559515 24439 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.050s
I20260709 12:25:55.600037 24401 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:25:55.602438 24392 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:55.632457 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54420.8us
I20260709 12:25:55.633401 13719 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:25:55.630079 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.634671 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.594117 23957 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.616377 24457 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.615473 24194 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:55.627903 24297 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:55.629182 23970 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.628877 24288 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.619385 24451 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:55.618484 13643 pending_rounds.cc:85] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 11
I20260709 12:25:55.681253 13643 pending_rounds.cc:107] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 12:25:55.681586 13643 raft_consensus.cc:2889] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:55.636576 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100030us
I20260709 12:25:55.593073 13722 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:25:55.684026 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:55.684712 13722 raft_consensus.cc:2376] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27. Re-sending same reply.
I20260709 12:25:55.688678 13648 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:55.690057 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:55.691114 13648 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:55.637048 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.633270 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:55.592768 23901 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:25:55.611701 24438 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:55.566608 24412 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.636281 24392 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:55.643980 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:55.693400 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 119989us
I20260709 12:25:55.592911 24476 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:25:55.633847 24401 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.661695 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.621663 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:55.673007 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 105739us
I20260709 12:25:55.701478 13690 leader_election.cc:400] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:25:55.701866 13690 leader_election.cc:403] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.634745 13725 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:55.703101 13652 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:25:55.678678 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.700112 13726 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
W20260709 12:25:55.703775 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.669611 24373 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:55.643651 24456 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:25:55.703732 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70556.2us
W20260709 12:25:55.698325 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 61336.8us
I20260709 12:25:55.675607 24442 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:55.707545 13648 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:25:55.710732 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.720338 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.677386 24168 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:55.722071 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:55.722872 13719 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 31 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:25:55.677619 24194 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:55.672441 24463 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:55.741189 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:55.725651 24288 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:55.827221 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.828888 24412 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.828933 24173 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.829874 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.830819 13652 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:25:55.831982 24194 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.834975 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.835470 23957 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:55.835752 14424 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:25:55.836213 23957 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:55.827144 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.837891 14421 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:55.841732 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:55.802554 24496 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:55.742527 24470 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.706413 24401 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:55.705389 24457 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:55.609948 24422 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:55.736230 24500 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:25:55.842325 13617 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.998s (SENT)
I20260709 12:25:55.744728 24451 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
W20260709 12:25:55.859776 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 412628us
I20260709 12:25:55.788411 24462 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.735607 24373 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:55.895991 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 154902us
W20260709 12:25:55.899500 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 205194us
I20260709 12:25:55.900357 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:55.904091 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61809.6us
I20260709 12:25:55.903594 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:55.905409 13617 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.631s (SENT)
I20260709 12:25:55.906026 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:55.914654 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 85355.9us
I20260709 12:25:55.916376 13648 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:55.917462 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:55.919965 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:55.920614 13652 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:55.923408 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.925597 13722 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:55.926656 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:55.864475 24500 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.863510 24443 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:55.903172 24463 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:55.932094 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:55.918362 24340 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:55.912658 24326 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.948082 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.948899 14420 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:55.939764 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:55.949887 13648 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:55.923403 24442 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:55.938164 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 418171us
I20260709 12:25:55.949229 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:55.940689 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:55.978379 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.906378 24297 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:25:55.994951 24510 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:55.996009 24510 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.996547 24510 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:55.999207 14424 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
W20260709 12:25:55.999290 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50154.6us
I20260709 12:25:56.001158 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:56.015413 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:55.933902 24500 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:55.940088 24258 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:55.981863 24509 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.999938 23901 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:55.982861 24508 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.073513 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:56.093639 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 190142us
I20260709 12:25:55.999846 24340 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:56.097577 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.111908 13792 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:25:56.100430 13617 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.874s (SENT)
I20260709 12:25:56.113133 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:56.117621 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44214.1us
I20260709 12:25:56.122223 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:56.142742 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.142994 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.027896 24442 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:56.046329 24438 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:25:56.121296 24402 consensus_meta.cc:287] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.773s	user 0.013s	sys 0.000s
I20260709 12:25:56.095927 24470 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.149379 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:56.153692 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43556.7us
I20260709 12:25:56.153743 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.174085 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.109560 23901 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:56.189469 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 257513us
I20260709 12:25:56.190845 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:56.112498 24395 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.095494 24451 raft_consensus.cc:2764] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:25:56.180694 24422 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:55.938961 24456 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.124796 24462 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:25:56.172731 24258 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:56.188398 24373 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:56.198052 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 356401us
I20260709 12:25:56.230849 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:56.115044 24340 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:56.199713 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:56.003947 24501 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:56.146618 24457 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:56.206480 13652 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:25:56.201275 13795 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 40.
I20260709 12:25:56.195010 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:56.216243 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 115865us
I20260709 12:25:56.186822 24442 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.199007 23901 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:56.228320 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:56.231989 13648 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:56.197333 24412 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:56.247916 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.199036 24173 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:56.248641 14416 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:56.248701 24173 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.110488 24509 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.238198 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.243222 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.222910 24438 raft_consensus.cc:2764] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:25:56.241232 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:56.259446 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:56.128391 24508 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.215847 24395 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.247320 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:56.261976 14424 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:56.262331 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:56.263343 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.247738 24495 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 12:25:56.124934 24506 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.124s
I20260709 12:25:56.255679 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:56.266139 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:56.267087 24507 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:25:56.267899 24505 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:25:56.334976 24503 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:56.335642 24503 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.336031 24503 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:56.340385 24395 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:56.340881 24401 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.273759 14424 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:25:56.274461 24504 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:25:56.275328 24501 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.277328 24392 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:25:56.253485 24373 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:25:56.281916 24168 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.002s
W20260709 12:25:56.287876 24452 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.013s	user 0.013s	sys 0.001s
I20260709 12:25:56.199252 24470 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:56.291455 24479 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.141s	user 0.006s	sys 0.003s
W20260709 12:25:56.296159 24414 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.790s	user 0.011s	sys 0.000s
I20260709 12:25:56.298702 24508 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:56.299467 13794 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:56.304100 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.334118 24426 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:56.306530 24509 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:56.306907 24412 raft_consensus.cc:2764] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
W20260709 12:25:56.309978 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47784.5us
I20260709 12:25:56.315878 13652 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:56.232702 24340 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:25:56.321630 24192 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.958s	user 0.012s	sys 0.002s
I20260709 12:25:56.358621 24168 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31.
I20260709 12:25:56.318588 24439 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:25:56.323077 24484 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.886s	user 0.001s	sys 0.012s
I20260709 12:25:56.325016 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2186) took 1082 ms. Trace:
I20260709 12:25:56.394136 13726 rpcz_store.cc:276] 0709 12:25:55.242484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.704439 (+461955us) service_pool.cc:224] Handling call
0709 12:25:55.740943 (+ 36504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.857402 (+116459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.857455 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.900734 (+ 43279us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.152738 (+252004us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:56.152782 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.152858 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.324754 (+171896us) raft_consensus.cc:1685] finished
0709 12:25:56.324780 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.324944 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94988,"raft.queue_time_us":102624,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":8333,"thread_start_us":216044,"threads_started":1,"wal-append.queue_time_us":401058}
I20260709 12:25:56.346169 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:56.348335 14418 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:56.351068 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.352758 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:56.380491 24373 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:56.305608 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:56.414163 13727 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:56.387146 13721 raft_consensus.cc:2410] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 26 because replica has last-logged OpId of term: 25 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 20.
I20260709 12:25:56.390843 13798 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:25:56.322017 24297 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:25:56.388219 24414 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:56.354948 24392 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.416975 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:56.376650 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 121035us
I20260709 12:25:56.389461 24438 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:25:56.426595 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:56.427295 13727 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:56.428181 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:56.428624 13727 raft_consensus.cc:2376] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24. Re-sending same reply.
I20260709 12:25:56.396384 13690 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:56.418696 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:56.378914 24451 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:25:56.431562 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:56.424618 13648 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:56.430847 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:56.453912 13726 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 32: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:56.375190 24510 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:56.418848 24316 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.920s	user 0.018s	sys 0.007s
I20260709 12:25:56.438776 24506 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:25:56.393177 24192 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.418179 24439 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.442286 24373 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.445291 13793 raft_consensus.cc:2368] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26.
I20260709 12:25:56.462711 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5254) took 1731 ms. Trace:
I20260709 12:25:56.463714 24401 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:56.465030 24316 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.467924 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:56.355352 24501 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.551641 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:56.538596 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:56.585495 13726 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:25:56.534235 24490 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.124s	user 0.008s	sys 0.003s
I20260709 12:25:56.578661 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:56.478029 13765 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.009s	sys 0.002s
W20260709 12:25:56.587553 13765 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.011s	sys 0.003s
I20260709 12:25:56.588155 13648 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
W20260709 12:25:56.582118 13617 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.874s (SENT)
I20260709 12:25:56.418385 24463 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:56.546002 24526 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:25:56.473798 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:56.577487 13647 rpcz_store.cc:276] 0709 12:25:54.731783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.793011 (+ 61228us) service_pool.cc:224] Handling call
0709 12:25:54.857461 (+ 64450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.877686 (+ 20225us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:54.929094 (+ 51408us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.020780 (+ 91686us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.021495 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 12:25:55.021684 (+   189us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:55.021732 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.021834 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.290726 (+1268892us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:56.462334 (+171608us) raft_consensus.cc:1685] finished
0709 12:25:56.462465 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.462639 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26129,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":13710,"raft.queue_time_us":1133,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":35837,"spinlock_wait_cycles":125072128,"thread_start_us":60009,"threads_started":1,"wal-append.queue_time_us":133114}
W20260709 12:25:56.560132 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134177us
I20260709 12:25:56.711040 24393 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:56.709939 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.712637 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:56.668341 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55892.8us
I20260709 12:25:56.724414 13799 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:56.724784 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:25:56.724953 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.595867 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:56.709815 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.725534 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:56.725853 14424 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 40: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:56.471213 24506 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.509342 24414 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:56.704645 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:56.714898 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4383) took 1752 ms (client timeout 1874 ms). Trace:
W20260709 12:25:56.767028 13724 rpcz_store.cc:269] 0709 12:25:54.963006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.994888 (+ 31882us) service_pool.cc:224] Handling call
0709 12:25:56.714721 (+1719833us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x7b5400174154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:56.714836 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157633792}
W20260709 12:25:56.766808 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 66786.5us
I20260709 12:25:56.687911 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:56.616935 13801 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:56.720927 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:56.721215 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139137us
I20260709 12:25:56.770841 13720 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:25:56.771255 13617 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.991s (SENT)
I20260709 12:25:56.771704 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:56.676393 24464 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.723145 13727 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:56.722659 24463 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.731353 24510 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:56.732425 24501 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:56.733696 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:56.737335 24192 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:56.738499 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:56.739236 24520 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:25:56.743173 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5255) took 2003 ms. Trace:
W20260709 12:25:56.748311 14423 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1641) took 1643 ms (client timeout 2164 ms). Trace:
I20260709 12:25:56.797351 24483 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:56.798059 24483 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.801373 24483 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:56.749526 13795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4804) took 1416 ms (client timeout 1553 ms). Trace:
I20260709 12:25:56.802361 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.803725 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.804569 13799 raft_consensus.cc:2410] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 41 because replica has last-logged OpId of term: 39 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 37 index: 25.
I20260709 12:25:56.805367 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:56.751570 24439 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:56.752259 24500 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.000s
I20260709 12:25:56.806875 24500 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.782109 23901 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.809376 24500 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:56.753575 24457 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:56.722695 24522 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.754007 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.757128 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:56.785712 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:56.793694 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60083.6us
W20260709 12:25:56.802130 13795 rpcz_store.cc:269] 0709 12:25:55.333064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.354035 (+ 20971us) service_pool.cc:224] Handling call
0709 12:25:56.749345 (+1395310us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54002cb0d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:56.749464 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1825044736}
I20260709 12:25:56.766705 24412 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:25:56.810693 13792 raft_consensus.cc:2368] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
W20260709 12:25:56.797567 14423 rpcz_store.cc:269] 0709 12:25:55.104617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.104995 (+   378us) service_pool.cc:224] Handling call
0709 12:25:56.748072 (+1643077us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54005e4f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:56.748244 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2889860992}
I20260709 12:25:56.813928 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:56.781024 24438 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 12:25:56.814594 13617 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.023s (SENT)
I20260709 12:25:56.767457 24506 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:56.810084 23901 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:56.783529 24344 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:25:56.820851 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63829.7us
I20260709 12:25:56.816395 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.821460 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:56.822079 13727 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:56.796051 13646 rpcz_store.cc:276] 0709 12:25:54.739846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.816865 (+ 77019us) service_pool.cc:224] Handling call
0709 12:25:54.872508 (+ 55643us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:54.925041 (+ 52533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.925075 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:54.935207 (+ 10132us) log.cc:844] Serialized 52 byte log entry
0709 12:25:55.326167 (+390960us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b180037aa40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:25:55.409793 (+ 83626us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:55.410295 (+   502us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.449346 (+ 39051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.449405 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.533482 (+1084077us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:56.742852 (+209370us) raft_consensus.cc:1685] finished
0709 12:25:56.742886 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.743093 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9154,"prepare.queue_time_us":237,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":59914,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":26597,"spinlock_wait_cycles":46410496,"thread_start_us":122104,"threads_started":2,"wal-append.queue_time_us":450077}
I20260709 12:25:56.787536 24414 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:56.830278 24414 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:56.824721 13726 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:56.796001 24426 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:25:56.835492 13646 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:25:56.841658 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:56.795240 24393 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:56.826754 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:56.795970 24520 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:56.851926 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:56.859287 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:56.866776 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:56.867933 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:56.866931 24500 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:56.869307 24500 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.870457 24452 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:56.871534 24452 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:56.871773 24500 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:56.871891 24452 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.872319 24452 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:56.822815 24439 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:25:56.887077 14390 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.741s (SENT)
I20260709 12:25:56.887620 13799 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:56.874521 24412 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:56.893040 24316 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:56.893746 24457 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:56.893678 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.825073 24192 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:25:56.817828 24522 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:56.899905 24344 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:56.901350 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74685.2us
I20260709 12:25:56.902469 24494 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:25:56.903321 24169 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:56.908823 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2197) took 1316 ms. Trace:
I20260709 12:25:56.902094 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.915731 13644 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:25:57.028849 13726 rpcz_store.cc:276] 0709 12:25:55.592176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.907709 (+1315533us) service_pool.cc:224] Handling call
0709 12:25:56.908230 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.908473 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.908512 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:56.908568 (+    56us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:56.908595 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.908644 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.908736 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:56.917305 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.918519 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1698) took 1632 ms. Trace:
I20260709 12:25:56.920612 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.922489 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:56.931756 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2192) took 1636 ms. Trace:
I20260709 12:25:57.034545 13723 rpcz_store.cc:276] 0709 12:25:55.295085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.705441 (+410356us) service_pool.cc:224] Handling call
0709 12:25:55.721682 (+ 16241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.915033 (+193351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.915084 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.924795 (+  9711us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.211095 (+286300us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:56.211142 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.931145 (+720003us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54000276d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:56.931220 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.931301 (+    81us) raft_consensus.cc:1685] finished
0709 12:25:56.931321 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.931663 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15699,"raft.queue_time_us":14044,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12600,"spinlock_wait_cycles":412825984,"thread_start_us":228483,"threads_started":1,"wal-append.queue_time_us":416342}
W20260709 12:25:56.939221 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2188) took 1695 ms (client timeout 1741 ms). Trace:
W20260709 12:25:57.037094 13721 rpcz_store.cc:269] 0709 12:25:55.243490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.324339 (+ 80849us) service_pool.cc:224] Handling call
0709 12:25:56.938860 (+1614521us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:56.939123 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441287552}
I20260709 12:25:56.941808 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:56.943544 24462 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:56.945050 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4692) took 1021 ms. Trace:
I20260709 12:25:56.956701 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:57.046653 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90090.7us
W20260709 12:25:56.957609 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 161612us
W20260709 12:25:57.047653 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.026s (SENT)
I20260709 12:25:56.842286 24501 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:25:56.969426 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 231048us
W20260709 12:25:56.986963 24423 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.031s	sys 0.039s
I20260709 12:25:56.976125 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:57.024434 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 137407us
I20260709 12:25:57.054951 13799 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:57.027804 24483 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:57.070729 24483 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.070675 24344 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.073160 24483 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:57.030593 24494 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:56.792979 24464 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.045194 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.082690 13651 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:25:57.038789 13793 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:25:57.030499 13800 rpcz_store.cc:276] 0709 12:25:55.286288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.286430 (+   142us) service_pool.cc:224] Handling call
0709 12:25:56.918226 (+1631796us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:56.918439 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245746432}
I20260709 12:25:56.965579 24439 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.050957 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:56.973328 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.046274 13722 rpcz_store.cc:276] 0709 12:25:55.923974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.940424 (+ 16450us) service_pool.cc:224] Handling call
0709 12:25:56.944825 (+1004401us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54000294d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:56.944972 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164185344}
I20260709 12:25:57.052819 13801 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:56.963624 23957 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.056775 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:25:57.053504 24423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.031s	sys 0.039s
I20260709 12:25:57.054458 24393 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:25:57.062832 24518 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.368s	user 0.003s	sys 0.000s
I20260709 12:25:56.968423 24457 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:57.064214 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.074013 24494 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:25:57.066313 24497 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:25:57.070199 24340 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.656s	user 0.006s	sys 0.005s
I20260709 12:25:56.968984 24426 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:25:57.023490 23901 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:57.148509 23901 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.097262 13726 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:25:57.150827 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.152607 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:57.099503 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.154424 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.154716 13644 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:57.155191 13649 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:57.156176 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.135025 24169 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.157392 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.101838 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.157721 24169 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.158725 13651 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:57.046012 24462 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:57.049414 24412 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:57.050688 24192 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:57.116714 13797 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:25:57.117242 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:57.167327 14419 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
W20260709 12:25:57.167665 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50540.9us
W20260709 12:25:57.168732 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 117845us
I20260709 12:25:57.066514 24522 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:57.134699 13794 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:57.110562 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:57.177346 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66897.9us
I20260709 12:25:57.143741 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.178699 13718 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:57.095490 24439 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:57.105679 23957 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.147081 24340 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.153951 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:56.895650 24520 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.192198 24520 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:57.170502 13766 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:57.196281 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:57.197021 24050 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:57.197530 24050 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.197912 24050 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:57.199023 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.199769 13722 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:57.197058 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2190) took 1923 ms. Trace:
I20260709 12:25:57.202764 13719 rpcz_store.cc:276] 0709 12:25:55.273530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.635846 (+362316us) service_pool.cc:224] Handling call
0709 12:25:57.196829 (+1560983us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b54000285d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:57.196990 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48419072}
I20260709 12:25:57.203994 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.170120 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:57.141716 24503 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.800s	user 0.010s	sys 0.005s
I20260709 12:25:57.206846 24503 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.209396 24503 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.141215 24510 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:25:57.212177 24464 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.180747 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.214833 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3275) took 1897 ms. Trace:
I20260709 12:25:57.215365 13649 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:57.215328 13643 rpcz_store.cc:276] 0709 12:25:55.329047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.361913 (+ 32866us) service_pool.cc:224] Handling call
0709 12:25:55.411402 (+ 49489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.708704 (+297302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.214554 (+1505850us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001478d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:57.214754 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425667328}
I20260709 12:25:57.216780 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:57.217700 24478 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.547s	user 0.016s	sys 0.000s
I20260709 12:25:57.183210 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:57.218411 24478 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.219370 13796 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:57.221526 24168 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:57.221284 24457 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.222671 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4397) took 2060 ms. Trace:
I20260709 12:25:57.184101 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.139926 24393 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:57.207248 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3282) took 1107 ms. Trace:
I20260709 12:25:57.186120 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3275) took 1088 ms. Trace:
I20260709 12:25:57.225031 13792 rpcz_store.cc:276] 0709 12:25:56.099691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.153403 (+ 53712us) service_pool.cc:224] Handling call
0709 12:25:57.207001 (+1053598us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54000d16d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:57.207183 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448595328}
I20260709 12:25:57.227666 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.228495 13722 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:25:57.214900 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44877.1us
I20260709 12:25:57.229121 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.218048 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.226501 13794 rpcz_store.cc:276] 0709 12:25:56.097838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.922132 (+824294us) service_pool.cc:224] Handling call
0709 12:25:57.186027 (+263895us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.233350 13794 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:57.235652 13794 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:57.238081 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.182883 24426 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:25:57.223097 13725 rpcz_store.cc:276] 0709 12:25:55.162807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.703055 (+540248us) service_pool.cc:224] Handling call
0709 12:25:55.720559 (+ 17504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.843406 (+122847us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:55.936115 (+ 92709us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.393719 (+457604us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.418193 (+ 24474us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.418446 (+   253us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:25:56.418498 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.876894 (+458396us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:25:57.151764 (+274870us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b54003bf454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:57.151821 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.222414 (+ 70593us) raft_consensus.cc:1685] finished
0709 12:25:57.222436 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.222601 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167214,"prepare.queue_time_us":57422,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":6866,"raft.queue_time_us":149036,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":31355,"spinlock_wait_cycles":63882368,"thread_start_us":423946,"threads_started":2,"wal-append.queue_time_us":590781}
I20260709 12:25:57.245239 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.257290 13722 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:25:57.259792 24192 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.261970 24169 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.262701 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.263463 13650 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:25:57.264928 13650 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:25:57.267056 24462 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.267867 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 1965) took 2210 ms. Trace:
I20260709 12:25:57.268209 24462 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:57.255244 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.271322 13643 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:25:57.262949 24525 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.561s	user 0.003s	sys 0.000s
I20260709 12:25:57.274627 24412 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.278493 24443 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:57.278746 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:57.279301 24443 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.279749 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:57.281030 24503 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:25:57.281901 24503 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:25:57.260296 23957 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.284197 13792 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:57.284317 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:57.285895 13797 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: []
I20260709 12:25:57.189992 24522 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:57.268247 14419 rpcz_store.cc:276] 0709 12:25:55.057076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.057227 (+   151us) service_pool.cc:224] Handling call
0709 12:25:55.067564 (+ 10337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.168235 (+2100671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.267580 (+ 99345us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:57.267800 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4812731776}
I20260709 12:25:57.250885 13651 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:57.282292 24503 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:57.293522 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 97308.2us
I20260709 12:25:57.294220 13649 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:57.294410 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:25:57.289177 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 430008us
I20260709 12:25:57.299811 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.302507 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:57.303470 14418 raft_consensus.cc:2368] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:25:57.285673 24422 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:57.305721 13726 raft_consensus.cc:2393] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:57.283097 24426 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:57.263417 24169 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:57.281914 24443 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:57.288057 24500 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:25:57.285874 24476 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:57.291086 13644 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:57.292263 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5272) took 1087 ms. Trace:
I20260709 12:25:57.321111 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.326227 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.341909 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:57.342614 13643 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:57.311162 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.301321 23957 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:57.296344 24316 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:25:57.297552 24503 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:57.296167 24464 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:25:57.376179 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.342887 14416 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:57.343802 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 21.
I20260709 12:25:57.418217 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.342324 13647 rpcz_store.cc:276] 0709 12:25:56.204875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.728061 (+523186us) service_pool.cc:224] Handling call
0709 12:25:56.728474 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:56.728915 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.728954 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:56.729769 (+   815us) log.cc:844] Serialized 52 byte log entry
0709 12:25:56.821765 (+ 91996us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:57.291520 (+469755us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.291869 (+   349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.291962 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.292003 (+    41us) raft_consensus.cc:1685] finished
0709 12:25:57.292015 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.292192 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1590,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":530260,"threads_started":2,"wal-append.queue_time_us":410611}
I20260709 12:25:57.344619 13649 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:57.354569 24169 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.344769 24500 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.356109 13643 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:25:57.356746 24340 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.337127 24426 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:57.362440 24438 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.367558 24506 consensus_meta.cc:287] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.002s
I20260709 12:25:57.373605 24494 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:57.374536 24529 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:25:57.391350 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.481860 13644 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:57.418934 13726 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:25:57.484354 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.353832 24510 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:25:57.416898 24462 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.491647 24462 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.375756 23957 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:57.494807 23957 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:57.476871 24439 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.475538 24456 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:57.347497 23901 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.486229 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.341178 24476 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:57.422308 24478 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:57.423944 24500 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:57.506096 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1694) took 1149 ms. Trace:
I20260709 12:25:57.519645 13652 rpcz_store.cc:276] 0709 12:25:56.356520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.477788 (+1121268us) service_pool.cc:224] Handling call
0709 12:25:57.506005 (+ 28217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.482870 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:57.521740 13723 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:57.541555 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23.
I20260709 12:25:57.558248 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2199) took 1863 ms. Trace:
I20260709 12:25:57.507448 24340 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:57.474851 24169 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:57.615192 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.526714 24050 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.543442 24462 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:57.588724 13648 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.035s	sys 0.061s
W20260709 12:25:57.621543 13648 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.036s	sys 0.062s
I20260709 12:25:57.622443 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1691) took 1275 ms. Trace:
I20260709 12:25:57.483260 24438 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:57.518203 13727 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.007s	sys 0.024s
I20260709 12:25:57.599503 13722 rpcz_store.cc:276] 0709 12:25:55.694736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.556952 (+1862216us) service_pool.cc:224] Handling call
0709 12:25:57.557494 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.557753 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.557791 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.557870 (+    79us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:25:57.557919 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.557994 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.558157 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.481051 24506 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.498752 24456 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.628756 24438 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:57.629058 24456 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:57.426889 24393 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:57.629401 24523 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.245s	user 0.004s	sys 0.000s
I20260709 12:25:57.630501 13722 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:25:57.631201 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4674) took 1910 ms. Trace:
I20260709 12:25:57.631505 13722 rpcz_store.cc:276] 0709 12:25:55.720956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.629734 (+1908778us) service_pool.cc:224] Handling call
0709 12:25:57.630087 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.630897 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.631112 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.633020 13722 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:57.633411 13725 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:25:57.633694 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4676) took 1912 ms. Trace:
I20260709 12:25:57.634088 13722 rpcz_store.cc:276] 0709 12:25:55.721627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.632305 (+1910678us) service_pool.cc:224] Handling call
0709 12:25:57.632677 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.633306 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.633540 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.635243 24316 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:25:57.612787 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:57.636588 24510 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:57.637017 24510 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.637051 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:57.637521 24510 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:57.618688 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 458094us
W20260709 12:25:57.619136 13720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.006s	sys 0.008s
W20260709 12:25:57.586229 13799 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.009s
W20260709 12:25:57.625449 13727 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.008s	sys 0.024s
I20260709 12:25:57.609189 24340 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.623035 13648 rpcz_store.cc:276] 0709 12:25:56.347449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.759359 (+411910us) service_pool.cc:224] Handling call
0709 12:25:56.759686 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.759950 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:56.989027 (+229077us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.622035 (+633008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.622101 (+    66us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:57.622150 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.622225 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.622381 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":586164,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"thread_start_us":828799,"threads_started":2}
I20260709 12:25:57.635617 13722 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:25:57.646128 13644 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
W20260709 12:25:57.643888 13799 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.011s	sys 0.009s
W20260709 12:25:57.644169 13720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.007s	sys 0.008s
I20260709 12:25:57.634196 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4677) took 1912 ms. Trace:
I20260709 12:25:57.651561 13725 rpcz_store.cc:276] 0709 12:25:55.721905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.632795 (+1910890us) service_pool.cc:224] Handling call
0709 12:25:57.633095 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.633942 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.634143 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.638110 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:25:57.653540 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 423565us
W20260709 12:25:57.602383 13693 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.009s	sys 0.009s
W20260709 12:25:57.655936 13693 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.010s	sys 0.009s
W20260709 12:25:57.656424 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 584366us
I20260709 12:25:57.668941 13723 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:25:57.657349 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:57.774106 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 117864us
I20260709 12:25:57.671629 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4684) took 1781 ms. Trace:
I20260709 12:25:57.775494 13722 rpcz_store.cc:276] 0709 12:25:55.865008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.635129 (+1770121us) service_pool.cc:224] Handling call
0709 12:25:57.635448 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.646432 (+ 10984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.646672 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.672622 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:57.686956 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:57.689260 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:57.777976 13652 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:57.689491 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2195) took 2344 ms. Trace:
I20260709 12:25:57.690201 13725 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:25:57.693140 14425 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:25:57.621121 24499 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:25:57.784829 24499 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.785444 24499 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.698900 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:57.786911 14419 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:57.663599 24340 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:25:57.789070 24483 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.015s	sys 0.007s
I20260709 12:25:57.671489 23901 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:57.601660 24476 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:57.790197 24483 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.015s	sys 0.008s
I20260709 12:25:57.688340 24456 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.714089 24535 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:25:57.714695 24439 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:57.720961 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:57.807224 13721 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
W20260709 12:25:57.775060 24344 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.038s	sys 0.096s
I20260709 12:25:57.725116 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:57.709664 24168 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:25:57.742152 24520 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.548s	user 0.014s	sys 0.000s
I20260709 12:25:57.748365 24169 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:57.748797 24173 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.750841 24392 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:57.754667 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.754954 24470 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:57.856772 24520 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.857128 24462 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:57.858210 24520 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.858559 24462 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.757375 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:57.758123 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:57.737182 24494 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.862448 14423 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:25:57.759181 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:57.761058 24537 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:25:57.767647 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:57.866605 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2194) took 2531 ms. Trace:
I20260709 12:25:57.868489 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4811) took 1189 ms. Trace:
I20260709 12:25:57.768735 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:57.779241 13720 rpcz_store.cc:276] 0709 12:25:55.344718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.808604 (+1463886us) service_pool.cc:224] Handling call
0709 12:25:56.808944 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.809192 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:25:57.080252 (+271060us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.689101 (+608849us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.689163 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:57.689214 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.689288 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.689430 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37336,"prepare.queue_time_us":681325,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":40313,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":4900,"thread_start_us":836368,"threads_started":2,"wal-append.queue_time_us":691600}
I20260709 12:25:57.782099 24050 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:57.779639 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4685) took 1909 ms. Trace:
W20260709 12:25:57.870633 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103065us
I20260709 12:25:57.870900 13725 rpcz_store.cc:276] 0709 12:25:55.870429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.653787 (+1783358us) service_pool.cc:224] Handling call
0709 12:25:57.689612 (+ 35825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.779333 (+ 89721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.779579 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.793813 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:57.792843 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:57.872781 13722 raft_consensus.cc:2368] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:25:57.756505 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:57.874297 14417 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:57.857447 24442 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:25:57.848358 24344 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.038s	sys 0.096s
I20260709 12:25:57.791144 24456 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.867004 13724 rpcz_store.cc:276] 0709 12:25:55.334753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.781181 (+1446428us) service_pool.cc:224] Handling call
0709 12:25:56.781985 (+   804us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:56.822584 (+ 40599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.822625 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:56.842982 (+ 20357us) log.cc:844] Serialized 52 byte log entry
0709 12:25:57.283554 (+440572us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:57.283614 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.283808 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.866349 (+582541us) raft_consensus.cc:1685] finished
0709 12:25:57.866372 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.866530 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36977,"raft.queue_time_us":15873,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7379,"thread_start_us":416952,"threads_started":1,"wal-append.queue_time_us":707069}
W20260709 12:25:57.884460 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.803313 24522 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:57.884346 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1709) took 1486 ms. Trace:
I20260709 12:25:57.887990 13721 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:57.888944 13797 rpcz_store.cc:276] 0709 12:25:56.398472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.176269 (+777797us) service_pool.cc:224] Handling call
0709 12:25:57.226664 (+ 50395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.288084 (+ 61420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.884006 (+595922us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54003d40d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:57.884273 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129406208}
W20260709 12:25:57.888408 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 94717.3us
W20260709 12:25:57.891458 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.011s	sys 0.005s
I20260709 12:25:57.869046 13796 rpcz_store.cc:276] 0709 12:25:56.679539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.220479 (+540940us) service_pool.cc:224] Handling call
0709 12:25:57.220833 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.221175 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.221203 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.221785 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.603906 (+382121us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:25:57.603952 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.604140 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.868279 (+264139us) raft_consensus.cc:1685] finished
0709 12:25:57.868301 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.868431 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1337,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"thread_start_us":381632,"threads_started":1,"wal-append.queue_time_us":521667}
I20260709 12:25:57.892310 13800 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
W20260709 12:25:57.879030 13618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.038s
W20260709 12:25:57.896348 13618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.000s	sys 0.038s
W20260709 12:25:57.890977 13646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.004s	sys 0.012s
W20260709 12:25:57.897801 13646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.004s	sys 0.012s
I20260709 12:25:57.849391 24302 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:57.900282 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4689) took 1987 ms. Trace:
W20260709 12:25:57.898615 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.011s	sys 0.005s
I20260709 12:25:57.891379 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:57.920228 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:57.921656 13801 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:57.878718 24446 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:25:57.919767 13721 rpcz_store.cc:276] 0709 12:25:55.913283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.870791 (+1957508us) service_pool.cc:224] Handling call
0709 12:25:57.887788 (+ 16997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.899991 (+ 12203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.900219 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:57.931182 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1690) took 1610 ms. Trace:
I20260709 12:25:57.926740 14417 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:57.935523 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1695) took 1564 ms. Trace:
W20260709 12:25:57.919346 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 590709us
W20260709 12:25:57.937125 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45860.5us
I20260709 12:25:57.938710 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:57.936272 13646 rpcz_store.cc:276] 0709 12:25:56.371695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.839547 (+467852us) service_pool.cc:224] Handling call
0709 12:25:56.840466 (+   919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.840940 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:25:57.334792 (+493852us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.934983 (+600191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.935080 (+    97us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:57.935151 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.935254 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.935439 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24193,"prepare.queue_time_us":521944,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":944849,"threads_started":2}
I20260709 12:25:57.945020 13793 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:25:57.850802 24539 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 12:25:57.932044 13645 rpcz_store.cc:276] 0709 12:25:56.320968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.752971 (+432003us) service_pool.cc:224] Handling call
0709 12:25:56.754894 (+  1923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.755268 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.755297 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.793062 (+ 37765us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.292521 (+499459us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:25:57.292575 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.292668 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.930835 (+638167us) raft_consensus.cc:1685] finished
0709 12:25:57.930866 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.931088 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":16931,"thread_start_us":442127,"threads_started":1,"wal-append.queue_time_us":779834}
I20260709 12:25:57.948163 24344 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:25:57.948948 24558 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:57.900768 24463 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:57.949694 14416 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:57.992632 24522 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.994385 24173 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:25:57.995186 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:57.995833 24173 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:57.996685 24550 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:25:57.935380 24302 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:57.900983 24442 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:57.950790 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:57.891392 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.008s	sys 0.009s
I20260709 12:25:57.960503 13721 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:25:57.944644 24446 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:57.949378 24545 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:57.892894 23901 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:57.892410 24392 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:25:57.892361 24168 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:57.975634 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.975919 24478 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:25:57.909409 24494 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:25:58.008764 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2201) took 1965 ms. Trace:
I20260709 12:25:58.027041 13721 rpcz_store.cc:276] 0709 12:25:56.043925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.930720 (+1886795us) service_pool.cc:224] Handling call
0709 12:25:57.931230 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.008432 (+ 77202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.008700 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.028766 13721 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:25:58.029476 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4694) took 1877 ms. Trace:
W20260709 12:25:58.033056 24503 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.002s
I20260709 12:25:58.029814 13721 rpcz_store.cc:276] 0709 12:25:56.152637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.028003 (+1875366us) service_pool.cc:224] Handling call
0709 12:25:58.028471 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.029189 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.029414 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.007499 24538 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:58.035190 24538 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:58.035908 24538 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:58.035972 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4697) took 1769 ms. Trace:
I20260709 12:25:58.036612 13721 rpcz_store.cc:276] 0709 12:25:56.289390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.035061 (+1745671us) service_pool.cc:224] Handling call
0709 12:25:58.035406 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.035589 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.035611 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.035652 (+    41us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:25:58.035708 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.035764 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.035906 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:58.009182 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.009s
I20260709 12:25:57.959264 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:57.992772 24557 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:25:57.997714 24173 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:25:58.010387 24499 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:25:57.991816 24452 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.118s	user 0.015s	sys 0.004s
I20260709 12:25:57.957989 24551 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:57.968983 24344 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:57.934072 24439 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.079s
W20260709 12:25:58.043500 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 749112us
I20260709 12:25:58.005057 23957 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.015748 24446 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:25:58.016161 24483 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:58.070214 24392 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:58.080641 24462 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.025908 24506 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.044287 13721 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:58.033876 24503 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.061857 24173 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:58.089177 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 416662us
I20260709 12:25:58.120937 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2202) took 1411 ms. Trace:
I20260709 12:25:58.132220 13721 rpcz_store.cc:276] 0709 12:25:56.709570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.037642 (+1328072us) service_pool.cc:224] Handling call
0709 12:25:58.038094 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.119140 (+ 81046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.120746 (+  1606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.115103 24470 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.017s
I20260709 12:25:58.088943 24539 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.131857 24480 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.002s
I20260709 12:25:58.091383 24393 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:25:58.135668 24173 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.136270 13721 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:25:58.089304 24344 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:58.136999 24173 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:58.134353 24392 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.138388 24168 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.140025 24503 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:58.140794 24503 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:25:58.113366 24483 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:58.137053 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2203) took 1268 ms. Trace:
W20260709 12:25:58.119861 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 392860us
W20260709 12:25:58.088433 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 215263us
I20260709 12:25:58.092391 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1706) took 1772 ms. Trace:
I20260709 12:25:58.142549 13721 rpcz_store.cc:276] 0709 12:25:56.868967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.135505 (+1266538us) service_pool.cc:224] Handling call
0709 12:25:58.135939 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.136749 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.136968 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:58.145220 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 257429us
I20260709 12:25:58.144744 13799 rpcz_store.cc:276] 0709 12:25:56.320330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.066939 (+746609us) service_pool.cc:224] Handling call
0709 12:25:57.067273 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.067584 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:57.650983 (+583399us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.088027 (+437044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.088112 (+    85us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:58.088203 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.088297 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.092286 (+  3989us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51792,"prepare.queue_time_us":667224,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":1442,"raft.queue_time_us":48568,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":13740,"thread_start_us":924203,"threads_started":2}
I20260709 12:25:58.149767 13721 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:25:58.106665 24457 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:58.150481 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4416) took 1168 ms. Trace:
I20260709 12:25:58.086923 24452 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.154183 13721 rpcz_store.cc:276] 0709 12:25:56.981838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.149076 (+1167238us) service_pool.cc:224] Handling call
0709 12:25:58.149420 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.150215 (+   795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.150426 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:58.162098 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 165879us
I20260709 12:25:58.134750 24480 consensus_queue.cc:1243] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 28.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:25:58.115599 24499 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:25:58.195431 13719 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:25:58.164274 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2196) took 2811 ms. Trace:
I20260709 12:25:58.196780 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4687) took 2296 ms. Trace:
I20260709 12:25:58.188417 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4688) took 2281 ms. Trace:
I20260709 12:25:58.197306 13718 rpcz_store.cc:276] 0709 12:25:55.899942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.807324 (+1907382us) service_pool.cc:224] Handling call
0709 12:25:57.882552 (+ 75228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.883291 (+   739us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:57.943876 (+ 60585us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.196238 (+252362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.196320 (+    82us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:58.196393 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.196496 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.196688 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":647,"prepare.queue_time_us":114687,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"thread_start_us":251974,"threads_started":1}
I20260709 12:25:58.201283 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.132349 24050 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:25:58.196056 13727 rpcz_store.cc:276] 0709 12:25:55.353041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.848795 (+1495754us) service_pool.cc:224] Handling call
0709 12:25:56.849274 (+   479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.849577 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:57.643548 (+793971us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.163701 (+520153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.163919 (+   218us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:58.163978 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.164065 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.164210 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":641165,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":3685,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":54,"thread_start_us":1194701,"threads_started":3,"wal-append.queue_time_us":472729}
W20260709 12:25:58.163822 24560 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:25:58.212977 13718 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:25:58.197803 13720 rpcz_store.cc:276] 0709 12:25:55.906871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.870661 (+1963790us) service_pool.cc:224] Handling call
0709 12:25:57.888757 (+ 18096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.888980 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:25:57.973210 (+ 84230us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.187746 (+214536us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.187912 (+   166us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:25:58.187992 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.188110 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.188325 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65349,"prepare.queue_time_us":113853,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":28965,"thread_start_us":193259,"threads_started":1}
I20260709 12:25:58.210018 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.206192 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4429) took 1145 ms. Trace:
W20260709 12:25:58.211917 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55025.2us
I20260709 12:25:58.211993 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.213280 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.134296 24539 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.146327 24510 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:58.217046 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.238117 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:58.238662 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.215838 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4428) took 1155 ms. Trace:
I20260709 12:25:58.229684 13727 rpcz_store.cc:276] 0709 12:25:57.060471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.205021 (+1144550us) service_pool.cc:224] Handling call
0709 12:25:58.205462 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.205724 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.205768 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.205836 (+    68us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:25:58.205887 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.205966 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.206113 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.242695 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2205) took 1031 ms. Trace:
I20260709 12:25:58.243389 13727 rpcz_store.cc:276] 0709 12:25:57.211811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.241406 (+1029595us) service_pool.cc:224] Handling call
0709 12:25:58.241827 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.242074 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.242112 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.242175 (+    63us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:25:58.242221 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.242466 (+   245us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.242621 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.231613 13800 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:25:58.239593 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.199527 24506 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.312740 24539 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.240123 13718 rpcz_store.cc:276] 0709 12:25:57.060190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.200497 (+1140307us) service_pool.cc:224] Handling call
0709 12:25:58.210381 (+  9884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.214492 (+  4111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.215778 (+  1286us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:58.239711 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.247581 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 103613us
I20260709 12:25:58.232977 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:58.374444 24581 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:25:58.286213 24340 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.373205 24510 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:58.387836 24344 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.441161 24344 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.318588 23957 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.252957 13795 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:58.257753 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:58.294441 24568 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:25:58.347436 24500 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.003s
I20260709 12:25:58.447192 24500 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.340939 24452 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.274518 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.247017 24456 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.234870 24503 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:25:58.152064 23901 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:58.404742 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 2.889s (SENT) [suppressed 50 similar messages]
W20260709 12:25:58.452733 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48037.7us
I20260709 12:25:58.340652 24442 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:58.290066 24563 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:58.299394 24551 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:58.459221 24551 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:58.305281 13723 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:25:58.303280 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4432) took 1242 ms. Trace:
I20260709 12:25:58.372570 24443 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:58.463418 24443 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.463953 24443 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:58.366643 24556 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:58.466557 24556 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:58.467105 24556 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:58.365363 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 414744us
W20260709 12:25:58.368242 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 409033us
I20260709 12:25:58.471722 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:58.208863 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:58.376562 24483 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:58.341866 24562 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:25:58.297111 24302 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.312656 13721 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:25:58.476563 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2207) took 1142 ms. Trace:
I20260709 12:25:58.477056 13721 rpcz_store.cc:276] 0709 12:25:57.334506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.311630 (+977124us) service_pool.cc:224] Handling call
0709 12:25:58.312119 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.476183 (+164064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.476486 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.340061 24520 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:25:58.317170 24570 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.184s	user 0.002s	sys 0.000s
I20260709 12:25:58.273890 13724 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:58.257231 13727 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:58.478546 13719 rpcz_store.cc:276] 0709 12:25:57.061254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.203649 (+142395us) service_pool.cc:224] Handling call
0709 12:25:58.303003 (+1099354us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54000f7154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:58.303207 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278192384}
I20260709 12:25:58.479693 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2206) took 1219 ms. Trace:
I20260709 12:25:58.480173 13727 rpcz_store.cc:276] 0709 12:25:57.260406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.248442 (+988036us) service_pool.cc:224] Handling call
0709 12:25:58.256701 (+  8259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.479333 (+222632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.479613 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:58.255108 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:58.241801 24561 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 12:25:58.258845 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 111753us
I20260709 12:25:58.481606 13801 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:58.340255 24575 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.482758 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3295) took 1154 ms. Trace:
I20260709 12:25:58.482851 24575 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.483283 13647 rpcz_store.cc:276] 0709 12:25:57.328116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.475671 (+147555us) service_pool.cc:224] Handling call
0709 12:25:57.486623 (+ 10952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.487211 (+   588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.487245 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:57.488077 (+   832us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.807136 (+319059us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:57.807201 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.807300 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.482530 (+675230us) raft_consensus.cc:1685] finished
0709 12:25:58.482560 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.482693 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2346,"raft.queue_time_us":248769,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":3110,"thread_start_us":318462,"threads_started":1,"wal-append.queue_time_us":522805}
I20260709 12:25:58.485350 24575 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:58.393136 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 249251us
W20260709 12:25:58.426440 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 163818us
I20260709 12:25:58.431438 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:25:58.487079 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56060.7us
I20260709 12:25:58.207968 24480 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.204s
I20260709 12:25:58.490293 24585 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:25:58.490429 24302 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:25:58.490931 24585 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:58.491927 24585 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 33.25, Last appended by leader: 25, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.492198 24302 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:58.495642 24478 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:58.494714 13801 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:25:58.495071 24456 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:25:58.496225 24478 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.496790 24456 consensus_queue.cc:1243] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer b4f6bfcda63148e4965ca02c639336f1 log is divergent from this leader: its last log entry 25.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:25:58.498474 24573 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:25:58.485512 24520 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.398056 24412 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:25:58.501912 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 269027us
I20260709 12:25:58.422446 24169 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:58.422494 24571 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:25:58.503396 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:25:58.508170 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 357957us
I20260709 12:25:58.432480 24462 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:58.512570 13800 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:25:58.509650 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:58.487104 24483 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.436306 24422 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:58.506868 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:58.521165 13797 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 40.
I20260709 12:25:58.516242 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:58.517122 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4693) took 2585 ms. Trace:
I20260709 12:25:58.521795 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3276) took 2423 ms. Trace:
I20260709 12:25:58.521961 13724 rpcz_store.cc:276] 0709 12:25:55.931661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.880740 (+1949079us) service_pool.cc:224] Handling call
0709 12:25:57.908793 (+ 28053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.478955 (+570162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.516881 (+ 37926us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54003fedd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:58.517062 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838982528}
I20260709 12:25:58.523275 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4686) took 2625 ms. Trace:
I20260709 12:25:58.522228 13795 rpcz_store.cc:276] 0709 12:25:56.098116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.052093 (+953977us) service_pool.cc:224] Handling call
0709 12:25:57.640150 (+588057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.442605 (+802455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.514694 (+ 72089us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:25:58.521623 (+  6929us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b540006b150. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:58.521744 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750789248}
I20260709 12:25:58.523684 13723 rpcz_store.cc:276] 0709 12:25:55.898475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.804845 (+1906370us) service_pool.cc:224] Handling call
0709 12:25:58.305000 (+500155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.462785 (+157785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.523087 (+ 60302us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54000f7150. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:58.523221 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148937472}
I20260709 12:25:58.424512 24506 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:58.503207 24522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.004s	sys 0.010s
W20260709 12:25:58.513949 24426 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.029s	user 0.015s	sys 0.000s
I20260709 12:25:58.506994 24555 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:58.544661 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:58.536588 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:58.545449 13645 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:58.526598 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:58.544492 13648 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:58.505919 24169 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:58.513737 24571 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.545873 13618 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:58.537503 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:58.549413 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:58.555888 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.543844 24522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.005s	sys 0.011s
I20260709 12:25:58.558988 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:58.558480 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4690) took 2639 ms. Trace:
I20260709 12:25:58.505462 24520 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.525413 24344 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:58.513958 24579 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:25:58.562872 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.576063 13646 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:25:58.565091 13725 rpcz_store.cc:276] 0709 12:25:55.919086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.871812 (+1952726us) service_pool.cc:224] Handling call
0709 12:25:57.897652 (+ 25840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.897902 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:58.211156 (+313254us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.557960 (+346804us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.558041 (+    81us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:58.558109 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.558204 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.558396 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100292,"prepare.queue_time_us":153317,"prepare.run_cpu_time_us":296,"prepare.run_wall_time_us":18921,"raft.queue_time_us":16614,"thread_start_us":541814,"threads_started":2}
I20260709 12:25:58.566588 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:58.540889 24422 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.568238 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:58.570134 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:58.550586 24538 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.514s	user 0.016s	sys 0.000s
I20260709 12:25:58.574182 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:58.511152 24550 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.044s
I20260709 12:25:58.538928 24194 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:25:58.545159 24478 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.545176 24555 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.509871 24412 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.609812 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.610646 13801 raft_consensus.cc:2368] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:25:58.543391 24548 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:58.553268 24557 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:25:58.548157 24316 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:25:58.623080 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 134464us
I20260709 12:25:58.522437 24462 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.593724 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:58.595873 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:58.598518 13797 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:58.598838 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:58.560645 24168 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:25:58.598838 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.754513 13795 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:25:58.606284 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:58.608744 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:58.609113 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.756902 13793 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:58.580938 24499 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.572486 24439 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:58.571879 24344 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.614933 13796 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: []
W20260709 12:25:58.617120 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 110380us
I20260709 12:25:58.612329 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.744060 24557 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:25:58.789398 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 230477us
I20260709 12:25:58.575826 24579 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:58.591645 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:58.602398 24538 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.798385 24538 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.811794 24302 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.812582 24500 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:58.802141 23957 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:58.813866 23957 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:25:58.811836 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.815039 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.815447 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.610841 24555 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:58.743216 14422 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:58.657159 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:58.744966 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82604.9us
I20260709 12:25:58.735796 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:58.736513 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.736795 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:58.616990 24494 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:25:58.736006 24595 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:25:58.830478 24494 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:58.830951 24452 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.831913 13795 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:58.831575 24452 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.832716 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:58.832855 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.833320 13718 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:58.833540 13795 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:58.757154 14423 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:25:58.740476 24457 consensus_meta.cc:287] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.000s
I20260709 12:25:58.549779 24426 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.837627 24426 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.838351 13726 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:25:58.785378 13794 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:25:58.787976 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:58.792455 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:58.843448 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:58.842641 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:58.845690 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57822us
I20260709 12:25:58.812628 24302 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.816213 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.852072 13725 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:58.853163 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.816704 24510 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:25:58.817514 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.819124 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:58.820391 24476 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:58.821136 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:58.785290 24168 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:58.822136 13721 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:25:58.656221 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:58.618306 24578 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:25:58.834408 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:58.840564 24442 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.831003 24494 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:25:58.841643 13726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4427) took 1781 ms (client timeout 2208 ms). Trace:
I20260709 12:25:58.845489 24555 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.847704 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:58.848522 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56151us
I20260709 12:25:58.882673 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:58.847748 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87717.6us
I20260709 12:25:58.883596 13644 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:25:58.849587 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 258102us
I20260709 12:25:58.885452 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:58.887813 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:58.794701 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 248907us
I20260709 12:25:58.850939 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.784885 24499 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.611452 24169 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.857942 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:58.941033 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51544.7us
I20260709 12:25:58.941530 13796 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:58.789032 24344 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:25:58.848671 24597 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 12:25:58.820984 23957 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.853163 14422 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:58.859208 13724 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:58.859587 13615 leader_election.cc:400] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:25:58.953586 13615 leader_election.cc:403] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Cancelling election due to peer responding with higher term
I20260709 12:25:58.862769 13719 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:25:58.859285 13727 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:58.738070 24587 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.109s
I20260709 12:25:58.834087 24452 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.753440 24194 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:25:58.866945 13652 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:58.868770 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:58.857335 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.870594 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.795317 24579 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:25:58.878996 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:58.962759 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 108172us
I20260709 12:25:58.963290 13721 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:25:58.878650 13726 rpcz_store.cc:269] 0709 12:25:57.059946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.484008 (+424062us) service_pool.cc:224] Handling call
0709 12:25:58.840227 (+1356219us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54001a5754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:25:58.841582 (+  1355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3112167168}
I20260709 12:25:58.854090 24575 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:58.857199 24439 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:58.762454 24443 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.853832 24316 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:25:58.853879 13725 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:25:58.853771 24598 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.065s
I20260709 12:25:58.913569 24412 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.883973 24578 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 40 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:58.915732 24571 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.918705 24457 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.006635 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:59.007891 24443 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.009532 13650 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: [31.17-31.17]
I20260709 12:25:59.013275 24578 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 39.26, Last appended by leader: 26, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:59.017342 24173 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.880s	user 0.015s	sys 0.001s
I20260709 12:25:59.007906 14416 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:58.920920 13693 leader_election.cc:400] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:25:59.022866 13693 leader_election.cc:403] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:58.921933 24569 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:25:58.922765 24602 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:25:58.922468 24539 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:25:58.922479 24426 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:25:58.923769 24591 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:25:58.924588 24462 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.925629 24422 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:58.928194 24601 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:25:58.795161 24556 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:58.930815 24596 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:25:58.931867 24557 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:58.933169 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:58.855299 24480 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:59.030441 24602 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.934671 24168 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:58.935557 24302 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:59.034137 24426 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:58.937147 13718 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:59.006932 24457 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.059863 24571 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:25:58.940132 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:58.942203 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 179724us
I20260709 12:25:59.095245 13797 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:25:58.942754 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:58.946162 13643 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:25:58.947412 13645 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:58.877964 13547 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 24, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:59.099843 13547 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 25, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 25 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:58.967355 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:59.103410 14422 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 40: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:25:58.967573 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:58.975033 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:59.104596 13724 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.105445 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.977974 13801 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:58.989024 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:58.996423 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:58.886960 24442 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.111833 13547 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 31, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 31 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:58.953158 24456 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.047s
I20260709 12:25:58.980640 24604 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:25:59.000732 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.991863 24393 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:25:59.001971 24594 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 12:25:59.006405 24446 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.001s
I20260709 12:25:59.134682 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:25:59.031701 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 110933us
I20260709 12:25:59.044492 13727 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:25:59.045343 13652 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:59.047991 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:59.136754 13726 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:58.960317 24478 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.069s
I20260709 12:25:59.054891 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:59.022578 23957 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:25:59.059754 13720 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.047s	sys 0.040s
I20260709 12:25:59.060067 13719 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
W20260709 12:25:59.140061 13720 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.047s	sys 0.040s
I20260709 12:25:59.141078 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2204) took 2071 ms. Trace:
I20260709 12:25:59.060220 13796 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:25:59.028092 24555 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:58.878958 24340 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:59.081980 13795 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:59.162142 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:58.936045 24506 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:59.086535 13647 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.018s	sys 0.010s
W20260709 12:25:58.958170 24538 consensus_queue.cc:1175] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" responder_term: 30 status { last_received { term: 30 index: 21 } last_committed_idx: 21 last_received_current_leader { term: 30 index: 21 } }
W20260709 12:25:59.165289 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.089365 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:58.858795 24510 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:59.167416 24556 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:59.167373 24510 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.144846 24599 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.024s
I20260709 12:25:59.168812 24510 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:25:58.939483 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:59.170531 13793 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:25:59.171397 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:59.061657 24569 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:25:59.171926 13793 raft_consensus.cc:2376] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31. Re-sending same reply.
I20260709 12:25:59.172092 24569 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:58.999459 24520 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.066635 24539 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.172781 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:59.172943 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:59.173282 24539 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.115891 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:25:59.114184 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:59.174728 13801 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:25:59.174988 24599 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:25:59.180076 24548 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:25:59.180791 24548 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:59.183051 24499 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:59.183600 24173 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:59.182070 24548 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.184096 24173 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:25:59.184473 24587 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:25:59.184746 24548 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:59.185040 24173 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.186697 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4691) took 3265 ms. Trace:
I20260709 12:25:59.186975 24173 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:59.188544 24572 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.189121 24572 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.120931 24586 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 33.25, Next index: 26, Last known committed idx: 24, Time since last communication: 0.070s
I20260709 12:25:59.123871 24393 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:59.191524 24572 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:25:59.123936 14416 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:25:59.092284 24426 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:25:59.192499 24393 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.193687 24393 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:25:59.128486 24598 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.070s
I20260709 12:25:59.197072 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:25:59.199936 24169 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:25:59.145458 24302 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:25:59.201789 24302 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:25:59.124558 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 181898us
I20260709 12:25:59.202972 13617 leader_election.cc:400] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:25:59.204842 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:59.204664 24579 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.205946 13651 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:25:59.206689 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.088688 24480 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.210532 24480 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:59.173902 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:59.191383 24586 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:25:59.137521 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:25:59.138394 13723 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:25:59.105687 24470 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.143707 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:59.222659 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85212.4us
I20260709 12:25:59.223565 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:25:59.174760 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60667.6us
I20260709 12:25:59.231540 13650 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:25:59.233009 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:59.234447 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:59.239498 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:25:59.243345 13646 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:25:59.243674 13650 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:25:59.246791 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.247727 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:59.141592 13720 rpcz_store.cc:276] 0709 12:25:57.069604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.215083 (+1145479us) service_pool.cc:224] Handling call
0709 12:25:58.244208 (+ 29125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.244444 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:25:58.388860 (+144416us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.140590 (+751730us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.140665 (+    75us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:25:59.140749 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.140847 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.141006 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":143717,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"thread_start_us":751267,"threads_started":1}
I20260709 12:25:59.257310 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:59.258296 13720 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:25:59.259311 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:59.261605 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:59.261984 13720 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:25:59.263255 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:59.263507 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:25:59.131600 24439 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:25:59.150418 24551 consensus_meta.cc:287] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.003s
I20260709 12:25:59.155958 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:25:59.157034 24592 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.364s	user 0.003s	sys 0.000s
I20260709 12:25:59.159818 24575 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:25:59.187147 13722 rpcz_store.cc:276] 0709 12:25:55.921342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.873336 (+1951994us) service_pool.cc:224] Handling call
0709 12:25:57.873744 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.885901 (+ 12157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.885985 (+    84us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.303914 (+417929us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b2c00000a78. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:25:58.304431 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.514765 (+210334us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:25:58.551469 (+ 36704us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.557705 (+  6236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.557820 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.186343 (+628523us) raft_consensus.cc:1685] finished
0709 12:25:59.186460 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.186627 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5693,"prepare.queue_time_us":34031,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":70902,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":3260,"spinlock_wait_cycles":29574016,"thread_start_us":209903,"threads_started":1,"wal-append.queue_time_us":295063}
I20260709 12:25:59.146755 13724 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:59.082921 24602 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:25:59.248783 13793 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:59.221958 24586 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:25:59.231540 24520 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:25:59.302771 13793 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:25:59.251920 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:59.119230 13547 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 25, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 25 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:59.260493 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:59.304729 13719 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:25:59.147923 13543 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 241) took 100413 ms (client timeout 15000 ms). Trace:
I20260709 12:25:59.264742 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:25:59.305369 13547 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 26, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 26 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:59.273424 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100563us
I20260709 12:25:59.274677 13721 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:25:59.278483 13726 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:25:59.175418 24599 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:25:59.284242 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.341164 13547 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 27, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:59.184913 24587 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:59.289000 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:25:59.288213 24439 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.289820 24194 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:25:59.290112 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.291150 13648 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:25:59.287887 24457 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.359107 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.291689 13797 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 34. Current term is 35. Ops: []
I20260709 12:25:59.257792 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:25:59.219684 24600 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.076s
I20260709 12:25:59.295351 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.299126 13796 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
W20260709 12:25:59.361014 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103262us
W20260709 12:25:59.299590 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.202140 24302 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:25:59.305292 13543 rpcz_store.cc:269] 0709 12:24:18.620469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.622469 (+  2000us) service_pool.cc:224] Handling call
0709 12:25:59.027212 (+100404743us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":50}
I20260709 12:25:59.337340 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.146960 24446 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.308319 24538 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.258718 24462 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:25:59.340442 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:25:59.262763 24422 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:59.343668 13547 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 29, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:59.349913 24599 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:25:59.283900 24556 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:25:59.396286 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55952.4us
I20260709 12:25:59.360607 24439 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:59.353658 13718 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:25:59.220741 24423 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.090s
I20260709 12:25:59.357651 24194 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:59.242668 24470 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.337359 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:59.308094 24550 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:59.368134 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 171162us
I20260709 12:25:59.369918 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.414743 13646 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:25:59.300257 24575 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:59.307951 24506 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:25:59.376577 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.380447 13547 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 24, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:59.347633 14417 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:25:59.381742 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:25:59.383373 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.389952 13719 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:25:59.391760 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:25:59.351491 24316 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:59.395195 13797 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:25:59.397167 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:59.284548 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:25:59.401842 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.403501 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.404520 14420 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: [36.25-36.25]
I20260709 12:25:59.362895 13693 leader_election.cc:400] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:25:59.380539 24462 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.294514 24551 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.416501 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:25:59.417423 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.418685 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.546646 13646 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:25:59.420055 24607 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:25:59.549060 13720 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:25:59.420656 24616 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:25:59.412088 24194 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:25:59.428001 24446 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.396672 24452 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:25:59.435159 24439 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.404965 24556 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:59.454469 24483 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.934s	user 0.013s	sys 0.000s
I20260709 12:25:59.454680 24426 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.583074 24556 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.583541 24426 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.585048 13652 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:25:59.458819 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:25:59.465631 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.208s (SENT)
W20260709 12:25:59.588409 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122849us
W20260709 12:25:59.465960 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.408984 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:59.506335 13725 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:59.594638 24539 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.506502 13693 leader_election.cc:403] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:59.508023 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 110946us
I20260709 12:25:59.599735 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:25:59.509431 13647 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:25:59.400010 24050 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:25:59.571720 24443 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.537724 24462 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:25:59.603027 24443 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.606482 24575 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.608247 24575 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.467988 24550 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.535547 24192 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:25:59.612545 24192 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:25:59.613130 24192 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.470182 24587 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:25:59.470048 24302 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:25:59.623358 24620 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 12:25:59.406922 24423 consensus_queue.cc:1243] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:25:59.535589 13547 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 23 to 25, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 25 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:25:59.510716 13719 pending_rounds.cc:85] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 19
I20260709 12:25:59.640973 13719 pending_rounds.cc:107] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.20
I20260709 12:25:59.641399 13719 raft_consensus.cc:2889] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.393350 24538 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.571036 24579 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:25:59.576871 24611 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:25:59.577399 24615 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.577593 24452 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:25:59.647145 24615 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.647781 24452 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.406729 24464 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.142s
I20260709 12:25:59.407663 24586 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 33.25, Last appended by leader: 25, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:59.578593 24619 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:25:59.580150 24439 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.654157 24616 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:25:59.655597 24616 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.640185 13547 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 31 to 36, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 36 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:59.685022 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 276086us
W20260709 12:25:59.677367 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.686705 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.677560 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 314807us
I20260709 12:25:59.717666 24538 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:59.718091 24443 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.686170 13547 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:25:59.719515 13547 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 28 to 31, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 31 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:59.622027 24302 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:25:59.730060 13644 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
I20260709 12:25:59.732183 13547 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 24, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 24 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:59.723618 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.742894 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.740862 13547 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 26 to 31, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 31 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:59.748332 13547 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 23. New cstate: current_term: 23 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:59.749702 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:25:59.747205 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.460246 23901 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.259s
W20260709 12:25:59.758889 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72899us
I20260709 12:25:59.764456 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:25:59.768740 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:59.771759 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:25:59.764626 24615 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:25:59.775223 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:25:59.774757 24316 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:25:59.775283 24422 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:25:59.782968 13727 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:25:59.784233 14419 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 40 index: 27. (index mismatch)
I20260709 12:25:59.786038 24614 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 25, Time since last communication: 0.000s
W20260709 12:25:59.787982 24604 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.000s
I20260709 12:25:59.788628 24604 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:25:59.796690 13792 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
W20260709 12:25:59.800957 24510 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.632s	user 0.008s	sys 0.003s
I20260709 12:25:59.801156 13727 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:25:59.801735 24462 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:25:59.802716 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.015118 13643 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:26:00.016114 13652 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 22: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:25:59.753557 13547 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 22 to 24, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 24 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:59.804325 24344 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.846s	user 0.012s	sys 0.000s
I20260709 12:25:59.720570 24522 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:25:59.804924 24571 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.699s	user 0.013s	sys 0.000s
I20260709 12:25:59.699828 24457 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:25:59.811795 24550 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.536012 24602 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:25:59.821790 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 222169us
W20260709 12:26:00.063071 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 298708us
I20260709 12:26:00.064090 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:25:59.877746 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.807811 24442 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:25:59.719732 24426 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.122s
I20260709 12:25:59.698678 24600 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.153s
W20260709 12:26:00.006671 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.711527 24412 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:00.010936 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.819907 24464 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.122s
I20260709 12:26:00.017848 13547 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 32 to 37, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 37 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:00.017853 13725 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:00.117515 13645 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:00.117802 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4725) took 1054 ms. Trace:
I20260709 12:26:00.118463 13725 rpcz_store.cc:276] 0709 12:25:59.062952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.221404 (+158452us) service_pool.cc:224] Handling call
0709 12:26:00.117594 (+896190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.028640 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.058995 24571 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
W20260709 12:26:00.028708 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.025810 24169 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.823s	user 0.011s	sys 0.009s
I20260709 12:26:00.027669 24480 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:00.025203 24604 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.055301 24393 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.861s	user 0.015s	sys 0.002s
I20260709 12:26:00.082353 24050 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:00.056219 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:00.137327 13652 raft_consensus.cc:2376] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26. Re-sending same reply.
W20260709 12:26:00.122171 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58198.3us
I20260709 12:26:00.123780 13547 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 21, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 21 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:26:00.126075 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.063519 24614 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:00.140578 13547 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:00.064356 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.135470 13649 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:00.028334 24624 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.114941 24476 consensus_meta.cc:287] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.245s	user 0.007s	sys 0.010s
I20260709 12:26:00.139981 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.099283 24522 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.144088 13794 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:00.139783 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.045615 24344 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.146675 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.150775 13723 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:00.150879 13652 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
W20260709 12:26:00.151955 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.063144 24457 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:00.137331 24173 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.950s	user 0.012s	sys 0.008s
W20260709 12:26:00.157159 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92869.6us
I20260709 12:26:00.177484 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.159235 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:00.119539 24539 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:26:00.133379 24510 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:00.145589 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:00.134775 24550 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:00.146540 24480 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.194924 13648 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 32: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:00.145529 24558 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.130347 24604 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.200732 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:00.203207 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.202632 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.209059 24558 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.209404 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.210928 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.211874 13646 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:00.127522 24412 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.215667 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.216603 13646 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:26:00.218344 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:00.183334 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:00.172360 24050 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.172137 24464 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:26:00.197890 23957 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.041s	user 0.013s	sys 0.003s
I20260709 12:26:00.144551 24522 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:00.202744 24494 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.264s	user 0.017s	sys 0.000s
I20260709 12:26:00.224931 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:00.226774 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 403680us
I20260709 12:26:00.207011 24499 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:00.233021 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.238418 24499 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.238742 24503 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:00.239153 24499 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:00.239231 24503 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:00.242642 24503 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:00.233528 13652 raft_consensus.cc:2368] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:26:00.150532 24614 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:00.223318 24168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.015s	sys 0.010s
I20260709 12:26:00.215054 24470 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.154155 24624 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.236447 13646 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:00.195533 24590 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:26:00.226694 24510 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:00.226542 24610 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:00.232308 24476 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:00.320371 14420 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: [36.25-36.25]
I20260709 12:26:00.207571 24550 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:00.265148 24168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.016s	sys 0.010s
I20260709 12:26:00.273461 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.270125 24510 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.267882 24520 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:00.264196 24494 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.310806 24442 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.325139 24610 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.336050 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.325451 24476 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.268280 24452 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:00.344309 13724 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.002s	sys 0.026s
I20260709 12:26:00.347400 13646 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:00.347081 13724 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.002s	sys 0.026s
W20260709 12:26:00.340339 24629 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:26:00.339399 24510 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:00.348836 24476 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:00.342109 24050 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.367769 24412 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.367969 24610 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.368543 24412 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:00.369189 24412 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:00.371595 24632 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:26:00.372210 24494 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.373011 24494 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20260709 12:26:00.374320 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4717) took 1441 ms. Trace:
I20260709 12:26:00.374944 24602 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.375744 24602 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:00.375792 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4719) took 1442 ms. Trace:
I20260709 12:26:00.349604 24462 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.346433 24520 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.349956 24480 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.376492 13721 rpcz_store.cc:276] 0709 12:25:58.933689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.307669 (+373980us) service_pool.cc:224] Handling call
0709 12:25:59.308325 (+   656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.376020 (+ 67695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.376070 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.376878 (+   808us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.537604 (+160726us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:00.122438 (+584834us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.122737 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.122827 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.375522 (+252695us) raft_consensus.cc:1685] finished
0709 12:26:00.375545 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.375729 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7301,"prepare.queue_time_us":269535,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":52495,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":14438,"thread_start_us":655410,"threads_started":3,"wal-append.queue_time_us":361211}
I20260709 12:26:00.379189 13721 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:00.349879 24442 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.379477 24457 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.380033 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2216) took 1272 ms. Trace:
I20260709 12:26:00.380460 13721 rpcz_store.cc:276] 0709 12:25:59.107354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.378721 (+1271367us) service_pool.cc:224] Handling call
0709 12:26:00.379029 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.379645 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.379968 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.381631 24442 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.379221 24520 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.384197 13722 rpcz_store.cc:276] 0709 12:25:58.933170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.300677 (+367507us) service_pool.cc:224] Handling call
0709 12:25:59.301070 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:59.314296 (+ 13226us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:25:59.508194 (+193898us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.275677 (+767483us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:00.330204 (+ 54527us) log.cc:844] Serialized 52 byte log entry
0709 12:26:00.330455 (+   251us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:00.340928 (+ 10473us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.341230 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.341337 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.373946 (+ 32609us) raft_consensus.cc:1685] finished
0709 12:26:00.373972 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.374166 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11320,"prepare.queue_time_us":373992,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":98,"raft.queue_time_us":100734,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":65279,"thread_start_us":718504,"threads_started":3,"wal-append.queue_time_us":572927}
I20260709 12:26:00.401674 13722 raft_consensus.cc:1275] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 24. Preceding OpId from leader: term: 33 index: 25. (index mismatch)
I20260709 12:26:00.406740 13798 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
W20260709 12:26:00.374712 24633 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:26:00.413408 13722 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: [33.23-33.23]
I20260709 12:26:00.374482 24476 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.355554 24452 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.442821 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2211) took 1372 ms. Trace:
W20260709 12:26:00.418911 24548 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.234s	user 0.013s	sys 0.004s
I20260709 12:26:00.376585 24602 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:00.446251 14391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.009s	sys 0.001s
W20260709 12:26:00.454190 14391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.010s	sys 0.001s
W20260709 12:26:00.454013 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 115924us
I20260709 12:26:00.459599 13722 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:26:00.462210 13793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.002s	sys 0.009s
W20260709 12:26:00.462817 13793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.003s	sys 0.009s
I20260709 12:26:00.463075 13650 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:00.465106 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:00.466640 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.437282 24344 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.467886 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:00.477975 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.481945 14417 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:00.482870 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4830) took 1451 ms. Trace:
I20260709 12:26:00.483100 24548 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.486359 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.486570 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.487215 14424 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:00.487951 24302 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.002s
I20260709 12:26:00.488715 24548 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.492043 24624 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.492841 24168 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:00.494390 24344 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.494984 24344 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.495760 23901 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.496411 23901 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.483434 13793 rpcz_store.cc:276] 0709 12:25:59.032066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.303600 (+271534us) service_pool.cc:224] Handling call
0709 12:25:59.303897 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.304108 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 25
0709 12:25:59.730702 (+426594us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.481326 (+750624us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.481448 (+   122us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:00.481566 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.481716 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.482790 (+  1074us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17960,"prepare.queue_time_us":810546,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":71,"raft.queue_time_us":18131,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":3261,"thread_start_us":1113610,"threads_started":2}
I20260709 12:26:00.500095 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1701) took 1268 ms. Trace:
I20260709 12:26:00.500828 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:00.503682 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:00.504844 13793 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:00.505052 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:00.505836 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:00.507254 24637 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:00.507364 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:00.510409 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4721) took 1576 ms. Trace:
I20260709 12:26:00.510850 13726 rpcz_store.cc:276] 0709 12:25:58.948837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.310243 (+361406us) service_pool.cc:224] Handling call
0709 12:25:59.351902 (+ 41659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.355567 (+  3665us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:59.647224 (+291657us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:25:59.712752 (+ 65528us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.301022 (+588270us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.301716 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.301906 (+   190us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:00.301953 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.302052 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.510033 (+207981us) raft_consensus.cc:1685] finished
0709 12:26:00.510169 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.510340 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45745,"prepare.queue_time_us":450996,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":40866,"raft.run_cpu_time_us":2457,"raft.run_wall_time_us":280389,"spinlock_wait_cycles":44012288,"thread_start_us":736810,"threads_started":2,"wal-append.queue_time_us":698621}
I20260709 12:26:00.512864 24558 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.515333 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5316) took 1204 ms. Trace:
I20260709 12:26:00.515741 13648 rpcz_store.cc:276] 0709 12:25:59.298684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.381374 (+ 82690us) service_pool.cc:224] Handling call
0709 12:26:00.502600 (+1121226us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54001e7654. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.502813 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574688512}
I20260709 12:26:00.517035 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.500614 13645 rpcz_store.cc:276] 0709 12:25:59.231867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.233640 (+  1773us) service_pool.cc:224] Handling call
0709 12:25:59.233962 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.118174 (+884212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.499777 (+381603us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54001473d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:00.500015 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031349632}
I20260709 12:26:00.520038 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:00.520807 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 280903us
I20260709 12:26:00.507323 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1717) took 1945 ms. Trace:
I20260709 12:26:00.521901 13800 rpcz_store.cc:276] 0709 12:25:58.562338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.562532 (+   194us) service_pool.cc:224] Handling call
0709 12:26:00.507102 (+1944570us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.507263 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4041656192}
I20260709 12:26:00.478639 13722 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:00.479000 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.517658 13648 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:00.534365 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5317) took 1235 ms. Trace:
I20260709 12:26:00.536753 13645 rpcz_store.cc:276] 0709 12:25:59.298970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.533206 (+1234236us) service_pool.cc:224] Handling call
0709 12:26:00.533638 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.533891 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.533931 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.533992 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:00.534040 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.534124 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.534284 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.536098 13648 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:00.541280 13645 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:26:00.553323 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5319) took 1253 ms. Trace:
I20260709 12:26:00.482089 24615 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:00.544441 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2212) took 1460 ms. Trace:
I20260709 12:26:00.558012 13719 rpcz_store.cc:276] 0709 12:25:59.084252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.510006 (+425754us) service_pool.cc:224] Handling call
0709 12:25:59.510475 (+   469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:59.642677 (+132202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.642727 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:59.643502 (+   775us) log.cc:844] Serialized 52 byte log entry
0709 12:26:00.075501 (+431999us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:00.075569 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.075696 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.544151 (+468455us) raft_consensus.cc:1685] finished
0709 12:26:00.544176 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.544365 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3223,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":424591,"threads_started":1,"wal-append.queue_time_us":551706}
I20260709 12:26:00.493562 24168 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.546195 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:00.552269 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5318) took 1253 ms. Trace:
I20260709 12:26:00.563690 14423 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:26:00.564010 13648 rpcz_store.cc:276] 0709 12:25:59.299216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.535290 (+1236074us) service_pool.cc:224] Handling call
0709 12:26:00.535831 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.551408 (+ 15577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.552183 (+   775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.554248 13645 rpcz_store.cc:276] 0709 12:25:59.299472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.540411 (+1240939us) service_pool.cc:224] Handling call
0709 12:26:00.540828 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.552966 (+ 12138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.553251 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.531744 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.569615 13645 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:26:00.571262 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5320) took 1271 ms. Trace:
I20260709 12:26:00.571823 13645 rpcz_store.cc:276] 0709 12:25:59.299742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.568903 (+1269161us) service_pool.cc:224] Handling call
0709 12:26:00.569280 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.570884 (+  1604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.571183 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.569669 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:00.521227 24604 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.581130 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.582916 13722 raft_consensus.cc:2393] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 23: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:00.584465 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.584652 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:00.577207 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.586045 13726 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:26:00.565912 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.587303 13648 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:00.592213 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.593257 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.594282 13726 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:00.601456 13645 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:26:00.603883 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.480304 24503 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.603910 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5322) took 1286 ms. Trace:
W20260709 12:26:00.606034 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.620456 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:00.620183 24192 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.006s	user 0.015s	sys 0.000s
I20260709 12:26:00.626959 24192 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:00.627632 24635 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:26:00.628191 13800 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:00.629186 24604 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:00.629354 24192 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.629853 24604 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.633044 24604 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.633690 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.634531 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:00.635243 13648 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
W20260709 12:26:00.635392 24642 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:26:00.636983 24618 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.035s	user 0.005s	sys 0.000s
W20260709 12:26:00.641973 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.642288 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2214) took 1548 ms. Trace:
I20260709 12:26:00.642719 13720 rpcz_store.cc:276] 0709 12:25:59.094152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.684586 (+590434us) service_pool.cc:224] Handling call
0709 12:25:59.684972 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.685378 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.685409 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.711874 (+ 26465us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.276453 (+564579us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:00.276512 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.276796 (+   284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.642020 (+365224us) raft_consensus.cc:1685] finished
0709 12:26:00.642045 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.642207 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4221,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":36129,"thread_start_us":264914,"threads_started":1,"wal-append.queue_time_us":687650}
W20260709 12:26:00.645951 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3306) took 1888 ms (client timeout 2041 ms). Trace:
W20260709 12:26:00.647349 13794 rpcz_store.cc:269] 0709 12:25:58.757892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.842971 (+ 85079us) service_pool.cc:224] Handling call
0709 12:26:00.645638 (+1802667us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.645866 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2987174272}
I20260709 12:26:00.647805 13801 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:00.579871 24522 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:26:00.646920 13792 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3290) took 2035 ms (client timeout 1759 ms). Trace:
W20260709 12:26:00.648874 13792 rpcz_store.cc:269] 0709 12:25:58.611617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.611741 (+   124us) service_pool.cc:224] Handling call
0709 12:26:00.646725 (+2034984us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540012b454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.646842 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1738201472}
I20260709 12:26:00.590715 24539 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:00.651024 13794 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 39 index: 26. Preceding OpId from leader: term: 40 index: 27. (index mismatch)
I20260709 12:26:00.658914 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:00.619347 13645 rpcz_store.cc:276] 0709 12:25:59.317854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.572891 (+1255037us) service_pool.cc:224] Handling call
0709 12:26:00.573257 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.603599 (+ 30342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.603836 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.668848 13645 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:00.669688 13792 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:00.619674 24614 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.542801 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:00.574717 24499 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.619927 24412 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.675212 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.679068 13720 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:00.629390 24643 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:00.680478 13651 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
W20260709 12:26:00.672199 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 144553us
I20260709 12:26:00.682770 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.625377 24648 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:26:00.674053 24599 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.265s	user 0.008s	sys 0.007s
I20260709 12:26:00.647655 24590 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:00.658448 23901 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.650904 24653 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:00.696455 24510 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.697897 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3304) took 1404 ms. Trace:
I20260709 12:26:00.669059 24192 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 12:26:00.698355 13650 rpcz_store.cc:276] 0709 12:25:59.293732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.376261 (+ 82529us) service_pool.cc:224] Handling call
0709 12:26:00.697711 (+1321450us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.697829 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2198786816}
I20260709 12:26:00.699883 13651 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:00.701300 24503 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:00.702701 24575 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 1.094s	user 0.016s	sys 0.001s
I20260709 12:26:00.703649 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:00.704528 24652 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:00.706801 24602 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.707861 24483 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:00.708335 24483 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.709209 24483 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.17, Last appended by leader: 17, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.716179 24483 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:00.719651 24499 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.718109 23901 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.720489 23901 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.722980 23901 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.725621 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:00.725260 24522 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:00.735097 24551 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.726364 24499 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:26:00.726922 24658 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:26:00.727249 24500 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:00.735442 24522 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:00.736526 24522 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.728003 24344 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.737564 24522 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.673494 24539 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.738628 24586 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:00.734061 24548 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:00.741289 13727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4713) took 1809 ms (client timeout 2203 ms). Trace:
W20260709 12:26:00.741897 13727 rpcz_store.cc:269] 0709 12:25:58.932115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.261385 (+329270us) service_pool.cc:224] Handling call
0709 12:26:00.741046 (+1479661us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400157cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.741221 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1197473024}
I20260709 12:26:00.743532 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.744419 13727 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:00.746524 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3302) took 1453 ms. Trace:
I20260709 12:26:00.747148 13644 rpcz_store.cc:276] 0709 12:25:59.293194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.689690 (+396496us) service_pool.cc:224] Handling call
0709 12:25:59.729799 (+ 40109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.736879 (+  7080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.746014 (+1009135us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b54003159a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393b6bdc0 00007f5393bf7a02 00007f5393bf3dd6 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:26:00.746444 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59788672}
I20260709 12:26:00.748533 24556 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:00.749015 24556 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:00.749872 24556 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 36.25, Last appended by leader: 25, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.752184 13797 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1724) took 1068 ms. Trace:
I20260709 12:26:00.752743 24556 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:00.753384 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.755493 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:00.686045 24194 consensus_meta.cc:287] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.069s	user 0.008s	sys 0.008s
I20260709 12:26:00.731125 24452 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.735517 24590 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.743943 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3295) took 2126 ms. Trace:
I20260709 12:26:00.780277 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:00.793815 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94496.7us
I20260709 12:26:00.752640 13797 rpcz_store.cc:276] 0709 12:25:59.683714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.684180 (+   466us) service_pool.cc:224] Handling call
0709 12:25:59.733257 (+ 49077us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:59.887275 (+154018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.011940 (+124665us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:00.027263 (+ 15323us) log.cc:844] Serialized 52 byte log entry
0709 12:26:00.426645 (+399382us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:00.443695 (+ 17050us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.444954 (+  1259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.445076 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.505326 (+ 60250us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b18000f1a80. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:00.751783 (+246457us) raft_consensus.cc:1685] finished
0709 12:26:00.751899 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.752115 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":188869,"prepare.queue_time_us":16829,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":2918,"raft.queue_time_us":94661,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":90840,"spinlock_wait_cycles":51969280,"thread_start_us":327263,"threads_started":1,"wal-append.queue_time_us":394509}
I20260709 12:26:00.794502 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.795295 14422 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:26:00.783392 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.796993 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.787395 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3311) took 1029 ms. Trace:
I20260709 12:26:00.797613 13652 rpcz_store.cc:276] 0709 12:25:59.761394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.160473 (+399079us) service_pool.cc:224] Handling call
0709 12:26:00.787207 (+626734us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.787339 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126254464}
I20260709 12:26:00.793881 13801 rpcz_store.cc:276] 0709 12:25:58.617480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.253452 (+635972us) service_pool.cc:224] Handling call
0709 12:26:00.647424 (+1393972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.648223 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.743585 (+ 95362us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540012b450. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.743837 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3205052672}
I20260709 12:26:00.685803 24558 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:00.787676 13726 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:00.787968 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:00.792719 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67170.9us
W20260709 12:26:00.785640 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.801774 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:00.804734 14423 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: [36.25-36.25]
I20260709 12:26:00.805614 24500 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:00.805529 24192 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:00.806633 24500 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:00.800447 23957 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:00.798601 24499 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:26:00.807670 23957 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.808135 23957 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:00.811898 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:00.812885 24192 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:00.807106 24539 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.813331 24192 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.793272 24653 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.814145 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.817220 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.818015 13719 raft_consensus.cc:2410] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 29 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:26:00.790282 24439 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:00.818970 24439 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.799662 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:00.821205 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.800268 24194 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.822219 24650 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:00.822455 24624 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:00.823899 24194 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.824668 13796 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:00.826745 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4835) took 1570 ms. Trace:
I20260709 12:26:00.827450 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2217) took 1140 ms. Trace:
I20260709 12:26:00.827116 13796 rpcz_store.cc:276] 0709 12:25:59.257410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.367907 (+110497us) service_pool.cc:224] Handling call
0709 12:25:59.400987 (+ 33080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.825057 (+1424070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.826459 (+  1402us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54001004d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:00.826676 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3273644416}
I20260709 12:26:00.827854 13721 rpcz_store.cc:276] 0709 12:25:59.687352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.381331 (+693979us) service_pool.cc:224] Handling call
0709 12:26:00.381755 (+   424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:00.382182 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.382219 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:00.383742 (+  1523us) log.cc:844] Serialized 52 byte log entry
0709 12:26:00.672969 (+289227us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:00.673027 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.674044 (+  1017us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.827196 (+153152us) raft_consensus.cc:1685] finished
0709 12:26:00.827225 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.827374 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":808,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":118,"thread_start_us":288730,"threads_started":1,"wal-append.queue_time_us":441221}
I20260709 12:26:00.829527 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.830191 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.800277 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:00.830945 13721 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:00.831171 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:00.818389 24192 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:00.832106 13797 raft_consensus.cc:2376] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28. Re-sending same reply.
I20260709 12:26:00.834214 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:00.831704 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4718) took 1898 ms. Trace:
W20260709 12:26:00.835158 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55022.6us
I20260709 12:26:00.836637 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:00.838447 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:00.789238 24648 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.793440 24647 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.892132 24647 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.859455 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:00.895119 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:00.896704 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:00.897469 24624 raft_consensus.cc:2764] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
W20260709 12:26:00.859887 13649 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1704) took 1627 ms (client timeout 1713 ms). Trace:
W20260709 12:26:00.898442 13649 rpcz_store.cc:269] 0709 12:25:59.232951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.234810 (+  1859us) service_pool.cc:224] Handling call
0709 12:26:00.859652 (+1624842us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400199954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.859806 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821618944}
I20260709 12:26:00.881035 24344 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.018s
I20260709 12:26:00.899734 24412 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:00.900152 24412 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.900516 24412 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:00.881326 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:00.860996 13723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4723) took 1861 ms (client timeout 1966 ms). Trace:
I20260709 12:26:00.898453 24624 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:26:00.902501 13723 rpcz_store.cc:269] 0709 12:25:58.999864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.251634 (+251770us) service_pool.cc:224] Handling call
0709 12:26:00.860777 (+1609143us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b5400174154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:00.860917 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909905792}
I20260709 12:26:00.903786 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.815647 24590 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.904913 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.885568 24340 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.906682 24340 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.864477 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:00.692335 24599 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.837271 13724 rpcz_store.cc:276] 0709 12:25:58.933420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.301801 (+368381us) service_pool.cc:224] Handling call
0709 12:25:59.302212 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.399108 (+ 96896us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:00.350604 (+951496us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.683578 (+332974us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.685651 (+  2073us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.685812 (+   161us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:00.688881 (+  3069us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.778556 (+ 89675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.778673 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.831433 (+ 52760us) raft_consensus.cc:1685] finished
0709 12:26:00.831458 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.831622 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184129,"prepare.queue_time_us":411779,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":121537,"raft.run_cpu_time_us":505,"raft.run_wall_time_us":58901,"spinlock_wait_cycles":7308160,"thread_start_us":1055113,"threads_started":3,"wal-append.queue_time_us":326428}
I20260709 12:26:00.792428 24522 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:00.871397 24452 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:00.871659 24446 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.974849 24446 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.887262 24423 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:00.979197 24423 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:00.979640 24423 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:00.980764 24423 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:00.888159 24655 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:00.984727 24569 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:00.986434 24569 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:00.873919 24476 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.874363 24610 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:01.026547 24610 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.026763 24647 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.027225 24610 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.031287 24653 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:01.033398 24439 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.042297 24423 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:00.977977 24446 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:01.047214 13643 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1702) took 1814 ms (client timeout 1683 ms). Trace:
W20260709 12:26:01.047907 13643 rpcz_store.cc:269] 0709 12:25:59.232270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.234163 (+  1893us) service_pool.cc:224] Handling call
0709 12:26:01.046908 (+1812745us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54001b6a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.047105 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274727808}
I20260709 12:26:00.868708 13720 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:26:00.836594 24646 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:26:00.934290 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.049713 13722 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:01.050616 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2215) took 1944 ms. Trace:
I20260709 12:26:01.050833 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.934983 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:00.910470 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:01.052682 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 142295us
I20260709 12:26:01.052745 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.053256 24464 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:01.053642 24464 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:01.053947 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:01.051569 13722 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.054607 24464 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.055783 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:01.051163 13725 rpcz_store.cc:276] 0709 12:25:59.106355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.212563 (+1106208us) service_pool.cc:224] Handling call
0709 12:26:00.213031 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.213504 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.213557 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.214188 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.481133 (+266945us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:00.481180 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.481306 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.050088 (+568782us) raft_consensus.cc:1685] finished
0709 12:26:01.050113 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.050525 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1195,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":7268,"thread_start_us":249661,"threads_started":1,"wal-append.queue_time_us":407255}
I20260709 12:26:00.910948 13721 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:01.056967 24586 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:01.057379 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.057688 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.058019 13725 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:01.058878 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.059517 13721 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:26:00.807082 24500 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.860421 24494 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:01.060397 24494 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.060717 24500 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.062388 24500 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:01.062687 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.063846 24548 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:00.854935 24558 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:00.943495 24456 consensus_queue.cc:1048] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:00.935273 24393 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:26:00.935482 24585 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.010s
I20260709 12:26:00.945863 14418 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:00.947192 24599 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:01.074782 24494 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:00.932042 24648 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.075361 24550 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.949342 24340 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:00.952493 24622 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.324s	user 0.003s	sys 0.000s
I20260709 12:26:01.055805 24464 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:00.957016 24510 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:00.957304 14416 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:00.958860 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.078806 14422 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:00.908967 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.080415 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.080919 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:00.880088 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:00.898001 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:00.912165 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:00.910192 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.965623 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:00.451354 13718 rpcz_store.cc:276] 0709 12:25:59.070379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.402167 (+331788us) service_pool.cc:224] Handling call
0709 12:25:59.406384 (+  4217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.406731 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:25:59.931262 (+524531us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.442141 (+510879us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.442412 (+   271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.442459 (+    47us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:00.442521 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.442610 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.442748 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115111,"prepare.queue_time_us":402315,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":2029,"thread_start_us":801254,"threads_started":2}
I20260709 12:26:01.022689 24168 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:00.879817 13727 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:26:01.107824 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.108408 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.108943 13723 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:01.051429 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4836) took 1352 ms. Trace:
I20260709 12:26:01.049784 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.082240 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:01.109992 13719 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:26:01.065549 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.053650 13796 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:01.073006 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.111821 13721 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:26:01.075929 24558 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:01.093878 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.111570 13800 rpcz_store.cc:276] 0709 12:25:59.699000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.575159 (+876159us) service_pool.cc:224] Handling call
0709 12:26:00.575461 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.628655 (+ 53194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.051140 (+422485us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b54003d54d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:01.051325 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120670464}
W20260709 12:26:01.114244 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.114743 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 216809us
W20260709 12:26:01.115229 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 310163us
I20260709 12:26:01.072659 13725 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 22.
I20260709 12:26:01.115701 14423 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:26:01.116333 14425 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.063s	sys 0.134s
I20260709 12:26:01.116374 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:01.117087 14425 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.064s	sys 0.134s
I20260709 12:26:01.117373 13725 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.125110 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.122188 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:01.134342 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:01.135367 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:01.076828 13798 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3289) took 2466 ms (client timeout 1831 ms). Trace:
I20260709 12:26:01.110960 13720 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:26:01.136708 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:01.137833 13798 rpcz_store.cc:269] 0709 12:25:58.611219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.611375 (+   156us) service_pool.cc:224] Handling call
0709 12:26:01.076594 (+2465219us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54000d25d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.076717 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3303594368}
I20260709 12:26:01.131392 24655 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:01.140403 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.142450 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.143615 13798 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:01.144865 13725 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:01.083137 24446 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:01.147243 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.149044 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.160063 13727 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.161513 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.154397 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.163146 13796 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:01.152853 13722 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:01.148190 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1669) took 1359 ms. Trace:
I20260709 12:26:01.164592 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.156937 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1710) took 1453 ms. Trace:
I20260709 12:26:01.165933 13648 rpcz_store.cc:276] 0709 12:25:59.703780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.635963 (+932183us) service_pool.cc:224] Handling call
0709 12:26:01.052122 (+416159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.071608 (+ 19486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.071661 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.071732 (+    71us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:01.071786 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.071885 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.156727 (+ 84842us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54001b6a50. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.156868 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19030,"raft.queue_time_us":55104,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":17300,"spinlock_wait_cycles":956210048}
I20260709 12:26:01.167407 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.148383 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:01.169441 24575 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:01.169870 24575 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:01.170245 24575 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:01.171842 13727 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:01.172909 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.173528 13727 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:01.174455 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.176578 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.083241 24169 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:01.183235 13652 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:01.183990 14424 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:26:01.185247 13643 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: [31.17-31.17]
I20260709 12:26:01.187076 13793 raft_consensus.cc:606] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:01.187543 13793 raft_consensus.cc:614] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:01.177266 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.163620 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.163952 14419 rpcz_store.cc:276] 0709 12:25:59.788434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.788635 (+   201us) service_pool.cc:224] Handling call
0709 12:25:59.789060 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:59.789671 (+   611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.789701 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:59.790438 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 12:26:00.230164 (+439726us) raft_consensus.cc:1645] Marking committed up to 26
0709 12:26:01.085551 (+855387us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b180039d420. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf
0709 12:26:01.085784 (+   233us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.121032 (+ 35248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.121151 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.121216 (+    65us) raft_consensus.cc:1685] finished
0709 12:26:01.147919 (+ 26703us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.148118 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67199,"prepare.queue_time_us":252368,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":9381,"raft.queue_time_us":91,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":2866048,"thread_start_us":589476,"threads_started":2,"wal-append.queue_time_us":622317}
I20260709 12:26:01.192546 13650 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:01.176921 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.194660 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:01.201210 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.123687 24439 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:01.132128 24624 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:01.211627 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.201148 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.219376 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.220225 13644 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.220860 13693 leader_election.cc:400] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:26:01.221468 13693 leader_election.cc:403] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.207532 13727 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.207868 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.209481 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.133060 24426 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.217031 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.217888 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.229210 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.229849 13647 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
W20260709 12:26:01.218998 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.217300 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.206938 13723 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:26:01.151831 24615 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:01.224261 13643 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:01.153432 24644 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:01.228574 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.158766 24446 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:01.151746 24651 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:01.242548 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.139995 24599 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.086s
I20260709 12:26:01.241346 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.246228 13725 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:26:01.246325 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 39 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.247622 13798 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
W20260709 12:26:01.217365 24635 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:26:01.240005 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:01.252658 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.218760 24442 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:01.214867 24503 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:01.210386 24439 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:01.258051 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.290974 13801 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 32 because replica has last-logged OpId of term: 31 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 15.
I20260709 12:26:01.259367 13650 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:01.269595 13725 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:26:01.265048 13798 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:01.256558 13652 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
W20260709 12:26:01.278519 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 345586us
I20260709 12:26:01.186512 24647 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
W20260709 12:26:01.283042 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48513.3us
I20260709 12:26:01.153174 24340 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.084s
I20260709 12:26:01.244077 24539 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:26:01.357290 13724 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:01.357681 13723 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:01.357874 13720 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:26:01.297670 13644 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:01.239643 24644 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.240630 24587 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:01.250984 24446 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:01.367017 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.314416 13727 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 32 because replica has last-logged OpId of term: 31 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:26:01.347121 13798 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:26:01.373006 13727 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
W20260709 12:26:01.336674 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.218387 24169 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.352308 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3304) took 2610 ms. Trace:
I20260709 12:26:01.381004 13649 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:01.359021 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59142us
I20260709 12:26:01.361253 13718 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:26:01.363147 13801 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:26:01.302063 24483 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:01.233062 24194 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.069s
I20260709 12:26:01.283255 24657 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.238054 24426 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.377698 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.239010 24615 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.284788 24651 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:01.380601 13799 rpcz_store.cc:276] 0709 12:25:58.741964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.286664 (+544700us) service_pool.cc:224] Handling call
0709 12:26:00.646634 (+1359970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.675921 (+ 29287us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:00.675961 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.676014 (+    53us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:00.676050 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.676139 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.352091 (+675952us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54000d2ad0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.352235 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28857,"raft.queue_time_us":31299,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":13677,"spinlock_wait_cycles":3126759296}
I20260709 12:26:01.394335 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.397729 13649 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:01.398221 13725 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:01.400817 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1726) took 1679 ms. Trace:
I20260709 12:26:01.401257 13792 rpcz_store.cc:276] 0709 12:25:59.721111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.649931 (+928820us) service_pool.cc:224] Handling call
0709 12:26:00.650282 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.670230 (+ 19948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.400511 (+730281us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:01.400742 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44069632}
I20260709 12:26:01.282407 24442 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:01.403774 24442 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:01.388386 13723 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:01.301216 24648 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:01.386976 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.405357 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.407373 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.407963 24648 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:01.363312 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 296590us
I20260709 12:26:01.403807 24393 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:01.423391 24393 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:01.430011 13644 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:01.379886 24644 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:01.389906 24483 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.446952 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:01.401618 23957 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.000s
I20260709 12:26:01.394007 24615 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:01.448330 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 251041us
I20260709 12:26:01.459421 13799 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: [25.19-27.20]
I20260709 12:26:01.442185 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.389200 24616 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.063s
W20260709 12:26:01.445617 24640 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:26:01.461699 13644 raft_consensus.cc:2376] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 23. Re-sending same reply.
I20260709 12:26:01.469002 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:01.473718 13792 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.451972 24483 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:01.475538 13644 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:26:01.478305 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.397997 24657 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.479875 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
W20260709 12:26:01.471688 24670 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:01.397426 24439 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.459911 23957 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.505779 13644 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:01.493441 24393 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.509121 24393 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:01.482470 24675 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.020s
I20260709 12:26:01.520670 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1723) took 1839 ms. Trace:
I20260709 12:26:01.521096 13792 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
W20260709 12:26:01.522049 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 227087us
I20260709 12:26:01.535007 13796 raft_consensus.cc:2368] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:26:01.521191 13795 rpcz_store.cc:276] 0709 12:25:59.681307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.681561 (+   254us) service_pool.cc:224] Handling call
0709 12:25:59.729799 (+ 48238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.756155 (+ 26356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.756199 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.762878 (+  6679us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.875100 (+1112222us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b18004a7fa0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:26:01.186851 (+311751us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:01.186919 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.189829 (+  2910us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.515290 (+325461us) raft_consensus.cc:1685] finished
0709 12:26:01.515326 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.520584 (+  5258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66468,"raft.queue_time_us":120,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":25923,"spinlock_wait_cycles":125668096,"thread_start_us":311322,"threads_started":1,"wal-append.queue_time_us":1610660}
I20260709 12:26:01.522145 24439 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:01.554385 24674 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 12:26:01.554705 24667 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:26:01.561437 24192 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.715s	user 0.011s	sys 0.003s
W20260709 12:26:01.584584 13693 connection.cc:256] RPC call timeout handler was delayed by -1.1556724560000475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2041s for a response
W20260709 12:26:01.604158 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.499769 24651 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:01.585178 24665 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:26:01.613270 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.613270 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:01.618168 24569 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.631s	user 0.012s	sys 0.000s
I20260709 12:26:01.619019 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.623813 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:01.621120 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:01.585314 24683 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Stepping down as leader of term 33
W20260709 12:26:01.655825 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.654706 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108630us
I20260709 12:26:01.615234 24647 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:01.611019 24616 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:01.619971 24668 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:26:01.604735 24192 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.626523 24169 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.666479 24696 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
W20260709 12:26:01.673702 24679 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 12:26:01.673877 24522 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.000s
I20260709 12:26:01.629195 24675 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:01.693091 13794 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:01.628793 23901 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.752704 23901 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.657251 24683 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:01.664422 24694 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 35.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.038s
W20260709 12:26:01.715411 24671 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:26:01.716291 24681 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:26:01.754837 24694 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:01.754798 24683 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.755549 24586 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:01.755997 24586 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:01.727545 24644 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.756441 24683 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:01.756973 24586 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.758214 24644 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:01.759132 13722 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:01.728989 13726 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:01.764859 24168 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:01.765331 24168 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:01.730058 24412 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.829s	user 0.014s	sys 0.000s
I20260709 12:26:01.731056 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.766281 24168 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.767454 24168 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:01.738757 24464 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.661s	user 0.002s	sys 0.011s
W20260709 12:26:01.739858 24500 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.001s
W20260709 12:26:01.742973 24423 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.001s
I20260709 12:26:01.652182 24575 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.770584 13719 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:01.771912 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4455) took 1075 ms. Trace:
I20260709 12:26:01.716737 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1673) took 1252 ms. Trace:
I20260709 12:26:01.772559 14416 rpcz_store.cc:276] 0709 12:26:00.464054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.466333 (+  2279us) service_pool.cc:224] Handling call
0709 12:26:01.716446 (+1250113us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.716647 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126522752}
W20260709 12:26:01.775461 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.776012 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.776444 13800 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:01.776703 13796 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:01.772265 13719 rpcz_store.cc:276] 0709 12:26:00.696724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.159643 (+462919us) service_pool.cc:224] Handling call
0709 12:26:01.771700 (+612057us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.771851 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239453056}
I20260709 12:26:01.745774 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.776427 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.779119 13651 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 35
W20260709 12:26:01.746542 24677 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.197s	user 0.000s	sys 0.005s
I20260709 12:26:01.748584 24192 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.662164 24426 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:01.749596 24657 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:01.750293 24050 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.678738 24616 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.789052 24050 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.626235 24442 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:01.751398 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 269982us
I20260709 12:26:01.755426 24694 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:01.766955 24412 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:01.744133 24610 consensus_meta.cc:287] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.716s	user 0.014s	sys 0.001s
I20260709 12:26:01.780884 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.780470 24615 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:01.784751 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.859891 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.785653 13652 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:26:01.787098 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.862679 13646 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:01.788095 24569 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.876641 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.876960 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:01.877633 13796 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:01.713970 24556 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.961s	user 0.013s	sys 0.001s
I20260709 12:26:01.908061 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.909132 13801 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:26:01.736106 24446 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.791476 24483 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:01.795814 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:01.848168 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 187933us
I20260709 12:26:01.856217 13719 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:01.868666 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:01.916678 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:01.917160 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 121457us
I20260709 12:26:01.879479 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.883836 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.854992 24548 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:26:01.869905 24610 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.885782 24694 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 35.19, Last appended by leader: 19, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.935994 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.892207 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.892391 24412 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:01.936978 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:01.894610 24442 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.895597 13796 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 34 because replica has last-logged OpId of term: 33 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 22.
I20260709 12:26:01.896948 24615 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.897332 13723 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:01.894610 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:01.913842 13800 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:01.914841 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.881841 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.926245 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:01.927037 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:01.945748 13722 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:26:01.927384 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.934470 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:01.802198 24586 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:01.940945 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:01.947477 13792 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:01.870116 24575 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.950162 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:01.954363 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:01.956162 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:01.922685 24426 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:01.930883 24500 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:01.959290 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:42338 (request call id 112) took 2271 ms. Trace:
I20260709 12:26:01.931664 24675 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:01.966768 13793 rpcz_store.cc:276] 0709 12:25:59.698407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.542604 (+844197us) service_pool.cc:224] Handling call
0709 12:26:01.948070 (+1405466us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b540012a554. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bee853 00007f5398bbc60a 00007f53921427d4 00007f5392142762 00007f5392142691 00007f539214260c 00007f53921418c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:01.948198 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1181137280}
I20260709 12:26:01.967026 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:01.930735 24616 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.972812 13766 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:01.974463 13649 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:01.945741 24470 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.975980 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:01.951232 24439 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:01.964964 24556 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:01.978922 24476 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:01.959677 24169 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:26:01.960517 24464 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:01.916852 24569 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:01.960785 24655 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:01.941146 24683 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.036s
I20260709 12:26:01.970526 24615 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:01.995198 13646 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
W20260709 12:26:01.988756 24624 consensus_meta.cc:287] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.770s	user 0.015s	sys 0.000s
I20260709 12:26:01.975425 24548 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:02.014611 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:02.003665 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:01.975085 23957 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:01.984577 24587 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:01.996176 24590 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:02.017457 13793 raft_consensus.cc:606] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:02.017807 13646 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:01.995519 24556 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:02.024133 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:02.002470 24701 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.154s	user 0.000s	sys 0.005s
I20260709 12:26:02.003496 24655 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.030155 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:02.001564 24426 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:02.004527 24169 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:26:02.037847 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.038384 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:02.023042 24624 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:02.042332 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:01.984015 24470 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.028362 24657 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:02.051928 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:02.055069 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:02.056068 13793 raft_consensus.cc:606] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:02.056726 13793 raft_consensus.cc:614] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:02.058812 24616 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:02.059355 24446 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:02.060635 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72276.8us
I20260709 12:26:02.061908 24673 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:02.061995 24590 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:02.062732 24673 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:02.063648 24590 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.061358 24587 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:02.066083 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:02.067289 24587 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.029883 24548 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:02.071719 13720 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:02.041141 24655 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:02.073513 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.043954 24169 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:02.040856 24344 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.081473 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:02.082321 13648 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:02.071249 24653 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:02.071244 24657 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:02.096896 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.065385 24673 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:02.084900 24548 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:26:02.100739 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 160789us
I20260709 12:26:02.107940 13726 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:02.079615 24616 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:26:02.091785 24653 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:02.097990 24657 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:02.120674 13648 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:26:02.105307 24464 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:26:02.086130 24344 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.106029 24483 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:02.110034 13618 leader_election.cc:400] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
I20260709 12:26:02.317999 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4745) took 1471 ms. Trace:
W20260709 12:26:02.810241 13721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.011s
W20260709 12:26:02.810951 13721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.000s	sys 0.011s
I20260709 12:26:02.810643 13724 rpcz_store.cc:276] 0709 12:26:00.845652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.358188 (+512536us) service_pool.cc:224] Handling call
0709 12:26:01.359949 (+  1761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.360226 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:01.360615 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.228407 (+867792us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b18000837a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:26:02.316539 (+ 88132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.316640 (+   101us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:02.316716 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.316827 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.317050 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30432,"prepare.queue_time_us":468,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":891591,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"spinlock_wait_cycles":11654016,"thread_start_us":87734,"threads_started":1}
I20260709 12:26:02.131978 13726 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:26:02.814674 13618 leader_election.cc:403] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:02.815425 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:02.816133 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 707670us
I20260709 12:26:02.816810 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:02.816761 13722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.005s	sys 0.005s
I20260709 12:26:02.818534 24439 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35.
W20260709 12:26:02.802595 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.001s	sys 0.010s
W20260709 12:26:02.232164 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.273102 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:02.391248 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2224) took 1544 ms. Trace:
I20260709 12:26:02.394893 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:02.822124 13793 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:02.829599 13723 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:02.830099 13793 raft_consensus.cc:606] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:02.830963 13793 raft_consensus.cc:614] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:02.236733 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4730) took 1403 ms. Trace:
W20260709 12:26:02.308040 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.449035 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:02.863958 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:02.864475 13795 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
W20260709 12:26:02.524209 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.177660 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.135885 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:02.877903 13719 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:02.193168 24470 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:02.393018 13794 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4847) took 1602 ms (client timeout 1843 ms). Trace:
I20260709 12:26:02.879482 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:02.880641 13719 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:02.282526 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:02.883046 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5331) took 2616 ms. Trace:
I20260709 12:26:02.883285 13718 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:02.252813 24688 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.096673 23957 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.016s
I20260709 12:26:02.080626 24556 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.304637 24702 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:02.890961 24539 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:02.890561 24702 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:02.892097 13724 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:02.891822 24539 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.193660 24708 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:26:02.375910 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:02.387893 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:02.898720 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:02.899297 14417 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:02.899720 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55697.8us
I20260709 12:26:02.395901 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:02.903946 14421 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:02.904107 24688 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:02.384881 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:02.386471 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:02.164552 24050 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:02.164064 24569 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:02.112511 24653 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:02.112378 24669 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.059s
W20260709 12:26:02.162993 24703 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:26:02.155047 24704 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:26:02.917970 24669 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:02.918488 24669 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:02.163620 24710 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:26:02.450563 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:02.362192 23901 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:02.107674 24169 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.195048 24503 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:03.006880 24503 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:02.114349 24548 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:03.014401 24548 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:03.015576 24548 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:03.020591 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:03.021716 13652 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:02.400377 24694 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:03.024717 13647 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:02.267076 24687 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:26:02.165118 24655 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.073315 24655 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:02.363651 24707 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.233s	user 0.000s	sys 0.003s
W20260709 12:26:02.111255 24709 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:26:03.087829 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1715) took 1215 ms. Trace:
W20260709 12:26:02.539392 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4850) took 1708 ms (client timeout 1841 ms). Trace:
I20260709 12:26:02.449736 13798 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:02.254709 13801 raft_consensus.cc:2410] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
W20260709 12:26:02.799129 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.001s	sys 0.020s
I20260709 12:26:02.382763 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:03.166754 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.095s	user 0.002s	sys 0.020s
I20260709 12:26:03.167295 14419 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.168685 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3335) took 1287 ms. Trace:
W20260709 12:26:03.168897 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 1.11697e+06us
I20260709 12:26:03.169378 13801 rpcz_store.cc:276] 0709 12:26:01.881437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.910028 (+ 28591us) service_pool.cc:224] Handling call
0709 12:26:03.168212 (+1258184us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54000e60d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:03.168439 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714017152}
I20260709 12:26:02.228075 24344 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:02.458878 24476 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.374855 24510 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.203287 24510 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.304136 24587 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.208962 24587 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.212602 24586 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.357510 24610 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:02.278556 24644 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.647612 24657 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.002s
I20260709 12:26:03.217177 24644 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:02.333083 24675 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:02.402631 24699 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:03.219765 24699 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:02.370376 24716 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:26:02.819208 13722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.006s	sys 0.005s
W20260709 12:26:02.230856 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 801408us
W20260709 12:26:02.833871 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.002s	sys 0.011s
I20260709 12:26:02.843521 13725 rpcz_store.cc:276] 0709 12:26:00.840794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.293149 (+452355us) service_pool.cc:224] Handling call
0709 12:26:01.293459 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.398781 (+105322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.236356 (+837575us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400175054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:02.236643 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240031104}
I20260709 12:26:02.852924 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5356) took 1433 ms. Trace:
I20260709 12:26:02.853942 24721 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.856511 24590 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:02.856794 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 583730us
I20260709 12:26:02.829021 13720 rpcz_store.cc:276] 0709 12:26:00.846962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.358772 (+511810us) service_pool.cc:224] Handling call
0709 12:26:01.359053 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.073754 (+714701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.390824 (+317070us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b54001a5254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:02.391155 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1638487424}
I20260709 12:26:02.859550 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4741) took 2001 ms. Trace:
I20260709 12:26:02.245049 24456 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.260975 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2230) took 1251 ms. Trace:
I20260709 12:26:03.261665 24456 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.264635 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:03.265146 24456 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.251551 13721 rpcz_store.cc:276] 0709 12:26:00.844375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.357756 (+513381us) service_pool.cc:224] Handling call
0709 12:26:01.358083 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.460857 (+102774us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:01.481150 (+ 20293us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.236581 (+755431us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.811653 (+575072us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.813065 (+  1412us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.820588 (+  7523us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:02.820646 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.827908 (+  7262us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.827971 (+    63us) raft_consensus.cc:1685] finished
0709 12:26:02.827979 (+     8us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.845086 (+ 17107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":718105,"prepare.queue_time_us":21249,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":118,"raft.queue_time_us":1578034,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":20201,"spinlock_wait_cycles":1510144,"thread_start_us":730892,"threads_started":3,"wal-append.queue_time_us":183380}
I20260709 12:26:03.268211 24733 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:02.883621 13645 rpcz_store.cc:276] 0709 12:26:00.266590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.682932 (+416342us) service_pool.cc:224] Handling call
0709 12:26:00.788032 (+105100us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:00.839013 (+ 50981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.839062 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:00.873065 (+ 34003us) log.cc:844] Serialized 52 byte log entry
0709 12:26:01.267648 (+394583us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:01.267710 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.268667 (+   957us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.763913 (+495246us) raft_consensus.cc:1685] finished
0709 12:26:01.763937 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.872642 (+1108705us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400097750. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:02.882770 (+ 10128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50473,"raft.queue_time_us":72319,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":58747,"spinlock_wait_cycles":240768384,"thread_start_us":314464,"threads_started":1,"wal-append.queue_time_us":544067,"wal-append.run_cpu_time_us":3821,"wal-append.run_wall_time_us":1258954}
I20260709 12:26:02.906214 14422 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:02.879007 13794 rpcz_store.cc:269] 0709 12:26:00.790779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.833948 (+ 43169us) service_pool.cc:224] Handling call
0709 12:26:02.392763 (+1558815us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54002cc4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:02.392924 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952286592}
I20260709 12:26:02.907219 14420 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:02.920503 24669 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:02.898001 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 521958us
I20260709 12:26:02.945524 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5357) took 1204 ms. Trace:
I20260709 12:26:02.909267 24393 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:02.894157 24702 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:02.911448 24556 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:03.275327 14423 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:03.275527 24393 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:03.276911 24393 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.261772 13725 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:02.911156 24688 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.278787 24729 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.972651 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:03.280216 13725 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:26:02.999209 24539 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.093029 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:03.280548 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4759) took 1136 ms. Trace:
I20260709 12:26:03.281253 13727 rpcz_store.cc:276] 0709 12:26:02.144099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.822490 (+678391us) service_pool.cc:224] Handling call
0709 12:26:03.280430 (+457940us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.973500 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:03.282709 13795 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:03.282804 24510 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.119886 23901 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.283594 24510 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.283905 23901 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.284340 24510 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.088604 13651 rpcz_store.cc:276] 0709 12:26:01.872729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.917216 (+ 44487us) service_pool.cc:224] Handling call
0709 12:26:01.917559 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.918017 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.918048 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.937947 (+ 19899us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.046301 (+108354us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:02.046346 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.046418 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.087163 (+1040745us) raft_consensus.cc:1685] finished
0709 12:26:03.087208 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.087556 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14383,"raft.queue_time_us":22267,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2944,"thread_start_us":76280,"threads_started":1,"wal-append.queue_time_us":85752}
I20260709 12:26:03.120939 24687 consensus_queue.cc:1243] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 28.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:03.259891 24590 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.162878 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:03.288375 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125948us
I20260709 12:26:03.164227 24169 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:03.163898 24728 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:03.290673 23957 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:03.291294 23957 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.290892 24728 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.291708 23957 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:02.384130 14418 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:03.171830 24470 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:03.173111 24558 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:03.193332 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4757) took 1051 ms. Trace:
I20260709 12:26:03.195955 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:03.294864 24558 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:03.162055 13797 rpcz_store.cc:269] 0709 12:26:00.830505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.881005 (+ 50500us) service_pool.cc:224] Handling call
0709 12:26:02.539042 (+1658037us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400103454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:02.539288 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3753939328}
I20260709 12:26:03.191089 24476 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.213660 24586 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:03.230398 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 143741us
I20260709 12:26:02.384716 24673 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:03.299204 24673 consensus_queue.cc:1243] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:03.299988 24673 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:03.300024 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:03.300572 24673 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:03.301712 24551 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:03.217295 24657 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.303308 24708 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:03.303815 24708 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.304252 24708 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.306150 24657 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.260039 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:03.311271 13720 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.294756 13726 rpcz_store.cc:276] 0709 12:26:02.141844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.865600 (+723756us) service_pool.cc:224] Handling call
0709 12:26:02.868825 (+  3225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.000310 (+131485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.000347 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.059739 (+ 59392us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.098888 (+ 39149us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:03.098945 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.099043 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.192943 (+ 93900us) raft_consensus.cc:1685] finished
0709 12:26:03.192978 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.193222 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":221629,"raft.run_cpu_time_us":506,"raft.run_wall_time_us":87945,"thread_start_us":138014,"threads_started":2,"wal-append.queue_time_us":91005}
W20260709 12:26:03.312386 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.313545 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:02.967247 24426 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:03.314530 24426 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.315038 24426 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:03.316356 24050 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:03.316890 24669 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:03.273430 13643 rpcz_store.cc:276] 0709 12:26:01.740745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.773833 (+ 33088us) service_pool.cc:224] Handling call
0709 12:26:01.774209 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.774668 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:01.775028 (+   360us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.975601 (+200573us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.980024 (+  4423us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.980178 (+   154us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:01.980212 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.980283 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.934755 (+954472us) raft_consensus.cc:1685] finished
0709 12:26:02.944951 (+ 10196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.945263 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27991,"prepare.queue_time_us":6972,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":34473,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":5887,"thread_start_us":172120,"threads_started":1,"wal-append.queue_time_us":255361}
I20260709 12:26:03.281948 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:03.318181 24522 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:03.308514 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:03.279708 24729 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.281410 13649 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:26:03.115813 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 232096us
I20260709 12:26:03.285923 24720 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:03.248328 24721 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:03.296496 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126323us
I20260709 12:26:03.247862 13644 rpcz_store.cc:276] 0709 12:26:01.419686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.494751 (+ 75065us) service_pool.cc:224] Handling call
0709 12:26:01.495899 (+  1148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.507596 (+ 11697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.852151 (+1344555us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b54002b48d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:02.852682 (+   531us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21628672}
W20260709 12:26:03.244621 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.307142 13721 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:03.308475 24569 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:03.309381 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:03.309708 24551 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:03.240556 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.273427 14417 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:03.274377 14419 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:03.333423 24728 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.333693 24721 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:03.334993 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70562.2us
I20260709 12:26:03.261523 13722 rpcz_store.cc:276] 0709 12:26:02.009535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.009715 (+   180us) service_pool.cc:224] Handling call
0709 12:26:02.010079 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.010324 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:02.038814 (+ 28490us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.243486 (+1204672us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.260364 (+ 16878us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:03.260443 (+    79us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:03.260602 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.260720 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.260906 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146314,"prepare.queue_time_us":39674,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":149582,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":6168,"spinlock_wait_cycles":872192,"thread_start_us":1046028,"threads_started":2,"wal-append.queue_time_us":152035}
I20260709 12:26:03.336074 24456 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:26:03.350051 24738 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.002s
W20260709 12:26:03.350564 13690 connection.cc:256] RPC call timeout handler was delayed by -1.1414509530000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:26:03.351076 13690 connection.cc:256] RPC call timeout handler was delayed by -1.0417811150000489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 12:26:03.353096 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:03.357249 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48309.1us
I20260709 12:26:03.358222 24569 raft_consensus.cc:2764] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:26:03.358712 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:03.359771 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:03.360144 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:03.362653 24590 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.361948 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:03.370901 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.376099 14422 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:03.365063 24456 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:03.370976 24539 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.379508 24456 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.222124 24699 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.354072 24558 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.387735 13720 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:03.388764 24721 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:03.388999 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:03.389565 24735 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:03.390159 24551 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.391491 24736 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.391897 24644 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.392450 24344 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:03.399896 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:03.404158 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 2008) took 1025 ms. Trace:
I20260709 12:26:03.358194 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:03.405315 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4760) took 1162 ms. Trace:
I20260709 12:26:03.347697 24657 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:03.424237 14422 rpcz_store.cc:276] 0709 12:26:02.378225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.384586 (+  6361us) service_pool.cc:224] Handling call
0709 12:26:03.383170 (+998584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.385872 24050 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:03.371384 24688 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:03.419703 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.420207 14423 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:03.421048 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:03.388901 24625 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:03.421748 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:03.302073 24673 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:03.423902 13723 consensus_meta.cc:287] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.593s	user 0.013s	sys 0.006s
I20260709 12:26:03.423839 24344 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:03.380113 24539 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.433936 13723 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
W20260709 12:26:03.431064 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72955.6us
I20260709 12:26:03.414997 24586 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.428066 24699 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.447377 14424 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:03.449378 14418 raft_consensus.cc:2368] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:03.436697 24344 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:03.450846 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51162.7us
I20260709 12:26:03.452044 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:03.459640 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 70825.7us
I20260709 12:26:03.449558 24688 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.467550 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4758) took 1306 ms. Trace:
I20260709 12:26:03.468267 13723 rpcz_store.cc:276] 0709 12:26:02.142955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.394514 (+251559us) service_pool.cc:224] Handling call
0709 12:26:03.438719 (+1044205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.436610 24510 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.474819 14419 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32. Ops: [29.19-31.20]
I20260709 12:26:03.464942 24688 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:03.480748 24653 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.571s	user 0.015s	sys 0.004s
I20260709 12:26:03.481287 24653 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.482508 24749 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:03.483047 24749 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.483445 24749 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:03.483726 24539 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.429914 24736 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.443112 24644 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:03.424718 13719 rpcz_store.cc:276] 0709 12:26:02.243419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.881916 (+638497us) service_pool.cc:224] Handling call
0709 12:26:02.954789 (+ 72873us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.005155 (+ 50366us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:03.064580 (+ 59425us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.229782 (+165202us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.327496 (+ 97714us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.327672 (+   176us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:03.327708 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.327779 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.405014 (+ 77235us) raft_consensus.cc:1685] finished
0709 12:26:03.405042 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.405241 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12559,"prepare.queue_time_us":49843,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":9757,"raft.queue_time_us":252746,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":4243,"thread_start_us":159826,"threads_started":2,"wal-append.queue_time_us":264243}
I20260709 12:26:03.469197 24439 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:03.500535 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:03.506842 13690 leader_election.cc:400] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:03.521922 13690 leader_election.cc:403] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:26:03.508608 24050 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.486817 24687 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:03.500489 24688 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:03.530788 24736 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.534505 24708 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:03.521342 14391 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.683s (SENT)
W20260709 12:26:03.560376 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 54127.3us
I20260709 12:26:03.535555 24751 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:26:03.541534 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51044.4us
I20260709 12:26:03.561510 24708 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.561205 24344 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.561695 24751 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:03.563167 24728 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:03.563431 24751 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.546141 13647 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 12:26:03.488761 24657 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:03.561233 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.713s (SENT)
W20260709 12:26:03.550483 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 242420us
I20260709 12:26:03.551501 24653 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:03.556636 24548 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.005s
I20260709 12:26:03.557622 24169 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:03.558964 24736 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.543280 24590 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:03.570451 24590 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.570861 24590 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:03.564510 13647 pending_rounds.cc:85] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 16
I20260709 12:26:03.567233 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:03.572770 13647 pending_rounds.cc:107] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 23.17
W20260709 12:26:03.584954 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 181452us
W20260709 12:26:03.582317 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 116154us
I20260709 12:26:03.578693 13647 raft_consensus.cc:2889] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.564208 24344 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:03.624359 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 172496us
I20260709 12:26:03.593649 24688 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:03.568864 24548 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.656512 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4763) took 1398 ms. Trace:
W20260709 12:26:03.660257 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 47902us
I20260709 12:26:03.658646 13724 rpcz_store.cc:276] 0709 12:26:02.257782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.893143 (+635361us) service_pool.cc:224] Handling call
0709 12:26:02.893654 (+   511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.894064 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.894092 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.967167 (+ 73075us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.135166 (+167999us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:03.135376 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.135524 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.656220 (+520696us) raft_consensus.cc:1685] finished
0709 12:26:03.656243 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.656427 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":167315,"threads_started":1,"wal-append.queue_time_us":415783}
I20260709 12:26:03.579270 23957 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:03.665844 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.593351 24736 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.645138 24439 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:03.668994 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:03.669713 13720 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:03.583066 24687 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.706740 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.707844 14423 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 36: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:03.672207 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:03.666746 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:03.678356 24750 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.106s
I20260709 12:26:03.685047 24456 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.662215 24755 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:03.666553 24728 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 35.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.020s
I20260709 12:26:03.690784 13652 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:03.644148 24169 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.693176 24757 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:03.692996 24439 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:03.694342 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 133150us
I20260709 12:26:03.694742 24657 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.666364 24720 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 12:26:03.698791 24688 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:03.701347 24548 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.702152 24765 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:03.707729 24687 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:03.677964 24503 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.005s
I20260709 12:26:03.711987 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:03.692777 24735 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:03.723734 24755 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:03.794216 24755 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:03.732383 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 168327us
I20260709 12:26:03.744033 24625 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 12:26:03.734329 13764 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:03.705637 24749 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:03.749265 13617 connection.cc:256] RPC call timeout handler was delayed by -1.8157712330003051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26680000000000004s for a response
I20260709 12:26:03.797958 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:03.798758 13617 connection.cc:256] RPC call timeout handler was delayed by -1.6561631509998733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2311s for a response
I20260709 12:26:03.799065 24751 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.799748 24751 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.800294 23957 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.801038 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:03.801826 13643 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:03.802711 24764 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.803498 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:03.803398 24764 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.805773 13766 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:03.785264 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.786187 24733 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:26:03.725411 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 294959us
W20260709 12:26:03.786358 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55281.7us
I20260709 12:26:03.807194 24768 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:03.807523 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:03.825212 13649 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:03.808086 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:03.808300 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:03.809026 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:03.781304 24675 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.810491 24749 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.819730 24728 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.004s
I20260709 12:26:03.811617 13652 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 25. Current term is 26. Ops: []
W20260709 12:26:03.812068 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.559s (SENT) [suppressed 23 similar messages]
I20260709 12:26:03.823869 24757 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.805914 24764 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.841018 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.841948 14418 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:03.834169 24675 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.843199 24750 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:03.715098 24426 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.845523 24675 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.845616 24426 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.847419 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:03.816284 24736 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.778894 24503 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.779426 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:03.850785 13725 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:03.818545 24050 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:03.850117 24503 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.812839 24590 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:03.853114 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.854249 24590 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.818938 24754 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
W20260709 12:26:03.867781 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62136.7us
I20260709 12:26:03.875730 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:03.822664 14417 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:03.826974 13644 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.880373 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:03.833397 13651 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:03.881898 13792 raft_consensus.cc:2376] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27. Re-sending same reply.
I20260709 12:26:03.821923 24733 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:03.847538 24169 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.806741 14424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:43168
I20260709 12:26:03.883872 14424 raft_consensus.cc:606] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:03.884342 14424 raft_consensus.cc:614] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:03.884476 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.884097 24733 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.885427 24733 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:03.860598 13720 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
W20260709 12:26:03.892725 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49782.5us
I20260709 12:26:03.892618 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.802083 24751 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.894155 14422 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:03.895303 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:03.898917 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.899438 24729 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.900609 24456 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:03.901250 24675 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:03.901515 24522 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:03.902168 24522 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:03.902489 24675 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:03.902984 24675 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:03.903339 24522 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.904533 24625 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:03.905316 24625 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.896147 13724 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:26:03.907249 24587 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.907889 24625 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:03.909366 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 111397us
I20260709 12:26:03.904168 24675 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:03.921809 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48537.3us
I20260709 12:26:03.930169 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:03.895275 24736 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:03.935461 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:03.937898 24673 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:03.942519 24169 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:03.943357 24169 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.943782 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4762) took 1699 ms. Trace:
I20260709 12:26:03.943941 24657 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.937810 24764 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:03.944480 24657 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:03.944806 24764 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.946579 24759 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.947468 24764 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.947327 24759 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:03.944211 13718 rpcz_store.cc:276] 0709 12:26:02.244074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.884371 (+640297us) service_pool.cc:224] Handling call
0709 12:26:02.885069 (+   698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.334887 (+449818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.334931 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.335649 (+   718us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.613534 (+277885us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:03.613599 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.943357 (+329758us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b5400157cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:03.943409 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.943458 (+    49us) raft_consensus.cc:1685] finished
0709 12:26:03.943470 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.943715 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":733,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":132,"spinlock_wait_cycles":1033013632,"thread_start_us":277323,"threads_started":1,"wal-append.queue_time_us":330366}
I20260709 12:26:03.949782 23901 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:03.923704 24750 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:03.951460 24750 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 33.25, Last appended by leader: 25, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:03.967598 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.972962 13649 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:03.955623 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:03.979900 13792 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:03.923589 24688 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:03.978971 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:03.919850 24687 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:03.988598 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:03.926213 24757 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:03.991075 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65504.6us
I20260709 12:26:03.994215 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:03.995222 13721 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:26:03.994442 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:03.976960 24610 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:04.007654 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:04.003809 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:03.946579 24169 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:03.990502 24765 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:04.001427 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:04.007829 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73239.5us
I20260709 12:26:03.934154 24736 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:04.017931 24539 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:04.020017 13644 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:04.020239 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:03.988869 24687 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:26:04.029462 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 305176us
I20260709 12:26:04.018497 24539 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:04.019788 24736 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:04.038801 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.033823 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.036509 24169 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.031805 24558 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:04.039987 24548 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.040748 13719 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:04.042487 24476 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.019264 24765 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.049505 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.050454 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.081779 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5372) took 1055 ms. Trace:
I20260709 12:26:04.039218 24439 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:04.059926 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.060586 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.050084 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:04.036943 24610 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:04.088373 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:04.088737 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.037449 24699 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:04.075645 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.091900 14423 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
W20260709 12:26:04.096015 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73837.6us
I20260709 12:26:04.082317 13647 rpcz_store.cc:276] 0709 12:26:03.026195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.026324 (+   129us) service_pool.cc:224] Handling call
0709 12:26:03.038599 (+ 12275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.651281 (+612682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.081490 (+430209us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b5400113d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:04.081708 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166099200}
I20260709 12:26:04.028647 24759 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:04.049546 24757 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:04.103904 13795 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:04.104562 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55181us
W20260709 12:26:04.103232 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 95678.3us
I20260709 12:26:04.091228 24675 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:04.121157 13792 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:04.123651 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:04.119172 24558 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:04.129791 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:04.091728 24765 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:04.128325 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.107698 24586 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:04.078295 24169 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.112049 24751 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:04.107990 24757 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.134469 13764 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:04.137478 13644 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 22.
I20260709 12:26:04.129645 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.141410 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:04.143319 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:04.143992 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.144212 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:04.144881 13719 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 12:26:04.144865 13643 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:04.145510 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:04.146128 13726 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:04.149775 14425 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:04.150630 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.119395 24699 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:04.151659 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.152726 13797 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:04.156105 24694 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:04.155750 24539 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.157552 24539 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:04.162524 24657 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:04.166985 13724 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:04.153059 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.137099 24586 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.172921 13801 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:26:04.115715 24548 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.136791 24751 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.173327 13801 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 16
I20260709 12:26:04.140228 24757 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:04.179530 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 142156us
I20260709 12:26:04.126196 24721 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:26:04.191310 24699 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.25, Last appended by leader: 25, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.210808 13725 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:04.199851 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:04.213160 13801 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:26:04.213600 13801 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.213343 13797 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:04.125871 24439 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:04.209673 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 72453us
W20260709 12:26:04.217605 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 93067.8us
I20260709 12:26:04.170367 24694 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:04.175006 24586 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:04.221560 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52918.4us
I20260709 12:26:04.223331 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:04.224337 13719 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:04.227669 24721 consensus_queue.cc:1243] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 28.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:04.228312 24721 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:04.228633 24721 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:04.229488 24721 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.250963 24721 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:04.250371 24610 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:04.259881 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.261322 24556 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:04.261971 24765 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.220726 13721 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:04.265395 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.266191 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 22.
I20260709 12:26:04.267161 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.267808 13721 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:04.268720 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.269367 13721 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:04.214324 13797 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:04.270799 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:04.246397 24569 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:04.271819 24569 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:04.271580 24558 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.272996 24569 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.271183 24169 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:04.271951 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:04.274449 24729 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:04.274919 24729 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:04.275182 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:04.275707 24470 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:04.276156 24470 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.276515 24470 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:04.276616 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:04.277621 23901 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:04.278007 23901 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.278411 23901 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:04.278465 24587 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:04.279018 24587 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.279524 24587 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:04.247399 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.287832 24655 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:04.126533 24687 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.288360 24655 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.185065 24751 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:04.270152 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.185266 24754 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:04.288848 24655 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:04.299129 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 172269us
W20260709 12:26:04.305279 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.301121 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:04.275310 24729 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:04.226660 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:04.311875 13721 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:26:04.315590 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89053.5us
W20260709 12:26:04.372320 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66587.1us
I20260709 12:26:04.294818 24774 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:04.390731 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.388520 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:04.389331 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:04.392832 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.403268 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:04.404376 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.307468 24733 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.089s
I20260709 12:26:04.405049 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.411167 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.418021 14421 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:04.422111 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:26:04.425115 24744 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.934s	user 0.007s	sys 0.004s
I20260709 12:26:04.426285 24764 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:04.426788 24764 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.427170 24764 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:04.427289 24673 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.427845 24673 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.395921 24754 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.432886 24754 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:04.395639 24774 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:26:04.434080 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.434340 24774 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.434548 24590 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:04.438772 24772 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.438476 24694 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.439136 24548 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:04.439368 24772 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:04.439762 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 214567us
I20260709 12:26:04.387899 24720 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:04.399852 24657 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:04.459151 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 87858.7us
I20260709 12:26:04.413122 24749 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.465289 24587 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.466632 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:04.467664 24470 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.471381 24728 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.471709 24772 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:04.479092 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:04.481125 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.453258 24673 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.487056 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:04.455196 24750 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:04.480451 14425 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:04.493018 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1680) took 1193 ms. Trace:
I20260709 12:26:04.460933 24653 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:04.497829 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:04.506587 13801 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:04.502208 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:04.505008 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.513705 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:04.520046 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43738.4us
I20260709 12:26:04.513283 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:04.526101 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:04.528329 13645 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:04.529521 13794 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:04.536743 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.493642 14420 rpcz_store.cc:276] 0709 12:26:03.292652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.298202 (+  5550us) service_pool.cc:224] Handling call
0709 12:26:03.298784 (+   582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.299536 (+   752us) raft_consensus.cc:1534] Early marking committed up to index 27
0709 12:26:04.100271 (+800735us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.485231 (+384960us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:04.485292 (+    61us) raft_consensus.cc:1645] Marking committed up to 27
0709 12:26:04.485399 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.485492 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.485683 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":716790,"prepare.queue_time_us":759756,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":8158,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":8569,"thread_start_us":417063,"threads_started":2}
I20260709 12:26:04.491909 24749 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.540980 13795 raft_consensus.cc:2410] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 38 because replica has last-logged OpId of term: 36 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 34 index: 24.
I20260709 12:26:04.487468 23901 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.517782 24702 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:04.525676 24625 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:04.539575 24759 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:04.581490 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.582512 13797 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:04.576890 24169 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:04.595045 24784 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:26:04.597028 24795 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:04.567101 24765 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.597918 24795 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.525117 24653 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.554236 24702 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.530719 24750 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.538877 24590 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.046s
I20260709 12:26:04.530499 24728 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:04.570667 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 177944us
I20260709 12:26:04.617062 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:04.567677 24721 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:04.579483 24759 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:04.617794 24755 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.823s	user 0.009s	sys 0.004s
I20260709 12:26:04.606889 24548 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.600642 24749 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.648533 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.643888 13797 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:04.607924 24439 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:04.614727 24702 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:04.667461 24587 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.684478 13794 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:04.656471 24755 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.675742 13795 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:04.663436 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.671204 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.604482 24625 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:04.707556 13792 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.000s
I20260709 12:26:04.674002 24720 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:04.709687 13792 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:04.656953 24754 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:26:04.696178 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.697155 24470 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.697822 24728 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.700489 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:04.650789 24556 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:26:04.703125 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.703879 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:04.707909 24625 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:04.653409 24759 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:04.720752 13719 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:04.721760 13801 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:04.753237 24751 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.757730 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.766019 13645 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:04.705741 24729 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.679703 24610 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:04.762557 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46084.1us
I20260709 12:26:04.813510 13727 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:04.822749 24653 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:04.827519 13799 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 35. Current term is 36. Ops: []
W20260709 12:26:04.813024 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 205487us
I20260709 12:26:04.812361 13796 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:04.685590 24765 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.079s
W20260709 12:26:04.789055 13726 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.003s
I20260709 12:26:04.835772 13726 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:04.668345 24548 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:04.814185 13721 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:04.828552 13722 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:26:04.811497 14424 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:04.685402 24586 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:04.817808 24790 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 12:26:04.804065 24169 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:04.851392 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:04.855774 13719 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:04.824928 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:04.859280 13647 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 38: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:04.818534 24720 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:04.866968 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.865784 13721 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:26:04.870388 24788 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
I20260709 12:26:04.872700 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:04.873540 13726 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:26:04.884517 13719 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:26:04.910501 14419 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:04.911221 24764 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.868659 24720 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:04.913553 24791 log.cc:927] Time spent T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:26:04.852250 24673 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:04.916340 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91483.8us
I20260709 12:26:04.917598 13645 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:04.914034 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:04.919317 24805 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:26:04.828073 24610 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.917441 24811 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:26:04.961791 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 99796.2us
I20260709 12:26:04.916002 24720 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:04.994479 24794 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:26:04.995746 24806 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:26:04.956959 23901 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:04.929839 24673 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:04.932753 24733 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:26:05.011409 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.013113 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.014793 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:05.014899 13725 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:05.013473 13797 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:05.016520 13727 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
W20260709 12:26:05.014858 13617 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.530s (SENT)
I20260709 12:26:05.019483 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:05.002080 24811 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.042587 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.046597 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.048105 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.049808 13643 raft_consensus.cc:1240] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:26:05.050915 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.054858 24814 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:05.055545 24586 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:05.059453 24610 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.060379 24771 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:26:05.061796 24771 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:05.062357 24771 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.062616 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.063750 24771 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.065224 24771 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:05.065276 24673 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:05.067101 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75080.5us
W20260709 12:26:05.067600 24736 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.970s	user 0.014s	sys 0.000s
W20260709 12:26:05.067950 13617 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.585s (SENT)
I20260709 12:26:05.068037 24736 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.068480 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:05.069227 14422 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:05.069394 24556 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:05.071400 24736 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.074834 24548 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.075664 24751 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.077311 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:05.078176 14419 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:05.032246 24749 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:26:05.086822 24755 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.077333 24811 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.090965 13725 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:05.094067 14419 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:05.095669 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.097420 24750 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:05.098424 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.099076 24669 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:05.099476 24669 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:05.099638 13646 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:26:05.099682 13650 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:05.100507 24669 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.101879 24669 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:05.103145 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.140297 13646 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:05.106202 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.142369 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.143656 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.144284 13648 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:05.108251 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:05.145387 24752 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:05.145828 24752 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.147143 24752 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.25, Last appended by leader: 25, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.148526 24752 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:05.111690 24764 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.112116 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:05.114002 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:05.153928 24764 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:05.116653 24795 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.156193 24764 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:05.159421 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:05.033143 24757 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.811s	user 0.017s	sys 0.000s
W20260709 12:26:05.037499 24655 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.005s
I20260709 12:26:05.122721 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.175633 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.125077 13643 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:05.125772 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.127736 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.178043 13651 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:05.115200 13617 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.651s (SENT)
I20260709 12:26:05.179113 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:05.154175 24675 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.145851 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:05.181972 13645 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:26:05.188328 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:05.189745 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:05.190829 14421 raft_consensus.cc:2410] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:26:05.198292 14418 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 25. Preceding OpId from leader: term: 35 index: 26. (index mismatch)
I20260709 12:26:05.180589 23957 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:05.180450 24754 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:05.198453 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:05.223662 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.228749 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:05.250113 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:05.250903 14419 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:05.250979 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.184183 24675 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.183913 24503 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:05.244719 14416 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:26:05.248010 13764 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:05.253123 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.263458 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:05.263449 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.157783 24764 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 36.25, Last appended by leader: 25, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.194906 24814 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:05.212136 24510 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.270008 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:05.270658 24728 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:05.275997 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:05.264685 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 66346.1us
I20260709 12:26:05.220906 24610 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.281255 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:05.117842 24702 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.181330 24757 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.283876 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:05.183125 24655 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.298609 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.268299 24768 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.309973 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 194854us
I20260709 12:26:05.268000 23957 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.306057 13643 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:05.314733 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:05.315701 14417 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:26:05.280345 24749 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:05.282124 24503 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.247599 24759 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.322153 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:05.277709 24456 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.043s
I20260709 12:26:05.308485 24754 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:05.308785 24765 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:05.343330 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93664us
W20260709 12:26:05.338205 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 88192.3us
I20260709 12:26:05.321003 24728 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:05.324913 24503 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:05.335379 14420 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:05.320107 24749 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:05.348603 13617 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.672s (SENT)
I20260709 12:26:05.319463 24587 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:05.326809 24736 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:05.349155 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:05.308145 24610 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.337428 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.333824 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:05.346850 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:05.310338 24510 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.355068 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.355178 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.378593 13722 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:05.337357 24558 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:05.377804 13725 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
W20260709 12:26:05.380615 24818 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:26:05.350518 24749 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:05.387578 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 106441us
I20260709 12:26:05.320706 24768 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.340227 24811 consensus_queue.cc:1059] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:26:05.356880 24750 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:05.400535 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.397267 13801 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:26:05.401703 13643 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:05.376307 24587 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.404346 24587 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.405932 24768 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:05.404366 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.349684 24720 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.043s
I20260709 12:26:05.391491 24694 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:05.413406 13726 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
W20260709 12:26:05.416157 24820 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:26:05.391148 24736 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:05.393674 24795 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:05.396843 24699 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.396040 24688 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.416162 24675 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.449741 13648 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:05.450134 24765 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.450455 24768 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:05.450583 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 114006us
W20260709 12:26:05.451696 24824 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:05.452014 13643 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:05.449754 13646 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:26:05.453136 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.454686 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:05.466498 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.022s	sys 0.052s
W20260709 12:26:05.468775 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.950s	user 0.022s	sys 0.052s
I20260709 12:26:05.416988 24821 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.023s
W20260709 12:26:05.469895 13617 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.790s (SENT)
I20260709 12:26:05.473169 24759 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.475148 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:05.441736 24828 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.479193 13652 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:05.443370 24694 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:05.480490 13649 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:26:05.453327 24558 raft_consensus.cc:3055] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:05.393126 23957 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.486331 24558 raft_consensus.cc:740] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.495322 13643 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 25. Preceding OpId from leader: term: 35 index: 26. (index mismatch)
I20260709 12:26:05.459117 24688 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.459667 24699 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.466709 24765 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:05.507756 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 174020us
I20260709 12:26:05.520613 13767 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:05.497252 13652 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:05.492976 24811 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.528285 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 1.03321e+06us
I20260709 12:26:05.533303 13797 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:26:05.534587 24688 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:05.539623 24830 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.120s	user 0.002s	sys 0.000s
I20260709 12:26:05.541770 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.542550 24510 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.544073 24522 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:05.544623 24558 consensus_queue.cc:260] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.546481 24558 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:05.549027 24503 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.542582 13724 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:26:05.552618 24503 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:05.497803 24759 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:05.486331 24828 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.556679 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:05.572741 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.573297 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4794) took 1328 ms. Trace:
I20260709 12:26:05.574347 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:05.575815 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:05.577323 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.578656 24795 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.579409 24795 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.580607 13793 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:26:05.581787 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:05.582477 13792 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:05.586725 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.587742 13794 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:05.590972 24610 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.591701 24610 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.592238 24610 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.594484 24811 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:05.602866 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 138745us
W20260709 12:26:05.603796 13617 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.799s (SENT)
I20260709 12:26:05.604976 24694 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:05.515594 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123465us
I20260709 12:26:05.573690 13720 rpcz_store.cc:276] 0709 12:26:04.244557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.861149 (+616592us) service_pool.cc:224] Handling call
0709 12:26:04.869652 (+  8503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.924455 (+ 54803us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:04.925698 (+  1243us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.036441 (+110743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:05.036507 (+    66us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:05.036561 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.572038 (+535477us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b1800350fe0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419
0709 12:26:05.573025 (+   987us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b5400174154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:05.573091 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.573231 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63328,"prepare.queue_time_us":159,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":34122,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":1956,"spinlock_wait_cycles":128885376,"thread_start_us":75753,"threads_started":1,"wal-append.queue_time_us":258762}
I20260709 12:26:05.616333 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:05.629878 13651 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:05.647544 13618 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:05.534924 24819 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 12:26:05.652814 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 118986us
W20260709 12:26:05.655784 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52028.7us
I20260709 12:26:05.658996 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.597434 24811 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.581789 24795 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.645166 24759 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.644377 24721 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:05.522938 24736 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:05.674456 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.675927 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.676759 13651 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:05.682607 24702 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:05.685696 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.689363 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.690831 24590 raft_consensus.cc:993] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: : Instructing follower b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:26:05.692471 24610 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:05.692895 24801 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.690s	user 0.001s	sys 0.001s
I20260709 12:26:05.695223 24687 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:05.695607 24765 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.700137 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:05.751672 14418 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:05.671123 24759 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.701066 24828 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:05.701645 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.757885 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.758177 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.701442 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:05.702860 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.706578 24699 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:05.709549 24814 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:05.712445 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2022) took 1323 ms. Trace:
I20260709 12:26:05.814484 14424 rpcz_store.cc:276] 0709 12:26:04.389393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.390223 (+   830us) service_pool.cc:224] Handling call
0709 12:26:04.398351 (+  8128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.852071 (+453720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.712175 (+860104us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400516b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:05.712375 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789842304}
I20260709 12:26:05.815506 13727 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:05.720865 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.685987 24757 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.816579 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:05.816630 13720 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:05.817642 14424 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:05.659832 24755 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:26:05.722397 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:05.723089 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.820192 13719 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:05.674928 24795 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:05.661823 24050 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 33.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:26:05.821882 23957 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:05.741143 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:05.754968 13718 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 20. (term mismatch)
I20260709 12:26:05.822733 13718 pending_rounds.cc:85] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 19
I20260709 12:26:05.823073 13718 pending_rounds.cc:107] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 31.20
I20260709 12:26:05.823366 13718 raft_consensus.cc:2889] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.757560 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:05.748193 24590 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:05.722901 24837 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.077s
I20260709 12:26:05.747764 24510 raft_consensus.cc:1081] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Signalling peer b4f6bfcda63148e4965ca02c639336f1 to start an election
I20260709 12:26:05.806241 13722 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:26:05.806494 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.808643 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.787686 13801 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:05.724347 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.719470 24503 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.074s
I20260709 12:26:05.810148 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:05.789120 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 141713us
I20260709 12:26:05.838876 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.839625 13727 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:05.839920 13719 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:05.826967 13720 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:05.824221 13794 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:05.674870 24721 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.822733 14423 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:26:05.829031 13651 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 23.
I20260709 12:26:05.724771 24673 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.078s
I20260709 12:26:05.854406 24625 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:05.856642 13617 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.559s (SENT)
I20260709 12:26:05.857219 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:05.858742 13617 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.883s (SENT)
I20260709 12:26:05.788825 24728 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:05.868741 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:05.802507 24771 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.737s	user 0.016s	sys 0.007s
I20260709 12:26:05.823875 23957 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:05.844408 24749 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.841581 24590 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:05.852794 24721 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.892760 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.893390 13651 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:05.893755 13648 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:26:05.859341 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:05.864521 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.895095 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55689.2us
I20260709 12:26:05.824786 24720 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:05.908252 13644 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 36 index: 26. (index mismatch)
I20260709 12:26:05.872773 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.912223 13724 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:05.844071 13721 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:05.870157 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:05.877501 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:05.885541 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:05.914319 13646 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 37: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:05.887827 24764 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:05.889671 24590 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:05.890540 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:05.919083 14423 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
W20260709 12:26:05.920521 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50477us
I20260709 12:26:05.894639 13795 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:05.923210 24688 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:05.923698 24688 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.924104 24688 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:05.924562 23957 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.894939 24795 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:05.926427 24644 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:05.907193 24759 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.925048 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:05.926466 24795 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:05.928355 23957 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:05.930334 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.906741 24655 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:05.927356 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2018) took 1539 ms. Trace:
I20260709 12:26:05.932628 14425 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:05.897374 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:05.942914 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45624.7us
I20260709 12:26:05.936602 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.944106 13721 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 37
I20260709 12:26:05.937736 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:05.944761 13724 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:05.933820 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:05.938618 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55896.4us
I20260709 12:26:05.945363 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:05.947134 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:05.947542 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:05.948911 24754 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:05.949338 24754 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:05.949725 24754 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:05.941936 14417 rpcz_store.cc:276] 0709 12:26:04.388113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.393908 (+  5795us) service_pool.cc:224] Handling call
0709 12:26:04.403860 (+  9952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.316396 (+912536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.927081 (+610685us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b5400498c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:05.927290 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096090880}
I20260709 12:26:05.965628 13651 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:26:05.967321 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.972246 14418 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:05.974045 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.974931 13723 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
W20260709 12:26:05.980156 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77308us
W20260709 12:26:05.980147 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 278819us
I20260709 12:26:05.955853 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:05.983944 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50265.3us
I20260709 12:26:05.990427 13718 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
W20260709 12:26:05.976402 24844 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.003s
I20260709 12:26:05.993817 14416 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:05.931134 13722 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:34464
I20260709 12:26:05.999571 13722 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:06.000005 13722 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:05.990953 13718 pending_rounds.cc:85] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:26:06.004382 13718 pending_rounds.cc:107] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:26:06.004765 13718 raft_consensus.cc:2889] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.001952 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:06.011157 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:05.935549 24644 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:05.974468 24590 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:05.973773 24851 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:05.995711 24169 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:06.012043 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:06.041352 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:05.975031 24848 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:26:06.069891 14419 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
W20260709 12:26:06.069990 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 211296us
I20260709 12:26:05.998684 24828 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:06.043027 23901 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:06.074451 23901 consensus_queue.cc:1243] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:05.935971 24720 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.025424 24702 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:06.063390 13726 raft_consensus.cc:2410] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 33 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:26:06.063527 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:06.133831 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70411us
W20260709 12:26:06.064114 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52119.9us
I20260709 12:26:06.058318 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:06.059020 24169 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.077342 24792 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.426s	user 0.003s	sys 0.003s
I20260709 12:26:06.023439 24833 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
W20260709 12:26:06.083525 24842 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:26:06.054859 24476 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.029s
I20260709 12:26:06.091615 24749 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:06.093272 14418 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: [32.22-32.22]
I20260709 12:26:06.093514 24729 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:05.994676 24764 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 35.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.069s
I20260709 12:26:06.073422 24757 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:06.098213 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:06.100097 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2020) took 1711 ms. Trace:
I20260709 12:26:06.107053 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60832 (request call id 1729) took 1989 ms. Trace:
I20260709 12:26:06.186435 14422 rpcz_store.cc:276] 0709 12:26:04.388761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.394001 (+  5240us) service_pool.cc:224] Handling call
0709 12:26:04.394891 (+   890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.069658 (+674767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.099783 (+1030125us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54005302d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:06.100013 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550331776}
I20260709 12:26:06.076327 24811 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:06.123631 24848 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:06.062654 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:06.022745 24610 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:26:06.137238 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 212284us
I20260709 12:26:06.014863 24699 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.060s
I20260709 12:26:06.135646 14390 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:06.143038 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:06.198676 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 55792.6us
I20260709 12:26:06.150223 13722 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:06.194693 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 96631us
I20260709 12:26:06.167294 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:06.169976 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:06.163501 13542 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:06.186506 13647 rpcz_store.cc:276] 0709 12:26:04.117995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.123362 (+  5367us) service_pool.cc:224] Handling call
0709 12:26:06.106771 (+1983409us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54002e4ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:06.106947 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676359936}
W20260709 12:26:06.111500 24558 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.001s
I20260709 12:26:06.190507 23901 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:06.025923 24759 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.112s
I20260709 12:26:06.125806 24503 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.068s
I20260709 12:26:06.172957 24050 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:06.025719 24836 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.128s
I20260709 12:26:06.137758 24702 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.189277 13767 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:06.196173 24757 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:06.184415 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:06.211334 13800 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 37: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:06.153019 24169 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:06.200282 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:06.225945 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 90367.4us
I20260709 12:26:06.195233 24848 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:06.231045 24470 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.005s
I20260709 12:26:06.208315 24755 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.236439 24831 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.006s
I20260709 12:26:06.198493 24733 raft_consensus.cc:2764] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
W20260709 12:26:06.247361 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 176248us
I20260709 12:26:06.226068 13722 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:06.201618 24610 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.282560 24610 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:06.285466 24854 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:26:06.287434 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5407) took 1008 ms. Trace:
I20260709 12:26:06.245169 24702 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.208273 24510 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:26:06.264278 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 75093.2us
I20260709 12:26:06.265062 24558 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:06.265666 24050 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.266631 24470 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:26:06.278465 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 78245.1us
I20260709 12:26:06.281447 24759 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:26:06.281813 13617 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.034s (SENT)
I20260709 12:26:06.290647 24831 consensus_queue.cc:1243] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 28.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:26:06.293363 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.244833 24590 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.255043 13795 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:06.303547 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:06.320824 13647 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:06.310006 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:06.247372 24736 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:06.295848 24510 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.195117 24644 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.287830 13652 rpcz_store.cc:276] 0709 12:26:05.278938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.524008 (+245070us) service_pool.cc:224] Handling call
0709 12:26:05.639695 (+115687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.716996 (+ 77301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.717032 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.741287 (+ 24255us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.978677 (+237390us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:06.024050 (+ 45373us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.036688 (+ 12638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.036773 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.287178 (+250405us) raft_consensus.cc:1685] finished
0709 12:26:06.287202 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.287366 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12334,"prepare.queue_time_us":45393,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":157790,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":15173,"thread_start_us":150529,"threads_started":1,"wal-append.queue_time_us":218555}
W20260709 12:26:06.301280 24802 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.449s	user 0.000s	sys 0.005s
I20260709 12:26:06.278031 24728 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:26:06.324172 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:06.334411 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 270064us
I20260709 12:26:06.306108 24470 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.288115 24755 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.338575 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5406) took 1072 ms. Trace:
I20260709 12:26:06.324162 24736 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.336488 24754 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.392313 24754 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:06.397631 24736 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:06.334375 24851 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:06.398864 24590 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.320164 24759 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:06.303660 24169 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.350416 24853 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:26:06.351764 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69999.1us
I20260709 12:26:06.353308 13721 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:26:06.404728 13722 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:06.405118 13724 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:06.333590 24510 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:06.371621 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.320086 24558 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.340684 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:06.384414 13643 rpcz_store.cc:276] 0709 12:26:05.266166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.501146 (+234980us) service_pool.cc:224] Handling call
0709 12:26:05.525310 (+ 24164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.525734 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:05.546707 (+ 20973us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.882330 (+335623us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.986131 (+103801us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.986989 (+   858us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.987242 (+   253us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:06.337464 (+350222us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.337972 (+   508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.338209 (+   237us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.338285 (+    76us) raft_consensus.cc:1685] finished
0709 12:26:06.338297 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.338497 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":346434,"prepare.queue_time_us":131136,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":134,"raft.queue_time_us":228758,"raft.run_cpu_time_us":655,"raft.run_wall_time_us":55043,"thread_start_us":516436,"threads_started":2,"wal-append.queue_time_us":260540}
I20260709 12:26:06.301889 24757 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.389737 24733 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:26:06.355038 24856 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 12:26:06.406755 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 107564us
I20260709 12:26:06.360955 24050 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:06.420711 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55357us
W20260709 12:26:06.421484 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80887.6us
W20260709 12:26:06.427100 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.427186 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:06.388362 24644 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.438405 13645 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: [35.26-35.26]
W20260709 12:26:06.439764 13617 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.873s (SENT)
I20260709 12:26:06.411439 24853 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:06.485687 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:06.485893 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58808.3us
I20260709 12:26:06.486876 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:06.487443 23901 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
W20260709 12:26:06.487851 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 49134.3us
I20260709 12:26:06.489177 23901 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.437724 24864 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:06.460762 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:06.498131 13651 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:06.424207 24720 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:06.483057 13647 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:06.491084 23901 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.502556 23901 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:06.453841 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:06.439108 24694 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.058s
I20260709 12:26:06.418852 24610 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.525938 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:06.516266 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:06.513053 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:06.529405 24590 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:06.530241 24169 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:06.532418 24687 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.777s	user 0.012s	sys 0.000s
I20260709 12:26:06.533661 24853 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.536253 24853 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:06.534978 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 95248.2us
W20260709 12:26:06.539256 13617 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.900s (SENT)
I20260709 12:26:06.494921 24868 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:06.547010 23957 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:06.547763 23957 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:06.523841 24832 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.101s	user 0.006s	sys 0.002s
I20260709 12:26:06.514859 24851 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:06.557631 13651 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:06.559046 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:06.519516 24720 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:06.454651 24759 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.537238 24721 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.537520 24864 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.568514 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:06.544144 24655 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:26:06.570010 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 127806us
I20260709 12:26:06.452999 24470 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.580235 24470 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.317245 24848 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.583039 24848 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:06.542671 24752 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.586809 24752 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.589666 24752 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:06.558897 24772 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.590437 24772 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.592696 24772 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:06.593883 24755 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:06.574579 24721 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.598063 24702 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.612557 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5389) took 1508 ms. Trace:
W20260709 12:26:06.614297 24859 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:06.613288 13650 rpcz_store.cc:276] 0709 12:26:05.104285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.104483 (+   198us) service_pool.cc:224] Handling call
0709 12:26:05.104977 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.105336 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:05.287041 (+181705us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.156618 (+869577us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b18004d0fe0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:26:06.158790 (+  2172us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.159482 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.159662 (+   180us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:06.162316 (+  2654us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.162607 (+   291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.162733 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.327598 (+164865us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:06.612161 (+284563us) raft_consensus.cc:1685] finished
0709 12:26:06.612201 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.612455 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43442,"prepare.queue_time_us":243450,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":1311,"raft.queue_time_us":9408,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":16803,"spinlock_wait_cycles":29555200,"thread_start_us":239242,"threads_started":2,"wal-append.queue_time_us":117658}
I20260709 12:26:06.616523 24610 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:06.564893 24868 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.618047 24868 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.620263 24755 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.620855 24755 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:06.622149 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82927.9us
W20260709 12:26:06.623289 13617 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.120s (SENT)
I20260709 12:26:06.623692 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:06.623929 24755 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:06.624580 24755 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:26:06.624980 13617 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.129s (SENT)
I20260709 12:26:06.625439 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:06.626463 24851 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:06.627461 24702 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:06.628355 24750 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.628921 24750 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.631263 24750 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:06.632380 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:06.633401 13725 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:06.632539 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:06.635406 13724 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:26:06.636278 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:06.638829 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:06.639437 13798 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:06.640650 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:06.641209 13798 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:06.643915 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:06.644156 13725 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:26:06.645892 24749 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:06.615690 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:06.650070 24720 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:06.650556 24720 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:06.650950 24720 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:06.652128 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83730.3us
I20260709 12:26:06.653090 23901 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:06.572610 23957 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:06.647603 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74072.3us
I20260709 12:26:06.662964 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:06.663715 13719 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:06.655285 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:06.644870 13618 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:06.668766 23901 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.671520 13724 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:06.674830 13727 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:26:06.684943 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:06.708900 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:06.770592 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:06.771519 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:06.771754 13723 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 36 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
W20260709 12:26:06.777976 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 162356us
I20260709 12:26:06.684065 24851 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:06.786276 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:26:06.549006 24558 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:06.788816 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.799026 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 174048us
I20260709 12:26:06.677181 24755 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:06.674830 24655 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:06.792301 23901 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.810055 13724 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:06.813153 24751 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:06.791705 24851 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.829072 24688 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.904s	user 0.012s	sys 0.003s
W20260709 12:26:06.829563 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.834575 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.773327 24694 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:06.770773 24837 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:06.862073 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:06.862879 13651 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:06.864490 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:06.866317 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 33 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:06.867434 13652 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 36: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:06.860729 24688 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.805212 24755 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.873172 13800 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:06.876116 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:06.880517 13719 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:06.886471 13652 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:26:06.882968 13651 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:06.852782 24851 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:06.839570 24751 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:06.878862 13617 leader_election.cc:336] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.165s (SENT)
I20260709 12:26:06.902225 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:06.696077 24702 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:06.891783 13647 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:26:06.856786 24721 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:06.860749 24837 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:06.907599 13720 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:06.917768 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.900615 24736 consensus_meta.cc:287] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.001s
I20260709 12:26:06.875207 24694 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:06.892794 24811 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.931095 13719 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:06.889410 24755 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:06.947104 24866 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.087s	user 0.001s	sys 0.001s
I20260709 12:26:06.933080 24702 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:07.017494 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139209us
W20260709 12:26:07.004366 24840 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:26:06.960501 24811 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.940831 24736 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.211658 24848 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:07.033687 24694 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:06.929008 24764 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:06.961609 24868 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:26:07.219246 24699 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:26:07.220525 24720 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.002s
W20260709 12:26:07.221187 24869 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.282s	user 0.000s	sys 0.007s
W20260709 12:26:07.221746 24872 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.361s	user 0.000s	sys 0.007s
W20260709 12:26:07.222832 24510 consensus_meta.cc:287] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.798s	user 0.007s	sys 0.013s
I20260709 12:26:07.006253 24751 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 35.18, Last appended by leader: 18, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:06.893628 24864 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.223471 13644 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:26:07.290711 13644 pending_rounds.cc:85] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 13
I20260709 12:26:07.291033 13644 pending_rounds.cc:107] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:26:07.291314 13644 raft_consensus.cc:2889] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.947770 24865 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:26:07.227731 13722 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.029s	sys 0.037s
W20260709 12:26:07.294421 13722 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.030s	sys 0.037s
I20260709 12:26:06.934307 24655 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:07.228598 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 280197us
W20260709 12:26:07.239619 13617 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.311s (SENT)
W20260709 12:26:07.240327 24875 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.527s	user 0.004s	sys 0.003s
I20260709 12:26:07.240527 24702 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.241216 24837 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.241837 24811 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:07.255810 24853 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.008s
I20260709 12:26:07.316323 13652 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:07.267877 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:07.274078 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.297545 13795 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 4895) took 1742 ms (client timeout 1888 ms). Trace:
I20260709 12:26:07.296250 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:07.304212 13794 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 36 index: 25. Preceding OpId from leader: term: 38 index: 26. (index mismatch)
I20260709 12:26:07.305573 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.308017 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:07.312585 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:07.293977 24871 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.325s	user 0.003s	sys 0.005s
I20260709 12:26:07.253278 24851 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.285848 24610 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.319720 13795 rpcz_store.cc:269] 0709 12:26:05.555234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.556401 (+  1167us) service_pool.cc:224] Handling call
0709 12:26:07.297213 (+1740812us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000e47d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:07.297451 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775452544}
I20260709 12:26:07.322924 13647 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:26:07.345856 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.346838 13643 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 36 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:26:07.331048 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.326000 24751 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:07.333232 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2261) took 1440 ms (client timeout 1796 ms). Trace:
I20260709 12:26:07.336498 24771 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 32.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:07.337312 24558 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.359004 24558 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.361850 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1756) took 1220 ms. Trace:
I20260709 12:26:07.362751 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2266) took 1036 ms. Trace:
I20260709 12:26:07.363862 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5416) took 1061 ms. Trace:
I20260709 12:26:07.337461 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:07.338749 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.290169 24510 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.322152 24814 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:07.258376 24848 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.350642 13723 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 33: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:07.350046 13720 rpcz_store.cc:269] 0709 12:26:05.892908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.929986 (+ 37078us) service_pool.cc:224] Handling call
0709 12:26:07.333003 (+1403017us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54003bea54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:07.333164 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223205760}
I20260709 12:26:07.371315 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5418) took 1039 ms. Trace:
I20260709 12:26:07.285815 24720 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.285413 24699 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.343281 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.362421 13792 rpcz_store.cc:276] 0709 12:26:06.129130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.129393 (+   263us) service_pool.cc:224] Handling call
0709 12:26:06.153105 (+ 23712us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:06.172113 (+ 19008us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.172171 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.197668 (+ 25497us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.520128 (+322460us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:06.520176 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.520251 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.349302 (+829051us) raft_consensus.cc:1685] finished
0709 12:26:07.349329 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.349484 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":184,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":270044,"threads_started":1,"wal-append.queue_time_us":445802}
I20260709 12:26:07.364359 13648 rpcz_store.cc:276] 0709 12:26:06.302568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.308712 (+  6144us) service_pool.cc:224] Handling call
0709 12:26:06.309115 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.312383 (+  3268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.312433 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.320389 (+  7956us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.795133 (+474744us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:06.795185 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.795274 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.363608 (+568334us) raft_consensus.cc:1685] finished
0709 12:26:07.363634 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.363794 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2727,"raft.queue_time_us":7331,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5964,"thread_start_us":436085,"threads_started":1,"wal-append.queue_time_us":559560}
I20260709 12:26:07.363197 13726 rpcz_store.cc:276] 0709 12:26:06.326345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.332953 (+  6608us) service_pool.cc:224] Handling call
0709 12:26:06.360019 (+ 27066us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.401406 (+ 41387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.401473 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.411930 (+ 10457us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.519199 (+107269us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:06.519251 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.519332 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.362466 (+843134us) raft_consensus.cc:1685] finished
0709 12:26:07.362492 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.362685 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40757,"raft.queue_time_us":42846,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":10981,"thread_start_us":64065,"threads_started":1,"wal-append.queue_time_us":119091}
I20260709 12:26:07.329674 24837 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:07.383955 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.384740 13727 raft_consensus.cc:2368] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:26:07.379947 13649 rpcz_store.cc:276] 0709 12:26:06.332305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.332477 (+   172us) service_pool.cc:224] Handling call
0709 12:26:06.346724 (+ 14247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.347177 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.347218 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.347822 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.563578 (+215756us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:06.563652 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.563773 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.371080 (+807307us) raft_consensus.cc:1685] finished
0709 12:26:07.371105 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.371255 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1410,"spinlock_wait_cycles":33183616,"thread_start_us":215214,"threads_started":1,"wal-append.queue_time_us":535841}
I20260709 12:26:07.344367 24811 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:07.349656 24853 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:07.385365 24757 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:07.397951 14422 pending_rounds.cc:85] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 18
I20260709 12:26:07.395066 24755 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.398636 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:07.399957 14422 pending_rounds.cc:107] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:26:07.402541 14422 raft_consensus.cc:2889] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:07.402336 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 162740us
W20260709 12:26:07.403975 13617 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.668s (SENT)
I20260709 12:26:07.404518 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:07.407317 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.408370 13727 raft_consensus.cc:2368] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23.
I20260709 12:26:07.362085 24558 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.367043 24688 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.344513 24569 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:26:07.413172 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.414361 13727 raft_consensus.cc:2368] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24.
I20260709 12:26:07.391148 24811 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.354990 24610 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.423713 13646 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 37: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:06.344321 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:07.427724 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.428613 13726 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.412966 24851 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.433393 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.435398 13727 raft_consensus.cc:2368] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 32. Current term is 33.
W20260709 12:26:07.438551 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 170975us
I20260709 12:26:07.422705 24764 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:07.440430 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:07.424525 24811 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:07.440754 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.442101 13727 raft_consensus.cc:2410] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:26:07.400272 24814 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.430022 24655 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:07.449215 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45283.8us
W20260709 12:26:07.450886 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.451891 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.452365 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.452216 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.452975 13727 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37.
W20260709 12:26:07.447132 24887 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:26:07.458817 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.460608 24673 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:26:07.469995 13651 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
W20260709 12:26:07.464802 24878 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:26:07.472595 13727 raft_consensus.cc:2368] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26.
W20260709 12:26:07.470865 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.485317 13726 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:26:07.486869 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.488555 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.489439 13727 raft_consensus.cc:2410] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 35 because replica has last-logged OpId of term: 34 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:26:07.462733 24892 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:07.500171 13649 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:07.518853 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.520143 13727 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:07.456714 24655 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.477854 24755 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.527565 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:07.513223 24889 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:26:07.533385 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:07.530105 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77148.8us
I20260709 12:26:07.537153 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:07.540382 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:07.535588 24883 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.152s	user 0.000s	sys 0.002s
I20260709 12:26:07.547672 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.545476 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:07.546612 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.548612 13725 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:07.545559 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.531401 24851 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:07.545078 24755 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.548935 24702 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.002s
I20260709 12:26:07.549479 13726 raft_consensus.cc:2376] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26. Re-sending same reply.
I20260709 12:26:07.535424 24510 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.551321 13727 raft_consensus.cc:2368] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27.
I20260709 12:26:07.544287 24892 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:07.541224 24720 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.559460 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.563232 24655 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.547636 24694 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:07.567721 13726 raft_consensus.cc:2368] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27.
I20260709 12:26:07.565017 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.570055 13719 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:07.571435 13726 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 12. (term mismatch)
I20260709 12:26:07.573073 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:07.574386 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.571803 13726 pending_rounds.cc:85] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 11
I20260709 12:26:07.575410 13726 pending_rounds.cc:107] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 24.12
I20260709 12:26:07.575773 13726 raft_consensus.cc:2889] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:07.555418 24699 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:07.576318 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:07.573644 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.580861 13725 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:07.582981 13727 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 23. Current term is 24. Ops: []
W20260709 12:26:07.574400 24895 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:26:07.591027 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.585136 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:07.565969 24851 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.586561 24893 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:26:07.572570 24900 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:07.572326 24694 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.580555 24655 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:07.590528 24848 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.594635 24892 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:07.594517 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.596974 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:07.598027 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:07.603251 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:07.612504 13651 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:07.605137 24894 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:26:07.604362 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.556424 24755 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.609170 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.610431 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.612314 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.617017 13719 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 33: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:07.616724 13726 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:07.618006 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
W20260709 12:26:07.606195 24896 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:26:07.613705 13725 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 30 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:07.604020 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.591401 24699 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:07.618848 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.608134 24694 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.620864 13643 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:07.621640 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.622578 13719 raft_consensus.cc:2368] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37.
I20260709 12:26:07.622471 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.614792 24610 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.625133 13651 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:26:07.632045 24755 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.635283 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.640650 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.637116 13725 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:07.642441 13727 raft_consensus.cc:2368] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 32. Current term is 33.
I20260709 12:26:07.643314 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.643916 13727 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:07.645660 13719 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:07.646637 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.647330 13719 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.648155 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.649222 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.649951 24169 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:07.642114 13725 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:07.650209 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:07.650238 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.653614 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:07.671641 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.672070 14421 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
W20260709 12:26:07.672151 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.654003 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:07.661288 13652 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:26:07.652062 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:07.652350 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:07.668231 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.673401 14417 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:07.654778 14420 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 24. Preceding OpId from leader: term: 38 index: 26. (index mismatch)
I20260709 12:26:07.657527 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:07.673346 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:07.683768 14424 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.689245 13726 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:26:07.690317 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.650619 24169 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.691458 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.659188 24470 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:07.695277 24848 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:07.678583 13725 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 15, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.716624 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.717553 13721 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.718467 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.719692 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.720414 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:07.720892 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.721096 14423 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:07.722911 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:07.723371 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.723698 24848 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:07.724123 24848 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:07.724476 24848 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:07.718905 24169 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.715764 24470 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:07.728168 24470 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.729446 24470 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:07.713331 24699 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:07.733639 13725 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:07.702410 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4906) took 1069 ms. Trace:
I20260709 12:26:07.702036 24811 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.702879 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.664363 24851 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.743155 24811 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:07.747108 24587 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.747826 24587 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.751957 13652 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 36 index: 19. (index mismatch)
W20260709 12:26:07.704802 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.707049 14417 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:07.756330 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.666201 24814 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.712448 24751 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:07.712899 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4823) took 1052 ms. Trace:
I20260709 12:26:07.714968 13726 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:07.739423 13794 rpcz_store.cc:276] 0709 12:26:06.632760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.635179 (+  2419us) service_pool.cc:224] Handling call
0709 12:26:07.297455 (+662276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.320004 (+ 22549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.702168 (+382164us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54000e47d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:07.702344 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522113792}
I20260709 12:26:07.756997 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:07.733579 24848 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.765581 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:07.757578 13722 rpcz_store.cc:276] 0709 12:26:06.660424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.660592 (+   168us) service_pool.cc:224] Handling call
0709 12:26:06.662879 (+  2287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.663117 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:07.294848 (+631731us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.446386 (+151538us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.664646 (+218260us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b18001bf400. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:07.712086 (+ 47440us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b18001bf400. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:26:07.712334 (+   248us) log.cc:844] Serialized 14 byte log entry
0709 12:26:07.712537 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:07.712601 (+    64us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:07.712654 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.712726 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.712840 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":259802,"prepare.queue_time_us":853522,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":118,"raft.queue_time_us":3243,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":20773,"spinlock_wait_cycles":24762496,"thread_start_us":746104,"threads_started":2,"wal-append.queue_time_us":189898}
I20260709 12:26:07.768071 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.768051 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.771430 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.774978 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.775329 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:07.775952 13795 raft_consensus.cc:2376] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38. Re-sending same reply.
I20260709 12:26:07.749130 24728 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:07.749153 24882 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.333s	user 0.001s	sys 0.001s
I20260709 12:26:07.790832 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.797153 13722 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:07.796584 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:07.779428 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:07.783346 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:07.803725 13800 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:07.783988 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:07.784003 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:07.785306 13725 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 12:26:07.804728 13794 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:07.787964 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.805528 13726 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.807663 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.808350 13721 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:07.774854 24814 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.788692 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.804874 13725 pending_rounds.cc:85] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 16
I20260709 12:26:07.837072 13725 pending_rounds.cc:107] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:26:07.838128 13792 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:07.838042 13649 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:26:07.844326 13725 raft_consensus.cc:2889] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:07.810132 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.837531 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.787396 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.849969 24510 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.854013 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:07.836365 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57087us
I20260709 12:26:07.855705 13719 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:07.855590 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.783099 24755 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:07.863973 13726 raft_consensus.cc:2410] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:07.851227 24764 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.869982 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.870004 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.871855 13722 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:07.874325 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:07.876245 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.806717 24749 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.877004 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:07.835647 24851 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.878278 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:07.887013 13649 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:07.876282 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.837896 24752 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:07.838032 24833 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:07.882579 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.883195 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.896541 13726 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:07.896863 13725 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:26:07.884090 13721 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:07.884256 13798 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:07.786931 24754 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.088s
I20260709 12:26:07.851578 24169 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.907263 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4903) took 1888 ms. Trace:
I20260709 12:26:07.887274 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.908293 13722 raft_consensus.cc:2368] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:26:07.816579 24050 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:26:07.888497 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.898910 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.914350 13721 raft_consensus.cc:2410] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:26:07.910008 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:07.915306 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.916035 13721 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:07.917007 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.917834 13725 raft_consensus.cc:2368] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27.
I20260709 12:26:07.918802 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.919145 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.919175 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.920174 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.920511 13792 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:07.920959 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.921689 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.923123 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.924222 24865 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:07.924329 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.924988 24865 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:07.926546 13720 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:07.927249 24587 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:07.877157 24755 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:07.928102 13720 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:07.929322 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.789960 24694 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.850611 24510 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.931249 24848 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.932482 24510 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.932118 24169 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.933568 14418 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.934857 24169 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.935628 13720 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:07.790562 24728 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.937026 13720 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:07.862596 24814 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.938244 24728 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.938926 13720 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:07.906198 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:07.940539 13795 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:07.889596 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:07.907740 13801 rpcz_store.cc:276] 0709 12:26:06.018916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.019069 (+   153us) service_pool.cc:224] Handling call
0709 12:26:06.060686 (+ 41617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.141727 (+ 81041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.141763 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.811026 (+669263us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b2c0011ee98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:26:06.811568 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.935156 (+123588us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:06.935299 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.935394 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.748520 (+813126us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:07.907010 (+158490us) raft_consensus.cc:1685] finished
0709 12:26:07.907038 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.907196 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73092,"raft.queue_time_us":95440,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":22745,"spinlock_wait_cycles":28333184,"thread_start_us":123169,"threads_started":1,"wal-append.queue_time_us":537958}
I20260709 12:26:07.942090 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.942423 13721 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:07.928794 24755 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.807125 24772 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 33.25, Next index: 26, Last known committed idx: 24, Time since last communication: 0.077s
W20260709 12:26:07.939080 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.959071 13793 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:07.961531 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.959954 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:07.963135 13722 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 30 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:07.964498 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:07.964761 24814 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:07.965705 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:07.873229 24764 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:07.966434 13725 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:07.968433 24764 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:07.968827 13725 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:26:07.969197 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.970122 13801 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:07.939208 24853 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:26:07.974433 13650 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:07.974800 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5423) took 1548 ms. Trace:
I20260709 12:26:07.975585 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:07.976519 13646 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:07.974661 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:07.978132 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:07.975291 13644 rpcz_store.cc:276] 0709 12:26:06.426422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.426614 (+   192us) service_pool.cc:224] Handling call
0709 12:26:06.451045 (+ 24431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.291813 (+840768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.974546 (+682733us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400095454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:07.974739 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236645248}
I20260709 12:26:07.978978 13725 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:26:07.987733 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.986892 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:07.988837 13650 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:07.988859 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:07.988111 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 110892us
I20260709 12:26:07.991588 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.991923 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:07.994056 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:07.998219 13793 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:07.999650 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:08.000437 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.001053 13725 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:26:07.958889 24751 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:08.003578 13726 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:08.007885 13726 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:08.009251 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:08.009685 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.010712 13719 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:08.011668 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:08.005326 13648 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 34: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:08.006075 13646 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 36. Current term is 37. Ops: [36.26-36.26]
I20260709 12:26:08.016408 13795 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:07.892146 24749 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:07.912475 24426 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.085s
I20260709 12:26:07.897918 24851 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.032150 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:08.033210 13722 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 18. Preceding OpId from leader: term: 36 index: 19. (index mismatch)
I20260709 12:26:08.047084 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.038985 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:08.045090 24749 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:08.047881 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.041225 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.058677 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.048377 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:08.041765 13726 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.061131 13725 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:08.069303 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78908.2us
I20260709 12:26:08.071666 24759 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:08.073380 13726 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:08.074860 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:08.075230 13646 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:08.075994 13722 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:08.002920 24694 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:08.077700 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.078627 13795 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:26:08.081013 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.083104 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.084120 13649 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:08.040421 24673 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:26:08.062628 24901 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:08.056674 24828 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:08.087453 24884 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.826s	user 0.001s	sys 0.001s
I20260709 12:26:08.103926 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.087870 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:08.105134 13719 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:08.079582 24655 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 35.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.011s
I20260709 12:26:08.006628 24893 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.072s
I20260709 12:26:08.111545 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.094781 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.079077 24610 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:08.102716 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.079487 24644 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.107699 13646 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:26:08.109325 13649 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:08.002720 24900 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.071s
W20260709 12:26:08.114589 13693 connection.cc:256] RPC call timeout handler was delayed by -5.0787191449999227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 12:26:08.115885 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:08.106599 24814 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.121332 24851 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:08.124819 13721 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:26:08.127130 24907 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:26:08.128545 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1742) took 1275 ms. Trace:
I20260709 12:26:08.128592 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.129043 13645 rpcz_store.cc:276] 0709 12:26:06.853214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.853392 (+   178us) service_pool.cc:224] Handling call
0709 12:26:06.871665 (+ 18273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.883909 (+ 12244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.883958 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.722529 (+838571us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:26:07.723024 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.859544 (+136520us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:07.859630 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.859733 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.128231 (+268498us) raft_consensus.cc:1685] finished
0709 12:26:08.128255 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.128468 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1465,"raft.queue_time_us":128,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":10782,"spinlock_wait_cycles":3747200,"thread_start_us":135826,"threads_started":1,"wal-append.queue_time_us":271620}
I20260709 12:26:08.132650 13719 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:26:08.132611 13800 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:08.133234 13725 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:08.129410 13796 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:08.094254 24901 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:08.135648 13618 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:08.158612 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.114821 24828 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:08.115825 24688 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.119335 24814 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:08.179931 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 76094.4us
I20260709 12:26:08.181067 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:08.166715 13693 connection.cc:256] RPC call timeout handler was delayed by -4.8987191450000864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
W20260709 12:26:08.182533 13693 connection.cc:256] RPC call timeout handler was delayed by -4.8897191450000719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
W20260709 12:26:08.182862 13693 connection.cc:256] RPC call timeout handler was delayed by -4.8285191449999729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260709 12:26:08.183161 13693 connection.cc:256] RPC call timeout handler was delayed by -4.7547191449998536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 12:26:08.183457 13693 connection.cc:256] RPC call timeout handler was delayed by -4.6674191449997124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19970000000000002s for a response
W20260709 12:26:08.183744 13693 connection.cc:256] RPC call timeout handler was delayed by -4.6305191450001075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 12:26:08.188285 13693 connection.cc:256] RPC call timeout handler was delayed by -4.6008191450000595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20710000000000003s for a response
W20260709 12:26:08.188612 13693 connection.cc:256] RPC call timeout handler was delayed by -4.0865264839999327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:08.171142 24754 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.018s
I20260709 12:26:08.169543 24426 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:08.165154 24901 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:08.195604 13796 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:26:08.138366 24644 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.233203 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.175406 24771 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.257158 24721 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:08.257409 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63721us
I20260709 12:26:08.256937 13650 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 27.
I20260709 12:26:08.258546 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.259493 24721 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:08.260200 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:08.261253 24721 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:08.262728 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.263336 24736 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:08.263754 13644 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:08.267228 13643 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:26:08.268090 24891 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.833s	user 0.000s	sys 0.002s
I20260709 12:26:08.268522 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:08.268860 13645 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:26:08.269220 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:08.272094 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.272714 13645 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:08.274497 13646 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:08.275345 13794 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:08.276300 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.276880 13794 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:08.242359 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:08.277438 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.243671 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:08.278895 13651 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:08.245787 13648 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:26:08.247408 13647 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:26:08.279110 13793 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 36. Current term is 37. Ops: [36.26-36.26]
I20260709 12:26:08.286434 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.290592 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.183391 24688 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.292384 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.299877 13645 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:08.300151 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:08.293581 13647 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:08.298048 13651 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:08.304806 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.305156 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:08.306938 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.319006 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.320654 24798 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:08.328503 24644 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:08.251103 24587 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:26:08.330348 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:08.331239 13648 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:08.332266 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:08.273047 24771 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.334573 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.335458 24771 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:08.252357 24865 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.288555 24590 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:08.336839 13792 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:08.338451 24865 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:08.339743 24688 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:08.340204 13617 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.891s (SENT)
I20260709 12:26:08.340704 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:08.340965 24470 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
I20260709 12:26:08.342131 24426 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.332456 24814 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:08.336237 13796 raft_consensus.cc:1217] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Deduplicated request from leader. Original: 29.19->[31.20-31.20]   Dedup: 31.20->[]
I20260709 12:26:08.328276 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:08.342224 13617 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.936s (SENT)
W20260709 12:26:08.347198 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 161257us
W20260709 12:26:08.349514 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.351572 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.343545 13799 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:08.353391 13794 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:08.374989 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:08.344856 13796 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32. Ops: [31.20-31.20]
I20260709 12:26:08.387727 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:08.353693 14419 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
W20260709 12:26:08.399829 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57631.4us
W20260709 12:26:08.398376 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.352646 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.345566 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.343914 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: true dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:08.404330 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:08.409092 13647 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:08.410079 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:08.411011 14419 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:08.410068 24893 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.007s
I20260709 12:26:08.412665 24900 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.412815 24688 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.413214 24900 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.414505 24694 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:08.410141 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:08.418581 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 43723.4us
I20260709 12:26:08.419659 13618 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:08.346642 24655 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:08.420159 24900 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:08.359319 24590 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.422366 24814 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:08.422319 24590 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.374562 13542 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 16 to 27, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:08.423216 24814 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:08.424312 24814 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.25, Last appended by leader: 25, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.425052 24590 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:08.420676 24655 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:08.422902 24868 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:08.388334 13645 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
W20260709 12:26:08.442128 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.451648 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.346807 24610 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:08.452692 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:08.457603 14420 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:08.453047 13801 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
W20260709 12:26:08.455137 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50889.2us
W20260709 12:26:08.461081 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.468042 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122592us
I20260709 12:26:08.469064 13801 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:08.470698 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:08.469666 13647 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:08.395258 24699 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:26:08.482410 14416 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:08.453925 24853 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:08.449044 24868 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:08.347442 24917 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.116s
I20260709 12:26:08.496366 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:08.497689 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:08.506078 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:08.354465 24901 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.456511 24729 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:08.501331 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:08.517529 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:08.540634 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.543809 13723 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:08.557579 13718 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:08.547525 24868 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:08.555315 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57716.9us
I20260709 12:26:08.501343 24558 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:08.502310 24751 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:08.568003 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 71749.2us
I20260709 12:26:08.541610 24688 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.572829 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:08.558774 24899 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:26:08.574472 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:08.495740 24865 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.082s
W20260709 12:26:08.585005 24864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.044s	sys 0.012s
I20260709 12:26:08.565034 24510 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:08.515380 24470 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:26:08.565636 24694 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:08.600368 13648 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:08.601316 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:08.608035 13650 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:26:08.588305 24864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.227s	user 0.045s	sys 0.012s
I20260709 12:26:08.553686 24610 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.625913 13644 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: [35.26-35.26]
I20260709 12:26:08.591715 24510 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:08.629695 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 217450us
W20260709 12:26:08.630872 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 177356us
I20260709 12:26:08.598934 24721 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:08.608167 24768 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:08.652494 14419 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:08.672540 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:08.616132 24868 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:08.673911 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80290.3us
I20260709 12:26:08.586141 24688 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.610772 24862 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:08.685683 13650 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
W20260709 12:26:08.676260 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83962.7us
I20260709 12:26:08.650806 24510 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:08.653757 24768 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.573724 24655 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 36.26, Last appended by leader: 26, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:08.691371 13617 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.512s (SENT)
I20260709 12:26:08.692734 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:08.647871 24736 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.039s
I20260709 12:26:08.693074 24768 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:08.692574 24721 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.682209 24772 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.020s
W20260709 12:26:08.715072 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.683799 24699 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.021s
W20260709 12:26:08.716573 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 215333us
I20260709 12:26:08.719061 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:08.720284 13647 raft_consensus.cc:2410] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:26:08.751868 24752 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:08.752645 24752 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:08.753750 24752 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 35.26, Last appended by leader: 26, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:08.757828 24752 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:08.762776 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4849) took 1029 ms. Trace:
I20260709 12:26:08.763283 13720 rpcz_store.cc:276] 0709 12:26:07.733830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.955887 (+222057us) service_pool.cc:224] Handling call
0709 12:26:07.956275 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:07.975507 (+ 19232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.975540 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:07.976280 (+   740us) log.cc:844] Serialized 52 byte log entry
0709 12:26:08.080585 (+104305us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:08.107647 (+ 27062us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.107869 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.107942 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.747108 (+639166us) spinlock_profiling.cc:243] Waited 9.93 ms on lock 0x7b18001bab40. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:08.762383 (+ 15275us) raft_consensus.cc:1685] finished
0709 12:26:08.762494 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.762699 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8714,"prepare.queue_time_us":22633,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":21140,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":3812,"spinlock_wait_cycles":22846336,"thread_start_us":103873,"threads_started":1,"wal-append.queue_time_us":140670}
I20260709 12:26:08.714815 24655 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:08.699069 24426 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.038s
W20260709 12:26:08.771538 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.703246 24892 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
W20260709 12:26:08.775514 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 93829.8us
I20260709 12:26:08.776995 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:08.776167 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84821.9us
W20260709 12:26:08.781307 13617 leader_election.cc:336] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.596s (SENT)
I20260709 12:26:08.781806 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:08.770480 24928 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260709 12:26:08.831084 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.730556 24721 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.702474 24694 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:08.870906 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 180242us
W20260709 12:26:08.883018 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 112090us
I20260709 12:26:08.838537 24901 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:08.775854 24944 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.882921 13718 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:08.885326 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:08.830976 24759 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:08.829263 24945 raft_consensus.cc:2764] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
W20260709 12:26:08.889460 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59068.4us
W20260709 12:26:08.878962 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 86173.4us
W20260709 12:26:08.904592 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.862143 24939 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:08.905323 13617 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.788s (SENT)
W20260709 12:26:08.886672 24931 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 12:26:08.913247 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:08.920259 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.920259 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:08.921105 13648 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:08.914276 24924 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:26:08.918947 24759 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.940516 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45139.1us
W20260709 12:26:08.941356 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.940991 24926 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
W20260709 12:26:08.970050 24927 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
W20260709 12:26:08.953715 24925 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
W20260709 12:26:08.982666 13617 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 2.007s (SENT)
I20260709 12:26:08.983274 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:08.950802 24688 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:08.979411 24947 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:26:08.945984 24759 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:08.895925 24944 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.048792 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.049690 24688 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:08.993086 24940 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:09.055006 13648 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
W20260709 12:26:09.067102 24951 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:26:08.979811 24959 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.088s
I20260709 12:26:09.073855 14419 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:09.050132 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.080150 24759 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.098654 24721 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.100934 24944 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.108762 24853 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.109398 24853 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:09.111831 24798 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:09.112524 24798 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:09.112706 24967 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:09.113106 24798 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.113256 24967 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:09.115377 24967 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.117694 24967 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:09.105957 24921 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.547s	user 0.000s	sys 0.003s
W20260709 12:26:09.121218 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71146.9us
I20260709 12:26:09.134234 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.135586 24721 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:09.136405 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.137349 24721 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:09.137476 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.137838 24721 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:09.138063 24955 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:26:09.138348 24721 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:09.138968 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.142577 24950 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:09.145850 24965 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:09.148288 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.152920 24950 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:09.153789 24965 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:09.154927 24965 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.160036 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.166240 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.171273 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.169373 24950 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.174762 24848 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:09.176460 24699 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:09.180287 24944 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:26:09.238088 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.245229 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.178848 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:09.246814 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.247769 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69031.8us
I20260709 12:26:09.249053 13617 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:09.237607 24768 consensus_meta.cc:287] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.005s
I20260709 12:26:09.257647 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.250957 24939 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:09.251510 24644 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.264191 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.264045 24644 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.265306 24688 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:09.255390 24959 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.257808 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.266760 13722 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:26:09.266607 24959 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.268011 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.260390 24768 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.269079 24959 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.269215 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.250689 24759 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:09.270782 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.271934 24959 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:09.272086 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.272871 24644 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.254912 24969 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.260785 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.274286 24969 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.275084 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:09.253772 24950 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.276551 24969 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.253088 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.262627 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:09.262730 24970 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:26:09.263509 24868 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.576s	user 0.016s	sys 0.001s
I20260709 12:26:09.273439 24959 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:09.260515 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.277194 13722 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:09.273478 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.281242 24959 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:09.281880 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.177613 24510 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.283113 24759 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:09.259387 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.261564 24961 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.278016 24950 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.259268 24699 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:09.284997 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.283253 24768 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:09.256598 24964 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:09.284088 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:09.285161 24959 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:09.305305 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.312071 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.314005 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.300127 24736 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.315340 24736 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.316169 13617 leader_election.cc:400] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:26:09.316506 13617 leader_election.cc:403] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:26:09.317509 24736 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.317734 24759 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:09.318163 24759 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:09.318635 24759 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.322576 24644 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:09.322739 24939 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 24. Current term is 25.
I20260709 12:26:09.317199 24853 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.327080 24853 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:09.335274 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.336818 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.337528 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1766) took 1155 ms. Trace:
I20260709 12:26:09.337667 24610 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.337741 24644 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:09.338184 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.332564 24759 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.341602 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.371975 13647 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:09.363973 13792 rpcz_store.cc:276] 0709 12:26:08.182193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.337992 (+155799us) service_pool.cc:224] Handling call
0709 12:26:08.383360 (+ 45368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.383923 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.383967 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:08.396797 (+ 12830us) log.cc:844] Serialized 52 byte log entry
0709 12:26:08.588849 (+192052us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:08.610995 (+ 22146us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.651525 (+ 40530us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.133095 (+481570us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b48001b0258. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:26:09.289728 (+156633us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18003ce7a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:09.336937 (+ 47209us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b18003ce7a0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:26:09.337154 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.337239 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.337291 (+    52us) raft_consensus.cc:1685] finished
0709 12:26:09.337305 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.337464 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276026,"prepare.queue_time_us":676038,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":2218,"raft.queue_time_us":713520,"raft.run_cpu_time_us":1984,"raft.run_wall_time_us":488057,"spinlock_wait_cycles":17356672,"thread_start_us":136085,"threads_started":1,"wal-append.queue_time_us":545270}
I20260709 12:26:09.342661 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.342748 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:09.343582 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 660316us
I20260709 12:26:09.375111 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:09.344864 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 658083us
I20260709 12:26:09.345329 24862 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:09.309587 24893 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.001s	sys 0.047s
I20260709 12:26:09.346400 24558 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:09.353502 24853 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:09.359269 24729 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:09.363808 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:09.364975 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:09.366861 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1767) took 1179 ms. Trace:
I20260709 12:26:09.395118 13794 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:09.365069 13617 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.397881 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:09.368400 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:09.368559 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.399055 13795 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:09.399168 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:09.309814 24950 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:09.369781 13801 pending_rounds.cc:85] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 21
I20260709 12:26:09.404449 13801 pending_rounds.cc:107] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 32.22
I20260709 12:26:09.404745 13801 raft_consensus.cc:2889] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:09.370150 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.380569 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:09.390810 24893 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.001s	sys 0.047s
W20260709 12:26:09.411986 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47082.6us
I20260709 12:26:09.412865 13617 leader_election.cc:400] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27.
I20260709 12:26:09.356355 24798 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.313591 24969 raft_consensus.cc:2764] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:26:09.398079 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:09.396992 24050 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:09.409605 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.412801 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.423574 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.424333 13649 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:26:09.413354 13617 leader_election.cc:403] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:26:09.385982 13799 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:09.383499 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.392360 24967 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:09.418509 13795 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:09.395196 13798 rpcz_store.cc:276] 0709 12:26:08.187262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.365971 (+178709us) service_pool.cc:224] Handling call
0709 12:26:08.754784 (+388813us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:26:08.755085 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.755400 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:08.755868 (+   468us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.358405 (+602537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.366474 (+  8069us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.366533 (+    59us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:09.366590 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.366672 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.366799 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":508834,"prepare.queue_time_us":1449,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":512540,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":10821,"spinlock_wait_cycles":42087296,"thread_start_us":100830,"threads_started":1,"wal-append.queue_time_us":297209}
I20260709 12:26:09.428053 13650 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:09.371284 24939 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.048s
I20260709 12:26:09.303385 24961 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.438827 13726 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:09.439236 24944 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 26. Current term is 27.
I20260709 12:26:09.410988 24688 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:09.372385 24610 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.443822 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.407040 24946 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:09.418787 24814 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:09.448550 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:09.423802 24050 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.402810 24721 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.478734 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.463038 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.419582 24470 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:26:09.424157 24833 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:26:09.481283 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.484232 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.485018 24848 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:26:09.485482 24976 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:26:09.486424 24969 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:09.429149 24752 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:26:09.493695 24969 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.465205 24865 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.429803 24768 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:26:09.491159 24764 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:09.497823 24865 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.433629 24868 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:26:09.464316 24946 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.498181 24764 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:09.500234 24865 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:09.490885 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.500221 24946 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.501219 24764 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.497352 24969 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:09.508175 13718 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:09.483974 24811 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:09.477131 24050 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:09.467383 24967 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.407284 24699 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.483250 24915 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:09.506512 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.487778 24944 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:09.469084 24853 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.044s
W20260709 12:26:09.581073 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 157587us
W20260709 12:26:09.581667 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61702.8us
I20260709 12:26:09.514843 24688 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:09.516856 24917 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.582540 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.516089 24950 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:09.520403 24811 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.510130 24910 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:26:09.532397 24959 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:09.532917 24510 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.472018 24862 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:09.595907 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 197929us
I20260709 12:26:09.620774 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:09.622395 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:09.581562 24965 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:09.627691 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:09.525302 24755 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.096s
I20260709 12:26:09.667166 13794 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:09.628314 13793 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:09.630966 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:09.591460 24688 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.639693 24759 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.640100 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:09.640935 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.777s (SENT) [suppressed 28 similar messages]
I20260709 12:26:09.536451 24967 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.599320 24917 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.643210 24954 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:09.644299 13800 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:26:09.645606 24978 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 12:26:09.646577 24980 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:26:09.591818 24644 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.625646 24944 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:09.652899 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:09.654574 14419 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 37 index: 26. (term mismatch)
I20260709 12:26:09.597527 24811 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:09.667491 24979 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:26:09.677089 24982 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:09.685753 24798 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.679204 24721 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.683771 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.686422 24764 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:09.700177 24644 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.706583 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.740211 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:09.729054 24967 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.727516 24811 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.725265 24946 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.745715 24811 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:09.689360 24050 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.666749 24965 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.673233 24688 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:09.699275 14419 pending_rounds.cc:85] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 25
I20260709 12:26:09.691478 24954 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.755841 24954 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:09.758416 24764 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.759080 24764 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.706043 24995 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:09.747242 13647 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:09.720865 24961 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.731071 24644 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.753422 14419 pending_rounds.cc:107] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 36.26
I20260709 12:26:09.722554 24610 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.764405 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:09.771539 13795 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.749221 24751 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:09.748984 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.761961 24764 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.769505 14419 raft_consensus.cc:2889] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.752969 24965 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:09.772275 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 131367us
I20260709 12:26:09.754845 24946 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.753727 24939 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:09.754308 24755 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.784034 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.785231 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.785910 13649 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
W20260709 12:26:09.788064 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.789240 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.790158 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:09.791040 13650 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:09.792646 24959 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:09.793152 24959 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.793494 24959 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.793888 24946 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:09.794909 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.795518 13644 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:09.796757 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.797531 24798 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.798583 24967 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:09.798296 24798 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.799075 24967 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.799515 24967 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:09.801653 24798 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:09.804363 24917 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.805193 13652 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:09.781435 24961 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.802433 24944 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.789381 13795 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:09.780959 24644 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:09.807894 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59030.9us
W20260709 12:26:09.803025 24983 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:26:09.826779 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:09.833838 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.835283 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:09.836150 13719 pending_rounds.cc:85] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 16
I20260709 12:26:09.836477 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:09.837165 13724 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.840215 24963 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.840883 24963 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.843106 13721 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:26:09.843502 24963 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.844360 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.845057 13721 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:09.845522 24915 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:09.836501 13719 pending_rounds.cc:107] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 26.17
I20260709 12:26:09.846144 13719 raft_consensus.cc:2889] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.829931 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:09.830996 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:09.832057 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:09.849766 24944 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.852103 24944 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.774662 24969 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.856142 24969 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.856962 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.831840 24644 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.859684 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:09.865492 13651 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:09.861056 13724 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:09.861150 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:09.863593 24940 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:09.867841 24940 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:09.868857 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.869133 24940 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.867391 13721 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:09.857749 13644 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
W20260709 12:26:09.881177 24962 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.631s	user 0.011s	sys 0.004s
I20260709 12:26:09.883881 24644 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.826624 24798 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.067s
I20260709 12:26:09.901628 24969 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:09.900480 24959 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.911804 13651 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
W20260709 12:26:09.920549 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 324391us
I20260709 12:26:09.922160 13644 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
W20260709 12:26:09.923386 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 289698us
I20260709 12:26:09.926466 24833 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:09.928273 24833 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:09.929328 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.933878 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.935638 13647 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:26:09.938558 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:09.939402 13647 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:09.941685 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:09.943107 13724 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:26:09.951834 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.963447 13650 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:09.790531 24755 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:09.777567 24995 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.153s
I20260709 12:26:09.966970 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:09.975360 13649 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:09.969842 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:09.973870 13651 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:09.979971 24988 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:09.980672 24988 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:09.981279 24981 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.546s	user 0.016s	sys 0.000s
I20260709 12:26:09.982564 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2289) took 1098 ms. Trace:
I20260709 12:26:09.983035 13727 rpcz_store.cc:276] 0709 12:26:08.884236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.884494 (+   258us) service_pool.cc:224] Handling call
0709 12:26:08.884841 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.950046 (+ 65205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.950075 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.957314 (+  7239us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.326726 (+369412us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b18001c55e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:26:09.414589 (+ 87863us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:09.414642 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.414694 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.923476 (+508782us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:09.982289 (+ 58813us) raft_consensus.cc:1685] finished
0709 12:26:09.982315 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.982477 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73759,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":13251,"spinlock_wait_cycles":65777408,"thread_start_us":152435,"threads_started":2,"wal-append.queue_time_us":477356}
I20260709 12:26:09.987123 24900 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:09.987634 24900 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:09.988646 24900 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.991822 24952 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.992527 24952 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.993888 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:09.994349 24959 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.996560 13792 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:10.000602 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:09.859119 24961 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:10.003963 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 376371us
I20260709 12:26:10.005034 24961 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:09.945132 24644 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:10.006927 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.009533 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:10.008622 13644 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:26:10.010150 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:10.015590 14419 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:10.008438 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:10.017318 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:09.951879 24757 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:09.973529 24721 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:10.018960 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.025641 13726 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:10.014786 14416 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:10.025950 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.017503 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.018913 13801 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:10.018118 13542 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 27 to 33. New cstate: current_term: 33 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:09.921389 24967 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:09.943418 24940 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:10.032795 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63043.2us
I20260709 12:26:09.981873 24988 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:10.024602 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 79147.4us
I20260709 12:26:10.028702 13718 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:09.971999 24050 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:10.031397 24736 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:10.031903 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:10.033033 13725 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:10.023197 13647 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:10.034557 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:10.016204 24946 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:26:10.025311 24971 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:26:10.045365 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:10.049209 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:10.025208 24721 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.058666 24721 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:10.031174 24961 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.062161 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:10.065259 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 142934us
I20260709 12:26:10.025357 24644 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:10.016530 24965 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.042362 24426 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:10.078577 14419 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:10.057294 24848 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:10.090775 24952 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.090647 24929 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:10.030716 24757 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.094033 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:10.096134 24757 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.099939 24050 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:10.100576 24050 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:10.084336 24988 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:10.109691 24959 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.111994 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.067458 24798 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:10.070380 24751 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:10.068545 24946 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:10.072388 24961 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.115147 24943 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:10.164314 24943 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.088611 24937 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:10.117739 13723 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:10.034547 24954 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.120492 13647 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [24.17-26.18]
I20260709 12:26:10.173817 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:10.110684 24959 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.129246 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:10.174667 13647 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:10.175015 13649 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:10.130323 13645 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: [31.17-33.18]
W20260709 12:26:10.132303 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49248.4us
I20260709 12:26:10.132504 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.176189 13721 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 26.
I20260709 12:26:10.176553 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:10.177162 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:10.137514 24967 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:10.137291 24644 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.17, Last appended by leader: 17, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.131389 24798 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:10.138579 24755 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.089375 24764 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.140692 24965 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.131366 24929 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.010s
I20260709 12:26:10.075506 24953 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.063s
I20260709 12:26:10.144162 24946 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.085564 24848 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.153142 24688 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.151317 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:10.122400 24751 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.158217 13725 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 17. (term mismatch)
I20260709 12:26:10.158571 14416 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:10.235131 13725 pending_rounds.cc:85] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 16
I20260709 12:26:10.159116 24721 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.163724 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:10.164775 24943 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:10.101562 24050 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.179795 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.181572 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:10.081321 24939 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.084s
I20260709 12:26:10.190426 13721 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:10.181265 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.192885 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:10.199231 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.200773 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.235625 13725 pending_rounds.cc:107] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:26:10.204166 13647 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:10.246308 13725 raft_consensus.cc:2889] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:10.247269 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48564.1us
I20260709 12:26:10.228468 13652 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:10.248636 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:10.248525 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.250011 13723 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:10.237111 13764 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:10.250989 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:10.251602 13723 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:26:10.251848 13649 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:10.227463 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.131773 24893 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.066s
W20260709 12:26:10.241578 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 67441.7us
I20260709 12:26:10.241731 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.258311 13797 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:10.226536 24848 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:10.244014 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.243537 13643 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:10.260591 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:10.265355 13647 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:10.265882 13720 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:10.241904 24751 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:10.234911 24862 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:10.248454 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:10.204733 24755 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.268915 13727 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:10.277925 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:10.281108 13725 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:10.281420 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:10.270124 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:10.270311 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.196861 24828 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:10.283385 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:10.196344 24610 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.059s
W20260709 12:26:10.292665 25001 log.cc:927] Time spent T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:26:10.285091 13767 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:10.272662 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.263057 24655 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.010s
I20260709 12:26:10.259938 24965 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:10.208526 24975 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.067s
I20260709 12:26:10.285754 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.245357 24757 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.040s
I20260709 12:26:10.273813 24868 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:10.291447 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:10.283986 13798 raft_consensus.cc:2376] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33. Re-sending same reply.
I20260709 12:26:10.299541 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.322034 13718 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:10.322403 13796 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:26:10.206235 24764 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.278903 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.263051 24752 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:26:10.312188 13647 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:10.287879 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:10.316329 13723 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:10.273415 24967 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.323894 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:10.329370 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81207.6us
I20260709 12:26:10.288753 24828 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.332137 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.299533 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:10.276360 24990 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:10.338778 13721 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:10.292699 24610 consensus_queue.cc:1243] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:26:10.340775 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.341015 13643 pending_rounds.cc:85] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 16
I20260709 12:26:10.356577 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:10.348169 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:10.399359 13798 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 36: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:10.399443 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.373872 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.375380 13643 pending_rounds.cc:107] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:26:10.402212 13643 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:10.309029 24771 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:10.411069 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:10.412669 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:10.399039 13542 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 22 to 25, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 25 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:10.318456 24893 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:10.340258 24694 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:26:10.408933 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109480us
I20260709 12:26:10.412761 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:10.379455 13723 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:10.318626 24989 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:10.407800 13725 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:10.416041 13796 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
W20260709 12:26:10.419711 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 226918us
I20260709 12:26:10.299104 24833 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 12:26:10.285701 24862 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.295583 24755 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.491519 13801 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:10.434026 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:10.496057 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 208282us
I20260709 12:26:10.372545 24828 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:10.433256 13766 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:10.356626 24759 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.355649 24967 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.425901 24868 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:10.293737 24798 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.423393 24694 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:10.424254 24893 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:10.578472 24965 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:10.498178 24752 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.134s
W20260709 12:26:10.709297 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 335871us
I20260709 12:26:10.410990 24990 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.614849 24759 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.702570 24893 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:10.639894 24961 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.753252 24918 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:10.753713 24993 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.754171 24721 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.755237 24759 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:10.706696 24772 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:10.703358 24954 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:10.738194 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.710554 24768 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:10.784543 24989 raft_consensus.cc:697] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:10.786136 24989 consensus_queue.cc:237] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:10.790877 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 357672us
I20260709 12:26:10.798565 13766 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:10.766858 24940 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.001s
I20260709 12:26:10.764109 24918 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.850723 24721 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.852160 24862 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:10.806659 24694 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.859668 24940 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.772451 24688 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.869953 24940 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.871778 24967 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:10.872365 24940 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:10.769948 24772 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:10.873616 24772 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 33.18, Last appended by leader: 18, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:10.848172 25018 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.865195 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 58593.9us
I20260709 12:26:10.876644 13649 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
W20260709 12:26:10.859511 25004 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 12:26:10.867830 25007 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.452s	user 0.001s	sys 0.000s
W20260709 12:26:10.869118 24988 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.766s	user 0.011s	sys 0.000s
W20260709 12:26:11.002637 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52838us
W20260709 12:26:10.866955 25012 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:26:10.861534 24918 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:10.965445 24721 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:10.849432 24833 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:26:11.002700 14389 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.008s	sys 0.015s
W20260709 12:26:11.146003 14389 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.008s	sys 0.015s
W20260709 12:26:10.866365 25011 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
W20260709 12:26:10.867403 25009 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.467s	user 0.001s	sys 0.000s
I20260709 12:26:11.152403 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
W20260709 12:26:11.154381 13643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.021s	sys 0.051s
W20260709 12:26:11.167479 13643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.022s	sys 0.051s
I20260709 12:26:10.771304 24868 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.163712 13725 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.012s	sys 0.062s
I20260709 12:26:10.796941 24993 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:11.176318 13725 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.012s	sys 0.062s
I20260709 12:26:10.982807 24772 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:11.169854 13617 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.016s	sys 0.043s
W20260709 12:26:11.182788 13617 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.016s	sys 0.043s
W20260709 12:26:11.183372 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.183652 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:10.878213 24755 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.132804 24967 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.177516 13644 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.019s	sys 0.041s
I20260709 12:26:11.027377 24764 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.028669 24694 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:10.859527 24768 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:11.195881 13644 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.020s	sys 0.041s
W20260709 12:26:11.196250 13652 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.947s	user 0.014s	sys 0.000s
I20260709 12:26:11.208055 13652 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:11.208815 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5477) took 1078 ms. Trace:
I20260709 12:26:11.209220 13652 rpcz_store.cc:276] 0709 12:26:10.130083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.149020 (+ 18937us) service_pool.cc:224] Handling call
0709 12:26:11.208606 (+1059586us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.185881 13693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.042s	sys 0.025s
W20260709 12:26:11.210341 13693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.043s	sys 0.025s
I20260709 12:26:11.196846 13793 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4949) took 1209 ms. Trace:
I20260709 12:26:11.135339 25027 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.144248 24865 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:10.874647 24965 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.185077 25010 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.582s	user 0.002s	sys 0.000s
I20260709 12:26:11.226559 24764 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:11.185577 24833 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:11.227043 25002 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.241s	user 0.002s	sys 0.004s
I20260709 12:26:11.259106 24833 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.227837 13652 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:11.230414 25027 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.263991 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.231290 24865 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.267336 24865 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.267894 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5473) took 1139 ms. Trace:
I20260709 12:26:11.234694 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:11.268853 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4943) took 1563 ms. Trace:
I20260709 12:26:11.235430 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:11.235790 24915 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.236887 25035 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:11.238550 24688 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.241003 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:11.212299 13793 rpcz_store.cc:276] 0709 12:26:09.987808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.999109 (+ 11301us) service_pool.cc:224] Handling call
0709 12:26:11.195748 (+1196639us) spinlock_profiling.cc:243] Waited 7.86 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:26:11.196128 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.196453 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.196486 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.196533 (+    47us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:11.196563 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.196624 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.196773 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18087680}
I20260709 12:26:11.247252 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:11.241743 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:11.177281 25022 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.250068 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:11.241391 25029 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.974066 25018 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.268343 13648 rpcz_store.cc:276] 0709 12:26:10.128956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.130239 (+  1283us) service_pool.cc:224] Handling call
0709 12:26:10.130558 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.147040 (+ 16482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.147088 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.147643 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.246743 (+ 99100us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:10.246808 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.246890 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.267664 (+1020774us) raft_consensus.cc:1685] finished
0709 12:26:11.267688 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.267827 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16022,"raft.queue_time_us":17421,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":98344,"threads_started":1,"wal-append.queue_time_us":197972}
I20260709 12:26:11.250689 24764 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.274682 25019 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.264529 25027 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.276201 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:11.276643 24755 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.278805 24768 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 35.19, Last appended by leader: 19, Current term: 38, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.281764 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1773) took 1273 ms. Trace:
I20260709 12:26:11.333025 13792 rpcz_store.cc:276] 0709 12:26:10.014088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.014429 (+   341us) service_pool.cc:224] Handling call
0709 12:26:11.281510 (+1267081us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54000d2fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:11.281684 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589032832}
I20260709 12:26:11.269384 13800 rpcz_store.cc:276] 0709 12:26:09.705139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.705372 (+   233us) service_pool.cc:224] Handling call
0709 12:26:09.714997 (+  9625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.983470 (+268473us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:09.983906 (+   436us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.164835 (+180929us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.174605 (+  9770us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.174822 (+   217us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:10.174863 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.175017 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.268559 (+1093542us) raft_consensus.cc:1685] finished
0709 12:26:11.268608 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.268784 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":305862,"prepare.queue_time_us":1953,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":275066,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":247,"thread_start_us":178591,"threads_started":1,"wal-append.queue_time_us":208628}
I20260709 12:26:11.313164 13722 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: [33.18-33.18]
I20260709 12:26:11.206771 24967 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.324923 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83322.4us
I20260709 12:26:11.249749 24862 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:11.328579 13648 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:26:11.353690 13615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.005s	sys 0.012s
I20260709 12:26:11.338850 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:11.377399 13615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.009s	sys 0.015s
W20260709 12:26:11.377715 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 615283us
I20260709 12:26:11.336089 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5476) took 1206 ms. Trace:
I20260709 12:26:11.306651 24915 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.372629 24990 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.315729 24764 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:11.323091 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73917.1us
I20260709 12:26:11.324854 24644 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.386324 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5480) took 1231 ms. Trace:
I20260709 12:26:11.394196 13643 rpcz_store.cc:276] 0709 12:26:10.154462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.303129 (+148667us) service_pool.cc:224] Handling call
0709 12:26:10.340781 (+ 37652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.402903 (+ 62122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.402954 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.403550 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.168090 (+764540us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.168158 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.168261 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.386050 (+217789us) raft_consensus.cc:1685] finished
0709 12:26:11.386084 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.386245 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":764105,"threads_started":1,"wal-append.queue_time_us":864430}
I20260709 12:26:11.310611 24688 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.339561 25027 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:11.376233 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:11.390102 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.413014 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:11.399816 25018 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.401268 24993 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.402880 24721 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.315382 25022 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.306885 25035 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.315260 25029 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.405495 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 4948) took 1418 ms. Trace:
I20260709 12:26:11.326495 25019 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.409504 24644 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.382856 13646 rpcz_store.cc:276] 0709 12:26:10.129814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.130418 (+   604us) service_pool.cc:224] Handling call
0709 12:26:10.130839 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.144574 (+ 13735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.144625 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.145361 (+   736us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.295005 (+149644us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:10.295072 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.295174 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.335356 (+1040182us) raft_consensus.cc:1685] finished
0709 12:26:11.335389 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.335911 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13121,"raft.queue_time_us":15098,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":424,"thread_start_us":149204,"threads_started":1,"wal-append.queue_time_us":200521}
I20260709 12:26:11.438488 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.416203 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.417423 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:11.416188 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5481) took 1243 ms. Trace:
I20260709 12:26:11.337397 24755 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:11.410215 24828 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.793s	user 0.008s	sys 0.000s
I20260709 12:26:11.416719 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:11.424355 13795 rpcz_store.cc:276] 0709 12:26:09.987469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.998596 (+ 11127us) service_pool.cc:224] Handling call
0709 12:26:10.253270 (+254674us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:26:10.253513 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.616958 (+363445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.617004 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.617075 (+    71us) raft_consensus.cc:1645] Marking committed up to 27
0709 12:26:10.617119 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.405241 (+788122us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54002cb0d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:11.405295 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.405435 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854122752}
I20260709 12:26:11.391376 25037 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.436236 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:11.445408 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:11.443586 24828 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.444572 24721 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:11.446617 25027 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:11.447983 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:11.447994 24828 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.444391 25037 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.450050 24721 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:11.450387 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:11.450480 25037 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.453303 13648 raft_consensus.cc:2393] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:11.460076 25026 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.461031 25026 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.460124 25024 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.466565 25027 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:11.466694 25024 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.469341 25027 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:11.424484 24993 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:11.383807 24862 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.434219 24961 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:11.424902 25018 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.477105 24772 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:11.484812 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:11.458465 24943 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.190s	user 0.013s	sys 0.004s
I20260709 12:26:11.435259 24558 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:26:11.479779 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4877) took 1344 ms. Trace:
W20260709 12:26:11.474905 24848 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.197s	user 0.014s	sys 0.000s
I20260709 12:26:11.473186 24865 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:11.521466 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.522336 13649 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:26:11.528479 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.484174 25024 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.530036 13646 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:11.480576 24993 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.531950 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:11.517138 24751 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.221s	user 0.017s	sys 0.000s
I20260709 12:26:11.517760 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2310) took 1386 ms. Trace:
I20260709 12:26:11.567931 13723 rpcz_store.cc:276] 0709 12:26:10.131826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.428027 (+296201us) service_pool.cc:224] Handling call
0709 12:26:10.428713 (+   686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.798469 (+369756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.798503 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.804786 (+  6283us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.300781 (+495995us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:11.300848 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.300954 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.517420 (+216466us) raft_consensus.cc:1685] finished
0709 12:26:11.517455 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.517676 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":408871,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":3091,"thread_start_us":503891,"threads_started":2,"wal-append.queue_time_us":358263}
I20260709 12:26:11.526173 13725 rpcz_store.cc:276] 0709 12:26:10.135097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.491989 (+356892us) service_pool.cc:224] Handling call
0709 12:26:10.601553 (+109564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.601823 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:11.177028 (+575205us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.451583 (+274555us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.451657 (+    74us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:11.451714 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.451964 (+   250us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.479685 (+ 27721us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24420,"prepare.queue_time_us":138795,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":15021,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":808390,"threads_started":2}
I20260709 12:26:11.512444 25026 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.583666 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.586812 13649 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:26:11.587648 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.516865 25027 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:11.583300 24721 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.600701 24965 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.605140 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:11.608911 25005 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.906s	user 0.003s	sys 0.000s
W20260709 12:26:11.610847 25045 log.cc:927] Time spent T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 12:26:11.611864 25042 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:26:11.601038 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 125501us
W20260709 12:26:11.616111 13727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4553) took 1599 ms (client timeout 1516 ms). Trace:
W20260709 12:26:11.601606 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 119727us
I20260709 12:26:11.566533 24993 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:11.616983 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:11.623566 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:11.531836 25040 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.512161 25018 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.626652 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 35 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:11.616670 13727 rpcz_store.cc:269] 0709 12:26:10.017109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.017242 (+   133us) service_pool.cc:224] Handling call
0709 12:26:11.615817 (+1598575us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400026cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:11.615990 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539860864}
I20260709 12:26:11.630040 13724 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: [24.17-26.18]
I20260709 12:26:11.517127 24943 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.636699 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:11.634290 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:11.643025 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:11.643764 14421 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:11.645682 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:11.647228 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3364) took 1624 ms. Trace:
I20260709 12:26:11.647660 13651 rpcz_store.cc:276] 0709 12:26:10.022795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.022966 (+   171us) service_pool.cc:224] Handling call
0709 12:26:10.023367 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.026781 (+  3414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.026813 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.039704 (+ 12891us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.262482 (+222778us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:10.262562 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.262660 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.608239 (+1345579us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:11.634074 (+ 25835us) raft_consensus.cc:1685] finished
0709 12:26:11.634100 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.647173 (+ 13073us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3028,"raft.queue_time_us":3596,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":6475,"spinlock_wait_cycles":49864960,"thread_start_us":180523,"threads_started":1,"wal-append.queue_time_us":431413}
I20260709 12:26:11.534443 24848 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.663724 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:11.666325 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:11.586975 24751 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.632115 25027 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.645309 25026 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.634284 25046 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.682394 13650 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:11.645119 25024 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.644281 25038 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.655160 24699 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:11.655412 24798 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.702054 24798 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.703114 25050 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:26:11.704990 25044 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:26:11.706215 25051 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:11.708271 25019 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.709540 24755 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.675091 24967 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.711601 25022 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.705915 24798 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.718998 24644 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.721022 25018 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.722846 24943 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.724270 24862 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:11.638327 25040 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.726555 24688 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.696153 25024 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.729928 25024 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.605218 24965 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.660624 24963 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:11.695235 13721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 4876) took 1560 ms (client timeout 1692 ms). Trace:
I20260709 12:26:11.699479 13647 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:11.683902 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.630764 24721 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.731550 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4881) took 1582 ms. Trace:
I20260709 12:26:11.662894 25052 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.730134 25029 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:11.749215 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80281us
W20260709 12:26:11.750197 13721 rpcz_store.cc:269] 0709 12:26:10.134460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.176857 (+ 42397us) service_pool.cc:224] Handling call
0709 12:26:11.694997 (+1518140us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b54001f0754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:11.695150 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29039360}
I20260709 12:26:11.757412 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.758376 13721 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:11.759505 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.760183 13721 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:11.761153 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.764349 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.750921 13726 rpcz_store.cc:276] 0709 12:26:10.149538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.755338 (+605800us) service_pool.cc:224] Handling call
0709 12:26:10.755707 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.183761 (+428054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.183797 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.185771 (+  1974us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.430469 (+244698us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:11.430519 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.430613 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.731214 (+300601us) raft_consensus.cc:1685] finished
0709 12:26:11.731249 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.731466 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114513,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":109,"thread_start_us":637327,"threads_started":2,"wal-append.queue_time_us":465112}
I20260709 12:26:11.726843 25040 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.750887 13727 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:11.699991 24699 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:11.759728 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:11.771582 13725 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:11.668777 25037 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.751560 24915 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.683519 25027 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.779438 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.686535 25026 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.786093 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.730338 25038 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.776062 24946 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:26:11.813697 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.776137 24764 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:11.804415 13720 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4545) took 1973 ms (client timeout 1762 ms). Trace:
W20260709 12:26:11.815538 13720 rpcz_store.cc:269] 0709 12:26:09.830585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.830736 (+   151us) service_pool.cc:224] Handling call
0709 12:26:11.804174 (+1973438us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b5400173c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:11.804333 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957858560}
I20260709 12:26:11.775848 25035 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:11.805148 24893 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 1.000s	user 0.014s	sys 0.000s
I20260709 12:26:11.823680 24721 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.824154 24965 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.823947 24893 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.825416 14419 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:11.803637 25027 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.776084 24862 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.699761 25046 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:11.821409 13801 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3364) took 1615 ms (client timeout 1935 ms). Trace:
I20260709 12:26:11.823308 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4884) took 1669 ms. Trace:
W20260709 12:26:11.793699 25049 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:26:11.825896 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5472) took 1697 ms. Trace:
I20260709 12:26:11.825924 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4548) took 1990 ms. Trace:
I20260709 12:26:11.831003 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:11.834015 13719 rpcz_store.cc:276] 0709 12:26:09.835413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.835546 (+   133us) service_pool.cc:224] Handling call
0709 12:26:09.835942 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.846727 (+ 10785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.846761 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.847251 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.027664 (+180413us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:10.051304 (+ 23640us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.051563 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.051650 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.801148 (+1749498us) spinlock_profiling.cc:243] Waited 6.79 ms on lock 0x7b1800501880. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:11.808906 (+  7758us) raft_consensus.cc:1685] finished
0709 12:26:11.825728 (+ 16822us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.825862 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28033,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":4642,"spinlock_wait_cycles":40144512,"thread_start_us":179959,"threads_started":1,"wal-append.queue_time_us":218578}
W20260709 12:26:11.806133 25055 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:11.755518 24963 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.830786 13801 rpcz_store.cc:269] 0709 12:26:10.208310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.278599 (+ 70289us) service_pool.cc:224] Handling call
0709 12:26:11.821111 (+1542512us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:11.821327 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222845824}
I20260709 12:26:11.831194 13720 rpcz_store.cc:276] 0709 12:26:10.153929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.822320 (+1668391us) service_pool.cc:224] Handling call
0709 12:26:11.822709 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.822960 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.822991 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.823051 (+    60us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:11.823087 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.823142 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.823248 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.835676 13726 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:11.833907 13650 rpcz_store.cc:276] 0709 12:26:10.128651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.128785 (+   134us) service_pool.cc:224] Handling call
0709 12:26:10.129152 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.694816 (+1565664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.825668 (+130852us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:11.825838 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3571519232}
I20260709 12:26:11.842448 13646 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:11.810110 25040 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.819797 24946 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.768163 25052 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:11.850171 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 209203us
I20260709 12:26:11.851752 13720 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:26:11.856215 13726 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:11.857070 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4889) took 1695 ms. Trace:
I20260709 12:26:11.857482 13726 rpcz_store.cc:276] 0709 12:26:10.161722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.845350 (+1683628us) service_pool.cc:224] Handling call
0709 12:26:11.855949 (+ 10599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.856740 (+   791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.856998 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.861624 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4888) took 1700 ms. Trace:
I20260709 12:26:11.862135 13720 rpcz_store.cc:276] 0709 12:26:10.161477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.844781 (+1683304us) service_pool.cc:224] Handling call
0709 12:26:11.851516 (+  6735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.861256 (+  9740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.861550 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.864019 13719 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:26:11.864874 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4887) took 1703 ms. Trace:
I20260709 12:26:11.865185 13719 rpcz_store.cc:276] 0709 12:26:10.161207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.835910 (+1674703us) service_pool.cc:224] Handling call
0709 12:26:11.863681 (+ 27771us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.864547 (+   866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.864804 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.785609 25037 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:11.833420 25024 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.874300 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2312) took 1692 ms. Trace:
I20260709 12:26:11.910712 13720 rpcz_store.cc:276] 0709 12:26:10.182130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.863155 (+1681025us) service_pool.cc:224] Handling call
0709 12:26:11.873526 (+ 10371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.873794 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.873835 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.873901 (+    66us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:11.873952 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.874036 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.874192 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.912858 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4891) took 1632 ms. Trace:
I20260709 12:26:11.913261 13720 rpcz_store.cc:276] 0709 12:26:10.280552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.912019 (+1631467us) service_pool.cc:224] Handling call
0709 12:26:11.912339 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.912514 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.912539 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.912581 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.912613 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.912682 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.912796 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.854734 24946 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:11.928202 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2313) took 1736 ms. Trace:
I20260709 12:26:11.928841 13719 rpcz_store.cc:276] 0709 12:26:10.192328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.865924 (+1673596us) service_pool.cc:224] Handling call
0709 12:26:11.927636 (+ 61712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.927840 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.927865 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.927913 (+    48us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:11.927949 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.928006 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.928137 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.860189 25068 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:11.917174 25048 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 12:26:11.953532 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2314) took 1461 ms. Trace:
I20260709 12:26:11.954284 13720 rpcz_store.cc:276] 0709 12:26:10.491906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.952008 (+1460102us) service_pool.cc:224] Handling call
0709 12:26:11.952514 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.952783 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.952823 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.953059 (+   236us) raft_consensus.cc:1645] Marking committed up to 26
0709 12:26:11.953152 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.953248 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.953429 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.842644 24862 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.957108 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4892) took 1676 ms. Trace:
I20260709 12:26:11.832688 24699 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.961716 13719 rpcz_store.cc:276] 0709 12:26:10.280859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.929952 (+1649093us) service_pool.cc:224] Handling call
0709 12:26:11.956387 (+ 26435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.956585 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.956706 (+   121us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.956778 (+    72us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.956827 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.956889 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.957040 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.850742 25040 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.931031 25027 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:11.972626 13720 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:26:11.974846 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2315) took 1181 ms. Trace:
I20260709 12:26:11.979122 13720 rpcz_store.cc:276] 0709 12:26:10.793973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.960698 (+1166725us) service_pool.cc:224] Handling call
0709 12:26:11.972219 (+ 11521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.974497 (+  2278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.974781 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.979689 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4893) took 1182 ms. Trace:
I20260709 12:26:11.988685 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:11.990703 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:11.924532 25024 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:11.993155 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.994290 24558 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:11.996466 24959 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.965457 24848 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.001827 25065 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.011157 24764 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.011368 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5484) took 1855 ms. Trace:
I20260709 12:26:12.013175 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.013769 25046 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.015388 24892 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.020164 25040 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.956044 25068 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:11.982532 25027 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.025439 25052 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.026557 25026 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.031526 24963 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.443521 13644 rpcz_store.cc:276] 0709 12:26:10.154747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.309407 (+154660us) service_pool.cc:224] Handling call
0709 12:26:10.346714 (+ 37307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.346973 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:11.207622 (+860649us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.394965 (+187343us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.395207 (+   242us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:11.395327 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.395477 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.398227 (+  2750us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19814,"prepare.queue_time_us":146307,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":15148,"raft.queue_time_us":179,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":21481,"thread_start_us":974836,"threads_started":2}
I20260709 12:26:12.032665 24944 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:12.031431 25017 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.261s	user 0.005s	sys 0.000s
I20260709 12:26:12.035506 25072 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:11.969733 24752 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.043426 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.321403 13794 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:12.044278 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:11.983765 13719 rpcz_store.cc:276] 0709 12:26:10.797140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.963172 (+1166032us) service_pool.cc:224] Handling call
0709 12:26:11.979088 (+ 15916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.979290 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.979315 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.979357 (+    42us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:11.979390 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.979455 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.979563 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.046640 13723 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:12.261588 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.036695 24939 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.094060 24993 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.034060 24893 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.984501 25060 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.087831 13795 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 35 index: 25. Preceding OpId from leader: term: 37 index: 26. (index mismatch)
I20260709 12:26:12.265780 25064 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:12.268503 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.187786 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 427395us
I20260709 12:26:12.342018 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:12.035593 25037 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:11.972420 24751 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.270841 13796 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
I20260709 12:26:12.045892 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.296837 13647 rpcz_store.cc:276] 0709 12:26:10.155464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.374396 (+218932us) service_pool.cc:224] Handling call
0709 12:26:10.374709 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.750426 (+1375717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.011095 (+260669us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400198f54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:12.011295 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3046105856}
W20260709 12:26:12.353132 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 307262us
I20260709 12:26:12.306800 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4895) took 1166 ms. Trace:
I20260709 12:26:12.309480 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.266804 24917 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:12.310016 13721 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:26:12.302103 25065 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.309480 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.268929 24848 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.363215 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:12.272046 24918 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 1.108s	user 0.018s	sys 0.003s
I20260709 12:26:12.326179 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:12.299312 25029 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.338683 24939 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.376196 25029 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.377918 24939 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.302177 24959 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.350807 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:12.314355 25046 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.408456 25046 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.269374 24558 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.354239 13725 rpcz_store.cc:276] 0709 12:26:11.140320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.017298 (+876978us) service_pool.cc:224] Handling call
0709 12:26:12.017684 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.017951 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:12.211005 (+193054us) log.cc:844] Serialized 14 byte log entry
0709 12:26:12.306165 (+ 95160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.306274 (+   109us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:12.306346 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.306463 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.306685 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6190,"prepare.queue_time_us":8910,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":28,"raft.queue_time_us":2544,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":4724,"thread_start_us":234190,"threads_started":2}
I20260709 12:26:12.320514 25079 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.354758 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.352099 25060 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.361828 24917 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.368160 25027 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:12.368466 25025 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.514s	user 0.005s	sys 0.004s
I20260709 12:26:12.371028 25065 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.320037 24944 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.329464 24930 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:12.329334 25072 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.329209 24752 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.329878 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.305572 24892 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.333124 24975 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:12.352706 25064 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.379251 24772 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:12.379571 24993 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.358203 25052 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:12.380221 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:12.381948 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:12.382123 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.382715 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.348099 25040 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.386574 25068 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.386220 24946 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.451984 25040 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.387718 24848 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.455649 25040 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.456288 24848 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.363317 24833 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:12.457405 24833 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.457945 24833 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.458765 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4894) took 1318 ms. Trace:
I20260709 12:26:12.392086 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4890) took 2112 ms. Trace:
I20260709 12:26:12.459887 24944 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.459527 13726 rpcz_store.cc:276] 0709 12:26:10.280197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.866136 (+1585939us) service_pool.cc:224] Handling call
0709 12:26:11.919017 (+ 52881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.919363 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:11.919390 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.925337 (+  5947us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.175129 (+249792us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:12.175171 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.175256 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.390420 (+215164us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:12.391783 (+  1363us) raft_consensus.cc:1685] finished
0709 12:26:12.391810 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.392018 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51599232,"thread_start_us":205205,"threads_started":1,"wal-append.queue_time_us":366666}
I20260709 12:26:12.393298 13721 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:12.459187 13720 rpcz_store.cc:276] 0709 12:26:11.140023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.998928 (+858905us) service_pool.cc:224] Handling call
0709 12:26:11.999322 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.999693 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.999721 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.000423 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.269901 (+269478us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:12.269944 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.270015 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.458522 (+188507us) raft_consensus.cc:1685] finished
0709 12:26:12.458551 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.458696 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":269022,"threads_started":1,"wal-append.queue_time_us":426490}
W20260709 12:26:12.394240 13799 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3369) took 2183 ms (client timeout 1777 ms). Trace:
I20260709 12:26:12.394963 24918 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.396179 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4874) took 2291 ms. Trace:
I20260709 12:26:12.398537 25029 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.402631 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.404645 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:12.412645 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61976.4us
I20260709 12:26:12.468626 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:12.440351 24772 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:12.434878 24862 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.417131 24961 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:12.428504 25035 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:12.452236 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.417555 24959 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:12.463330 13799 rpcz_store.cc:269] 0709 12:26:10.210626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.262750 (+ 52124us) service_pool.cc:224] Handling call
0709 12:26:12.394019 (+2131269us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003d45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:12.394144 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2904531840}
I20260709 12:26:12.464663 13722 rpcz_store.cc:276] 0709 12:26:10.104930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.106002 (+  1072us) service_pool.cc:224] Handling call
0709 12:26:10.106458 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.335025 (+1228567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.395798 (+1060773us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540021fcd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:12.396068 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2774182784}
I20260709 12:26:12.467341 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1729) took 2418 ms. Trace:
I20260709 12:26:12.470235 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:12.418387 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64213.7us
I20260709 12:26:12.444730 25024 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.477295 25064 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.440227 24930 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.463917 25038 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.483839 14420 rpcz_store.cc:276] 0709 12:26:10.044817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.050395 (+  5578us) service_pool.cc:224] Handling call
0709 12:26:10.050821 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.079300 (+ 28479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.079335 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.079790 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.387872 (+308082us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b18004a6e00. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:26:10.628814 (+240942us) raft_consensus.cc:1645] Marking committed up to 25
0709 12:26:10.628936 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.629067 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.367855 (+1738788us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:12.463186 (+ 95331us) raft_consensus.cc:1685] finished
0709 12:26:12.463213 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.463342 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34959,"raft.queue_time_us":28147,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6980,"spinlock_wait_cycles":36754560,"thread_start_us":240222,"threads_started":1,"wal-append.queue_time_us":676565}
I20260709 12:26:12.429131 25079 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.507511 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.477681 25072 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:12.538916 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.464908 24510 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:12.545318 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.483821 25035 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.551783 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.552138 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.552527 13645 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:26:12.552764 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:12.553741 25072 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:12.552778 13647 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:12.554198 25072 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.554652 25072 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:12.555395 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.556281 25024 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:12.536551 24862 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:26:12.558529 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:12.558717 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.559665 24965 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:12.560093 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2051) took 2237 ms. Trace:
I20260709 12:26:12.560518 24759 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:12.561113 14419 rpcz_store.cc:276] 0709 12:26:10.322974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.323146 (+   172us) service_pool.cc:224] Handling call
0709 12:26:10.323520 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.825819 (+1502299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.559700 (+733881us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400018954. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:12.559875 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3453569408}
I20260709 12:26:12.533186 25040 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:12.564801 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:12.565623 24759 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:12.565995 24759 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.566486 24759 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.574337 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4882) took 2423 ms. Trace:
I20260709 12:26:12.574872 13718 rpcz_store.cc:276] 0709 12:26:10.151116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.816982 (+665866us) service_pool.cc:224] Handling call
0709 12:26:10.944784 (+127802us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.239458 (+294674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.239509 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.240218 (+   709us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.429989 (+189771us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:11.430055 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.573862 (+1143807us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54001a6154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:12.573917 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.573968 (+    51us) raft_consensus.cc:1685] finished
0709 12:26:12.573981 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.574244 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21434,"raft.queue_time_us":23110,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":9716,"spinlock_wait_cycles":627123072,"thread_start_us":189239,"threads_started":1,"wal-append.queue_time_us":271646}
I20260709 12:26:12.557802 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.556725 25024 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.579149 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.579907 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.579173 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:12.597898 14419 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 34
I20260709 12:26:12.598017 13644 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:26:12.596876 13647 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:26:12.597543 13645 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:12.479324 25029 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:26:12.580369 13693 leader_election.cc:336] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.777s (SENT)
I20260709 12:26:12.593551 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.608052 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.608850 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:12.477823 24752 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.608989 13645 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:12.558482 25064 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:26:12.611132 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 598076us
I20260709 12:26:12.544533 25052 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.614127 13799 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:12.553052 24946 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.616294 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.629484 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.630455 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:12.631266 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.621100 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:12.631946 13725 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.621279 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:12.633147 13648 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:12.621970 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:12.546770 24510 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:12.560216 24965 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.639119 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.626063 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.544976 24990 consensus_queue.cc:1059] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 12:26:12.537329 24961 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:26:12.613448 25040 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:12.627529 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.627669 13652 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 12:26:12.610906 25087 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
W20260709 12:26:12.605636 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 272325us
I20260709 12:26:12.577531 25035 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.629113 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.619207 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.634595 13650 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:12.660079 13726 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 26: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:12.636189 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.640005 13647 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:12.639959 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.662240 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:12.663273 13651 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:12.642102 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.663610 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 4875) took 2558 ms. Trace:
I20260709 12:26:12.642611 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.640604 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.645167 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.664079 13724 rpcz_store.cc:276] 0709 12:26:10.105242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.111326 (+  6084us) service_pool.cc:224] Handling call
0709 12:26:10.111718 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:11.636293 (+1524575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.663351 (+1027058us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54003fedd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:12.663551 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3490974592}
I20260709 12:26:12.664745 13718 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:12.665506 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:12.665781 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:12.666556 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.666908 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.654300 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.668042 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.668180 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.669088 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.669226 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.669677 13718 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:12.671854 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.666154 13792 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:12.674086 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5490) took 1048 ms. Trace:
I20260709 12:26:12.674496 13643 rpcz_store.cc:276] 0709 12:26:11.625906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.626096 (+   190us) service_pool.cc:224] Handling call
0709 12:26:11.654615 (+ 28519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.859054 (+204439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.859089 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.859144 (+    55us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:11.859178 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.673847 (+814669us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400113d54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:12.673893 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.674017 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469511424}
I20260709 12:26:12.656157 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.678903 24918 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:12.683178 24558 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.664685 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.632325 25052 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.684371 13647 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:12.685662 25052 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.609969 24848 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:26:12.646095 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:12.535163 24865 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.076s
I20260709 12:26:12.651917 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.614562 24969 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.644888 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.708227 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.709168 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:26:12.624010 25064 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.606473 25024 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:12.641201 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.689970 13648 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:26:12.597489 24772 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.653977 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:12.698139 13725 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:12.701205 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.689388 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.712118 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.626585 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:12.721115 13721 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:12.648113 24965 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:12.724601 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.691720 13792 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
W20260709 12:26:12.737936 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48605.2us
I20260709 12:26:12.738777 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:12.741397 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89891.2us
I20260709 12:26:12.742554 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:12.739651 13650 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:12.746122 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:12.760854 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.783243 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46162us
I20260709 12:26:12.787689 14420 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 12:26:12.788204 14420 pending_rounds.cc:85] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 19
I20260709 12:26:12.788431 14420 pending_rounds.cc:107] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 12:26:12.788678 14420 raft_consensus.cc:2889] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:12.796088 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 53664.7us
I20260709 12:26:12.651892 24759 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.730516 24969 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.732120 24892 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.722982 25046 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.738989 25060 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.792344 24990 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.745759 25052 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:12.793140 24510 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.810933 24990 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.732796 25064 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:12.803601 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.782833 24851 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:12.783818 24917 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.813225 24772 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:12.819295 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.814428 13646 raft_consensus.cc:2393] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:12.748021 24946 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.815562 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.815817 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.820235 13648 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:12.831008 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.792435 24953 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.813989 24851 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:12.792511 24764 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:12.849727 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.814976 25037 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:12.848287 13648 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:26:12.859225 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.853422 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:12.860759 13644 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:12.854962 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.793534 24960 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.045s
I20260709 12:26:12.792783 24848 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.053s
I20260709 12:26:12.872007 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.883229 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:12.885182 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:12.887957 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 337369us
I20260709 12:26:12.858611 25091 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:12.856105 24959 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.836702 24946 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.895006 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.885187 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.864156 25080 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:12.903767 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.905347 13617 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.906540 24969 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.865491 24759 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:12.901392 24934 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:12.911314 24930 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.911445 24934 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:12.889654 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:12.850780 24953 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.879140 25027 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:12.881186 24851 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.914714 13724 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:12.903088 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.878669 25046 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:12.891577 24751 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:12.864339 24764 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.921241 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.927366 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.903170 25028 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:12.935365 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.939672 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.891266 24828 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.031s
I20260709 12:26:12.940881 13617 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:12.919554 24939 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:12.915122 25026 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:12.912405 25037 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:12.926892 25080 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:12.933857 24868 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:26:12.933007 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:12.936908 24751 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.971880 24751 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:12.936918 24969 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:12.974483 24969 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.974965 24969 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:12.975872 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.976586 13727 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:12.949338 24939 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.978920 24939 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:12.958987 24990 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:12.983399 13720 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:12.960860 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:12.950100 24934 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 35.26, Last appended by leader: 26, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.955210 25037 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.923750 25027 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:12.925725 25079 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.999138 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.963835 24764 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:12.900568 25091 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:12.945566 25046 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:26:12.988456 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55613.3us
I20260709 12:26:12.990640 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.003360 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:13.004102 13727 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:12.991257 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:12.994935 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:12.995095 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.995942 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.997087 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.962900 25072 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.005995 13798 raft_consensus.cc:2410] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 36 because replica has last-logged OpId of term: 33 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 22.
I20260709 12:26:12.983269 24961 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:13.001755 13693 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:13.007812 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:13.009756 13800 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 33 index: 18. Preceding OpId from leader: term: 34 index: 18. (term mismatch)
I20260709 12:26:13.010203 13800 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 17
I20260709 12:26:13.010525 13800 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 33.18
I20260709 12:26:13.010839 13800 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.013159 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:12.949630 25028 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:13.015020 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.015277 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.015844 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.010641 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:13.016716 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.022523 13799 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:12.938927 24851 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:12.987015 24960 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:13.021495 13726 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:13.029870 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:13.023770 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.023634 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:13.032357 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:13.052100 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.999619 24755 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:13.078636 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.086534 13724 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:26:13.078887 13726 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:13.027359 25052 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:13.081737 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.083051 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.140396 25052 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:13.142408 25052 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:13.142508 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:13.143676 13801 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27.
I20260709 12:26:13.135694 13793 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:13.145092 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.027411 25029 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:13.146309 25029 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.146796 25029 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:13.014086 24961 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:13.085909 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:12.996574 25037 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:12.999805 25027 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.065623 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.152813 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.153354 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.156044 25103 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:13.156697 25103 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.161139 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.162613 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.167019 24965 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.167693 25064 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.168612 13721 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:13.169802 24965 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.162626 14424 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:13.088778 25028 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.169535 25024 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.174903 25080 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:13.177124 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.178392 25064 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.178944 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:13.065872 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.021338 24868 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.020578 24862 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.020038 25079 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:13.033946 24764 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.051467 24959 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:13.191684 24959 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:13.028792 25063 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:13.191795 24868 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.192147 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:13.192885 13727 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:13.192617 25063 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.193463 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:13.194020 13799 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:13.187386 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.189102 14418 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 19. Preceding OpId from leader: term: 38 index: 20. (index mismatch)
I20260709 12:26:13.189661 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.178424 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:13.086282 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 189826us
I20260709 12:26:13.197145 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:13.176654 24900 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:13.194181 24868 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.192205 24959 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:13.196691 13793 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:13.092319 24755 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:13.201197 24965 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:13.225011 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.154340 24961 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:13.229158 24833 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.002s
I20260709 12:26:13.179483 25035 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.195559 25059 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:26:13.216974 24862 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.267890 24969 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.233217 24965 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.278218 25103 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:13.278297 24965 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:13.276397 25063 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.277575 25024 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.292701 24939 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.295022 24939 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.296275 24959 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.291594 25072 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.307991 25091 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:11.164953 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:13.362455 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 330029us
I20260709 12:26:13.363265 13647 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:13.081224 13649 raft_consensus.cc:2393] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:13.371817 13646 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:13.379668 13792 pending_rounds.cc:85] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 18
I20260709 12:26:13.380043 13718 raft_consensus.cc:2410] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 31 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:26:13.380303 13792 pending_rounds.cc:107] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 26.19
W20260709 12:26:13.380806 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 216768us
I20260709 12:26:13.381013 13792 raft_consensus.cc:2889] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.383787 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:13.387141 13794 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:13.387560 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.388470 13797 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:13.389235 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.379727 13720 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:13.194152 13727 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:26:13.401669 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:13.407148 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.405969 13690 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:13.408654 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:13.409848 13648 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.411561 13724 raft_consensus.cc:2410] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 32 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:13.371326 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.359090 13542 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:13.436909 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:13.440774 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.441996 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:13.445698 13645 raft_consensus.cc:1240] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:13.442154 13649 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
W20260709 12:26:13.447954 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46821.3us
I20260709 12:26:13.450040 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.450731 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:13.443284 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.458320 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:13.476850 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:13.479130 24959 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.476836 24944 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:13.301419 24969 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.490053 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:13.495143 24050 raft_consensus.cc:3055] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:13.495981 24050 raft_consensus.cc:740] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:13.458947 24755 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.460660 24610 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.463392 24851 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.464043 24862 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.250797 24833 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:13.465924 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 60513.6us
I20260709 12:26:13.501844 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:13.501924 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:13.479645 24865 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:13.571568 13645 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:13.580955 13647 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:13.584854 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54153.6us
I20260709 12:26:13.498131 24050 consensus_queue.cc:260] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.570387 24959 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:13.577796 24865 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:13.530162 24944 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:26:13.591254 24862 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:13.609083 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.611475 13797 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:13.590121 24969 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.625356 24050 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.625547 24768 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:13.626096 24768 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:13.630182 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 117420us
I20260709 12:26:13.631063 24865 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.632941 25072 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election won for term 34
W20260709 12:26:13.634343 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 198000us
I20260709 12:26:13.635926 24694 consensus_queue.cc:1059] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:13.639050 25072 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:13.627422 24768 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.643226 24768 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:13.651458 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 180073us
I20260709 12:26:13.622330 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.659132 24865 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.641201 25072 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.660709 24510 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.661641 25065 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:13.659349 25102 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:13.664898 25064 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:13.716536 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 25 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.674536 24969 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.610690 24610 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.670830 24946 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.662359 24699 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:13.665756 25075 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.017s
I20260709 12:26:13.675524 13800 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 33: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:13.774514 25065 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:13.775177 25065 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:13.776286 25065 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.776912 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.777910 13801 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:13.637025 24772 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.038s
I20260709 12:26:13.677856 25035 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.684779 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.610905 25040 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:13.717221 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 68960.7us
W20260709 12:26:13.721398 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 103837us
I20260709 12:26:13.716840 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:13.725296 13793 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:13.741071 13795 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:26:13.743117 13719 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:26:13.745698 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.741608 24953 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:13.752655 25119 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.199s	user 0.004s	sys 0.000s
I20260709 12:26:13.756049 24833 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:13.763386 25052 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.009s
I20260709 12:26:13.810647 25052 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.811673 13690 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:13.766630 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 316176us
I20260709 12:26:13.816220 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:13.770629 25111 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:13.721083 24752 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:13.818694 24752 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:13.820197 24752 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:13.821058 25029 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.005s
I20260709 12:26:13.817762 25111 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.774061 25102 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:13.824906 25102 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:13.777998 25065 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:13.785648 24610 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.785893 13724 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:13.827272 25114 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:13.828218 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:13.828927 25040 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:13.788914 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.830850 13651 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:13.771615 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:13.719924 24694 raft_consensus.cc:3055] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:13.798465 13795 pending_rounds.cc:85] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 16
I20260709 12:26:13.835479 13795 pending_rounds.cc:107] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:26:13.835808 13795 raft_consensus.cc:2889] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.799795 13646 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:13.806525 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 88791.7us
I20260709 12:26:13.741938 24969 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:13.830878 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.774148 13647 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:26:13.826603 13720 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:13.834211 24772 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:13.824180 25111 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:13.835569 13648 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:13.835891 13645 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:26:13.841821 13690 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:13.849786 13795 pending_rounds.cc:107] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:26:13.850203 13795 raft_consensus.cc:2889] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:13.851342 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.804502 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2324) took 1004 ms. Trace:
I20260709 12:26:13.821621 25029 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.814091 25052 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.821625 24752 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:13.832106 25040 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.864558 13723 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:13.838735 24694 raft_consensus.cc:740] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:13.874341 13721 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
W20260709 12:26:13.874993 13799 consensus_meta.cc:287] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.675s	user 0.018s	sys 0.002s
I20260709 12:26:13.876825 13643 raft_consensus.cc:1275] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
W20260709 12:26:13.878880 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62792.7us
I20260709 12:26:13.826139 25102 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.843132 24772 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:13.882711 24961 consensus_meta.cc:287] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.012s
W20260709 12:26:13.883939 24900 consensus_meta.cc:287] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.002s
I20260709 12:26:13.884845 25111 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:13.884709 24969 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:13.889424 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:13.891855 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51996us
W20260709 12:26:13.893375 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 122059us
I20260709 12:26:13.893808 13722 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:13.885823 13690 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:13.874433 25068 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:13.912092 13618 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:13.915369 25033 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.916025 25033 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.918363 24759 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:13.928970 13645 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:13.911545 13799 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:26:13.927402 25070 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:13.935388 24965 consensus_meta.cc:287] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.006s
I20260709 12:26:13.935292 25070 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:13.935951 24965 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.905974 24961 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:13.941035 24961 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.947649 24510 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:13.948244 24510 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.948606 24510 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:13.938498 24965 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:13.938560 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 221995us
I20260709 12:26:13.952847 13724 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:13.953081 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:13.955708 13647 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 19. Preceding OpId from leader: term: 38 index: 20. (index mismatch)
I20260709 12:26:13.930693 25111 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:13.977957 14425 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:13.980544 25033 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:13.981855 13648 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:13.969038 25102 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:13.987028 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 101384us
W20260709 12:26:13.990481 24751 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.009s	user 0.010s	sys 0.017s
I20260709 12:26:13.991034 24751 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:13.996440 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89361.2us
I20260709 12:26:13.853021 13725 rpcz_store.cc:276] 0709 12:26:12.728531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.735077 (+  6546us) service_pool.cc:224] Handling call
0709 12:26:12.735331 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.735525 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:12.851342 (+115817us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.447193 (+595851us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b18004ca800. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a 00007f5393bfea2a
0709 12:26:13.613668 (+166475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.620699 (+  7031us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:13.658379 (+ 37680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.724156 (+ 65777us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.733193 (+  9037us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29654,"prepare.queue_time_us":114063,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":958,"raft.queue_time_us":11748,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":7061,"spinlock_wait_cycles":24291712,"thread_start_us":135731,"threads_started":1,"wal-append.queue_time_us":794127}
I20260709 12:26:14.010471 24848 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.010869 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.011322 24848 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:14.013334 25027 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.824s	user 0.000s	sys 0.015s
I20260709 12:26:14.013818 24848 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.014415 25038 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:14.016139 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:14.028929 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:14.016671 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:14.013928 25027 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.030968 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:14.031219 24764 consensus_meta.cc:287] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.831s	user 0.014s	sys 0.004s
I20260709 12:26:14.031913 24764 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:14.034129 25027 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.016844 25064 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.017375 25111 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.018220 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.035207 24768 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:14.036690 24752 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:14.035488 25115 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:14.042595 24969 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:14.043228 24969 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:14.044788 13693 leader_election.cc:400] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27.
I20260709 12:26:14.045188 13693 leader_election.cc:403] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:26:14.045771 13649 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:14.036320 25111 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.039324 25029 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:14.056512 24969 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:14.064870 13618 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:14.070585 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.040277 13652 raft_consensus.cc:1240] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:14.036461 14420 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 38.
I20260709 12:26:14.045874 24961 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:14.046664 25027 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 26. Current term is 27.
I20260709 12:26:14.047237 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34450 (request call id 4581) took 1404 ms. Trace:
I20260709 12:26:14.047521 24736 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.079463 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:13.959334 24990 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.134s
I20260709 12:26:14.031255 13727 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:26:14.049460 25070 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:14.049038 24510 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.052909 24764 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:14.035899 25064 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.034881 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:13.925017 24772 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:14.038091 25037 consensus_meta.cc:287] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.843s	user 0.012s	sys 0.003s
I20260709 12:26:14.076927 13722 rpcz_store.cc:276] 0709 12:26:12.642656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.394198 (+751542us) service_pool.cc:224] Handling call
0709 12:26:14.046875 (+652677us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:14.047160 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109803392}
I20260709 12:26:14.089442 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:14.107079 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 72395us
I20260709 12:26:14.110337 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:14.086092 25029 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.114557 24961 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:14.114781 25029 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.116051 25111 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.117129 25029 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.119421 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:14.121532 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:14.123450 13722 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:14.122781 25122 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:14.124469 25064 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.122539 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.125391 14419 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:14.108928 24510 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:14.124642 24969 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.128546 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.130167 25091 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:14.131184 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.131541 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:14.070887 24694 consensus_queue.cc:260] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.133006 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:14.134119 14416 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:14.133980 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3394) took 1332 ms. Trace:
I20260709 12:26:14.135607 13650 rpcz_store.cc:276] 0709 12:26:12.801851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.802088 (+   237us) service_pool.cc:224] Handling call
0709 12:26:12.802486 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.959505 (+157019us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:26:13.360476 (+400971us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.637828 (+277352us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.637949 (+   121us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:13.638082 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.922428 (+284346us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b480033c558. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
0709 12:26:14.133470 (+211042us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001b6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:14.133524 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.133727 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42351,"prepare.queue_time_us":291022,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2716,"raft.queue_time_us":52430,"raft.run_cpu_time_us":3540,"raft.run_wall_time_us":284058,"spinlock_wait_cycles":489380864,"thread_start_us":602233,"threads_started":2,"wal-append.queue_time_us":255873}
I20260709 12:26:14.138388 24969 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:14.140208 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 75526.9us
I20260709 12:26:14.141215 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:14.142140 25091 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:14.144251 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3398) took 1078 ms. Trace:
I20260709 12:26:14.144717 13795 rpcz_store.cc:276] 0709 12:26:13.065846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.066331 (+   485us) service_pool.cc:224] Handling call
0709 12:26:13.066707 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.902148 (+835441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.143933 (+241785us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54000d2ad4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:14.144145 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548915072}
I20260709 12:26:14.103158 25037 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.153210 25037 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.154073 13718 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:14.154197 24694 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:14.102964 24764 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.157123 24764 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.092258 24736 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.224135 13646 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:14.225992 13648 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: [28.21-29.22]
I20260709 12:26:14.228173 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.241640 13644 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
W20260709 12:26:14.250653 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79125.5us
I20260709 12:26:14.255184 25073 consensus_queue.cc:1059] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:14.256704 25073 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:14.258489 24510 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:14.260229 24510 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:14.260915 24510 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:14.257212 13542 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 26 to 28, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:14.222137 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81058.7us
I20260709 12:26:14.276176 13799 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
W20260709 12:26:14.263230 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99419.8us
I20260709 12:26:14.278342 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.270299 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 2059) took 1192 ms. Trace:
I20260709 12:26:14.277957 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.277539 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.278695 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.217788 24865 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:14.300012 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1781) took 1133 ms. Trace:
I20260709 12:26:14.296747 14425 rpcz_store.cc:276] 0709 12:26:13.078088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.080444 (+  2356us) service_pool.cc:224] Handling call
0709 12:26:14.269995 (+1189551us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b540010f4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:14.270182 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1771157760}
I20260709 12:26:14.291846 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:14.278098 13542 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 30 to 32, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 32 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:14.303238 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:14.309823 14421 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:14.311892 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57715.7us
I20260709 12:26:14.302594 13801 rpcz_store.cc:276] 0709 12:26:13.159165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.227484 (+ 68319us) service_pool.cc:224] Handling call
0709 12:26:13.777436 (+549952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.778571 (+  1135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.287447 (+508876us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54000e56d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:14.292718 (+  5271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263678336}
I20260709 12:26:14.317050 13646 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:14.319073 13792 raft_consensus.cc:2393] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:14.324035 25101 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.325052 25101 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:14.293123 25072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.069s	sys 0.109s
W20260709 12:26:14.326754 25072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.069s	sys 0.109s
W20260709 12:26:14.326859 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 160875us
I20260709 12:26:14.328265 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:14.330791 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:14.331810 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 38.
I20260709 12:26:14.336870 14421 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:14.353510 24937 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:14.354945 24900 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:14.355396 24900 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:14.356559 24900 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.361578 14423 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:14.305531 25029 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:14.363401 25029 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.378755 25037 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:14.388659 25132 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 12:26:14.300144 25027 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.393085 25027 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.394367 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48736 (request call id 1741) took 1990 ms. Trace:
I20260709 12:26:14.394820 14423 rpcz_store.cc:276] 0709 12:26:12.404178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.404330 (+   152us) service_pool.cc:224] Handling call
0709 12:26:14.394101 (+1989771us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54005965d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:14.394305 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4225051392}
I20260709 12:26:14.395515 25027 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.396155 25065 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:26:14.397125 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 105384us
I20260709 12:26:14.399005 13764 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:14.305395 25115 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:14.399856 24772 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:14.400487 25115 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.353835 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:14.263550 24751 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.315886 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:14.420454 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 101535us
W20260709 12:26:14.423321 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 69566.2us
I20260709 12:26:14.419684 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:14.431850 13793 raft_consensus.cc:2368] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34.
I20260709 12:26:14.429584 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:14.425204 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 350705us
I20260709 12:26:14.436779 13645 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5510) took 1036 ms. Trace:
W20260709 12:26:14.280653 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 260) took 24208 ms (client timeout 15000 ms). Trace:
I20260709 12:26:14.262233 24510 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.437259 13645 rpcz_store.cc:276] 0709 12:26:13.400356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.964570 (+564214us) service_pool.cc:224] Handling call
0709 12:26:13.997591 (+ 33021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.041271 (+ 43680us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:26:14.041646 (+   375us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.436360 (+394714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.436424 (+    64us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:14.436480 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.436564 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.436717 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":689,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":16687,"thread_start_us":377709,"threads_started":1}
I20260709 12:26:14.424772 24990 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:14.428793 13542 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 17 to 21, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 21 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:14.464612 24736 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:14.465091 25131 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.001s
I20260709 12:26:14.466933 25075 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:14.467443 24990 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:14.418972 25029 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.417625 24865 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:14.456660 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:14.481915 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.430382 24953 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:26:14.489348 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68034.9us
W20260709 12:26:14.485651 13651 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.014s	sys 0.029s
W20260709 12:26:14.510816 13651 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.014s	sys 0.029s
I20260709 12:26:14.439674 24900 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:14.511890 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:14.493505 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.503736 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:14.504571 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 174447us
I20260709 12:26:14.485845 25027 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.515496 13542 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 25 to 27, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:14.513242 13793 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:14.521265 13720 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:14.495889 25075 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:14.517845 13767 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.024s	sys 0.051s
I20260709 12:26:14.537163 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:14.565267 13767 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.024s	sys 0.051s
W20260709 12:26:14.565783 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 678014us
W20260709 12:26:14.566711 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.438519 25065 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.086s
W20260709 12:26:14.570134 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54546.6us
I20260709 12:26:14.506866 24969 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.572297 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:14.573017 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1789) took 1168 ms. Trace:
I20260709 12:26:14.516587 24953 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:14.594727 13542 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:14.595911 13724 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:14.516265 25037 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.610683 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:14.577348 24943 raft_consensus.cc:2764] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:26:14.569831 25075 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:14.579734 13649 rpcz_store.cc:276] 0709 12:26:13.404248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.079324 (+675076us) service_pool.cc:224] Handling call
0709 12:26:14.079685 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:14.095196 (+ 15511us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:26:14.112054 (+ 16858us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.522214 (+410160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.522314 (+   100us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:26:14.522379 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.522493 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.572926 (+ 50433us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48791,"prepare.queue_time_us":16327,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":63,"raft.queue_time_us":17214,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":8354,"thread_start_us":330934,"threads_started":1}
I20260709 12:26:14.569602 25115 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:14.599040 24953 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:14.620883 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.594791 25091 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.620019 13542 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:14.606287 24510 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:14.645346 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3398) took 1277 ms. Trace:
I20260709 12:26:14.645807 13651 rpcz_store.cc:276] 0709 12:26:13.368529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.831719 (+463190us) service_pool.cc:224] Handling call
0709 12:26:13.832121 (+   402us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:13.899385 (+ 67264us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:13.929550 (+ 30165us) log.cc:844] Serialized 14 byte log entry
0709 12:26:14.516953 (+587403us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:14.572466 (+ 55513us) log.cc:844] Serialized 52 byte log entry
0709 12:26:14.572645 (+   179us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:14.572680 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.572757 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.645070 (+ 72313us) raft_consensus.cc:1685] finished
0709 12:26:14.645097 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.645278 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94210,"prepare.queue_time_us":24506,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":73,"raft.queue_time_us":73410,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":38499,"thread_start_us":531849,"threads_started":1,"wal-append.queue_time_us":679819}
I20260709 12:26:14.624851 13797 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:14.648742 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:14.650386 24751 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.651278 24751 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.652607 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3399) took 1196 ms. Trace:
I20260709 12:26:14.653101 13647 rpcz_store.cc:276] 0709 12:26:13.456387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.146896 (+690509us) service_pool.cc:224] Handling call
0709 12:26:14.147372 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.155974 (+  8602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.156010 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.156633 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.572136 (+415503us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:14.572223 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.597206 (+ 24983us) spinlock_profiling.cc:243] Waited 8.34 ms on lock 0x7b5400097c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:14.597396 (+   190us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.652264 (+ 54868us) raft_consensus.cc:1685] finished
0709 12:26:14.652288 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.652529 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19178752,"thread_start_us":366291,"threads_started":1,"wal-append.queue_time_us":493598}
I20260709 12:26:14.577628 25033 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.654594 24755 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:14.655274 24755 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.654970 25033 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.655752 24755 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:14.657506 25033 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.564203 25027 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.659155 13618 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:14.660010 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.660389 25027 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.599751 24969 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.661759 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:14.636912 25101 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:14.637671 25091 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.640877 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.641906 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:14.643587 25115 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:14.643337 13542 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 18 to 22, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 22 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:14.630900 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:14.627357 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 196902us
W20260709 12:26:14.674111 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 63544.8us
I20260709 12:26:14.625391 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:14.662962 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:14.679016 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 519746us
I20260709 12:26:14.679030 13798 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:14.676378 13801 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:14.691586 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:14.694669 13724 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 19. Preceding OpId from leader: term: 38 index: 20. (index mismatch)
I20260709 12:26:14.620909 24558 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:14.662204 25070 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:14.688629 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:14.690827 13719 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:14.618645 24937 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 12:26:14.672888 24736 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:14.663141 24969 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:14.708448 13796 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:14.624347 24768 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:26:14.628048 25073 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.073s
I20260709 12:26:14.683172 25029 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:14.623553 25037 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.721084 13725 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: [35.20-35.20]
I20260709 12:26:14.725679 13720 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:14.643365 24862 consensus_queue.cc:1059] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.068s
I20260709 12:26:14.729183 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:14.729501 13801 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:14.707937 25070 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:14.737160 13718 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:14.738008 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 229216us
I20260709 12:26:14.709151 25027 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.751004 13793 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 35 index: 26. Preceding OpId from leader: term: 37 index: 27. (index mismatch)
I20260709 12:26:14.721972 25029 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:14.750967 24752 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:14.799650 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 103569us
I20260709 12:26:14.799779 14391 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:14.678570 24990 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.735363 25065 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.751542 25070 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:14.797030 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 126114us
I20260709 12:26:14.780846 24751 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:14.438071 13547 rpcz_store.cc:269] 0709 12:25:50.071541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.858137 (+4786596us) service_pool.cc:224] Handling call
0709 12:26:14.280094 (+19421957us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:00.714537 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:00.715411 (+   874us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:00.715488 (+    77us) write_op.cc:435] Acquired schema lock
0709 12:26:00.715517 (+    29us) tablet.cc:662] Decoding operations
0709 12:26:00.717941 (+  2424us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:00.718100 (+   159us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:00.718142 (+    42us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:26:01.028921 (+310779us) tablet.cc:701] Row locks acquired
0709 12:26:01.028969 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:26:01.029177 (+   208us) write_op.cc:270] Start()
0709 12:26:01.029464 (+   287us) write_op.cc:276] Timestamp: P: 1783599961029105 usec, L: 0
0709 12:26:01.029500 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 12:26:01.123284 (+ 93784us) log.cc:844] Serialized 7425 byte log entry
0709 12:26:02.008267 (+884983us) op_driver.cc:464] REPLICATION: finished
0709 12:26:02.149612 (+141345us) write_op.cc:301] APPLY: starting
0709 12:26:02.149755 (+   143us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:02.461579 (+311824us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:02.461768 (+   189us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:02.921492 (+459724us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:03.002940 (+ 81448us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:26:03.030157 (+ 27217us) write_op.cc:312] APPLY: finished
0709 12:26:03.089448 (+ 59291us) log.cc:844] Serialized 172 byte log entry
0709 12:26:03.310051 (+220603us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:03.322028 (+ 11977us) write_op.cc:454] Released schema lock
0709 12:26:03.322170 (+   142us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:03.330558 (+  8388us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:03.330595 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":460448,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":118729,"apply.run_cpu_time_us":87404,"apply.run_wall_time_us":1198124,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83699,"num_ops":19,"prepare.queue_time_us":341227,"prepare.run_cpu_time_us":70003,"prepare.run_wall_time_us":777222,"raft.queue_time_us":209087,"raft.run_cpu_time_us":11793,"raft.run_wall_time_us":84134,"replication_time_us":978536,"thread_start_us":839819,"threads_started":5,"wal-append.queue_time_us":708934,"wal-append.run_cpu_time_us":39388,"wal-append.run_wall_time_us":2525224}]]}
I20260709 12:26:14.796001 25029 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:14.737030 24558 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.827672 24751 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.828406 24751 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.829210 24558 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:14.830745 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:14.832546 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 244) took 23526 ms (client timeout 15000 ms). Trace:
W20260709 12:26:14.833045 13547 rpcz_store.cc:269] 0709 12:25:51.305960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.832258 (+23526298us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:14.832434 (+   176us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:26:14.834103 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 161) took 20984 ms (client timeout 15000 ms). Trace:
W20260709 12:26:14.837383 13547 rpcz_store.cc:269] 0709 12:25:53.850250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.833889 (+20983639us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:14.834032 (+   143us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:14.717401 24865 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.813931 24699 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:14.863133 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:14.858477 24969 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.798146 25027 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.823105 25102 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:26:14.876682 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:14.887202 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 291729us
W20260709 12:26:14.875801 25136 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 12:26:14.881098 24694 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.004s
W20260709 12:26:14.899012 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68366.9us
I20260709 12:26:14.909452 13724 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
W20260709 12:26:14.898766 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 239722us
I20260709 12:26:14.866179 25064 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:14.929136 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:14.928417 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:14.854388 25065 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.896466 25102 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:14.968767 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.934623 25033 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:14.923760 24751 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:14.970420 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:14.974833 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51417.4us
I20260709 12:26:14.978135 13647 raft_consensus.cc:1275] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 25. Preceding OpId from leader: term: 37 index: 27. (index mismatch)
I20260709 12:26:14.972309 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:14.923187 24969 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:14.977173 24694 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:14.997304 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 111390us
W20260709 12:26:14.990429 25140 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.281s	user 0.000s	sys 0.002s
W20260709 12:26:14.999450 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 92806.6us
W20260709 12:26:15.000149 13617 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.66:41841 timed out after 1.643s (SENT) [suppressed 8 similar messages]
I20260709 12:26:14.979490 25033 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:14.985380 25091 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:15.018695 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73404.2us
I20260709 12:26:15.026633 25027 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.027685 25027 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:15.031689 24953 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.033390 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:15.034782 24915 raft_consensus.cc:3055] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:15.035579 25073 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:15.037161 25135 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:14.998950 24862 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:15.037433 24752 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:15.048094 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:14.981820 25064 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:15.020980 25138 log.cc:927] Time spent T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:26:15.049504 25102 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:15.055855 25148 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:15.056865 25102 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:15.050053 24862 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.059500 24862 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:15.060464 25152 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:15.063342 25065 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:15.063853 25064 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.017732 25028 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:15.023205 25037 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.024196 24694 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.068821 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1616) took 1021 ms. Trace:
I20260709 12:26:15.064702 25064 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:26:15.069715 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 70032.6us
I20260709 12:26:15.068389 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:15.031777 24610 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:15.078279 24610 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:15.081313 24969 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.047667 25135 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:15.084832 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99371.1us
I20260709 12:26:15.042207 24915 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:15.069242 14418 rpcz_store.cc:276] 0709 12:26:14.047801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.047947 (+   146us) service_pool.cc:224] Handling call
0709 12:26:14.048347 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.048826 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.048857 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.049526 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.424118 (+374592us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:14.424169 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.424253 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.068558 (+644305us) raft_consensus.cc:1685] finished
0709 12:26:15.068583 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.068739 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1371,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":367787,"threads_started":1,"wal-append.queue_time_us":662682}
W20260709 12:26:15.098755 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58859.5us
I20260709 12:26:15.100131 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:15.102223 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:15.102599 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:15.073356 25064 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:15.095383 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:15.102835 25146 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:26:15.095325 25153 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:15.134290 25164 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:15.134837 25153 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.137310 25153 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.139411 24694 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.096688 25091 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.140837 25091 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:15.145582 25075 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.001s
I20260709 12:26:15.146087 25075 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.148403 25075 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.122495 25064 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.150589 24969 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:15.131306 25135 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.150611 24915 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.153477 25135 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.154090 24915 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:15.155654 24953 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:15.157251 25140 log.cc:927] Time spent T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.093s	user 0.003s	sys 0.000s
I20260709 12:26:15.159937 25101 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:15.160454 25101 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:15.161448 25101 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.162729 24510 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:15.163170 24510 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:15.164352 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.164127 24510 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.165462 24510 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:15.166015 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 97719.7us
I20260709 12:26:15.167352 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:15.167979 24900 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.650s	user 0.005s	sys 0.007s
I20260709 12:26:15.168538 25070 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:15.176856 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 81578.5us
W20260709 12:26:15.180615 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84138.2us
I20260709 12:26:15.182308 25101 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.186446 25155 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:15.187184 25155 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.187546 25155 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:15.188480 24755 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.009s
W20260709 12:26:15.189486 25149 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.319s	user 0.003s	sys 0.000s
I20260709 12:26:15.190555 25072 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:15.191787 25160 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:26:15.192133 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:15.189093 24755 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.200215 25161 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:15.200484 13649 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:15.200917 25161 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.203361 25161 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.203518 24050 raft_consensus.cc:3055] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:15.203922 24050 raft_consensus.cc:740] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:15.204988 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:15.205662 14422 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:15.207021 24755 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.207381 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:15.212857 25164 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:15.213915 25029 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.152578 25091 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:15.218057 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 115988us
I20260709 12:26:15.218456 13720 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:15.219245 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:15.220073 25027 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:15.220219 25029 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.220636 25027 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:15.221264 13693 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:15.221382 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 179940us
I20260709 12:26:15.221642 25027 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.222312 25164 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:15.217298 24862 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.223599 25033 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.224710 24862 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:15.179025 13718 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
W20260709 12:26:15.226007 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 48036.5us
W20260709 12:26:15.233173 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.226786 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:15.179009 13797 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:26:15.243134 24990 consensus_queue.cc:1048] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:15.244318 25037 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:15.244895 25037 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.246574 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.247257 13649 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:15.252286 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.253100 13647 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:15.259306 13719 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:15.252425 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:15.268047 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.272244 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:15.273428 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66201us
I20260709 12:26:15.276197 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.230562 25075 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:15.276888 13648 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:15.268913 13651 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 38.
I20260709 12:26:15.276444 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:15.287719 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 211042us
I20260709 12:26:15.235208 25164 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:15.288779 13727 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:15.236943 25115 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:15.290895 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.254050 24755 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:26:15.292424 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 123033us
I20260709 12:26:15.290200 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.293277 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.294616 13647 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.297384 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.272984 25153 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:15.325320 13722 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.273149 25161 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.333765 25070 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.288867 25073 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:15.344226 25073 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:15.344732 25073 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.351150 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.341545 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.352097 13651 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:15.354537 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.298498 25033 consensus_queue.cc:1059] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 32.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.056s
W20260709 12:26:15.357097 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.360352 13723 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.361500 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.362704 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.357815 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.293632 24755 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:15.272768 24764 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
I20260709 12:26:15.366134 13719 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 23.
I20260709 12:26:15.374382 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.284173 25163 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.072s
I20260709 12:26:15.378010 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.378845 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.379694 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.384655 13648 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 36.
I20260709 12:26:15.328469 25153 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:15.386739 13719 raft_consensus.cc:2393] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.385777 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.388443 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.390751 13651 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:15.361022 24736 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.418606 24736 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.398845 25037 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.395099 25153 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.421240 24736 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:15.373394 24755 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.371356 24900 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.054s
I20260709 12:26:15.425894 24900 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:15.426312 24900 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:15.427328 24900 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:15.427903 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55928.6us
I20260709 12:26:15.429419 25073 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.431077 25065 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.431660 25065 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.433935 25065 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:15.401763 24969 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:15.440155 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:15.430532 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49532.3us
I20260709 12:26:15.429594 25150 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 12:26:15.404661 24050 consensus_queue.cc:260] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.369228 25102 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.451160 25102 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.453657 25102 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.434350 25073 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.436193 24833 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.410331 25023 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:26:15.436828 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77896.3us
I20260709 12:26:15.437501 25176 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:15.388798 25075 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.072s
I20260709 12:26:15.430109 25164 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.444907 13795 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:15.449177 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.450081 25033 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:15.435654 25115 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 12:26:15.444473 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:15.455651 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:15.455574 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.355432 25161 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.483023 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:15.580858 13646 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.582340 25161 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:15.477258 24050 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.457412 24833 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:15.535408 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.551219 13723 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:15.482769 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.588267 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:15.592653 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.595378 13651 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:15.474736 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.597018 13645 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:15.458956 13649 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 25.
I20260709 12:26:15.487802 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.598553 13727 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.553092 13648 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:15.488235 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.609447 13724 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:15.460141 25170 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.101s
I20260709 12:26:15.561556 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:15.466236 25156 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.406s	user 0.000s	sys 0.003s
I20260709 12:26:15.567531 24755 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.573194 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.614558 13797 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:15.575170 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.617118 13726 raft_consensus.cc:2393] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.617808 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.618292 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.619087 13648 raft_consensus.cc:2410] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:26:15.619815 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.620832 13726 raft_consensus.cc:2393] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.621831 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.578925 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.636396 13646 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:15.596037 25163 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:15.478781 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:15.551108 14423 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:15.577811 25155 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.606709 25101 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 12:26:15.575873 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.623227 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:15.497509 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.650841 13650 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
W20260709 12:26:15.651782 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.460614 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:15.637925 13647 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:26:15.640059 13723 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:15.461896 25174 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:15.488891 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.644260 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.644675 13720 raft_consensus.cc:2393] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.639154 14422 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:26:15.480535 25173 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
W20260709 12:26:15.484623 25172 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:15.652899 24558 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
W20260709 12:26:15.653134 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 192631us
I20260709 12:26:15.569604 25153 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.468557 25169 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:15.646974 25115 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:15.660459 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:15.675810 14421 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.650388 25027 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.667809 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.670902 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.662915 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.648506 25072 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.702216 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 315659us
W20260709 12:26:15.675063 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 219503us
I20260709 12:26:15.709128 13719 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.706508 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.710974 13720 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.478446 25064 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.710709 24764 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:15.720974 24764 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.722833 14416 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:26:15.723425 24764 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.724592 25153 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.669806 25167 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.726843 24937 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:15.727312 24937 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:15.727406 25167 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.728282 24937 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.729684 25167 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.673722 25169 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:15.731556 13652 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:15.741570 24699 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:15.742070 24699 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:15.742830 25169 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.744403 25169 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:15.746387 25155 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.743198 24699 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.649010 25163 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.749099 24699 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:15.750789 25163 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.753170 24610 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.754778 24610 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:15.755934 25072 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.757665 13720 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:15.759987 24900 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:15.764070 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:15.778345 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:15.778990 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:15.769361 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:15.787583 14423 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:15.788746 14419 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:15.791625 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:15.798983 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.797672 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 35 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.809306 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:15.679477 25091 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.678544 24736 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:15.701671 25115 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:15.828547 14421 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:15.830982 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.831559 13725 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:15.708498 25176 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:15.831202 13796 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 34. Current term is 35. Ops: [34.18-34.18]
I20260709 12:26:15.772836 24833 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:15.840605 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 62169.1us
I20260709 12:26:15.840092 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.838948 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.841956 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.793299 25161 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:15.796558 25072 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.793650 25070 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:15.847177 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.676807 25135 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.156s
I20260709 12:26:15.817932 24510 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:15.863287 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.712299 25027 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.677815 25035 consensus_queue.cc:1059] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 33.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.167s
I20260709 12:26:15.910864 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.659308 25174 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.842473 24990 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:15.914299 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.914654 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.868801 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.922596 13794 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:15.921360 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.846328 25161 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:15.919334 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50833.5us
I20260709 12:26:15.838047 25091 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:15.921104 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 111932us
I20260709 12:26:15.932713 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:15.878063 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.865772 24510 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.836239 24755 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:15.845472 25157 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:15.936139 13794 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:15.942862 13724 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:15.943521 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.945298 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.863729 25150 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:15.936720 13652 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 37: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:15.929407 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:15.946022 13720 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:15.919363 24764 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:15.953205 14391 leader_election.cc:400] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34.
I20260709 12:26:15.953655 14391 leader_election.cc:403] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:15.957654 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.958701 13794 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:15.934897 25161 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:15.941872 24937 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:15.943215 24510 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:15.951736 25155 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.868939 25070 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.992214 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:15.968102 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:15.945084 24990 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:15.971060 13652 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 26. Current term is 28. Ops: [24.17-26.18]
I20260709 12:26:15.972018 25171 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.973376 13646 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:15.973799 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.999583 13645 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:15.974189 13648 raft_consensus.cc:1240] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:15.974822 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.975584 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:15.976753 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:15.977308 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.001874 13651 raft_consensus.cc:2393] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:15.978644 25179 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.043s
I20260709 12:26:15.979368 25064 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:15.981473 13799 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
W20260709 12:26:15.982432 25159 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:26:15.982898 25158 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.115s
I20260709 12:26:15.869310 25176 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:15.956086 25038 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:15.994740 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:15.997829 13793 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:15.977859 13644 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:16.001861 13647 raft_consensus.cc:2368] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34.
I20260709 12:26:15.937089 25115 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.956180 25169 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.042s
I20260709 12:26:16.024509 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:15.950296 25150 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:16.008318 13643 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:16.005119 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:15.955713 24764 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:15.987920 25023 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election lost for term 33. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34.
I20260709 12:26:16.003958 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:16.021924 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77981.9us
I20260709 12:26:16.025247 13795 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:15.949319 25157 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.027052 13722 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:16.031983 13649 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:16.030196 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:16.030364 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.038065 13720 raft_consensus.cc:2393] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:15.997066 25155 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:16.038727 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44123us
I20260709 12:26:16.039809 25174 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.040221 25091 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.054401 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.055109 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:16.056671 13792 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:16.056473 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.033054 25150 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.005875 25171 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.059866 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.061618 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:16.061685 13718 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:16.063220 13648 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 24.
I20260709 12:26:16.063083 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.037457 25038 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:16.066211 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:16.066860 13800 raft_consensus.cc:2410] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:26:16.066924 13643 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:16.066326 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.068312 13727 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:16.051723 25153 raft_consensus.cc:2764] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:26:16.069229 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.072526 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:16.070592 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:16.071563 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.072911 13793 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:16.035501 25167 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.026s
I20260709 12:26:16.076076 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:16.076727 14424 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:16.076644 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:16.079852 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:16.080631 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.082489 13797 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.060774 24915 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:16.094904 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.095811 13649 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
W20260709 12:26:16.097028 25186 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 12:26:16.076059 24990 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.099073 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.099771 13720 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:16.089891 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.090178 25155 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.106859 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:16.106974 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:16.108376 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.104846 13796 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:16.109500 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:16.111238 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.112957 13727 raft_consensus.cc:1240] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:16.113924 13726 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:26:16.059141 25033 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.029s
I20260709 12:26:16.111887 13721 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:26:16.118134 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46653.3us
I20260709 12:26:16.120466 13792 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:16.117050 25027 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.132365 25169 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.133122 25169 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.135386 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.136052 13645 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:16.137506 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.132474 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:16.138172 13720 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:16.138556 13796 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:16.139741 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.142815 13719 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:16.144696 13646 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: [35.20-35.20]
I20260709 12:26:16.146662 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.094910 24768 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:16.157589 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.150509 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.160190 13721 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
W20260709 12:26:16.174871 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.185470 24510 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.189049 13720 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [27.17-28.18]
I20260709 12:26:16.154973 25153 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.192620 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 72621.7us
I20260709 12:26:16.194432 24510 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:16.197800 13643 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 35 because replica has last-logged OpId of term: 34 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 17.
W20260709 12:26:16.208608 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 144550us
I20260709 12:26:16.208235 24510 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:16.205281 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.174350 24610 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.025s
I20260709 12:26:16.190188 24768 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:16.228734 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.223801 13797 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:16.233001 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.234704 25174 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:16.235394 25174 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.199054 25153 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.216785 25027 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.241748 25027 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:16.248132 25192 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:26:16.250844 25189 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:26:16.252899 25171 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:16.254380 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60836us
I20260709 12:26:16.254890 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.255615 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.255882 13800 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:16.257371 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.245956 25027 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.258472 24768 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.261368 24768 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:16.271579 25169 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:16.194013 25038 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.275717 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.276223 25157 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:16.216796 25070 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.277715 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:16.290683 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.286074 14418 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
W20260709 12:26:16.298097 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66375.3us
I20260709 12:26:16.298746 13796 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:16.301182 13794 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
W20260709 12:26:16.303184 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 133367us
I20260709 12:26:16.305774 13796 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:26:16.310426 14421 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:16.277771 24764 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.315729 14419 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:26:16.300879 25181 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:16.301035 25166 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:16.318018 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.308914 24990 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:16.326380 13652 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:16.327705 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:16.323670 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.336464 13643 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:16.331530 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.333139 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.307960 25174 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.303910 25027 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:26:16.324781 25064 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:16.334478 24990 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:16.319500 24900 consensus_queue.cc:1048] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 24, Time since last communication: 0.035s
I20260709 12:26:16.366835 24833 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:16.367369 24833 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:16.368363 24833 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 35.20, Last appended by leader: 20, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.369629 24833 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:16.370311 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:16.372200 25206 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:16.375066 25064 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:16.350740 25037 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:16.354205 25164 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:16.387952 13643 raft_consensus.cc:2468] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
W20260709 12:26:16.423134 25197 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:16.375533 25064 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:16.428630 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:16.435714 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 124501us
W20260709 12:26:16.438452 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 68243.5us
I20260709 12:26:16.437100 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.442140 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:16.427074 25064 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:16.443895 24937 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:16.444384 14391 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.452955 25027 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:16.453642 25027 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:16.456292 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:16.458942 14391 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.457616 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:16.460892 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 264706us
I20260709 12:26:16.425746 25164 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.461833 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.474004 24900 raft_consensus.cc:2804] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:16.475118 24900 raft_consensus.cc:697] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:16.476204 25161 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.477622 13798 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:26:16.476158 24900 consensus_queue.cc:237] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.478474 25153 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.478861 25206 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:16.491643 25206 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.492188 25206 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:16.493953 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.495060 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.495400 25035 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.496498 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.488014 13648 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:26:16.488653 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.490864 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.504153 13651 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:16.496708 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:16.503050 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.513906 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.511085 24937 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:16.522356 24937 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:16.522699 25211 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:16.523377 25211 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:16.524520 25211 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.529803 25181 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:16.506773 25073 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:16.531397 13652 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:16.532222 24050 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:16.533685 24510 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:16.543608 24510 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.543983 24510 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:16.536571 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 5004) took 1174 ms. Trace:
I20260709 12:26:16.442317 25075 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.065s
I20260709 12:26:16.539502 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.559306 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.544868 13799 rpcz_store.cc:276] 0709 12:26:15.361846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.362236 (+   390us) service_pool.cc:224] Handling call
0709 12:26:16.536319 (+1174083us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54003c0854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:16.536452 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382454656}
W20260709 12:26:16.561414 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51138.8us
I20260709 12:26:16.570567 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.566128 13766 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:16.577976 13647 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:16.578621 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82227.1us
I20260709 12:26:16.498793 25209 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:16.579803 13767 leader_election.cc:400] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34.
I20260709 12:26:16.454950 25027 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.581446 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.584097 13767 leader_election.cc:403] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:16.590581 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:16.591431 13726 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:26:16.592185 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 78377.4us
I20260709 12:26:16.593160 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.594170 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.594458 24900 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:16.594687 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:16.595539 13722 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:16.596571 24937 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:16.596854 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.597122 14423 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:16.597512 24937 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:16.598013 24937 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:16.598743 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:16.599130 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.599068 24937 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.598760 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:16.600406 13794 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:16.600711 24937 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:16.601907 13797 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:16.603060 14419 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 22. (term mismatch)
I20260709 12:26:16.603415 25181 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:16.604044 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.605417 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.605438 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.606813 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.607344 25181 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:16.607586 13792 raft_consensus.cc:2368] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:26:16.607769 25181 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:16.608214 25181 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:16.608729 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.609324 13792 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:16.609685 13794 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:16.613257 25037 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.005s
I20260709 12:26:16.603459 14419 pending_rounds.cc:85] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 21
I20260709 12:26:16.619201 14419 pending_rounds.cc:107] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 29.22
I20260709 12:26:16.612884 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:16.620241 25037 consensus_queue.cc:1243] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 29.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:26:16.611248 13794 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:26:16.621397 14422 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:16.610189 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.619637 25181 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.619626 14419 raft_consensus.cc:2889] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:16.603426 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.608187 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.625775 25037 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:16.610651 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:16.627115 13690 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.628628 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:16.629926 25212 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:16.630620 25181 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:16.630684 25212 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:16.630957 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.633248 13726 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:16.634512 25212 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:16.633505 25037 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:16.637630 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.646574 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:16.643782 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 64313.4us
I20260709 12:26:16.449707 24990 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.652842 24510 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.654788 24050 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:16.659785 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.582451 25209 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.582082 25102 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:16.531394 25150 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:16.587881 25166 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:16.666178 13800 raft_consensus.cc:2368] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:16.665755 25209 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:16.676476 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:16.680739 13647 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:16.584971 24915 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.086s
W20260709 12:26:16.667151 25214 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:26:16.686990 24510 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:16.651822 25073 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 33. Reason: Vote denied by peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075) with higher term. Message: Invalid argument: T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 33. Current term is 34.
I20260709 12:26:16.638758 24833 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 12:26:16.668151 25181 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:26:16.694819 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 66259.3us
I20260709 12:26:16.692260 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:16.688418 13767 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.697201 13719 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:26:16.666230 25212 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:16.698526 14424 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:16.669436 25102 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.699338 24510 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:16.700279 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:16.703578 25210 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:26:16.683317 25206 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.700245 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:16.714844 25166 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:16.716260 25166 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:16.666492 25199 consensus_queue.cc:1059] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:26:16.699106 25181 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:16.723392 25073 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:16.726465 25181 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:16.731246 25203 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:16.731901 25150 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.712709 25102 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:16.740267 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:16.741318 13724 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:26:16.741816 13647 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:16.710031 25206 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.754561 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.755627 13649 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:16.775964 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:16.776082 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.776749 13649 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:16.777287 13647 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:16.778643 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 119030us
I20260709 12:26:16.692173 25037 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.779825 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:16.786949 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:16.788017 13647 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:16.780594 25161 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:16.783246 25164 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.791720 13693 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.800555 25064 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.803380 25064 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:16.806042 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:16.727290 25212 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.808043 13652 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:26:16.813717 13652 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 34. Current term is 35. Ops: []
W20260709 12:26:16.823345 25217 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:26:16.811676 25035 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:16.846248 13799 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 33. Current term is 34. Ops: []
W20260709 12:26:16.847004 25207 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:16.828640 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3412) took 1015 ms. Trace:
W20260709 12:26:16.829159 25218 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:26:16.848505 13798 rpcz_store.cc:276] 0709 12:26:15.813698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.814001 (+   303us) service_pool.cc:224] Handling call
0709 12:26:16.825368 (+1011367us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:16.828562 (+  3194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463029120}
I20260709 12:26:16.851296 13643 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:16.862473 13647 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:16.864398 13767 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:16.958525 24050 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:16.959246 24050 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.960341 13652 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:16.961784 13652 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:16.962822 24050 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:16.834112 13724 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:16.794339 25225 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:16.841156 25211 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election won for term 34
W20260709 12:26:16.841975 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50372.1us
I20260709 12:26:16.975171 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:16.963092 13647 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:16.977833 13647 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:16.978562 25225 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:16.979677 25225 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:16.982712 25211 raft_consensus.cc:697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:16.983786 25211 consensus_queue.cc:237] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:16.986680 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122424us
I20260709 12:26:16.988910 25065 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:16.989415 25065 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:16.990455 25161 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:16.991144 25161 consensus_queue.cc:1243] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer 4dafe08a4f404502822a0732fe570de4 log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:26:16.994414 25184 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.015s	user 0.001s	sys 0.005s
I20260709 12:26:16.995990 25065 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.000579 13693 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.004539 13649 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:17.005241 25064 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:17.006832 25226 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:16.833739 13719 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:17.030381 25064 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:17.018304 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:17.031837 25226 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:17.031570 25064 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.032279 25226 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.032655 25226 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:17.033931 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.034143 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.034469 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.032661 25206 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 12:26:17.035634 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:17.035759 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.005198 25102 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.015939 25150 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.040350 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:17.040146 25150 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.042019 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:17.042927 25102 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:17.026582 24736 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:17.043704 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.039392 13798 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 34. Current term is 35. Ops: [34.18-34.18]
I20260709 12:26:17.045043 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:17.029435 24558 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.046064 24558 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.046140 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.048545 13719 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:26:17.050607 25198 log.cc:927] Time spent T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.689s	user 0.000s	sys 0.011s
I20260709 12:26:17.051473 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.053973 24969 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:17.054060 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:17.054669 24969 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:17.054944 13644 raft_consensus.cc:3060] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:17.043123 13801 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:17.050499 25150 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:17.014015 13652 pending_rounds.cc:85] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 13
I20260709 12:26:17.045401 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.056952 13652 pending_rounds.cc:107] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:26:17.057454 13724 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:17.057487 13652 raft_consensus.cc:2889] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:17.039662 14418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 2075) took 1869 ms (client timeout 1575 ms). Trace:
W20260709 12:26:17.058353 14418 rpcz_store.cc:269] 0709 12:26:15.170582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.170722 (+   140us) service_pool.cc:224] Handling call
0709 12:26:17.039485 (+1868763us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b5400600254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:17.039589 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107233664}
I20260709 12:26:17.067489 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:17.056468 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.006971 25181 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.072391 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:17.073763 13649 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:17.075809 24050 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:17.076707 13792 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:17.076529 24050 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.073568 24558 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:16.990163 25209 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.075165 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.043725 24736 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:17.091871 25181 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:17.091333 24736 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.089304 25035 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:17.106294 25035 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.106891 25035 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:17.116158 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.118445 24699 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.119228 24699 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.123234 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48712 (request call id 1621) took 1036 ms. Trace:
I20260709 12:26:17.123766 14421 rpcz_store.cc:276] 0709 12:26:16.086767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.109275 (+ 22508us) service_pool.cc:224] Handling call
0709 12:26:17.122950 (+1013675us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400555354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:17.123114 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362636160}
I20260709 12:26:17.127707 25226 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.134032 14424 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1770) took 1030 ms. Trace:
I20260709 12:26:17.134554 14424 rpcz_store.cc:276] 0709 12:26:16.104019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.105519 (+  1500us) service_pool.cc:224] Handling call
0709 12:26:16.136953 (+ 31434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.712883 (+575930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.133746 (+420863us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400515ed4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:17.133950 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280540416}
I20260709 12:26:17.136330 24755 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:17.136943 24755 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:17.140105 24969 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 34.18, Last appended by leader: 18, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.141687 24969 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:17.141989 25035 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.145017 25035 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:17.146701 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:17.147571 13646 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:17.151635 25226 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:17.155910 24990 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:17.160419 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.161456 13726 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:17.074608 25101 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:17.169404 25176 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:17.172070 13722 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:17.175997 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.176218 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:17.176761 13725 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:17.177040 13719 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:17.177697 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:17.177924 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.179385 13724 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:26:17.179949 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.180564 13725 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:17.191028 13646 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
W20260709 12:26:17.198951 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.212597 13767 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.200181 13690 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:17.214593 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.225206 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:17.227722 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:17.228811 14421 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:17.121994 24699 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:17.237771 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:17.222571 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.241957 13617 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:17.250418 14418 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:17.263284 13646 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:26:17.268853 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:17.274603 25091 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:17.138106 24755 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.275286 25091 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:17.275655 25209 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:17.275902 24755 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:17.276249 25091 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.271517 25037 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:17.277714 25037 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:17.278723 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 66271.5us
I20260709 12:26:17.239709 25155 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:17.280053 25037 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:17.280354 25220 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:17.280202 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84839.6us
I20260709 12:26:17.281160 25220 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.234548 25226 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:17.282752 25226 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:17.282832 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.280156 25155 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.283552 25155 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:17.284029 25115 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:17.283800 25226 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.279685 24558 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:17.237877 25176 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:17.277952 25101 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:17.265460 24610 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:17.336256 13726 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:17.293517 25219 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:17.365465 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 325601us
I20260709 12:26:17.367209 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.373091 13644 raft_consensus.cc:2468] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:17.323814 24610 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.374073 24050 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:17.407727 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:17.411878 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 186816us
I20260709 12:26:17.361932 25167 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:17.420475 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.421653 25064 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:17.422467 25064 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:17.371941 24699 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.425058 14416 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
W20260709 12:26:17.376390 25212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.031s	sys 0.041s
I20260709 12:26:17.300984 24915 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:17.405414 24610 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:17.432462 13693 leader_election.cc:400] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:26:17.323418 25176 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.417790 25219 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:17.458710 13693 leader_election.cc:403] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.457572 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:17.417312 25167 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:17.461055 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:17.417706 25166 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:17.431555 25212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.031s	sys 0.041s
I20260709 12:26:17.462754 13651 raft_consensus.cc:2393] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:17.464432 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 227718us
I20260709 12:26:17.464883 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:17.431932 25211 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:17.468811 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:17.468313 14418 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:26:17.481369 14416 raft_consensus.cc:1217] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Deduplicated request from leader. Original: 25.19->[28.20-28.20]   Dedup: 28.20->[]
I20260709 12:26:17.496833 14421 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:17.503397 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:17.505017 14416 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:26:17.480353 13767 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.471513 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:17.508358 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:17.465731 25167 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:17.530650 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50491.7us
I20260709 12:26:17.465967 25064 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 24. Current term is 25.
I20260709 12:26:17.455152 24915 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.542814 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:17.433534 24699 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.468288 25219 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:17.556861 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 334373us
I20260709 12:26:17.534672 25212 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:17.436686 25101 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:17.596846 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 328116us
W20260709 12:26:17.612141 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.560315 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:17.542773 25073 raft_consensus.cc:2804] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:17.616031 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.626456 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.628681 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.637880 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.642444 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:17.559458 25161 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.639818 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.647282 13798 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:17.646519 13801 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:17.609478 25212 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:17.610981 25169 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:17.657439 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 364648us
I20260709 12:26:17.658924 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.661454 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:17.608736 25115 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.652562 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:17.638238 25073 raft_consensus.cc:697] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:17.662879 13764 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:17.648082 25033 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:17.658489 25212 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:17.657482 25241 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.687988 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:17.661592 25199 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.703298 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 238515us
W20260709 12:26:17.683735 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 441876us
W20260709 12:26:17.732009 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 265882us
I20260709 12:26:17.733824 13647 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:17.734571 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:17.742298 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:17.651648 25161 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.747115 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:17.689334 25033 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.756685 13693 leader_election.cc:400] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
I20260709 12:26:17.748947 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.664637 25169 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:17.777832 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 269541us
I20260709 12:26:17.730232 25248 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:17.686955 25091 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.072s
I20260709 12:26:17.695757 25226 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:17.692214 24558 raft_consensus.cc:697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:17.782034 13693 leader_election.cc:403] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.759438 25245 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:17.761180 25033 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:17.799736 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:17.855424 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 55795.7us
W20260709 12:26:17.855831 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 99344.7us
I20260709 12:26:17.856935 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.691855 25115 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:17.796969 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 236729us
W20260709 12:26:17.799403 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 135231us
W20260709 12:26:17.854437 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105690us
I20260709 12:26:17.859800 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:17.859742 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:17.689107 25219 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.747588 25211 consensus_queue.cc:1059] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.065s
I20260709 12:26:17.868472 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:17.752807 25102 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.103s
I20260709 12:26:17.867869 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:17.872534 13726 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:17.730206 25241 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.876103 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:17.876852 13718 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:17.878788 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:17.879433 13723 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:17.741694 25199 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.798607 25257 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:17.854875 25245 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.859233 25064 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:17.875434 25065 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:17.870450 25166 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:17.899659 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:17.870877 25023 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:17.883913 25233 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.377s	user 0.000s	sys 0.001s
I20260709 12:26:17.872009 25135 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:17.914801 13693 leader_election.cc:400] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:17.915205 13693 leader_election.cc:403] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.911269 25023 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:17.918382 25243 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:17.918565 25023 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:17.918924 25243 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.919437 25243 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:17.855261 25226 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.925228 13726 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:17.892834 25245 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:17.855876 25248 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:17.851521 25073 consensus_queue.cc:237] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.898697 24915 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:17.899482 24699 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:17.938880 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.913451 25250 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.909076 25259 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:17.943640 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 83921.1us
W20260709 12:26:17.920472 24969 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.778s	user 0.013s	sys 0.002s
I20260709 12:26:17.896495 25211 consensus_queue.cc:1059] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 20, Last known committed idx: 19, Time since last communication: 0.033s
I20260709 12:26:17.916685 25065 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:17.925815 25064 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28.
W20260709 12:26:17.932542 25231 log.cc:927] Time spent T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:26:17.980720 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:17.945230 25232 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:26:17.889981 13798 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:17.983516 25250 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.932333 25158 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:17.916447 25253 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:17.989104 13798 raft_consensus.cc:606] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:17.986310 13618 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:18.002655 13798 raft_consensus.cc:614] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:17.993281 25249 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:26:17.994626 25256 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:17.953311 25166 raft_consensus.cc:3055] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:17.926550 25135 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:17.987730 25211 consensus_queue.cc:1243] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer b708d8f5f6ac4aabb1eb5cb821ace195 log is divergent from this leader: its last log entry 29.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:26:17.952987 25268 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
W20260709 12:26:18.009284 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 346567us
I20260709 12:26:17.987875 24969 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:26:18.012262 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.987839 24990 consensus_meta.cc:287] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.831s	user 0.013s	sys 0.004s
I20260709 12:26:18.004961 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:17.993299 24610 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.547s	user 0.012s	sys 0.000s
I20260709 12:26:18.019585 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.019520 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:18.022576 13797 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:18.006346 25209 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:18.006691 25102 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:26:18.041894 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:17.928701 24558 consensus_queue.cc:237] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.008726 25166 raft_consensus.cc:740] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:18.009811 25158 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:18.068363 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 48934.2us
I20260709 12:26:18.016695 24915 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.033864 25237 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
W20260709 12:26:18.033154 25244 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:26:18.077816 25199 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:18.079731 25238 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 12:26:18.080168 24736 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:18.018101 25064 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:18.028422 25220 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:17.984516 25259 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.081846 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5549) took 1079 ms. Trace:
I20260709 12:26:18.083483 25259 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.082337 13652 rpcz_store.cc:276] 0709 12:26:17.002635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.007664 (+  5029us) service_pool.cc:224] Handling call
0709 12:26:17.012196 (+  4532us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:17.079232 (+ 67036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.079267 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:17.079945 (+   678us) log.cc:844] Serialized 52 byte log entry
0709 12:26:17.364533 (+284588us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:17.410816 (+ 46283us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.411106 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.411218 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.081591 (+670373us) raft_consensus.cc:1685] finished
0709 12:26:18.081617 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.081776 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49718,"prepare.queue_time_us":11453,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":9565,"raft.queue_time_us":251905,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":22518,"thread_start_us":260497,"threads_started":1,"wal-append.queue_time_us":510936}
I20260709 12:26:18.040257 25161 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.084872 25252 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:18.085528 25252 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:18.085628 25242 raft_consensus.cc:2764] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:26:18.086051 25252 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:18.087989 25271 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.088593 25271 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.096217 25252 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:18.101315 25167 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.002s
I20260709 12:26:18.099100 25102 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:18.103340 25102 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:18.104266 25102 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:18.105252 25155 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.821s	user 0.009s	sys 0.002s
I20260709 12:26:18.105554 25102 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:18.101436 14424 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
mode: GRACEFUL
new_leader_uuid: "4dafe08a4f404502822a0732fe570de4"
 from {username='slave'} at 127.0.0.1:43168
I20260709 12:26:18.106688 14424 raft_consensus.cc:606] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Received request to transfer leadership to TS 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:18.107030 14424 raft_consensus.cc:614] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:18.107247 25176 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.030977 25169 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.109011 14390 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:18.025383 25070 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:18.110042 25167 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.116797 25167 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.111184 25169 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:18.033011 25253 raft_consensus.cc:2764] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:26:18.149204 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.069696 25065 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.160392 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.161330 13643 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:18.162456 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.126199 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:18.163736 13726 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:18.110858 25226 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:18.038939 25234 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.568s	user 0.000s	sys 0.004s
I20260709 12:26:18.111371 25115 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.039232 25241 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.067070 24969 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:26:18.193342 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47801.4us
I20260709 12:26:18.020313 25251 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.128s
I20260709 12:26:18.067515 25209 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.154521 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.155774 13646 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:18.157815 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.158671 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:18.105747 25155 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.160053 13801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28452187c3924de9940235e9b965477d"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:18.212846 13801 raft_consensus.cc:606] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.069744 24990 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:18.191716 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.192577 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.216418 13644 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:18.198004 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:18.201610 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:18.169656 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:18.193410 25169 raft_consensus.cc:697] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:18.217911 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 358233us
I20260709 12:26:18.069936 25262 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.141s
W20260709 12:26:18.164064 25266 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
I20260709 12:26:18.202033 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:18.028594 24610 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.109365 25023 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.163810 25070 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:18.209280 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.232546 13650 raft_consensus.cc:2368] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30.
I20260709 12:26:18.209493 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:18.211047 14424 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
W20260709 12:26:18.214252 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 310877us
W20260709 12:26:18.218497 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48984.8us
I20260709 12:26:18.118191 25248 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.218055 13727 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:18.220439 14425 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:18.249899 13718 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:18.256876 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:18.221134 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 409006us
I20260709 12:26:18.132922 24915 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:18.211606 25174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.000s	sys 0.007s
I20260709 12:26:18.225483 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:18.208247 25226 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.232460 13795 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:18.245342 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:18.316097 13726 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
W20260709 12:26:18.211161 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 449496us
I20260709 12:26:18.245963 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:18.236176 13767 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:18.273798 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:18.308524 13725 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:18.276676 14423 raft_consensus.cc:1240] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31. Ops: [30.22-30.22]
W20260709 12:26:18.309778 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78001.7us
I20260709 12:26:18.310464 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.311044 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:18.323951 13794 raft_consensus.cc:2376] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29. Re-sending same reply.
W20260709 12:26:18.324087 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51705.6us
I20260709 12:26:18.320779 13721 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:18.311975 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.313130 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.324934 13794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3934ae8ea567487da7a989adece218c2"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:18.325994 13794 raft_consensus.cc:606] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.229405 25258 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:26:18.313839 25174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.004s	sys 0.007s
W20260709 12:26:18.335062 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 98950.6us
I20260709 12:26:18.335464 25073 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:18.333907 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:18.337852 24833 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.328896 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:18.338572 24833 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.334772 25258 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.339412 13799 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:18.339546 25258 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:18.338819 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.327512 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:18.327314 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 292886us
I20260709 12:26:18.341020 24833 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.341632 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.329577 25226 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.343276 13652 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:18.344723 25226 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.329166 25064 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.021s
I20260709 12:26:18.335157 24050 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:18.346442 24050 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.346801 24050 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:18.273964 25259 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.350797 25259 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.354566 13723 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:18.274117 25251 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.089s
I20260709 12:26:18.337218 25073 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:18.356689 25073 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:18.359827 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63183us
I20260709 12:26:18.361147 24699 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.361336 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:18.361663 24699 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:18.363993 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48740.6us
I20260709 12:26:18.365175 24699 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.367424 25259 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.368489 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:18.248759 25101 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:18.369824 13801 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:26:18.369182 13725 raft_consensus.cc:2376] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29. Re-sending same reply.
I20260709 12:26:18.219645 25245 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.375797 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:18.376439 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.381685 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:18.384253 13794 raft_consensus.cc:1240] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:26:18.385360 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.389501 13795 raft_consensus.cc:2410] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:26:18.392247 13721 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:26:18.392532 13793 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:18.328492 25271 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.377928 13799 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:18.307835 25164 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 12:26:18.371984 25262 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:26:18.393813 25276 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.313519 25272 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.078s
I20260709 12:26:18.430321 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
W20260709 12:26:18.401836 25240 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.307s	user 0.000s	sys 0.002s
I20260709 12:26:18.232029 25166 consensus_queue.cc:260] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:18.402099 25267 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 12:26:18.489924 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 156169us
I20260709 12:26:18.492820 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:18.328244 24969 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.427335 24833 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.402784 25033 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.536s	user 0.000s	sys 0.012s
I20260709 12:26:18.490336 25271 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.493790 24050 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.274983 25158 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.511318 13724 raft_consensus.cc:3055] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:18.511780 13724 raft_consensus.cc:740] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:18.512573 13724 consensus_queue.cc:260] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.513557 13724 raft_consensus.cc:3060] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:18.395048 25243 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.390882 25169 consensus_queue.cc:237] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.502741 25166 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:18.539366 13724 raft_consensus.cc:1275] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:26:18.553653 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61245.5us
I20260709 12:26:18.500718 25252 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.518525 25225 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.507059 24699 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.501207 25268 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:18.526753 25174 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.500372 24833 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.519173 25155 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.560585 24050 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.567408 14417 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:18.567598 14419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2088) took 1385 ms. Trace:
I20260709 12:26:18.428490 25070 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.561597 25239 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.601650 25265 consensus_queue.cc:1175] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4dafe08a4f404502822a0732fe570de4" responder_term: 28 status { last_received { term: 27 index: 17 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:18.603452 24736 consensus_meta.cc:287] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.007s
W20260709 12:26:18.604842 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.605597 25287 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:26:18.605960 24833 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.601706 25277 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:18.568028 24699 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.602205 25239 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.611985 24699 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.488126 25276 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.627475 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.628472 13798 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:18.635355 25291 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:18.637262 25245 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:18.641113 25212 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.910s	user 0.005s	sys 0.008s
I20260709 12:26:18.527241 25271 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.647408 14416 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:18.592516 25174 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.523584 25033 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.641633 25212 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.565479 25273 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.604447 25289 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:26:18.569403 14419 rpcz_store.cc:276] 0709 12:26:17.180061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.180303 (+   242us) service_pool.cc:224] Handling call
0709 12:26:17.180631 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.180980 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.181011 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.181626 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.479213 (+297587us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:17.479273 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.479371 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.565007 (+1085636us) raft_consensus.cc:1685] finished
0709 12:26:18.565032 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.565177 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7748,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":21227,"thread_start_us":297053,"threads_started":1,"wal-append.queue_time_us":558284}
W20260709 12:26:18.644004 25286 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:26:18.656157 25277 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 12:26:18.615317 25023 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.670056 13798 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:18.666011 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:18.566385 25225 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.674178 25297 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:26:18.684680 25295 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:18.685433 25276 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.685335 25295 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.689347 25298 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:18.690393 25295 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:18.684746 25254 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
I20260709 12:26:18.691190 25277 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:18.691736 25176 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:18.692157 25176 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:18.693902 25225 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.668421 24610 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.698378 25258 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:18.699091 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 351276us
I20260709 12:26:18.701454 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:18.704406 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 375619us
I20260709 12:26:18.657914 25268 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.017s
I20260709 12:26:18.673237 24915 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.705967 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.706847 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:18.697505 25102 consensus_meta.cc:287] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.005s
I20260709 12:26:18.712136 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:18.714330 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.715502 13651 raft_consensus.cc:1275] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:18.716702 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.663059 25273 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.718214 13643 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:26:18.696928 25209 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.723212 13800 raft_consensus.cc:2410] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:26:18.723418 13799 raft_consensus.cc:606] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.707736 25225 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:18.759696 13649 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:18.707824 25270 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:18.763584 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:18.768615 13651 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:26:18.718366 25277 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.714689 25241 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:18.767046 25225 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.771065 25212 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.771257 13651 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:26:18.772215 25225 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.782462 25169 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:18.783205 25169 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.785461 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.786871 13801 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:18.787137 25169 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.788443 13647 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:18.788836 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.789774 25284 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:18.790352 25284 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.792389 25284 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.793711 25174 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.795023 25033 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:18.796065 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 90423.2us
W20260709 12:26:18.799633 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93751.2us
I20260709 12:26:18.800590 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:18.799379 25282 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:18.804203 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.804950 13801 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:18.806036 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.806905 13801 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:18.823036 13793 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:18.759789 25239 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.831509 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.831761 13649 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:26:18.839354 13794 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 36 index: 20. (term mismatch)
I20260709 12:26:18.840009 13794 pending_rounds.cc:85] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 19
I20260709 12:26:18.840319 13794 pending_rounds.cc:107] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 35.20
I20260709 12:26:18.840945 13794 raft_consensus.cc:2889] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:18.840154 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.830049 25258 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.853142 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:18.854103 13794 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:18.855135 13648 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:26:18.829797 25270 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:18.860160 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.876379 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.828349 25277 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.765832 25176 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.896936 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:18.849798 25167 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:18.884898 25271 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:18.899768 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99299.4us
I20260709 12:26:18.901932 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.906355 25166 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.906961 25166 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.909260 25166 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.905675 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.912168 25282 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:18.912590 25279 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:18.914238 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.915256 25284 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:18.915570 25273 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.916664 25277 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.916599 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:18.917596 25070 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:18.856240 25073 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.918956 25270 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.919878 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:18.920841 25070 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:18.921224 25070 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.921409 25073 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:18.921605 25070 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:18.923955 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.924650 13799 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:18.925493 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:18.929772 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:18.935135 13799 raft_consensus.cc:2410] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:26:18.935580 25239 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:18.935204 14418 raft_consensus.cc:1217] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Deduplicated request from leader. Original: 25.19->[28.20-28.20]   Dedup: 28.20->[]
I20260709 12:26:18.937304 14418 raft_consensus.cc:1240] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:26:18.935524 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:18.940254 14420 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:18.938763 25158 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:18.936512 25258 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.942193 25158 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:18.942359 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:18.942637 25258 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.943291 14418 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:18.944948 25258 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:18.943264 25158 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:18.944775 13795 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:18.946450 25277 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.010s
I20260709 12:26:18.953795 25070 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:18.936429 25239 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:18.947726 13793 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: []
W20260709 12:26:18.974509 13650 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54756 (request call id 3436) took 1514 ms (client timeout 1882 ms). Trace:
W20260709 12:26:18.975485 13650 rpcz_store.cc:269] 0709 12:26:17.459991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.460108 (+   117us) service_pool.cc:224] Handling call
0709 12:26:18.974053 (+1513945us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54002e4fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:18.974327 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673143680}
I20260709 12:26:18.976290 25239 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.972285 25070 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:18.981448 13794 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:18.990765 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:18.949720 13799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f22eab7d8372432799f894a161194acd"
dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
mode: GRACEFUL
new_leader_uuid: "b4f6bfcda63148e4965ca02c639336f1"
 from {username='slave'} at 127.0.0.1:42338
I20260709 12:26:18.993775 13799 raft_consensus.cc:606] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Received request to transfer leadership to TS b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:18.994351 13799 raft_consensus.cc:614] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:18.992445 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:19.000362 14420 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:18.941783 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:19.001852 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82779.9us
I20260709 12:26:19.004773 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:19.008813 13795 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:19.008493 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:19.010028 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68367.5us
I20260709 12:26:18.977309 25167 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 12:26:18.993800 25258 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:19.008653 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:26:19.007232 25265 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.016s
I20260709 12:26:18.992206 25282 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 34.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.031s
I20260709 12:26:19.006995 25277 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:19.024435 13801 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:19.024150 25135 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.029629 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:19.016364 25260 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:19.034806 25167 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:19.036717 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2091) took 1356 ms. Trace:
I20260709 12:26:19.025998 25253 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:19.037775 14421 rpcz_store.cc:276] 0709 12:26:17.680937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.681143 (+   206us) service_pool.cc:224] Handling call
0709 12:26:17.681467 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.863302 (+181835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.863349 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.863415 (+    66us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:17.863462 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.036450 (+1172988us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400498c54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:19.036500 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.036645 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417436544}
I20260709 12:26:19.039189 25282 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:19.028705 25258 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.028457 25277 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.046656 25070 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:19.047360 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2369) took 1170 ms. Trace:
I20260709 12:26:19.047853 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.051334 25167 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:19.055296 24755 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:19.061674 13724 rpcz_store.cc:276] 0709 12:26:17.876743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.876858 (+   115us) service_pool.cc:224] Handling call
0709 12:26:17.885467 (+  8609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.560738 (+675271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.047048 (+486310us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400156dd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:19.047262 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438959360}
I20260709 12:26:19.059644 25282 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:19.067348 24755 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:19.072110 24755 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.061121 25277 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.073330 24755 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.060808 25253 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:19.076040 25253 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.080204 25258 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.072043 25282 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:19.082545 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.063162 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:19.086923 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.090456 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.087939 13650 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
I20260709 12:26:19.093168 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.064954 25241 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:19.065124 25091 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:19.094688 25279 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:19.091868 25270 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:19.094417 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.096856 25250 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.099946 25241 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:19.137441 25158 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.140498 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.069175 25167 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.141258 13648 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:19.140357 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.142783 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.143070 25260 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:26:19.144469 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51379.8us
I20260709 12:26:19.145020 13644 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:19.146482 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.158334 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:19.160259 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.162547 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.135758 25282 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:19.134881 25258 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:19.169888 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:19.150794 25250 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:19.150568 25158 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.167639 25260 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:19.168812 25070 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:19.191061 24937 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.006s
I20260709 12:26:19.199119 25243 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.207070 13650 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:19.211840 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.212877 13644 raft_consensus.cc:2393] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:19.213575 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:19.213838 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.214496 13652 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:19.214740 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.215097 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.215466 13651 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:19.215301 25135 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:19.216204 13647 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:19.216316 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.215422 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.217098 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:19.217864 25135 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:19.218241 25135 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:19.218616 25135 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:19.222105 25284 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.222726 25284 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.225579 25284 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.227547 25135 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.231041 25273 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:19.189407 25245 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.044s
I20260709 12:26:19.233520 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:19.234193 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.234413 25273 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:19.234918 25273 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.235306 25273 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:19.186478 25091 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:19.236552 25091 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:19.237514 25091 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.241129 13648 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:19.214744 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.243053 13649 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:19.244773 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.245461 13649 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:19.249769 25273 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.254843 13649 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:19.262053 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.270610 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.272524 25273 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.274376 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.274734 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:19.275887 13647 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 39: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:19.276197 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:19.277019 25073 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:19.277088 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.277515 25073 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:19.277948 25073 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.278470 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.180316 25250 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.280339 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.280577 25250 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.281203 25250 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:19.281566 25250 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.281929 25250 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:19.282152 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.283114 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.232537 25277 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.284984 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:19.291471 13650 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:19.291791 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.309979 13645 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:19.293450 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.294379 25273 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:19.294883 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.295867 25073 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.316390 25280 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.317097 25273 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:19.317941 25073 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.317899 25273 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.318846 25027 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:19.319013 25273 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:19.319295 25027 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:19.320253 25027 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.321291 25027 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.317018 25280 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.322870 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60492 (request call id 5034) took 1682 ms. Trace:
I20260709 12:26:19.323274 13796 rpcz_store.cc:276] 0709 12:26:17.643282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.644312 (+  1030us) service_pool.cc:224] Handling call
0709 12:26:19.322629 (+1678317us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002887d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:19.322803 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440320896}
I20260709 12:26:19.323887 25280 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.314208 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:19.324743 14417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 2097) took 1125 ms. Trace:
I20260709 12:26:19.325140 14417 rpcz_store.cc:276] 0709 12:26:18.199683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.199862 (+   179us) service_pool.cc:224] Handling call
0709 12:26:19.324520 (+1124658us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54006a72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:19.324677 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761813248}
I20260709 12:26:19.326485 25239 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:19.326923 25239 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:19.327308 25239 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.272653 25135 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.330571 25277 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:19.328619 25070 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:19.334695 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.178690 25258 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.338472 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:19.339879 25258 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.340534 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48712 (request call id 1628) took 1927 ms. Trace:
I20260709 12:26:19.340976 14423 rpcz_store.cc:276] 0709 12:26:17.413312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.413464 (+   152us) service_pool.cc:224] Handling call
0709 12:26:17.430954 (+ 17490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.321040 (+890086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.340250 (+1019210us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54006f72d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:19.340463 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1944296320}
I20260709 12:26:19.330873 13792 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:26:19.351096 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.351928 13726 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:19.353571 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.354840 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.355486 13724 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:19.356359 13720 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:19.357724 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.359943 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.363205 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.364024 13646 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:19.365269 13718 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:26:19.366295 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.368863 24050 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:19.369416 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:19.372377 13645 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:19.373592 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:19.372663 24050 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: INVALID_TERM, Last received: 38.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:19.377398 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.378191 13647 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:19.375859 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.381532 13722 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:19.381516 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:19.383978 14417 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.340538 25258 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:19.393441 13723 raft_consensus.cc:1275] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:19.393908 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.396346 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.393550 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.397799 13726 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:19.405860 13724 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:19.367251 25277 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.370002 25070 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:19.410785 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.413842 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.381597 25263 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:19.381280 25073 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:19.429607 13649 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:19.451668 13646 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 26.
I20260709 12:26:19.440438 24969 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:19.458063 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.435235 25277 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.441924 25070 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.400761 25258 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:19.458248 13646 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:19.462181 13720 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:19.462584 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.497434 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.462641 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.498214 13725 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:19.466591 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.480392 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:19.482002 13643 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 36 index: 20. (index mismatch)
I20260709 12:26:19.488361 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.502840 13726 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 39: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:19.492847 13652 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:19.465102 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:19.505954 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:19.499804 13723 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:19.505896 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.458657 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:19.474125 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.508730 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.509531 13726 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:19.453848 25263 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:19.512998 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.520056 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:19.521454 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.523047 13647 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:19.455945 25073 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.524241 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.525528 13722 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:26:19.483242 25234 log.cc:927] Time spent T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:19.474009 25277 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:19.529994 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.527844 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.528772 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.528021 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.479401 25281 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:19.523872 25265 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:19.524042 25226 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.023s
W20260709 12:26:19.542078 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.543299 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.544559 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.512754 25280 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.474447 25275 consensus_queue.cc:1048] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.055s
I20260709 12:26:19.531930 13726 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:19.499804 25258 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:19.523633 25270 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:19.525830 25263 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:19.544760 25277 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:19.545408 13727 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.528439 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.547847 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.549225 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:19.528180 14417 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:19.552832 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:19.559880 25273 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.528826 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.568161 13725 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:19.529546 25073 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.473515 25239 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:19.596728 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 311789us
W20260709 12:26:19.605234 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 52548.8us
I20260709 12:26:19.617949 13725 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: [28.22-28.22]
I20260709 12:26:19.618080 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:19.614080 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 233322us
I20260709 12:26:19.603178 13724 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:19.609863 13723 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:19.459208 24969 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.613765 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 39: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:19.571692 25282 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:19.531724 25281 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:19.607389 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:19.619283 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.629156 13722 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:19.596299 25257 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:19.641690 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.649996 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.647701 13726 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:19.652850 13724 raft_consensus.cc:1275] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:19.648340 25265 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.683378 13721 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:19.683280 25281 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.684444 25257 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.685629 25065 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:19.613435 25221 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:19.686133 25241 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.001s
I20260709 12:26:19.714975 14417 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: [28.22-28.22]
W20260709 12:26:19.722726 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 202760us
I20260709 12:26:19.723225 13723 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:19.739507 24050 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:19.739761 13727 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:19.758651 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:19.700476 25221 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.744539 25281 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:19.772899 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:19.805650 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:19.770934 25226 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:26:19.771728 25295 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 12:26:19.775074 14421 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
W20260709 12:26:19.775799 25278 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.281s	user 0.000s	sys 0.007s
I20260709 12:26:19.778447 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3448) took 1468 ms. Trace:
I20260709 12:26:19.851950 25280 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:19.697124 25265 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.782291 25176 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:19.737443 25241 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:19.747196 25027 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:26:19.701198 25065 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:19.798583 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 179470us
I20260709 12:26:19.846442 24969 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:19.853212 14423 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 22. (term mismatch)
I20260709 12:26:19.884150 14423 pending_rounds.cc:85] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting all ops after (but not including) 21
I20260709 12:26:19.884480 14423 pending_rounds.cc:107] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Aborting uncommitted NO_OP operation due to leader change: 33.22
I20260709 12:26:19.884783 14423 raft_consensus.cc:2889] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.762316 25277 consensus_queue.cc:1059] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:26:19.803931 25221 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:19.803195 24050 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:19.888007 24050 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:19.780169 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:19.863996 13792 rpcz_store.cc:276] 0709 12:26:18.309720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.309853 (+   133us) service_pool.cc:224] Handling call
0709 12:26:19.330539 (+1020686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.343546 (+ 13007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.778153 (+434607us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002887d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:19.778356 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2346436352}
W20260709 12:26:19.771275 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.796180 25250 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.880769 25241 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:19.883435 13767 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:19.903879 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 281885us
W20260709 12:26:19.905227 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 255310us
I20260709 12:26:19.901903 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:19.868752 25280 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:19.876444 13726 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:19.906988 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:19.906759 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:19.910634 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:19.903630 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:19.882671 25273 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.878364 25329 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:19.926750 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:19.928208 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122663us
I20260709 12:26:19.932072 14417 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:19.938542 14422 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 35: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:19.901805 25239 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:19.950696 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47170.7us
I20260709 12:26:19.951797 13797 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:26:19.904981 25332 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:19.912525 25324 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:19.954993 13792 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:19.895622 25271 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 38.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.052s
I20260709 12:26:19.917081 25333 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.918552 25280 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:20.006445 25339 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:20.007102 25339 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.009344 25339 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:20.010427 25328 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 12:26:20.014868 25331 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:26:20.020033 25250 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:20.020814 25239 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:19.967705 25332 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.024933 25332 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:20.027684 25269 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:20.029266 25269 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:19.936178 25241 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:19.974231 24050 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:19.986976 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.987869 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45772.3us
W20260709 12:26:19.988904 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 78843.5us
I20260709 12:26:19.989576 25277 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:20.001091 25065 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:20.002329 25321 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:26:20.035385 13796 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:26:20.002957 25323 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:26:20.034695 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:20.061264 24050 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:19.971390 24969 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:26:20.073587 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:20.065404 25250 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:26:20.102109 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 161484us
I20260709 12:26:20.064247 25324 raft_consensus.cc:697] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.029817 25269 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.145761 25269 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.148751 25265 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.149485 25265 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:20.151695 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:20.151889 25344 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:20.152704 14390 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.153748 25344 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:20.034862 25333 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.154448 25329 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:20.155022 25329 raft_consensus.cc:697] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:20.155905 25329 consensus_queue.cc:237] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.156575 25333 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:20.165453 25258 consensus_meta.cc:287] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 12:26:20.153887 14390 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:20.166356 25349 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.027s
I20260709 12:26:20.172674 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.053985 25073 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.142984 25324 consensus_queue.cc:237] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:19.973969 25257 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.067778 25273 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:20.026108 25263 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.180096 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.149544 25221 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.078555 25347 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:20.062024 25277 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:20.226641 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46598.8us
I20260709 12:26:20.173390 25265 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.166354 25344 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:20.184845 25241 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:20.194033 25269 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.229759 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 93527us
I20260709 12:26:20.248492 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.237756 14423 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:20.166170 25258 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.258512 13766 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.232257 25295 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:20.261404 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:20.219671 25273 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.269584 25243 consensus_queue.cc:1048] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:20.275489 25263 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:20.275825 25277 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.229406 25341 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
W20260709 12:26:20.279089 25340 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:26:20.266489 25345 raft_consensus.cc:2764] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:26:20.246816 25176 consensus_queue.cc:1048] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:26:20.300400 24050 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.301061 24050 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.239416 25347 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.061s
I20260709 12:26:20.304670 25239 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:20.257666 25250 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:20.314680 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87092.3us
I20260709 12:26:20.315766 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:20.246304 25265 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:20.319173 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57893.8us
I20260709 12:26:20.320327 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:20.325079 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.325758 13726 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:20.329288 25277 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:20.329771 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:20.330605 25277 raft_consensus.cc:2764] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:26:20.331382 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:20.332012 25135 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:20.332409 25135 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.333406 25353 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:20.333323 25135 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 38.20, Last appended by leader: 20, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.303192 25295 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:20.342248 25295 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.345844 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1812) took 1092 ms. Trace:
I20260709 12:26:20.352578 13644 rpcz_store.cc:276] 0709 12:26:19.253168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.256175 (+  3007us) service_pool.cc:224] Handling call
0709 12:26:19.256528 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.256758 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 27
0709 12:26:19.257039 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.563677 (+306638us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.345247 (+781570us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b180015f640. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287 00007f5393c0f208
0709 12:26:20.345430 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.345484 (+    54us) raft_consensus.cc:1645] Marking committed up to 27
0709 12:26:20.345542 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.345628 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.345774 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60507,"prepare.queue_time_us":9948,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":104,"raft.queue_time_us":32956,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":14338,"spinlock_wait_cycles":3687168,"thread_start_us":292474,"threads_started":1,"wal-append.queue_time_us":390944}
I20260709 12:26:20.252372 25269 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.299697 25273 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:20.367425 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.362108 25269 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.356303 25135 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:20.357560 14422 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:20.357491 25241 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:20.375291 25241 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:20.375761 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 117042us
W20260709 12:26:20.395402 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48097.1us
I20260709 12:26:20.397920 13726 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:20.304700 25335 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:20.391095 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:20.323637 25350 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:20.413180 13766 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.347532 25333 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:20.403798 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:20.299949 25280 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.392509 25332 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:20.427232 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.427685 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.349831 25176 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:20.432838 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.429788 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.392285 25349 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:20.437613 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.435235 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:20.440164 14420 raft_consensus.cc:2393] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:20.437734 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.400663 25269 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.438649 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:20.442535 13652 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:20.443454 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.400784 25329 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:20.448863 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5586) took 1088 ms. Trace:
I20260709 12:26:20.449311 13650 rpcz_store.cc:276] 0709 12:26:19.360717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.361916 (+  1199us) service_pool.cc:224] Handling call
0709 12:26:19.376022 (+ 14106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.376253 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:26:19.741110 (+364857us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.448460 (+707350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.448528 (+    68us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:20.448589 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.448676 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.448799 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":742562,"prepare.queue_time_us":90606,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"thread_start_us":256326,"threads_started":1}
I20260709 12:26:20.450846 13718 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:20.451869 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:20.452486 13644 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:20.428313 25348 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.417997 25352 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:20.414888 25339 consensus_queue.cc:1048] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.037s
I20260709 12:26:20.444764 25253 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.444599 25239 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:26:20.484534 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.485666 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.465986 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:20.487881 13649 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:20.468312 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:20.318660 25344 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.480494 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.491205 14418 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 39.
I20260709 12:26:20.490964 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.492584 13645 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:20.482966 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.493566 13726 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:20.481389 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.446553 25269 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.480943 13647 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
W20260709 12:26:20.495558 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43756.1us
I20260709 12:26:20.499915 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:20.504925 13644 raft_consensus.cc:2468] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
W20260709 12:26:20.506729 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 94718.6us
I20260709 12:26:20.509590 13652 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:20.511499 25350 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.512106 13726 raft_consensus.cc:2468] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:20.457329 25176 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:20.480476 25351 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:20.519133 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:20.553510 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.562s (SENT)
W20260709 12:26:20.554596 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67062.3us
I20260709 12:26:20.522177 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:20.478876 25073 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.419041 25335 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.455233 25329 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.450143 25250 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.581463 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:20.559068 25354 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:20.447141 25349 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.521186 13649 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:20.570196 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:20.554879 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62427.8us
I20260709 12:26:20.586925 25355 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.600071 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5585) took 1239 ms. Trace:
I20260709 12:26:20.512908 25239 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:20.605969 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52539.1us
I20260709 12:26:20.467525 25348 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.575786 25353 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:20.597045 13647 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:26:20.619982 25349 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.620957 25239 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.560042 25285 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:20.605643 25355 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:20.631268 14391 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.67:40415 timed out after 1.701s (SENT) [suppressed 16 similar messages]
I20260709 12:26:20.600497 13648 rpcz_store.cc:276] 0709 12:26:19.360416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.361064 (+   648us) service_pool.cc:224] Handling call
0709 12:26:19.361496 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:19.370586 (+  9090us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:19.419388 (+ 48802us) log.cc:844] Serialized 14 byte log entry
0709 12:26:20.412184 (+992796us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.412745 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.412927 (+   182us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:20.412965 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.413043 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.599838 (+186795us) raft_consensus.cc:1685] finished
0709 12:26:20.599863 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.600004 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":796444,"prepare.queue_time_us":95947,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":819631,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":16502,"thread_start_us":156338,"threads_started":1,"wal-append.queue_time_us":332122}
I20260709 12:26:20.616205 24937 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:20.612131 25279 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.004s
I20260709 12:26:20.617061 25350 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.592037 25346 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.592317 25176 raft_consensus.cc:697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.477743 25352 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:20.635849 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54514.8us
I20260709 12:26:20.491374 24755 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.596652 25258 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.555138 25242 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.061s
I20260709 12:26:20.615360 25351 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.673187 13645 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:26:20.677079 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 249454us
I20260709 12:26:20.577700 25363 consensus_queue.cc:1059] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.084s
I20260709 12:26:20.639561 25282 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:20.640911 25353 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:20.693729 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:20.688810 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.694576 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.502961 25027 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.695564 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 107761us
I20260709 12:26:20.704020 13767 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:20.493391 25253 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.700671 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.708945 13767 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:20.675545 25265 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.668308 25352 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:20.648622 25346 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.743185 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.711190 14390 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:20.674433 25285 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.714723 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:20.695279 25351 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:20.723696 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 98771.5us
W20260709 12:26:20.718773 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 87538.6us
I20260709 12:26:20.715896 25366 raft_consensus.cc:3055] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:20.720913 25295 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:20.755054 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:20.755007 25295 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.756382 25279 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:20.712755 25367 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 12:26:20.753629 13767 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:20.743643 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:20.746389 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.760921 13796 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:20.742776 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:20.762105 13801 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:20.744051 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.763386 13793 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:20.765030 13793 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:20.743260 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.742776 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.750948 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:20.744482 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:20.754356 25366 raft_consensus.cc:740] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:20.710449 25280 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.771795 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:20.739591 25364 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:26:20.722647 25359 raft_consensus.cc:2804] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:20.717619 25242 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:20.760944 13794 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 34: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:20.760450 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:20.782006 13726 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:26:20.772265 13793 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:26:20.685177 24755 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:20.739023 25356 log.cc:927] Time spent T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 12:26:20.740689 25281 consensus_meta.cc:287] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.922s	user 0.011s	sys 0.002s
I20260709 12:26:20.774603 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:20.738701 25269 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.726591 25369 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:20.782949 13793 pending_rounds.cc:85] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 17
I20260709 12:26:20.790619 13793 pending_rounds.cc:107] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:26:20.720382 25256 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.061s
I20260709 12:26:20.790962 13793 raft_consensus.cc:2889] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:20.760571 24937 raft_consensus.cc:2804] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:26:20.783394 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 97694.6us
I20260709 12:26:20.739354 25329 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.787947 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:20.734130 25348 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.716219 25027 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.737489 25176 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.783946 14417 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:26:20.787858 25358 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 12:26:20.760035 25339 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:20.806602 13648 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:20.807384 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:20.758126 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 250115us
I20260709 12:26:20.814761 13649 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:20.737053 25273 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.781358 25242 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:20.811985 25369 raft_consensus.cc:697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.818847 25379 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:20.824679 13801 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:20.826329 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.828487 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.775867 25270 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:26:20.832129 14416 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:20.791757 25366 consensus_queue.cc:260] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.837998 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.838908 13723 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:20.803470 25373 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:20.835536 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.810441 25359 raft_consensus.cc:697] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 37 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.812338 24937 raft_consensus.cc:697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:20.844090 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:20.737941 25353 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.845942 14391 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:20.847759 13722 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 34: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:26:20.847939 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 93403.2us
I20260709 12:26:20.848552 13648 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:26:20.808610 25256 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:26:20.853274 13644 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:26:20.852954 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:20.853820 13650 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 36. Current term is 37. Ops: [36.20-36.20]
W20260709 12:26:20.854137 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.857754 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:20.875787 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:20.847208 25250 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.879375 13725 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:20.879495 13649 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:26:20.886633 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:20.887408 13727 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:20.887293 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:20.887780 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.890749 25273 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:20.891053 25027 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.891992 25250 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.893436 24937 consensus_queue.cc:237] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.894068 25335 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.894692 25369 consensus_queue.cc:237] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.897294 25335 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:20.834424 25285 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.898712 25346 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:20.898537 25359 consensus_queue.cc:237] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 20, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.847334 25379 raft_consensus.cc:697] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:20.899997 25135 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.000s
I20260709 12:26:20.900333 25379 consensus_queue.cc:237] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:20.901635 25239 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:20.822790 25376 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:26:20.903759 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.904611 13644 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 39.
I20260709 12:26:20.902429 13800 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 39.
I20260709 12:26:20.854236 25295 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.855392 25373 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:20.920004 13649 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:26:20.873697 25073 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:20.921392 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.880283 25342 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:26:20.948846 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 95980.5us
I20260709 12:26:20.951149 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.902127 25239 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:20.962337 25239 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:20.902575 25279 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:20.907509 25258 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:20.958364 13648 raft_consensus.cc:1240] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:26:20.958067 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:20.919679 25380 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:20.964764 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.968256 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:20.922736 25329 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:20.968921 14419 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:20.927356 25070 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.982995 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2389) took 1446 ms. Trace:
I20260709 12:26:20.984531 13721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34464 (request call id 5006) took 1594 ms. Trace:
W20260709 12:26:20.958909 25360 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:26:20.992781 14423 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2099) took 1087 ms. Trace:
I20260709 12:26:21.026335 14423 rpcz_store.cc:276] 0709 12:26:19.905450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.906251 (+   801us) service_pool.cc:224] Handling call
0709 12:26:19.906600 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.249938 (+343338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.978123 (+728185us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400600254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:20.992693 (+ 14570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761091328}
I20260709 12:26:20.945603 25342 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:20.998554 13720 rpcz_store.cc:276] 0709 12:26:19.536378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.626508 (+ 90130us) service_pool.cc:224] Handling call
0709 12:26:19.626790 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.743767 (+116977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:19.743815 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:19.744045 (+   230us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:19.744093 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.982661 (+1238568us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54002f6054. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:20.982730 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.982907 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268433152}
I20260709 12:26:20.973757 25221 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.957576 25355 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.958956 25250 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:21.111166 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122362us
I20260709 12:26:20.965174 25253 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:20.954843 25073 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:21.103417 25328 log.cc:927] Time spent T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:26:20.902149 25242 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:20.986693 25279 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:21.201422 25388 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:26:20.994378 25380 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:21.361084 25381 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.401s	user 0.003s	sys 0.005s
I20260709 12:26:20.995426 24969 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:21.371208 25333 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.938s	user 0.015s	sys 0.005s
I20260709 12:26:21.182291 25373 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.355221 25342 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:21.365212 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:21.360532 25383 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.399s	user 0.004s	sys 0.010s
I20260709 12:26:21.366377 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:21.368252 25314 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.973s	user 0.006s	sys 0.000s
I20260709 12:26:21.344039 25295 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:21.377877 24755 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.343926 25070 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.342343 25027 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:20.959352 25377 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.001295 13721 rpcz_store.cc:276] 0709 12:26:19.390136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.393633 (+  3497us) service_pool.cc:224] Handling call
0709 12:26:20.984259 (+1590626us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54000f6754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:20.984430 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663964032}
I20260709 12:26:21.347067 25335 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.350984 25269 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:21.405230 13766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.013s
I20260709 12:26:21.407596 13796 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:26:21.443540 13766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.005s	sys 0.013s
I20260709 12:26:21.383200 25250 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:21.434816 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:21.445526 13793 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:21.405740 25333 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:21.449708 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:21.450565 13798 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:21.451944 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:21.460839 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:21.467615 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:21.468604 13794 raft_consensus.cc:2410] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 30 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:26:21.471917 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:21.475376 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:21.478089 25351 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.705s	user 0.015s	sys 0.006s
I20260709 12:26:21.478632 25351 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.480432 13721 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:26:21.481119 25351 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.482610 25404 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:21.482841 13797 raft_consensus.cc:2393] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:21.483318 25404 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:21.483866 25404 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:21.442703 25377 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.457787 25166 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:21.487540 25166 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.458701 25250 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.495343 25250 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:21.496654 25279 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.499300 13643 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:26:21.405284 24969 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.442757 25073 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.513010 25377 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.508903 25380 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.517756 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:21.519455 24969 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.521567 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:21.523685 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:21.525576 25386 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:26:21.527302 13794 raft_consensus.cc:1240] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 25. Current term is 26. Ops: []
W20260709 12:26:21.527650 25400 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:21.536832 25166 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.454397 25269 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.525828 13795 raft_consensus.cc:1240] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 37 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 36. Current term is 37. Ops: [36.20-36.20]
I20260709 12:26:21.436291 25295 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.438534 25070 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.541725 13766 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:21.547125 25269 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.447184 25335 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.549657 25295 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.440591 25027 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.558543 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5589) took 1122 ms. Trace:
I20260709 12:26:21.560216 13651 rpcz_store.cc:276] 0709 12:26:20.436478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.440667 (+  4189us) service_pool.cc:224] Handling call
0709 12:26:20.456454 (+ 15787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.730007 (+273553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.730047 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.730682 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.908525 (+177843us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:20.908568 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.553580 (+645012us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54002e45d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:21.553641 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.553689 (+    48us) raft_consensus.cc:1685] finished
0709 12:26:21.553701 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.558438 (+  4737us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18053,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":27016,"spinlock_wait_cycles":620767488,"thread_start_us":120783,"threads_started":1,"wal-append.queue_time_us":332566}
W20260709 12:26:21.563745 25319 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 1.677s	user 0.000s	sys 0.004s
I20260709 12:26:21.567338 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:21.550089 25239 consensus_meta.cc:287] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.003s
W20260709 12:26:21.552532 25352 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.805s	user 0.014s	sys 0.004s
I20260709 12:26:21.557602 25176 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:21.572425 13799 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:21.552879 25226 raft_consensus.cc:3055] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:21.572921 13647 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:21.574795 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:21.560981 25377 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.576864 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:21.545363 25250 consensus_queue.cc:1059] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.031s
I20260709 12:26:21.632512 14424 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:21.632740 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.622545 13764 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:21.577930 25226 raft_consensus.cc:740] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:21.571276 25404 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.571534 25352 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.572602 25239 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:21.667577 25387 log.cc:927] Time spent T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.318s	user 0.000s	sys 0.002s
I20260709 12:26:21.704125 25405 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:21.703908 25073 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:21.704895 25405 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.660245 25377 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.683182 24755 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.713276 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:21.714391 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:21.716535 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:21.719388 13651 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:21.718134 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:21.710604 25350 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.736958 25350 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.717504 25351 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:21.705818 25073 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:21.743392 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:21.745661 13723 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:21.714160 25027 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.712916 25377 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.716717 25335 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:21.734573 25353 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:21.750854 13645 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 37 index: 20. (index mismatch)
I20260709 12:26:21.752708 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:21.691870 25342 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.753538 13723 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:21.759598 25353 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.745249 25073 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:21.763362 25027 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.764837 25027 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.766666 25073 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:21.768823 25073 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:21.746831 25350 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:21.724071 25398 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:26:21.717068 24755 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.711325 25411 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 12:26:21.792153 25295 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.748323 25352 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:21.801966 13796 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:21.802466 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:21.831197 13723 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:21.854396 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.745386 25226 consensus_queue.cc:260] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.002060 13722 raft_consensus.cc:3055] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:22.002873 13722 raft_consensus.cc:740] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:21.888962 25070 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.886521 25295 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:21.984406 25027 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.929153 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1811) took 1235 ms. Trace:
I20260709 12:26:21.985292 25405 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:21.982946 13721 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:21.996337 14416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2106) took 1218 ms. Trace:
I20260709 12:26:22.004161 13722 consensus_queue.cc:260] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.008400 25284 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.011620 25256 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.013583 13722 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:22.014004 25256 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.013934 25284 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.016526 25256 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:21.886466 25243 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.016685 25242 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.017247 25243 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:22.008386 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81338.4us
I20260709 12:26:22.019820 25243 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:22.022714 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2397) took 1272 ms. Trace:
I20260709 12:26:22.023077 13724 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:22.023269 13726 rpcz_store.cc:276] 0709 12:26:20.750392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.750665 (+   273us) service_pool.cc:224] Handling call
0709 12:26:22.022502 (+1271837us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54000294d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:22.022650 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33319296}
I20260709 12:26:22.024045 13721 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:22.024638 13726 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:22.025007 13726 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:22.025203 13724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5018) took 1199 ms. Trace:
I20260709 12:26:22.026852 13718 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2393) took 1702 ms. Trace:
I20260709 12:26:22.013177 14416 rpcz_store.cc:276] 0709 12:26:20.777628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.777776 (+   148us) service_pool.cc:224] Handling call
0709 12:26:20.778053 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.844547 (+ 66494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.995953 (+1151406us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b540010f4d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:21.996239 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123822976}
I20260709 12:26:22.025553 13724 rpcz_store.cc:276] 0709 12:26:20.826117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.827071 (+   954us) service_pool.cc:224] Handling call
0709 12:26:22.022642 (+1195571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.023570 (+   928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.025000 (+  1430us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000294d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:22.025143 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2733161216}
I20260709 12:26:22.032966 13722 raft_consensus.cc:1275] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:22.011763 13723 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:22.037606 14424 raft_consensus.cc:2468] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:22.027228 13718 rpcz_store.cc:276] 0709 12:26:20.324378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.324536 (+   158us) service_pool.cc:224] Handling call
0709 12:26:20.324887 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.454420 (+129533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.026605 (+1572185us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003fd9d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:22.026786 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281207040}
I20260709 12:26:22.013175 13800 rpcz_store.cc:276] 0709 12:26:20.694018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.694245 (+   227us) service_pool.cc:224] Handling call
0709 12:26:21.928886 (+1234641us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000e51d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:21.929067 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460035456}
I20260709 12:26:22.027369 13726 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.051208 13726 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:22.053279 13793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.009s	sys 0.008s
W20260709 12:26:22.053761 13793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.009s	sys 0.008s
W20260709 12:26:22.074497 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.079474 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 381261us
I20260709 12:26:22.084375 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.016729 25284 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.012192 25226 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:22.043081 25070 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.116340 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 53205.9us
I20260709 12:26:22.116433 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.120348 13726 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:26:22.121013 13726 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:26:22.121289 13726 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:26:22.117389 13764 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:22.121594 13726 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.123342 13718 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:22.124209 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:22.127907 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.144722 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:22.145714 13718 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:22.146762 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.013336 25065 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:22.120043 25239 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.159371 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.158589 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.157986 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.118245 25404 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.119853 25284 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:22.164597 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:22.161790 25281 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:22.159410 25336 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:22.165457 25282 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.166544 25256 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.167791 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:22.170897 25342 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.172640 25295 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.173683 25353 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.175855 24755 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.177299 25398 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:22.243815 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.244184 25398 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.164608 25426 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.246618 25426 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.182320 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.174635 25284 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:22.195410 25369 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.026s	sys 0.057s
I20260709 12:26:22.195166 25027 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.197546 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 80241.3us
I20260709 12:26:22.121667 25352 consensus_queue.cc:1059] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.063s
I20260709 12:26:22.261256 13725 raft_consensus.cc:3055] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:22.261819 13725 raft_consensus.cc:740] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:22.163746 25065 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.262867 13725 consensus_queue.cc:260] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 36.20, Last appended by leader: 20, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.273329 13725 raft_consensus.cc:3060] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:22.216490 25336 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:22.256662 25369 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.868s	user 0.027s	sys 0.057s
I20260709 12:26:22.118813 25070 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.212956 25281 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.216187 25406 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.237425 25424 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:22.238853 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.242128 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.242944 25282 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.180712 25431 raft_consensus.cc:2804] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:26:22.235303 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67652.1us
I20260709 12:26:22.293033 13722 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:22.250610 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.250999 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.294406 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:22.277853 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73270.3us
I20260709 12:26:22.279426 13651 raft_consensus.cc:1275] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:26:22.288630 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.294715 13723 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
I20260709 12:26:22.275866 13650 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:26:22.296612 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:22.209292 13718 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:22.235270 25256 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.305029 25398 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:22.306020 25416 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:22.306811 25416 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.309598 25426 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:22.310637 25423 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:26:22.310019 25416 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.312256 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.304226 13645 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:22.312635 25414 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:22.313586 25425 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:26:22.316784 13801 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5061) took 1281 ms. Trace:
I20260709 12:26:22.318840 13719 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5024) took 1489 ms. Trace:
I20260709 12:26:22.317305 13801 rpcz_store.cc:276] 0709 12:26:21.035708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.342754 (+307046us) service_pool.cc:224] Handling call
0709 12:26:21.343183 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:21.578140 (+234957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.578178 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:21.578965 (+   787us) log.cc:844] Serialized 52 byte log entry
0709 12:26:22.000157 (+421192us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:22.000197 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.000271 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.316463 (+316192us) raft_consensus.cc:1685] finished
0709 12:26:22.316497 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.316698 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9989,"raft.queue_time_us":39707,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":12604,"thread_start_us":438406,"threads_started":2,"wal-append.queue_time_us":461712}
I20260709 12:26:22.320502 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:22.319525 13719 rpcz_store.cc:276] 0709 12:26:20.829703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.372712 (+543009us) service_pool.cc:224] Handling call
0709 12:26:21.373007 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.373438 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:21.619618 (+246180us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.852813 (+233195us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.917533 (+ 64720us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.917833 (+   300us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:21.917891 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.918020 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.318507 (+400487us) raft_consensus.cc:1685] finished
0709 12:26:22.318538 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.318749 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":328869,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":740,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":35253,"thread_start_us":437171,"threads_started":2,"wal-append.queue_time_us":520645}
I20260709 12:26:22.321664 13801 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:22.323871 25437 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:22.324473 25437 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:22.324892 25437 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:22.327378 13718 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
I20260709 12:26:22.334820 25336 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.336702 25411 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:22.337266 13644 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
I20260709 12:26:22.342185 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.342746 25284 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.297964 25406 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.366344 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:22.367532 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:22.344110 25065 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:22.346959 25431 raft_consensus.cc:697] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:22.328722 25281 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.300359 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:22.302337 24755 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:22.347056 13726 raft_consensus.cc:1275] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:26:22.399369 13726 pending_rounds.cc:85] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 14
I20260709 12:26:22.399712 13726 pending_rounds.cc:107] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:26:22.399976 13726 raft_consensus.cc:2889] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.347354 24050 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:22.327864 25070 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.352003 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.353502 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.401369 13723 raft_consensus.cc:2410] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:26:22.401381 13795 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:26:22.348790 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.358453 13725 raft_consensus.cc:1275] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 36 index: 20. Preceding OpId from leader: term: 37 index: 20. (term mismatch)
W20260709 12:26:22.359464 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 118926us
I20260709 12:26:22.402348 13725 pending_rounds.cc:85] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 19
I20260709 12:26:22.402418 13719 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:22.402796 13725 pending_rounds.cc:107] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 36.20
I20260709 12:26:22.402628 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.403249 13725 raft_consensus.cc:2889] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.403546 13795 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:22.403707 13690 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:22.404450 25226 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.404451 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.405156 25226 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.405586 25226 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.406153 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.406629 14424 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:26:22.407904 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.408401 25353 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:22.409312 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.359586 13801 raft_consensus.cc:2468] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:26:22.357043 25352 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.004s
W20260709 12:26:22.415261 25382 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.578s	user 0.004s	sys 0.000s
I20260709 12:26:22.367792 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54768 (request call id 5608) took 1021 ms. Trace:
I20260709 12:26:22.269506 25027 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.416879 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:22.416736 13647 rpcz_store.cc:276] 0709 12:26:21.346732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.402301 (+ 55569us) service_pool.cc:224] Handling call
0709 12:26:22.367595 (+965294us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54001482d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:22.367727 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286657792}
I20260709 12:26:22.418238 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:22.419226 25442 consensus_queue.cc:1048] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:22.420207 25406 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:22.421394 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.422236 13725 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:22.422480 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.423579 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:22.426626 25256 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:22.429168 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 128932us
W20260709 12:26:22.388731 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.336709 25282 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:22.421396 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65306.1us
I20260709 12:26:22.432824 13725 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:26:22.410048 13795 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 39.
I20260709 12:26:22.436640 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:22.400334 13646 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
W20260709 12:26:22.413132 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 160) took 48787 ms (client timeout 15000 ms). Trace:
I20260709 12:26:22.413340 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.358477 25411 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.385952 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.438772 14391 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:22.418432 25027 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.437332 14416 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:22.442975 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:22.441346 14391 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:22.444154 13799 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:26:22.449227 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:22.451051 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.452247 13801 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:22.453234 13647 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:26:22.453316 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.458848 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:22.437801 13542 rpcz_store.cc:269] 0709 12:25:33.628872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.078752 (+10449880us) service_pool.cc:224] Handling call
0709 12:25:44.716697 (+637945us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b54000021b8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999b2e43 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:22.413016 (+37696319us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:15.444074 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:15.450442 (+  6368us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:15.450501 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:26:15.450522 (+    21us) tablet.cc:662] Decoding operations
0709 12:26:15.452148 (+  1626us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:15.452277 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:15.452304 (+    27us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:26:15.453536 (+  1232us) tablet.cc:701] Row locks acquired
0709 12:26:15.453613 (+    77us) write_op.cc:260] PREPARE: finished
0709 12:26:15.453817 (+   204us) write_op.cc:270] Start()
0709 12:26:15.453969 (+   152us) write_op.cc:276] Timestamp: P: 1783599975453739 usec, L: 0
0709 12:26:15.454004 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:26:15.843674 (+389670us) log.cc:844] Serialized 4772 byte log entry
0709 12:26:16.495096 (+651422us) op_driver.cc:464] REPLICATION: finished
0709 12:26:16.706350 (+211254us) write_op.cc:301] APPLY: starting
0709 12:26:16.706480 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:16.805302 (+ 98822us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:16.805447 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:17.095958 (+290511us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:17.096501 (+   543us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:26:17.096596 (+    95us) write_op.cc:312] APPLY: finished
0709 12:26:17.097344 (+   748us) log.cc:844] Serialized 113 byte log entry
0709 12:26:17.102546 (+  5202us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:17.102803 (+   257us) write_op.cc:454] Released schema lock
0709 12:26:17.102913 (+   110us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:17.104542 (+  1629us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:17.104591 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9652148,"reported_tablets":28,"spinlock_wait_cycles":5717376,"child_traces":[["op",{"apply.queue_time_us":210905,"apply.run_cpu_time_us":23394,"apply.run_wall_time_us":398155,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54383,"num_ops":12,"prepare.queue_time_us":298212,"prepare.run_cpu_time_us":31858,"prepare.run_wall_time_us":654654,"raft.queue_time_us":244674,"raft.run_cpu_time_us":60122,"raft.run_wall_time_us":195711,"replication_time_us":1040890,"thread_start_us":523308,"threads_started":4,"wal-append.queue_time_us":202207,"wal-append.run_cpu_time_us":17375,"wal-append.run_wall_time_us":2017636}]]}
I20260709 12:26:22.446651 25342 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:22.468714 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.470646 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.472136 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.446810 25070 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:22.448145 25347 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.475698 13795 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:22.455348 25339 raft_consensus.cc:2804] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:22.479197 13798 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:22.484603 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.486321 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.486506 13798 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:22.462955 25365 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
W20260709 12:26:22.483963 13617 connection.cc:256] RPC call timeout handler was delayed by -1.0494751890000771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 12:26:22.491086 13798 raft_consensus.cc:1240] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:22.499446 13800 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:22.503575 13798 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:22.486390 25339 raft_consensus.cc:697] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:22.504926 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.505170 25339 consensus_queue.cc:237] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 36.26, Last appended by leader: 26, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.506764 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.507615 13647 raft_consensus.cc:2410] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 36 because replica has last-logged OpId of term: 34 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 17.
I20260709 12:26:22.508780 13647 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:22.509397 13800 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:22.484934 25226 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.512437 25226 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.515419 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:22.513880 25431 consensus_queue.cc:237] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.531114 25027 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:22.531780 25027 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.532688 13796 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60504 (request call id 3481) took 1049 ms. Trace:
W20260709 12:26:22.531265 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.533190 13796 rpcz_store.cc:276] 0709 12:26:21.483688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.522663 (+ 38975us) service_pool.cc:224] Handling call
0709 12:26:21.522978 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.817203 (+294225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.532312 (+715109us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001ca7d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:22.532578 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641088256}
W20260709 12:26:22.537725 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.544409 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.549115 13723 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:22.549767 13723 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:22.552703 13718 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:22.480950 25347 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.554379 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.556291 25070 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.559168 13725 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:22.559890 25282 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.561684 25070 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.562112 25282 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.551136 13723 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.577142 13723 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:22.553335 25295 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.579171 13798 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 19. (term mismatch)
I20260709 12:26:22.579650 13798 pending_rounds.cc:85] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 18
I20260709 12:26:22.579867 13798 pending_rounds.cc:107] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 29.19
I20260709 12:26:22.580116 13798 raft_consensus.cc:2889] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:22.582811 13800 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:26:22.583180 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.585613 13722 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:26:22.589107 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.594493 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.595502 13722 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 23.
I20260709 12:26:22.597606 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.600153 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.564401 25282 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.564997 25411 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:22.604521 25070 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.605598 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.606798 13723 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 18. (term mismatch)
I20260709 12:26:22.607803 13723 pending_rounds.cc:85] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 17
I20260709 12:26:22.607956 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.609048 13722 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:22.611956 13725 raft_consensus.cc:1240] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:22.611660 13723 pending_rounds.cc:107] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 29.18
I20260709 12:26:22.612628 13723 raft_consensus.cc:2889] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:22.615576 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.620263 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:22.620563 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.621176 13722 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:22.621413 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:22.622083 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.622128 13725 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:22.622481 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.623220 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.623282 13719 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:22.623772 13718 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:22.588606 25295 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.636878 25332 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.648983 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:22.655697 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.655573 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.656450 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.676342 25027 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.677699 13723 raft_consensus.cc:1240] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: []
I20260709 12:26:22.679244 25295 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.679751 25070 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:22.627810 25437 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.680372 25070 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.687594 14425 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:22.688575 13722 raft_consensus.cc:2410] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:26:22.689283 13725 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:22.710040 14422 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1790) took 2354 ms. Trace:
I20260709 12:26:22.711449 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:22.711592 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.690245 13795 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:22.719687 13725 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:22.689726 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:22.711092 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.718421 14422 rpcz_store.cc:276] 0709 12:26:20.349017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.351235 (+  2218us) service_pool.cc:224] Handling call
0709 12:26:20.351604 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.374469 (+ 22865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.702786 (+2328317us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b5400508a54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:22.703004 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13076224}
I20260709 12:26:22.711092 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.726141 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:22.702239 25282 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:22.725744 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.730806 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:22.723150 25332 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:22.731999 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:22.724464 25437 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:22.728940 25453 log.cc:927] Time spent T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.074s	user 0.003s	sys 0.009s
I20260709 12:26:22.729693 25027 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:22.726564 25295 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:26:22.735030 25332 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:22.728510 25226 consensus_queue.cc:1059] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:26:22.742501 25027 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.732901 25282 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.767961 25332 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:22.749854 25347 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.781967 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.783174 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.783773 13645 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:22.784514 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.785475 14418 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:22.774916 25027 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:22.789983 13651 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:22.791407 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.793045 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.793736 13644 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.789077 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:22.796334 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:22.787220 25347 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.800143 25243 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:22.807473 25070 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.807555 25282 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:22.805239 25347 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:22.811421 25419 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.813s	user 0.000s	sys 0.002s
I20260709 12:26:22.811695 25347 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.800621 25239 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:22.785470 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.816301 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:22.817018 13643 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:22.819694 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:22.824761 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:22.825753 25376 raft_consensus.cc:3055] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:22.826848 25376 raft_consensus.cc:740] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:22.828217 25376 consensus_queue.cc:260] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.829591 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:22.830658 13648 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:26:22.832134 25408 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.832588 25408 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.832937 25408 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.834362 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:22.837414 25385 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:22.855732 13801 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:22.825791 13647 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 32 because replica has last-logged OpId of term: 30 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:26:22.841317 25027 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.866546 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:22.869207 25431 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.869665 25431 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.870002 25431 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.877264 13644 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:22.878993 25351 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:22.882136 25335 raft_consensus.cc:3055] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:22.882701 25335 raft_consensus.cc:740] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:22.883873 25335 consensus_queue.cc:260] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.885334 25335 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:22.866050 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:22.866928 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:22.891613 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:22.894461 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 95522.7us
I20260709 12:26:22.895759 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:22.896260 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:22.896951 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:22.897311 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:22.902858 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:22.891178 25344 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.868237 25437 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.035s
I20260709 12:26:22.900311 25357 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:26:22.918807 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.901345 25424 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.913709 25462 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:22.920475 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:22.902092 25385 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:22.890198 25243 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.018s
I20260709 12:26:22.911581 25344 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:22.921160 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:22.930320 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.910156 25091 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.942476 25380 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:22.942358 25091 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.943138 25380 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.943578 25380 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.908938 25239 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:26:22.946599 25463 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:22.946935 13797 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:44068 (request call id 1810) took 2258 ms (client timeout 1701 ms). Trace:
W20260709 12:26:22.947468 13797 rpcz_store.cc:269] 0709 12:26:20.688321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.688452 (+   131us) service_pool.cc:224] Handling call
0709 12:26:22.946652 (+2258200us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54002882d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:22.946827 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794966400}
I20260709 12:26:22.948691 13792 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: [34.22-34.22]
I20260709 12:26:22.900502 25241 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.023s
I20260709 12:26:22.950693 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:22.925899 25167 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:22.923549 25275 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:22.924577 25424 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.926608 25462 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:22.957304 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:22.904040 25332 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.954555 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:22.935083 25344 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.966225 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:22.953114 25385 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:22.977224 25070 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:22.978905 25070 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.979287 25070 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:22.979511 13643 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:22.980897 13643 raft_consensus.cc:3055] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:22.981199 13643 raft_consensus.cc:740] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:22.962922 25424 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:22.988651 25385 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 37.27, Last appended by leader: 27, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.989468 25332 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.971071 25463 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:22.964787 25462 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:22.995042 25451 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:22.995497 25451 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:22.996276 25027 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 40 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:22.996488 25451 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.963162 25167 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:22.999754 25167 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.959873 25466 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:22.975975 25347 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:26:23.013490 25417 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.753s	user 0.001s	sys 0.000s
W20260709 12:26:23.015980 25470 log.cc:927] Time spent T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:23.017926 25431 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:22.998075 25451 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:23.025362 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:23.033254 14422 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:22.981931 13643 consensus_queue.cc:260] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.24, Last appended by leader: 24, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:22.977600 25241 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.035107 13643 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:23.038091 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:22.986280 25091 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:23.044806 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 144158us
I20260709 12:26:23.053499 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5023) took 2224 ms. Trace:
I20260709 12:26:23.054139 13720 rpcz_store.cc:276] 0709 12:26:20.829466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.364595 (+535129us) service_pool.cc:224] Handling call
0709 12:26:21.365039 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.365463 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.365492 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.365977 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.824015 (+458038us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:21.824069 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.824162 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.012851 (+1188689us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:23.049830 (+ 36979us) raft_consensus.cc:1685] finished
0709 12:26:23.049866 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.053406 (+  3540us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":194,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":5107072,"thread_start_us":457581,"threads_started":1,"wal-append.queue_time_us":789974}
I20260709 12:26:23.046650 25431 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:23.073254 25462 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.075773 25462 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:23.079190 13720 raft_consensus.cc:1240] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:23.080334 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:23.088832 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:23.043813 25241 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.095944 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:23.096345 13718 raft_consensus.cc:1240] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:23.097836 13727 raft_consensus.cc:1240] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:23.117735 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.093966 24937 consensus_queue.cc:1059] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 35.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.009s
I20260709 12:26:23.094045 25380 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.106905 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:23.108250 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5060) took 2345 ms. Trace:
I20260709 12:26:23.111400 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:23.111467 25437 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:23.113706 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5607) took 1767 ms. Trace:
I20260709 12:26:23.091883 25282 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:26:23.120314 14424 raft_consensus.cc:1240] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: [34.22-34.22]
I20260709 12:26:23.118060 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.126362 14419 raft_consensus.cc:1240] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:23.150601 13725 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:26:23.133255 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.123596 13643 raft_consensus.cc:1275] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 24. Preceding OpId from leader: term: 34 index: 24. (term mismatch)
I20260709 12:26:23.190482 13643 pending_rounds.cc:85] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 23
I20260709 12:26:23.190886 13643 pending_rounds.cc:107] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 33.24
W20260709 12:26:23.190951 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 121384us
I20260709 12:26:23.127786 13722 raft_consensus.cc:2410] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 29 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
W20260709 12:26:23.192149 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.147488 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 170471us
W20260709 12:26:23.150791 25419 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:26:23.199913 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.200977 14420 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:23.173635 13649 rpcz_store.cc:276] 0709 12:26:21.346418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.347311 (+   893us) service_pool.cc:224] Handling call
0709 12:26:21.365965 (+ 18654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.424307 (+ 58342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.424483 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.424568 (+    85us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:21.424649 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.113401 (+1688752us) spinlock_profiling.cc:243] Waited 58 ms on lock 0x7b5400148cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:23.113451 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.113627 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133468416}
I20260709 12:26:23.156792 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.191155 13643 raft_consensus.cc:2889] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.203976 13643 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:23.204381 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.206156 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:23.172089 25437 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.209118 24937 consensus_queue.cc:1059] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 35.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:23.211153 25243 raft_consensus.cc:3055] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:23.211333 25437 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:23.211597 25243 raft_consensus.cc:740] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:23.212527 25243 consensus_queue.cc:260] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.201211 25158 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:26:23.173712 13792 rpcz_store.cc:276] 0709 12:26:20.762779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.771709 (+  8930us) service_pool.cc:224] Handling call
0709 12:26:22.948401 (+2176692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.949098 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.107960 (+158862us) spinlock_profiling.cc:243] Waited 2.17 s on lock 0x7b54002882d0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:23.108134 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4994233728}
I20260709 12:26:23.219432 14418 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:23.223688 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.096689 25424 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:23.226421 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.232802 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.141804 25408 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.235451 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:23.215296 25472 raft_consensus.cc:3055] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:23.238531 14422 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 40.
I20260709 12:26:23.244611 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.217202 25167 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:26:23.231065 25260 consensus_queue.cc:1059] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 36.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.026s
I20260709 12:26:23.250067 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.251061 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:23.252732 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 98884.8us
I20260709 12:26:23.254197 13648 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:23.192179 13617 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:23.255577 14418 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:23.258880 14419 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:23.213526 25070 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.275003 13727 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:23.278029 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5041) took 1121 ms. Trace:
I20260709 12:26:23.282984 13726 rpcz_store.cc:276] 0709 12:26:22.156162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.156466 (+   304us) service_pool.cc:224] Handling call
0709 12:26:22.156803 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.400417 (+243614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.272178 (+871761us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400027bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:23.277957 (+  5779us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436636928}
W20260709 12:26:23.288559 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96466us
I20260709 12:26:23.240356 25472 raft_consensus.cc:740] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:23.289745 13617 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:23.304283 14416 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:23.281589 25380 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:23.305523 25365 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:23.312109 25466 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.317098 25158 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:26:23.372242 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 171568us
I20260709 12:26:23.376595 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.378631 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.380828 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.372279 25176 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:23.372126 25158 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:23.382728 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.383054 25158 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:23.383795 25241 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:23.384764 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.385985 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.387472 25167 consensus_queue.cc:1059] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 36.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.000s
I20260709 12:26:23.382863 25176 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.392313 13793 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:23.392809 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.390618 25176 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:23.346531 25257 consensus_queue.cc:1059] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.050s
I20260709 12:26:23.400943 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.314411 25295 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.059s
I20260709 12:26:23.365185 25380 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:23.421434 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 131760us
I20260709 12:26:23.422646 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.422869 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.426811 13797 raft_consensus.cc:2393] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 37: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:23.359831 25462 consensus_queue.cc:1059] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:26:23.429404 13793 raft_consensus.cc:2393] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:23.429606 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.414078 25347 raft_consensus.cc:3055] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:23.430936 13795 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:23.370011 25365 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.390693 25477 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
W20260709 12:26:23.442204 25480 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:26:23.443900 25158 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.445868 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5605) took 2591 ms. Trace:
I20260709 12:26:23.446422 13652 rpcz_store.cc:276] 0709 12:26:20.854305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.854472 (+   167us) service_pool.cc:224] Handling call
0709 12:26:20.854796 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.876589 (+ 21793us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:22.947265 (+2070676us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b180039d660. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:26:23.047666 (+100401us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.290458 (+242792us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.291067 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.291221 (+   154us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:23.291253 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.291347 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.445414 (+154067us) raft_consensus.cc:1685] finished
0709 12:26:23.445524 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.445730 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35526,"prepare.queue_time_us":535451,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":65026,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":44980,"spinlock_wait_cycles":2743552,"thread_start_us":287281,"threads_started":2,"wal-append.queue_time_us":312559}
I20260709 12:26:23.450008 25460 raft_consensus.cc:3055] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:23.450577 25460 raft_consensus.cc:740] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:23.451567 25460 consensus_queue.cc:260] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.455837 13650 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5618) took 1253 ms. Trace:
I20260709 12:26:23.456326 13650 rpcz_store.cc:276] 0709 12:26:22.202863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.244813 (+ 41950us) service_pool.cc:224] Handling call
0709 12:26:22.245128 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.296369 (+ 51241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.455521 (+1159152us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54001e6754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:23.455761 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69527424}
I20260709 12:26:23.460034 25408 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:23.463364 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.464427 14423 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 40: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:23.464730 25478 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:26:23.467080 13646 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5625) took 1192 ms. Trace:
I20260709 12:26:23.467516 13646 rpcz_store.cc:276] 0709 12:26:22.274577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.301315 (+ 26738us) service_pool.cc:224] Handling call
0709 12:26:22.301667 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.437344 (+135677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.466821 (+1029477us) spinlock_profiling.cc:243] Waited 98.2 ms on lock 0x7b5400097754. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:23.467012 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225764224}
W20260709 12:26:23.470652 25476 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:26:23.472460 13651 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5629) took 1035 ms. Trace:
W20260709 12:26:23.472695 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.473805 25353 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.474511 25353 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.475723 25243 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:23.432317 25347 raft_consensus.cc:740] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:23.479377 25347 consensus_queue.cc:260] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.480650 25347 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:23.488114 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.391163 25466 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.500346 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:23.440317 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:23.497876 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.498469 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 114196us
I20260709 12:26:23.486027 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:23.504920 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.507249 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84255.5us
I20260709 12:26:23.507411 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:23.509169 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:23.510075 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.512888 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:23.416685 25479 consensus_queue.cc:1059] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.025s
I20260709 12:26:23.514569 14423 raft_consensus.cc:1240] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:23.515396 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:23.520406 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:23.472947 13651 rpcz_store.cc:276] 0709 12:26:22.437444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.508547 (+ 71103us) service_pool.cc:224] Handling call
0709 12:26:22.786928 (+278381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:22.790427 (+  3499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.472210 (+681783us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54001e7650. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf38ad 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:23.472380 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596880256}
I20260709 12:26:23.437793 25462 consensus_queue.cc:1243] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 30.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:26:23.529505 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.531988 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:23.538146 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.543228 14425 raft_consensus.cc:1240] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:23.530984 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:23.518707 25357 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:23.540154 13793 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:23.558823 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:23.561822 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54501.2us
I20260709 12:26:23.588708 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.589694 13800 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 39.
I20260709 12:26:23.593269 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:23.594095 13800 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:23.595125 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.551334 25281 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:23.526738 25256 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:26:23.619951 25256 consensus_queue.cc:1059] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:23.621011 25256 consensus_queue.cc:1243] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 35.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
I20260709 12:26:23.621641 25256 raft_consensus.cc:3055] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:23.622092 25256 raft_consensus.cc:740] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:23.622960 14418 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:23.595500 25486 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:26:23.556037 25091 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:23.626542 25091 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.627020 25091 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:23.628705 25456 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.859s	user 0.006s	sys 0.000s
I20260709 12:26:23.536118 25424 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:23.568029 25365 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:23.609848 25353 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:23.568126 13801 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:23.570015 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:23.570935 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.636603 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:23.571473 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:23.637857 13799 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:26:23.572829 25344 consensus_meta.cc:287] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.590s	user 0.011s	sys 0.001s
I20260709 12:26:23.549470 25460 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:23.573933 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:23.575067 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:23.646901 13727 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:23.576012 13722 raft_consensus.cc:3055] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:23.647799 13722 raft_consensus.cc:740] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:23.576835 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:23.578596 13720 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:23.579795 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:23.649420 13726 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:23.580617 25167 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:23.558226 25342 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.035s
I20260709 12:26:23.587083 25485 consensus_queue.cc:1059] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:26:23.659204 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:23.660117 13801 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:23.420356 25472 consensus_queue.cc:260] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 35.26, Last appended by leader: 26, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.605281 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:23.605895 25422 log.cc:927] Time spent T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.336s	user 0.002s	sys 0.000s
I20260709 12:26:23.591112 25070 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:23.669621 13796 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:23.670439 25373 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:23.671190 13796 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 37 index: 27. (index mismatch)
I20260709 12:26:23.617121 25376 raft_consensus.cc:3060] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:23.672560 25367 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:23.672969 25367 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:23.673889 25367 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.675271 25367 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:23.675722 25333 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:23.565393 14425 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:26:23.595927 25158 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:23.591217 25477 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:23.678427 25484 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:23.568293 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:23.635151 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:23.639153 14419 raft_consensus.cc:2393] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:26:23.679840 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 111641us
I20260709 12:26:23.625780 25256 consensus_queue.cc:260] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:23.681317 25490 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:26:23.681521 25256 raft_consensus.cc:3060] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:23.681676 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:23.678432 25402 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.004s
I20260709 12:26:23.683468 25482 raft_consensus.cc:3055] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:23.683887 25482 raft_consensus.cc:740] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:23.684945 25333 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:23.684756 25482 consensus_queue.cc:260] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.670176 25350 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:23.687286 25350 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:23.687709 25350 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.644821 25486 consensus_queue.cc:1059] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 37.27, Next index: 28, Last known committed idx: 27, Time since last communication: 0.044s
I20260709 12:26:23.689749 25486 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:23.690117 25486 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:23.637590 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77004.3us
W20260709 12:26:23.643314 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.684779 13766 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:23.639384 13797 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:26:23.597836 25281 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:23.654472 25344 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.663048 25257 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:23.608047 25466 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:23.703558 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.686120 25482 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:23.652554 25167 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.707504 25281 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:23.672821 25373 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:26:23.709867 25257 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:23.737979 25282 raft_consensus.cc:3055] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:23.718089 25167 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:23.741567 25282 raft_consensus.cc:740] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:23.744963 25282 consensus_queue.cc:260] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.746603 25282 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:23.748528 25366 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:23.750309 25257 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.751333 25472 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:23.751721 25257 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:23.748548 25380 raft_consensus.cc:3055] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:23.753046 25380 raft_consensus.cc:740] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:23.754168 25380 consensus_queue.cc:260] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 36.26, Last appended by leader: 26, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.780951 25489 raft_consensus.cc:3055] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:23.781723 25489 raft_consensus.cc:740] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:23.782903 25489 consensus_queue.cc:260] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.784757 25489 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:23.732255 25486 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 37.27, Last appended by leader: 27, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:23.798202 25486 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:23.897985 13547 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 29. New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:23.910048 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5642) took 1020 ms. Trace:
W20260709 12:26:23.911024 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.914947 13719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.033s	sys 0.032s
W20260709 12:26:23.915673 13719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.033s	sys 0.032s
I20260709 12:26:23.915328 13722 consensus_queue.cc:260] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:23.919529 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.601297 13718 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
W20260709 12:26:23.693959 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 57468.1us
I20260709 12:26:23.693481 13796 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
W20260709 12:26:23.928272 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.929876 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.910815 13647 rpcz_store.cc:276] 0709 12:26:22.889460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.313236 (+423776us) service_pool.cc:224] Handling call
0709 12:26:23.356068 (+ 42832us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.365811 (+  9743us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:23.382748 (+ 16937us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.561699 (+178951us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.585574 (+ 23875us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.585750 (+   176us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:23.585785 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.585856 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.623249 (+ 37393us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003a73e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:23.665938 (+ 42689us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b18003a73e0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:23.909331 (+243393us) raft_consensus.cc:1685] finished
0709 12:26:23.909398 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.909805 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60341,"prepare.queue_time_us":30706,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":79591,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":26928,"spinlock_wait_cycles":33277184,"thread_start_us":127382,"threads_started":1,"wal-append.queue_time_us":214794}
I20260709 12:26:23.931470 13801 raft_consensus.cc:1275] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
W20260709 12:26:23.925676 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 240753us
W20260709 12:26:23.941890 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.945041 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:23.946614 13650 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:23.948068 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.959636 13718 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
W20260709 12:26:23.959736 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.917878 13722 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:23.998082 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 60701.4us
I20260709 12:26:24.014169 25260 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:24.015225 25260 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.013027 25392 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.022501 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:24.026249 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.028719 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5646) took 1002 ms. Trace:
I20260709 12:26:24.032102 13652 rpcz_store.cc:276] 0709 12:26:23.026383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.448938 (+422555us) service_pool.cc:224] Handling call
0709 12:26:23.449376 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.449655 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:23.450082 (+   427us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.527016 (+ 76934us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.619532 (+ 92516us) spinlock_profiling.cc:243] Waited 5.11 ms on lock 0x7b18004df5c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:23.982495 (+362963us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.998584 (+ 16089us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:24.006128 (+  7544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.008399 (+  2271us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.028411 (+ 20012us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100053,"prepare.queue_time_us":34487,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":1834,"raft.queue_time_us":163633,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":4234,"spinlock_wait_cycles":11756928,"thread_start_us":415130,"threads_started":1}
I20260709 12:26:24.051736 25326 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.052969 25326 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.070904 25380 raft_consensus.cc:3060] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:24.075664 25344 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:24.077298 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 123245us
I20260709 12:26:24.078783 25376 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:24.083640 25392 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.086546 25250 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:24.087297 25250 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:24.087622 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 567094us
I20260709 12:26:24.087792 25250 raft_consensus.cc:3060] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:24.097215 25509 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:26:24.098953 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.106803 25373 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:24.111158 25466 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.112113 25466 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:23.908226 13547 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 33, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 33 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:24.119196 25466 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.121147 25285 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:24.121471 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45961.5us
W20260709 12:26:24.127722 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.143632 25437 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:24.159970 25326 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:24.160815 13799 consensus_meta.cc:287] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.005s
I20260709 12:26:24.161633 13799 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 40.
I20260709 12:26:24.148231 25466 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:24.171434 25466 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:24.189844 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 100931us
W20260709 12:26:24.191654 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54551.9us
W20260709 12:26:24.185117 13793 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.628s	user 0.008s	sys 0.010s
I20260709 12:26:24.185525 13547 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 31 to 35. New cstate: current_term: 35 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:24.174194 25466 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:24.200870 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.209957 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.213791 13722 raft_consensus.cc:1275] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
W20260709 12:26:24.215106 25461 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 30 current term: 30 is higher.
I20260709 12:26:24.215890 13766 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:24.204828 13793 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 36.
I20260709 12:26:24.217120 25437 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
I20260709 12:26:24.209843 25411 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.026s
I20260709 12:26:24.227699 13547 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 24, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.240140 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.237498 25326 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:24.254653 25407 consensus_queue.cc:1048] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 12:26:24.255319 25393 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.254814 25273 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:24.256505 25393 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.234308 25466 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.261477 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49883.7us
W20260709 12:26:24.267632 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.255213 13547 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 27 to 34, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 34 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:24.260823 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.629s (SENT)
I20260709 12:26:24.261317 25392 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:24.271241 13720 consensus_meta.cc:287] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.563s	user 0.018s	sys 0.001s
W20260709 12:26:24.275074 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.277510 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:24.259757 25326 raft_consensus.cc:697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:24.278697 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:24.286196 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:24.286631 25350 consensus_meta.cc:287] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.595s	user 0.014s	sys 0.006s
I20260709 12:26:24.287448 25393 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:24.288360 25511 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:26:24.268437 25281 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.292865 25460 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.288843 25466 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:24.299358 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.287438 25350 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.304936 25460 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.314132 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:24.311115 13720 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:26:24.318099 13651 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
W20260709 12:26:24.319463 25366 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.570s	user 0.013s	sys 0.007s
I20260709 12:26:24.313993 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.322005 13724 raft_consensus.cc:2410] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 32 because replica has last-logged OpId of term: 30 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:26:24.322374 25472 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:24.322739 24969 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.322861 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 134065us
I20260709 12:26:24.320353 13547 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 26. New cstate: current_term: 26 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.324961 24969 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:24.326810 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 94580.7us
I20260709 12:26:24.332185 25326 consensus_queue.cc:237] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:24.335772 25257 consensus_meta.cc:287] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.013s
I20260709 12:26:24.336371 25466 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:24.336570 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.337145 25437 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:24.351836 13766 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:24.352900 25407 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:26:24.338044 25091 consensus_meta.cc:287] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.710s	user 0.007s	sys 0.014s
I20260709 12:26:24.353619 25407 raft_consensus.cc:697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:24.353901 25091 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.355012 25407 consensus_queue.cc:237] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.357057 25091 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.336153 25281 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.339380 25460 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.339926 13547 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 20 to 28. New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.340983 25411 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:24.346550 25256 consensus_meta.cc:287] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.014s
I20260709 12:26:24.348321 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:24.349180 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.741s (SENT)
I20260709 12:26:24.362371 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.366873 13547 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 29 to 38, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 38 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:24.376454 25472 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:26:24.377981 25282 consensus_meta.cc:287] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.631s	user 0.007s	sys 0.012s
I20260709 12:26:24.383025 25285 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:24.383311 25167 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.008s
I20260709 12:26:24.384279 13547 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:24.370865 25091 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:24.360708 25250 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.378533 25350 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:24.398555 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:24.399531 13720 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 29: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:24.402645 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 127962us
I20260709 12:26:24.391356 25281 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:24.404683 14390 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:24.405771 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 54091.2us
W20260709 12:26:24.397981 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48944.4us
W20260709 12:26:24.407140 14391 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.405468 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:24.391503 25285 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:24.404853 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 56754.3us
W20260709 12:26:24.410099 25512 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:26:24.411382 25472 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:24.411893 25472 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.412238 25472 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:24.403083 25167 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.414124 25281 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:24.423808 25344 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
I20260709 12:26:24.420648 25250 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:24.426717 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.426865 13547 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 16 to 25, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 25 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.419189 25281 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:24.446375 25344 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:24.448858 25344 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:26:24.454576 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 50283.5us
I20260709 12:26:24.445258 25281 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:24.466219 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.466485 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:24.451354 25256 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:24.471472 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:24.477666 13547 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 29 to 33, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 33 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:24.472805 25256 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:24.482872 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.486126 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:24.487275 25282 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:24.487427 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.476325 25091 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:24.484628 25256 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:24.493108 25250 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:26:24.494614 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.496492 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 45581.9us
I20260709 12:26:24.497638 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.497735 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:24.490125 25091 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:24.495117 25256 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:24.506284 25282 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:24.502341 13547 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 27, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 27 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:24.500769 25513 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.235s	user 0.003s	sys 0.000s
I20260709 12:26:24.495711 25250 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:24.504110 25091 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:24.512701 25250 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:24.525594 13547 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 17 to 25, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 25 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.521646 25167 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:24.529276 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 49855.6us
I20260709 12:26:24.529241 13615 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:24.534375 13547 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 32 to 37. New cstate: current_term: 37 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:24.540736 25524 consensus_queue.cc:1048] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:24.544397 25282 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:24.545416 25379 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:24.554520 13547 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 27 to 33. New cstate: current_term: 33 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:24.558192 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:26:24.562464 13724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2402) took 2535 ms (client timeout 2527 ms). Trace:
I20260709 12:26:24.562999 25379 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:24.563671 25379 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:24.564107 25379 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:26:24.564177 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.557876 13547 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 33, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 33 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.564545 13615 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:24.566082 25167 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:24.563167 24969 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:24.566324 13547 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 33. New cstate: current_term: 33 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:26:24.568372 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.566895 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:24.563084 13724 rpcz_store.cc:269] 0709 12:26:22.027485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.047409 (+ 19924us) service_pool.cc:224] Handling call
0709 12:26:24.562038 (+2514629us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b5400174b54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:24.562294 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5139429120}
W20260709 12:26:24.575981 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.578616 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:24.579770 13720 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 40: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:24.579684 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:24.580670 13795 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:24.580188 25281 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.583104 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:24.584537 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:24.585655 13722 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:24.586850 25260 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:24.600776 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:24.601737 13647 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:24.604928 13722 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:24.597292 13547 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 31 to 36. New cstate: current_term: 36 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:24.546151 25472 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:24.658337 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.658375 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:24.660256 13643 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:24.659571 13547 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 27, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 27 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:24.660972 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 108438us
I20260709 12:26:24.606698 25260 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:24.560748 25285 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.660699 24969 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.673974 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.678707 13644 raft_consensus.cc:1275] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 35 index: 22. (term mismatch)
W20260709 12:26:24.682288 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.679265 13644 pending_rounds.cc:85] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 21
I20260709 12:26:24.685469 13644 pending_rounds.cc:107] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 34.22
I20260709 12:26:24.685784 13644 raft_consensus.cc:2889] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.681900 13547 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 19 to 28. New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.701406 25281 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.701671 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:24.705431 25344 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:24.701692 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:24.721901 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 67299.3us
I20260709 12:26:24.724640 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:24.738644 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:24.673705 25407 consensus_queue.cc:1048] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.067s
I20260709 12:26:24.740561 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:24.740921 25376 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:24.716605 25167 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:24.746434 25344 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:24.738865 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84211.6us
W20260709 12:26:24.755300 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 221565us
I20260709 12:26:24.757606 14419 raft_consensus.cc:2468] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:24.764976 25530 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.765308 25472 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.765676 25530 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.766847 25344 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.768147 25530 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:24.698753 24969 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:24.788861 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82911.2us
I20260709 12:26:24.755523 13547 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 30. New cstate: current_term: 30 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:24.756170 25281 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.797346 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:24.798331 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:24.798487 13764 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:24.807992 13649 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:26:24.811568 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:24.814320 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:24.814586 13547 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 28 to 36. New cstate: current_term: 36 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.815936 13724 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:24.818033 25344 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:24.753159 25260 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.826725 25285 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:24.774952 25407 consensus_queue.cc:1059] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:26:24.824322 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:24.840176 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.840327 14419 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:24.813292 25461 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:24.827831 25530 consensus_queue.cc:1048] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.006s
I20260709 12:26:24.854379 13547 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 23 to 27, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 27 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.835901 25263 raft_consensus.cc:2804] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:24.815966 25281 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:24.866508 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:24.868978 25176 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:24.818789 25344 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:24.880287 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 47130.7us
I20260709 12:26:24.874691 25176 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.905905 14419 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
W20260709 12:26:24.907824 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 102074us
I20260709 12:26:24.858062 25285 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.032s
I20260709 12:26:24.909755 25501 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:24.911079 25492 log.cc:927] Time spent T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:24.910908 25260 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:24.879046 25263 raft_consensus.cc:697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:24.916942 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:24.859555 25524 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.048s
W20260709 12:26:24.933590 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 105639us
W20260709 12:26:24.930749 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45319.1us
I20260709 12:26:24.878897 25250 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.908514 25176 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:24.908716 25411 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:24.939664 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:24.932158 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:24.885690 25461 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:24.955638 13792 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:24.955950 13766 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:24.920775 25407 consensus_queue.cc:1048] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.035s
I20260709 12:26:24.943082 25260 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.970640 13649 raft_consensus.cc:1275] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 36 index: 26. Preceding OpId from leader: term: 37 index: 27. (index mismatch)
W20260709 12:26:24.976812 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.972309 25411 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:24.937887 25437 consensus_queue.cc:1048] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:26:24.955968 13547 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 15 to 24. New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:24.957125 25499 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:24.970641 25542 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:24.940307 25285 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
W20260709 12:26:24.987663 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48164.6us
I20260709 12:26:24.944676 25501 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:25.015095 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.008299 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:25.023999 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:25.047640 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:25.018082 13547 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 27, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:25.052866 25549 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.012713 25411 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:26:25.042409 25091 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.003s
W20260709 12:26:25.056730 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 121674us
I20260709 12:26:25.051362 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.061350 13650 raft_consensus.cc:1240] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:25.006690 25260 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.076740 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:25.079954 13617 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:25.086328 25070 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:25.087128 25070 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.059187 25285 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:25.057891 24969 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.094213 25285 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:25.073532 25379 consensus_meta.cc:287] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.003s
I20260709 12:26:25.057045 25411 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:25.059809 25524 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.104948 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:25.078922 13547 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 37 to 40, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 40 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:25.109907 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.110869 13795 consensus_meta.cc:287] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.004s
I20260709 12:26:25.111260 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:25.094944 24969 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.094995 25285 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:25.079152 25281 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.126816 13547 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 24, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:25.121644 13800 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
W20260709 12:26:25.122607 14389 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.131074 13547 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 22 to 25, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 25 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:25.136643 25538 log.cc:927] Time spent T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:25.137943 25546 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:26:25.137874 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 80466.4us
I20260709 12:26:25.138495 25344 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.140901 25333 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:25.141378 25333 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:25.141754 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.142591 13725 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:25.142385 25333 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.144014 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:25.144981 13726 raft_consensus.cc:3060] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:26:25.145954 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:25.146584 25544 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:25.005748 25263 consensus_queue.cc:237] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:25.138752 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 203875us
I20260709 12:26:25.149832 13766 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:25.074118 25091 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.155215 25544 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:25.121181 25498 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:25.144014 25333 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:25.158177 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 53361.5us
W20260709 12:26:25.162494 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82622.9us
I20260709 12:26:25.102864 25548 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:25.126122 25551 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.155390 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.192776 13792 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:25.129618 25282 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:26:25.212738 25537 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:26:25.129611 25545 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:25.218374 13617 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:25.162751 25250 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.171267 25544 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.217420 13725 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:25.110778 25379 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.218775 25551 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:25.217227 25534 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.263s	user 0.002s	sys 0.000s
I20260709 12:26:25.124891 25549 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.232450 25501 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.231741 25411 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:25.233630 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83973.4us
I20260709 12:26:25.235073 25385 raft_consensus.cc:3055] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:25.236452 24969 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.237779 25554 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:25.151523 25524 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.213304 25498 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:25.242025 13617 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:25.209817 13547 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 21 to 29. New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:26:25.248292 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 68129.3us
I20260709 12:26:25.162781 25344 consensus_queue.cc:1048] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.058s
I20260709 12:26:25.249361 25548 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.253595 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:25.266482 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:25.267774 14391 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:25.270320 25542 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:25.271162 25542 raft_consensus.cc:2804] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:25.271644 25542 raft_consensus.cc:697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:25.272663 25542 consensus_queue.cc:237] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.275727 25070 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.276160 25524 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.277721 13799 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:25.284039 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.222390 25461 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.243487 25544 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:25.301977 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.311254 13795 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 38: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:25.302058 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.290014 13725 raft_consensus.cc:2393] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:25.286579 13726 raft_consensus.cc:2468] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:25.258889 25385 raft_consensus.cc:740] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:25.323102 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 88892.4us
I20260709 12:26:25.327672 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:25.328729 13726 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:25.330585 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:25.302347 25501 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.338605 13651 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:25.339097 13725 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:25.285550 25281 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:25.262475 25545 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.330653 25070 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:25.366976 13726 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:26:25.379973 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 162440us
W20260709 12:26:25.406137 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.356750 25437 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.411456 13651 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:25.412225 25367 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.053s
I20260709 12:26:25.414744 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.330746 25552 consensus_queue.cc:1048] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 12:26:25.426780 25549 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.427997 25561 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:26:25.433645 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.445111 25437 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.457381 25437 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.458201 13722 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:25.459273 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.458930 25498 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.461066 25559 consensus_queue.cc:1059] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 35.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.016s
I20260709 12:26:25.461737 13648 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:26:25.462455 13648 pending_rounds.cc:85] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 18
I20260709 12:26:25.462723 13648 pending_rounds.cc:107] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:26:25.462975 13648 raft_consensus.cc:2889] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:25.463246 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 414842us
I20260709 12:26:25.464136 25561 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:25.464655 25561 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:25.467311 25379 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.467576 25393 consensus_queue.cc:1059] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 35.24, Next index: 25, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:25.471706 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:25.476418 25551 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:25.477314 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.477989 13646 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:25.479255 13618 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:25.443948 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:25.490979 13726 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:25.503216 13767 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.369412 25501 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:25.524731 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 194210us
W20260709 12:26:25.504873 25256 consensus_meta.cc:287] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.972s	user 0.016s	sys 0.000s
W20260709 12:26:25.538197 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.489353 25091 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.545887 13795 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 37. Current term is 38. Ops: []
I20260709 12:26:25.552613 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:25.555052 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 139172us
I20260709 12:26:25.555410 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:25.543649 25176 consensus_meta.cc:287] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.001s
I20260709 12:26:25.553817 25260 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.562269 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.528120 25379 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:25.567236 24969 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.567813 25176 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.568845 25385 consensus_queue.cc:260] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.545632 25282 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:26:25.588629 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 45370.2us
I20260709 12:26:25.592351 25260 consensus_queue.cc:1059] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 37.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.001s
I20260709 12:26:25.610885 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.613313 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.621469 13722 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:25.614225 13801 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:25.582576 25091 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.545432 25256 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.630560 13726 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:25.626689 25385 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:25.647877 13643 raft_consensus.cc:1240] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:25.663034 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:25.676083 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 118678us
I20260709 12:26:25.678804 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:25.680256 13722 raft_consensus.cc:2393] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:25.616650 25379 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.682597 13726 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:25.669620 25557 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:26:25.685393 25568 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:25.688621 25501 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.690551 25176 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.693516 25548 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.695226 13726 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:26:25.631856 25542 consensus_queue.cc:1059] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.064s
I20260709 12:26:25.695581 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:25.700906 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.703565 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 100650us
W20260709 12:26:25.695914 25534 log.cc:927] Time spent T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:25.633433 25091 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:25.707120 13693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.12.45.65:33075 timed out after 1.566s (SENT) [suppressed 26 similar messages]
I20260709 12:26:25.726765 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:25.710851 25567 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.711244 25501 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:25.715663 25557 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:25.717082 25256 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.749571 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:25.750140 25566 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:26:25.750628 25569 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.751263 25569 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.753844 25569 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.756433 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:25.757947 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:25.758647 25260 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:25.759095 25260 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:25.759479 25260 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:25.760726 25582 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:25.761925 25569 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:25.762370 25569 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:25.762681 25576 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:25.763276 25576 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:25.763782 25576 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:25.765153 25572 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.765655 25572 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.746199 13727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4685) took 1029 ms. Trace:
I20260709 12:26:25.762789 25569 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:25.772790 25545 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:25.773551 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:25.775693 25282 raft_consensus.cc:3055] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Stepping down as leader of term 37
I20260709 12:26:25.777729 25565 raft_consensus.cc:3055] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:25.825999 25565 raft_consensus.cc:740] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:25.826411 25379 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.827198 25565 consensus_queue.cc:260] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.829010 25565 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:25.778245 25557 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.779932 14425 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
W20260709 12:26:25.787200 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.807824 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:25.809646 25499 consensus_meta.cc:287] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.800s	user 0.006s	sys 0.008s
I20260709 12:26:25.809821 25282 raft_consensus.cc:740] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:25.736541 25567 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.820623 13725 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:25.820757 25091 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.745155 25501 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.766745 13727 rpcz_store.cc:276] 0709 12:26:24.600582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.602222 (+  1640us) service_pool.cc:224] Handling call
0709 12:26:24.658831 (+ 56609us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:24.685303 (+ 26472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.685342 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:24.686139 (+   797us) log.cc:844] Serialized 52 byte log entry
0709 12:26:24.778179 (+ 92040us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:24.887209 (+109030us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.625935 (+738726us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b18004e3dc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390a81c0a 00007f539a1c1219 00007f5390b09a40 00007f5390b044e4 00007f5390b04455 00007f53975ec3c7 00007f5393bfe9a7 00007f5393c0e0bf 00007f5393c0e038
0709 12:26:25.626212 (+   277us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.626606 (+   394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.626713 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.629305 (+  2592us) raft_consensus.cc:1685] finished
0709 12:26:25.629335 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.629564 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":641269,"prepare.queue_time_us":809273,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":4419,"raft.queue_time_us":173805,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":20964,"spinlock_wait_cycles":20473600,"thread_start_us":249449,"threads_started":3,"wal-append.queue_time_us":156907}
W20260709 12:26:25.825497 25285 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.004s
I20260709 12:26:25.841114 13647 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:25.886364 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:25.896109 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53670.9us
I20260709 12:26:25.902134 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.903676 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:25.904170 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.880420 25499 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:25.907126 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.901103 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.918902 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.918690 25282 consensus_queue.cc:260] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 37.26, Last appended by leader: 26, Current term: 37, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.920305 25282 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:25.920434 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:25.921568 13719 raft_consensus.cc:2393] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 36: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:25.922544 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.922653 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.923950 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.925081 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.925659 13793 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:25.927968 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.929248 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:25.931039 25544 consensus_meta.cc:287] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.000s
I20260709 12:26:25.931559 25544 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.932289 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:25.932873 13798 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 40. Re-sending same reply.
I20260709 12:26:25.933743 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:25.934089 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.934561 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:25.893564 25285 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:25.935364 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.935611 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.936085 13797 raft_consensus.cc:2368] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 37. Current term is 38.
I20260709 12:26:25.938414 13796 raft_consensus.cc:1217] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Deduplicated request from leader. Original: 28.16->[29.17-29.17]   Dedup: 29.17->[]
I20260709 12:26:25.936903 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.935405 13650 raft_consensus.cc:2368] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 35. Current term is 36.
I20260709 12:26:25.940783 13798 raft_consensus.cc:2376] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 36. Re-sending same reply.
I20260709 12:26:25.937023 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 38 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.940310 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5110) took 1221 ms. Trace:
I20260709 12:26:25.943921 13796 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:26:25.945636 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:25.937378 13718 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:25.902388 25535 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.946897 13800 rpcz_store.cc:276] 0709 12:26:24.718666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.725609 (+  6943us) service_pool.cc:224] Handling call
0709 12:26:24.753513 (+ 27904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.133591 (+380078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.940016 (+806425us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54003d3bd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:25.940220 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474907520}
I20260709 12:26:25.945043 13617 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:25.908012 25070 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:25.962353 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.939273 25544 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.966289 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.937409 25285 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:25.964408 13718 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:26:25.958823 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:25.964820 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.889659 25561 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:25.967031 13725 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:26:25.908757 25582 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:25.967327 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.967475 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:25.968885 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:25.966426 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.913143 25176 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.053s
I20260709 12:26:25.980393 13796 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:25.975421 13650 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 38.
I20260709 12:26:25.977761 13617 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:25.964072 25499 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:25.996608 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:25.969950 25070 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.009879 13650 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 36: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:26.011478 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44118.7us
I20260709 12:26:26.013747 25591 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.036s
I20260709 12:26:25.997282 25285 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:26.018492 13723 raft_consensus.cc:1240] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:26:25.994802 25256 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:26.020131 25561 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:25.982161 25535 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.024310 25260 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.047694 25260 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.035753 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.036067 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:25.995157 25499 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:26.036784 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.041416 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.037274 25285 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:26.051100 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.050199 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.040616 25576 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.059562 25576 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.054723 25285 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:26.064674 13650 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.065465 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:26.074435 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 79856.9us
I20260709 12:26:26.068061 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.079864 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:26.055362 25256 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:26.087227 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.085510 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:26.081267 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.070955 25385 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:26.063175 25569 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.093885 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.103690 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.104719 13723 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:26.088693 25535 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.089533 25576 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:26.093497 25522 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:26.118503 25522 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.118968 25522 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:26.112799 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.122954 13645 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:26.124310 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.125109 13645 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.125697 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.099602 25385 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.126726 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.128260 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.128844 25544 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.126680 13727 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:26.116709 25256 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.129875 13650 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 37.
I20260709 12:26:26.131345 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.125835 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.131711 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.134279 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.131043 25499 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.135262 13727 raft_consensus.cc:2468] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:26.136468 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.133533 13719 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:26.147010 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.148008 13727 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.151705 13723 raft_consensus.cc:2468] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:26:26.152796 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.083683 25582 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.153383 13723 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:26.129544 25385 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:26.129557 25544 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.154443 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.154578 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.142333 25565 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:26.158557 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.156695 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.159860 13650 raft_consensus.cc:2393] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.161180 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.161825 13719 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:26:26.163110 25565 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.166095 25565 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.166486 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.167838 13718 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 34: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:26.168272 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.168916 13646 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 36.
I20260709 12:26:26.169294 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:26.169853 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.174547 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:26.175410 25569 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.176343 25544 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.177728 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.178484 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.180567 25285 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.181543 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.183914 14419 raft_consensus.cc:2368] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37.
I20260709 12:26:26.168910 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.186758 25385 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.186831 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.187659 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.189337 13652 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5677) took 1080 ms. Trace:
I20260709 12:26:26.190136 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:26.190249 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:26.192490 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:26.193493 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:26.197305 13725 raft_consensus.cc:1275] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:26.198539 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:26.199481 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:26.202875 25603 consensus_queue.cc:1059] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:26.214216 13718 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:26.254408 13727 raft_consensus.cc:2468] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 33.
I20260709 12:26:26.224253 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.279634 13719 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:26.278132 13652 rpcz_store.cc:276] 0709 12:26:25.109102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.109230 (+   128us) service_pool.cc:224] Handling call
0709 12:26:26.188490 (+1079260us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:26:26.188709 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.188909 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.188942 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.189023 (+    81us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:26.189064 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.189125 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.189240 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8340352}
I20260709 12:26:26.223017 13651 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:26.255890 13723 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:26.282707 13645 raft_consensus.cc:2393] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 34: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.279717 14416 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:26.281143 13650 raft_consensus.cc:2468] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
W20260709 12:26:26.289659 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 107831us
I20260709 12:26:26.298913 14425 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 24.
I20260709 12:26:26.298434 13615 leader_election.cc:400] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37.
I20260709 12:26:26.300324 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.300910 13615 leader_election.cc:403] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:26.301290 13650 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.289721 13651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.301982 13652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.230331 25576 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:26.291146 13718 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:26.304260 13645 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
W20260709 12:26:26.223820 25553 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.974s	user 0.002s	sys 0.003s
I20260709 12:26:26.303577 13651 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 24.
I20260709 12:26:26.307508 13721 raft_consensus.cc:1275] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 36 index: 25. (index mismatch)
I20260709 12:26:26.309302 13652 raft_consensus.cc:2393] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 24: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.316007 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.316931 13650 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:26.318048 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 40 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.319813 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.320711 13650 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 36: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.322567 13693 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:26.323406 25608 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:26.322954 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.324268 25603 raft_consensus.cc:3055] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:26:26.312146 25344 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:26.324805 13650 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 37.
I20260709 12:26:26.325827 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.325778 25344 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.326659 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 38 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.326833 25602 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:26.327456 25602 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:26.328305 25344 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.328267 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.328562 25602 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 38.20, Last appended by leader: 20, Current term: 40, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.312288 25544 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:26.329996 25544 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.331121 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.332240 25544 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:26.323493 25285 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:26.212182 25522 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.324043 25608 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:26.333983 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.309244 25582 raft_consensus.cc:3060] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 37
I20260709 12:26:26.338605 13721 raft_consensus.cc:1240] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: [35.22-35.22]
I20260709 12:26:26.339200 13646 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 40.
I20260709 12:26:26.336822 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.340333 13718 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:26:26.324712 25603 raft_consensus.cc:740] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:26.341266 14423 raft_consensus.cc:2393] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 39: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:26.284523 25385 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.341789 13650 raft_consensus.cc:2393] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.342428 14420 raft_consensus.cc:1240] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 37. Current term is 38. Ops: []
W20260709 12:26:26.344414 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.345690 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.347112 25603 consensus_queue.cc:260] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.348533 25603 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:26.349700 13643 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5668) took 1679 ms. Trace:
I20260709 12:26:26.350090 13643 rpcz_store.cc:276] 0709 12:26:24.670765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.673431 (+  2666us) service_pool.cc:224] Handling call
0709 12:26:24.707373 (+ 33942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.648554 (+941181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.349415 (+700861us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b5400095454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:26.349635 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162331520}
I20260709 12:26:26.351423 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.354702 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:26.355657 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.357055 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.358800 25273 raft_consensus.cc:3055] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:26.359331 25273 raft_consensus.cc:740] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:26.360443 25582 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 36. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for earlier term 36. Current term is 37.
I20260709 12:26:26.361836 13650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.364440 14416 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:26.360443 25273 consensus_queue.cc:260] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.22, Last appended by leader: 22, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:26.366597 25590 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.257s	user 0.002s	sys 0.000s
I20260709 12:26:26.366801 25273 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:26.370442 25576 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.371100 25576 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.371562 25522 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.372895 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.373860 13648 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 36: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.352100 13643 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:26.374718 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.374854 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.375092 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.375375 13722 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:26.375581 13648 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:26.375718 13643 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 36.
I20260709 12:26:26.376535 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.376628 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.377673 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.377837 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.381954 14421 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:26.383407 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:26.373893 25576 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.397104 13646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.339895 25608 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.398108 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.399417 13643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.402938 13643 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:26.372923 25070 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:26.407205 14391 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.414005 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.415429 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.416584 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.418797 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.422626 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.424685 13793 raft_consensus.cc:1275] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:26:26.427479 14390 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:26.423364 13648 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 40.
I20260709 12:26:26.429095 13643 raft_consensus.cc:2468] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 39.
I20260709 12:26:26.427161 14391 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.433030 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.395828 25551 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:26.431469 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.395304 25385 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:26.445538 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.447428 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.444947 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.445119 14391 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:26.450938 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.451603 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68303.3us
I20260709 12:26:26.452773 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:26.455101 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.460548 25385 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:26.435081 25333 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:26.432430 25592 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:26.463721 13721 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:26.463908 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.464989 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.465808 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.449880 25545 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:26.442941 25587 raft_consensus.cc:2804] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election won for term 37
W20260709 12:26:26.467298 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 130545us
I20260709 12:26:26.468580 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.461256 25522 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:26.473086 13792 raft_consensus.cc:1240] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:26.463254 25576 raft_consensus.cc:2804] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:26.474159 13693 leader_election.cc:400] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 37. Current term is 38.
I20260709 12:26:26.490269 13648 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 26: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.497586 13693 leader_election.cc:403] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:26.485049 13721 raft_consensus.cc:1275] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:26.498288 25333 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:26.500710 13793 raft_consensus.cc:1240] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:26.496644 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.499392 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.488642 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:26.518872 13800 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:26:26.523119 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.529199 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.530030 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.472976 25582 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:26.503573 13797 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 38.
I20260709 12:26:26.467658 25592 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.470625 25545 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:26:26.538671 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 64668.6us
W20260709 12:26:26.540978 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 109584us
I20260709 12:26:26.497771 25522 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.532449 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.523286 25333 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:26.543771 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.524168 25499 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:26.554546 13793 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 35: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.588424 25411 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:26.589903 25560 log.cc:927] Time spent T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.344s	user 0.002s	sys 0.001s
I20260709 12:26:26.499595 25587 raft_consensus.cc:697] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:26.591140 25587 consensus_queue.cc:237] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 35.26, Last appended by leader: 26, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.544368 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.593075 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2131) took 1871 ms. Trace:
I20260709 12:26:26.552487 25582 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.596469 25582 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.599093 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.599428 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.552037 13648 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 34.
I20260709 12:26:26.550602 25545 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.548085 14390 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:26.605584 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57631.1us
I20260709 12:26:26.605917 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.553867 25522 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:26.541262 25576 raft_consensus.cc:697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:26.585582 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.584223 25326 raft_consensus.cc:3055] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:26.618417 25326 raft_consensus.cc:740] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:26.610555 25499 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.619472 25326 consensus_queue.cc:260] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 35.24, Last appended by leader: 24, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.620529 25326 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:26.621260 25499 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.553570 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:26.552371 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:26.600106 13800 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 40.
I20260709 12:26:26.603319 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5676) took 1494 ms. Trace:
I20260709 12:26:26.548506 25592 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:26.543586 25551 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Advancing to term 38
I20260709 12:26:26.608284 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:26.632321 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 80041.5us
I20260709 12:26:26.625236 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.624521 13649 rpcz_store.cc:276] 0709 12:26:25.108416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.108572 (+   156us) service_pool.cc:224] Handling call
0709 12:26:26.591867 (+1483295us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b5800120c98. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfa22 00007f53999e88d9 00007f5398b42ee0 00007f5398c0c0ab 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:26:26.592093 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.592280 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.592310 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.592358 (+    48us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:26.592390 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.603072 (+ 10682us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.603231 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7605504}
I20260709 12:26:26.631175 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:26.636502 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 46984.5us
W20260709 12:26:26.639762 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 91953.1us
I20260709 12:26:26.593447 14418 rpcz_store.cc:276] 0709 12:26:24.721340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.792360 (+ 71020us) service_pool.cc:224] Handling call
0709 12:26:24.817336 (+ 24976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.915296 (+ 97960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.915348 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.942018 (+ 26670us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.244339 (+302321us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:25.244383 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.244458 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.589229 (+1344771us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:26.592613 (+  3384us) raft_consensus.cc:1685] finished
0709 12:26:26.592636 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.592861 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11563,"raft.queue_time_us":57694,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":2630,"spinlock_wait_cycles":6517120,"thread_start_us":351042,"threads_started":2,"wal-append.queue_time_us":296598}
I20260709 12:26:26.634438 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:26.648029 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:26.640101 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.649036 25602 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election won for term 36
W20260709 12:26:26.650695 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.651391 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.610917 25535 raft_consensus.cc:2804] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:26.652864 14421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:26.650776 13798 raft_consensus.cc:2410] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:26:26.653411 13648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.623463 25544 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.655586 13648 raft_consensus.cc:2410] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:26:26.623889 25559 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.656436 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:26.656939 13649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
W20260709 12:26:26.658910 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50705us
I20260709 12:26:26.639464 25597 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:26.662025 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.662528 25597 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.662947 25597 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:26.663769 13615 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:26.683694 25091 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:26.684626 25091 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:26.639461 25376 consensus_queue.cc:1048] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.048s
I20260709 12:26:26.687656 25615 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.048s
I20260709 12:26:26.639511 25333 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 12:26:26.694851 25385 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:26.695611 25411 consensus_queue.cc:1048] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:26.697091 25619 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:26.697522 25619 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.697762 25619 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:26.655808 25285 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:26.699326 25285 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.699813 25285 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.701465 25567 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.667007 25544 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.676959 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.647343 25525 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.711777 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54768 (request call id 5688) took 1240 ms. Trace:
I20260709 12:26:26.713176 13648 rpcz_store.cc:276] 0709 12:26:25.471291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.710711 (+1239420us) service_pool.cc:224] Handling call
0709 12:26:26.711104 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.711347 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.711385 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.711449 (+    64us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:26.711497 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.711569 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.711699 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:26.715992 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.639343 25273 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 32.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.070s
I20260709 12:26:26.666601 25591 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:26.723183 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59518.4us
I20260709 12:26:26.724833 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:26.723707 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46852.1us
I20260709 12:26:26.708739 25569 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:26.658231 25582 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:26:26.712975 25379 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.714608 25385 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:26.736500 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.727278 25565 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.734329 25344 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.685958 25091 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.749586 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.733355 25544 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.756680 13721 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:26.758315 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.739739 25499 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:26.769754 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.770763 25273 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.737514 25602 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:26.774246 25273 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.765477 25569 consensus_queue.cc:1059] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 12:26:26.754956 25070 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:26:26.738277 25379 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.731076 25576 consensus_queue.cc:237] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 37.27, Last appended by leader: 27, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.730530 25525 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.801811 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.744928 25565 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.781665 25282 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:26.676671 25559 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.767854 25256 consensus_queue.cc:1059] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:26:26.805619 25070 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:26.752228 25344 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.814224 13797 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 29.
I20260709 12:26:26.740334 25592 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.785996 25273 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:26.829795 14416 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:26.781010 25582 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.042s
I20260709 12:26:26.821591 25498 raft_consensus.cc:3055] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:26:26.838521 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 127991us
I20260709 12:26:26.839969 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1850) took 1289 ms. Trace:
I20260709 12:26:26.836656 13690 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:26.820473 25624 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:26.825861 25535 raft_consensus.cc:697] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:26.846115 13647 rpcz_store.cc:276] 0709 12:26:25.550457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.707410 (+1156953us) service_pool.cc:224] Handling call
0709 12:26:26.707800 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.708037 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:26:26.728667 (+ 20630us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.839417 (+110750us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.839506 (+    89us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:26.839576 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.839694 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.839878 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18252,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":1244,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":148,"spinlock_wait_cycles":42240,"thread_start_us":117544,"threads_started":2,"wal-append.queue_time_us":105774}
I20260709 12:26:26.857214 14418 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 38 index: 20. Preceding OpId from leader: term: 40 index: 21. (index mismatch)
I20260709 12:26:26.822602 25545 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:26.850926 25610 log.cc:927] Time spent T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.242s	user 0.006s	sys 0.000s
I20260709 12:26:26.888283 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5105) took 2328 ms. Trace:
I20260709 12:26:26.888666 13799 rpcz_store.cc:276] 0709 12:26:24.559928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.563366 (+  3438us) service_pool.cc:224] Handling call
0709 12:26:24.563872 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.278158 (+714286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.887986 (+1609828us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b540006b154. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:26.888191 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641173888}
I20260709 12:26:26.855218 25628 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:26.890209 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:26.892292 25501 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:26.894095 25594 log.cc:927] Time spent T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.895s	user 0.000s	sys 0.004s
I20260709 12:26:26.890807 25619 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.895586 25608 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.896406 24969 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:26.897159 25619 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:26.867117 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.867851 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.899760 13800 raft_consensus.cc:2410] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 35 because replica has last-logged OpId of term: 34 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 21.
I20260709 12:26:26.899905 25551 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election lost for term 37. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 37. Current term is 38.
W20260709 12:26:26.902084 25614 log.cc:927] Time spent T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
I20260709 12:26:26.902871 25525 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:26.868479 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.905972 25522 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.905083 13647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.907462 25569 raft_consensus.cc:3055] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:26.907706 13647 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 33: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:26.907958 25569 raft_consensus.cc:740] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:26.898499 25608 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.869117 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:26.870942 25603 consensus_meta.cc:287] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.522s	user 0.016s	sys 0.000s
I20260709 12:26:26.909221 25569 consensus_queue.cc:260] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.911499 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.911643 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.911895 25569 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:26.915792 13727 raft_consensus.cc:2410] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:26.916558 13644 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 22. (term mismatch)
I20260709 12:26:26.916808 13647 raft_consensus.cc:1240] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:26.917572 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3484) took 1110 ms. Trace:
I20260709 12:26:26.917961 13647 rpcz_store.cc:276] 0709 12:26:25.807456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.908440 (+1100984us) service_pool.cc:224] Handling call
0709 12:26:26.916564 (+  8124us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.917308 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.917512 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:26.918565 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.870461 25535 consensus_queue.cc:237] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.920244 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1855) took 1010 ms. Trace:
I20260709 12:26:26.920790 13647 rpcz_store.cc:276] 0709 12:26:25.909742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.919107 (+1009365us) service_pool.cc:224] Handling call
0709 12:26:26.919507 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.919855 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.919885 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.919936 (+    51us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:26:26.919974 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.920034 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.920172 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:26.855836 25256 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:26.873611 25597 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.874899 25379 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:26.875559 25070 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.843649 25498 raft_consensus.cc:740] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:26.922667 25256 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.923553 25498 consensus_queue.cc:260] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.924103 25501 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:26.820719 25385 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:26.924695 25501 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.848060 25624 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.887283 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:26.931840 13792 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:26:26.827330 25602 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 34.18, Last appended by leader: 18, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.898500 25592 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:26.924963 25256 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:26.926211 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.884909 25285 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:26.927744 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:26.916925 13644 pending_rounds.cc:85] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Aborting all ops after (but not including) 21
I20260709 12:26:26.920894 25563 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.936460 13644 pending_rounds.cc:107] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: Aborting uncommitted NO_OP operation due to leader change: 28.22
I20260709 12:26:26.938795 13644 raft_consensus.cc:2889] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.922377 25522 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:26.924970 25498 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.942868 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:26.943964 14418 raft_consensus.cc:3060] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:26.952975 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.933727 25624 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.955196 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3490) took 1146 ms. Trace:
I20260709 12:26:26.953979 25559 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:26.955672 13644 rpcz_store.cc:276] 0709 12:26:25.809271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.915822 (+1106551us) service_pool.cc:224] Handling call
0709 12:26:26.916160 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.949706 (+ 33546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.955128 (+  5422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:26.973866 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.975230 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.976322 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4" is_pre_election: true
I20260709 12:26:26.976938 13644 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:26.970993 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.978542 13792 raft_consensus.cc:2410] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:26.980257 13644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "4dafe08a4f404502822a0732fe570de4"
I20260709 12:26:26.980993 13644 raft_consensus.cc:3060] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:26.985334 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.995623 25285 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:26.997414 25632 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:26.997982 25632 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.998389 25632 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:26.998610 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.999671 13796 raft_consensus.cc:2410] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
W20260709 12:26:27.000595 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 230942us
I20260709 12:26:26.942364 25256 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.001058 13797 raft_consensus.cc:2393] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:27.005066 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:27.014410 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.017441 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:27.019462 13799 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:27.020036 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:27.026826 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.945533 25273 consensus_queue.cc:1059] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 32.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.066s
I20260709 12:26:27.032330 13792 raft_consensus.cc:1240] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:26:27.035763 13793 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:27.039225 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:26.942755 25563 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.059546 13644 raft_consensus.cc:2468] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:27.061642 13644 raft_consensus.cc:1217] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Deduplicated request from leader. Original: 27.15->[28.16-28.16]   Dedup: 28.16->[]
I20260709 12:26:27.062233 13644 raft_consensus.cc:1240] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:26:27.063023 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3500) took 1123 ms. Trace:
W20260709 12:26:27.057713 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.063477 13644 rpcz_store.cc:276] 0709 12:26:25.939164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.061098 (+1121934us) service_pool.cc:224] Handling call
0709 12:26:27.061513 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.062769 (+  1256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.062961 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:27.036330 25603 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.031067 25256 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:27.079080 25588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.924s	user 0.007s	sys 0.019s
I20260709 12:26:27.104702 13644 raft_consensus.cc:1240] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:26:27.110450 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3501) took 1171 ms. Trace:
I20260709 12:26:27.110899 13644 rpcz_store.cc:276] 0709 12:26:25.939476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.086077 (+1146601us) service_pool.cc:224] Handling call
0709 12:26:27.104495 (+ 18418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.110056 (+  5561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.110368 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:27.045703 25499 consensus_queue.cc:1059] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
W20260709 12:26:27.105742 25588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.007s	sys 0.019s
I20260709 12:26:27.120786 13801 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:27.121749 13644 raft_consensus.cc:1217] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Deduplicated request from leader. Original: 28.16->[29.17-29.17]   Dedup: 29.17->[]
I20260709 12:26:27.126506 13644 raft_consensus.cc:1240] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:26:27.128198 13644 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3504) took 1174 ms. Trace:
I20260709 12:26:27.128760 13644 rpcz_store.cc:276] 0709 12:26:25.953069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.111933 (+1158864us) service_pool.cc:224] Handling call
0709 12:26:27.121602 (+  9669us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.127035 (+  5433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.127327 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:27.131703 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 172070us
I20260709 12:26:27.131811 13647 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54756 (request call id 3497) took 1193 ms. Trace:
I20260709 12:26:27.095171 25603 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.120851 25597 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:27.139727 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:27.140676 13801 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:27.142171 25501 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:27.152334 13644 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:27.157516 13801 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:27.175601 14421 raft_consensus.cc:1240] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:27.176240 25256 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:27.096721 25642 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.042s
W20260709 12:26:27.192801 25643 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:27.194415 25641 consensus_queue.cc:1059] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:27.195629 25641 raft_consensus.cc:3055] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:27.195995 25641 raft_consensus.cc:740] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:27.135597 25623 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
W20260709 12:26:27.198865 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.194962 25638 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:26:27.203182 25636 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:26:27.215957 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5067) took 1314 ms. Trace:
W20260709 12:26:27.218971 25634 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:26:27.219794 25591 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:27.220247 25591 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:27.221246 25591 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.196990 25641 consensus_queue.cc:260] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.216493 13726 rpcz_store.cc:276] 0709 12:26:25.901920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.962104 (+ 60184us) service_pool.cc:224] Handling call
0709 12:26:25.975567 (+ 13463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:25.976187 (+   620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.976232 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:26.027808 (+ 51576us) log.cc:844] Serialized 52 byte log entry
0709 12:26:26.109488 (+ 81680us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:26.138652 (+ 29164us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.139775 (+  1123us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.597896 (+458121us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b1800362bc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:27.211994 (+614098us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800362bc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b07522 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:27.215270 (+  3276us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b1800362bc0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b04455 00007f5393bec060 00007f5393bfe716 00007f5393c0f287
0709 12:26:27.215467 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.215567 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.215631 (+    64us) raft_consensus.cc:1685] finished
0709 12:26:27.215646 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.215839 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":645,"prepare.queue_time_us":44640,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":106,"raft.queue_time_us":48430,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":9239,"spinlock_wait_cycles":64825216,"thread_start_us":73367,"threads_started":2,"wal-append.queue_time_us":147553,"wal-append.run_cpu_time_us":3236,"wal-append.run_wall_time_us":1003258}
I20260709 12:26:27.257283 25603 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:27.266714 25273 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.266861 25545 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:27.267784 25273 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.271015 25273 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:27.267036 25641 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:27.281831 25135 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:27.282579 25135 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:27.283911 25135 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:27.285606 25640 log.cc:927] Time spent T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:26:27.286834 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.296721 14418 raft_consensus.cc:2468] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
W20260709 12:26:27.302726 13693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.304407 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.309186 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:27.285758 25135 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:27.242625 25591 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:27.314410 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:27.316183 13723 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:27.317075 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:27.317289 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:27.317983 13726 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:27.321421 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.316447 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:27.325436 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.327656 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:27.305302 25603 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:27.335997 13693 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:27.321904 25636 log.cc:927] Time spent T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:461: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 12:26:27.338600 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.341917 25626 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:27.343358 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
I20260709 12:26:27.350626 25344 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:27.356573 25593 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:27.319787 25608 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:27.367190 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:27.331784 25621 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:27.371994 14391 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:27.365724 25602 raft_consensus.cc:2764] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:26:27.374396 25535 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:27.377415 25559 raft_consensus.cc:3055] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:26:27.378124 25559 raft_consensus.cc:740] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:27.379134 25559 consensus_queue.cc:260] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.380405 25559 raft_consensus.cc:3060] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:27.380957 13723 raft_consensus.cc:2468] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:27.381668 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:27.334210 25551 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:27.395380 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59487.9us
I20260709 12:26:27.396216 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.397305 13690 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:27.398458 13726 raft_consensus.cc:2468] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:26:27.383167 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:27.391700 13794 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:27.368747 25603 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.392333 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:27.334015 25563 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:27.314509 25592 consensus_queue.cc:1048] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.045s
I20260709 12:26:27.394984 25621 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:27.393247 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:27.400192 13723 raft_consensus.cc:2393] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 32: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:27.360374 25626 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:27.412727 25626 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:27.414927 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60504 (request call id 3514) took 1497 ms. Trace:
I20260709 12:26:27.415354 13798 rpcz_store.cc:276] 0709 12:26:25.917188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.953593 (+ 36405us) service_pool.cc:224] Handling call
0709 12:26:27.414649 (+1461056us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b5400102cd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:27.414842 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553063296}
I20260709 12:26:27.419847 13693 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:27.420876 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.421964 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.354117 25632 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.411180 25621 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:27.408134 25385 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:27.402536 25545 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:27.426101 13724 raft_consensus.cc:2393] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:27.433641 13648 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1852) took 1882 ms. Trace:
I20260709 12:26:27.436966 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:27.438563 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 136265us
I20260709 12:26:27.439711 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.425591 25608 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:27.429028 25551 raft_consensus.cc:2804] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:27.434098 13648 rpcz_store.cc:276] 0709 12:26:25.551076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.714684 (+1163608us) service_pool.cc:224] Handling call
0709 12:26:26.715003 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.722416 (+  7413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.722453 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.731951 (+  9498us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.937300 (+205349us) raft_consensus.cc:1645] Marking committed up to 27
0709 12:26:27.230805 (+293505us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.433271 (+202466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.433370 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.433420 (+    50us) raft_consensus.cc:1685] finished
0709 12:26:27.433432 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.433573 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239397,"prepare.queue_time_us":83863,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":118,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9465,"thread_start_us":416879,"threads_started":2,"wal-append.queue_time_us":299507}
I20260709 12:26:27.435988 25561 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:27.442742 25501 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.466583 14421 raft_consensus.cc:1240] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:27.132406 13647 rpcz_store.cc:276] 0709 12:26:25.938092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.922142 (+984050us) service_pool.cc:224] Handling call
0709 12:26:26.926014 (+  3872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.926313 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:26:26.946081 (+ 19768us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.131368 (+185287us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:27.131432 (+    64us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:27.131500 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.131570 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.131733 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44166,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":28226,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3989,"thread_start_us":131303,"threads_started":2,"wal-append.queue_time_us":178372}
I20260709 12:26:27.430559 25624 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.468477 25572 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:27.453596 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.456084 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.471271 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:27.472810 13617 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.456240 25651 raft_consensus.cc:2804] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:27.452493 25597 raft_consensus.cc:2804] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:27.462208 14425 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:27.463605 25620 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.463690 25603 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:27.501874 25603 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:27.433733 25385 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.450959 25608 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.503050 25608 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:27.503293 25256 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:27.503803 25256 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.504187 25256 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:27.504513 25385 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:27.505280 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:27.466248 25501 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.506040 25642 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:26:27.508427 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.455832 25598 log.cc:927] Time spent T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.333s	user 0.006s	sys 0.000s
I20260709 12:26:27.494839 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:27.495213 25535 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:27.484114 25651 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.531342 25651 raft_consensus.cc:3060] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:27.531615 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:27.532290 14417 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:27.533576 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:27.534574 25499 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:27.535024 25499 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.535363 25499 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:27.535861 14424 raft_consensus.cc:1240] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:27.486596 25282 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:27.499377 25582 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:27.500102 25091 consensus_queue.cc:1059] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:27.500191 13618 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:27.486829 25597 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.486634 25498 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:27.502468 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:27.475080 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:27.510462 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 103958us
I20260709 12:26:27.561630 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:27.506073 14423 raft_consensus.cc:2368] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32.
I20260709 12:26:27.528666 25501 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:27.503062 25070 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
W20260709 12:26:27.509181 25569 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.000s
I20260709 12:26:27.560386 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:27.572211 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 72115.8us
I20260709 12:26:27.572490 25569 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:27.572817 14424 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:27.566138 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:27.575388 25608 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.570775 25642 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:27.571410 25557 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.576355 25569 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:27.577039 25659 log.cc:927] Time spent T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:27.578029 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:27.622787 13799 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:27.624382 13799 raft_consensus.cc:3055] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:27.624831 13799 raft_consensus.cc:740] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:27.580569 25621 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.581480 13792 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 15. (term mismatch)
I20260709 12:26:27.634882 13792 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 14
I20260709 12:26:27.635211 13792 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 28.15
I20260709 12:26:27.635553 13792 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:27.580595 13795 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 35 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:27.587749 25281 raft_consensus.cc:3055] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:27.588964 25632 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:27.560571 25498 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.560847 25597 raft_consensus.cc:3060] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:27.516105 25620 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.592577 14417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:27.570173 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 67794.4us
W20260709 12:26:27.602941 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.613801 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.614945 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.643416 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:27.653172 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57249.4us
I20260709 12:26:27.549571 25256 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.653982 14417 raft_consensus.cc:3060] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:27.674197 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.643004 13799 consensus_queue.cc:260] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 38.28, Last appended by leader: 28, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.654438 13617 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.634208 25563 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.675598 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.676530 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:27.678153 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.683297 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.688387 13799 raft_consensus.cc:3060] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:27.639982 25281 raft_consensus.cc:740] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:27.654552 25498 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:27.693945 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.729476 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 75178us
W20260709 12:26:27.733877 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56445.9us
I20260709 12:26:27.651368 25070 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:26:27.621995 25557 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.747901 13798 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:27.652910 25501 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.081s
I20260709 12:26:27.761819 14423 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:27.738370 25282 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:26:27.770974 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.762600 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:27.774062 13767 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:27.740252 25632 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:27.691246 25563 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.781745 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:27.788503 13726 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:27.819422 13719 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:26:27.768254 25626 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:27.805610 25551 raft_consensus.cc:697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
W20260709 12:26:27.844448 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81956.4us
I20260709 12:26:27.847092 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.800967 25651 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.925630 13764 leader_election.cc:400] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32.
W20260709 12:26:27.925545 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78553.5us
I20260709 12:26:27.956336 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:27.773274 25569 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.818296 25665 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:27.812387 25591 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.041s
I20260709 12:26:27.955328 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:27.978084 13764 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:27.840468 25285 consensus_queue.cc:1048] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.112s
I20260709 12:26:27.961524 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:27.982822 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 194972us
I20260709 12:26:27.927029 25661 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:26:27.988786 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.924913 25608 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:27.998353 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:27.999181 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.000304 14423 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:27.951253 25660 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.000170 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:27.964927 25501 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.953908 25091 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:27.953467 25666 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:28.011548 14417 raft_consensus.cc:2468] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:27.841363 25281 consensus_queue.cc:260] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:27.935742 25499 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:28.012410 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 244503us
I20260709 12:26:28.029175 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:27.993552 25621 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:28.031224 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 253275us
I20260709 12:26:28.029958 14423 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:28.009344 25665 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:28.035564 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.010056 25557 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.038296 13690 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:28.021493 25256 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.020610 25608 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:28.031708 25661 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:28.021270 25620 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.047907 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.034546 25070 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.054296 14417 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 40. Current term is 41. Ops: []
I20260709 12:26:28.015832 25660 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:28.058459 25670 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:26:28.059128 25667 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 12:26:28.060073 25664 log.cc:927] Time spent T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:28.048357 25666 raft_consensus.cc:2804] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:26:28.065140 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 103719us
I20260709 12:26:28.017122 25501 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:28.074023 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.045331 25671 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:28.031942 25091 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.052224 25661 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.051909 25608 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:28.053397 25273 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:28.094605 13693 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:28.085932 25285 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:28.112982 25680 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:28.064842 25070 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.096593 25666 raft_consensus.cc:697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:28.054005 25551 consensus_queue.cc:237] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.104564 25608 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:28.103235 25661 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:28.109916 25285 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.139967 25285 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:28.143002 25651 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.127779 25565 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.118403 25563 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:28.151654 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 204660us
W20260709 12:26:28.155319 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.158109 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.160444 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 391528us
I20260709 12:26:28.120704 25282 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:28.165778 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 79200.7us
I20260709 12:26:28.167465 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.171016 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:28.171717 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.187357 25681 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:28.190599 25070 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.196329 25282 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.205826 25501 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.209609 25624 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.216253 25666 consensus_queue.cc:237] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.223694 25591 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:28.109169 25273 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:28.228410 13764 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.224478 25591 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.232468 25660 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.232527 25685 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:28.233373 25685 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.234148 25626 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.235318 25499 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.236264 25685 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.238664 25632 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:28.240477 25273 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:28.240947 25641 consensus_meta.cc:287] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.929s	user 0.011s	sys 0.003s
I20260709 12:26:28.126379 25676 consensus_queue.cc:1059] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.127s
W20260709 12:26:28.238940 25662 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 12:26:28.249545 25281 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:28.253489 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 82608.6us
I20260709 12:26:28.254346 13767 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:28.247377 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.252823 25256 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:28.260751 25256 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.264207 25273 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:26:28.264071 25681 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:26:28.267016 25597 consensus_meta.cc:287] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 12:26:28.267499 25597 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:28.270664 25498 consensus_meta.cc:287] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.003s
I20260709 12:26:28.271150 25498 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:28.276180 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.234148 25591 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.293705 13727 raft_consensus.cc:2468] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:26:28.293959 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 122349us
I20260709 12:26:28.275597 25685 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.304268 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.307371 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.309341 25091 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.309037 25685 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.310034 25687 raft_consensus.cc:3055] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:28.310189 25597 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.310518 25687 raft_consensus.cc:740] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:28.311517 25687 consensus_queue.cc:260] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.312866 25687 raft_consensus.cc:3060] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:28.300375 25676 raft_consensus.cc:2764] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:26:28.319049 25285 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.321548 25285 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:28.323429 25665 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:28.325157 25608 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.327679 25641 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:28.330646 13767 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:28.331106 25653 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.944s	user 0.002s	sys 0.004s
I20260709 12:26:28.308657 25680 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:28.334335 25684 consensus_queue.cc:1059] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:28.335453 25671 raft_consensus.cc:2804] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:28.336038 25671 raft_consensus.cc:697] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:28.336181 25559 raft_consensus.cc:3055] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:28.336633 25559 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:28.337060 25671 consensus_queue.cc:237] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 36.21, Last appended by leader: 21, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.337474 25567 raft_consensus.cc:3055] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:28.338020 25567 raft_consensus.cc:740] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:28.338583 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:28.337607 25559 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.339562 25665 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:28.340071 25665 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:28.340533 25665 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:28.340883 25639 log.cc:927] Time spent T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 1.246s	user 0.000s	sys 0.004s
I20260709 12:26:28.339731 25559 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:28.338935 25567 consensus_queue.cc:260] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.343690 25567 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:28.346974 13721 raft_consensus.cc:1275] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:28.347506 13725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5094) took 1613 ms. Trace:
I20260709 12:26:28.347880 13725 rpcz_store.cc:276] 0709 12:26:26.733751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.734015 (+   264us) service_pool.cc:224] Handling call
0709 12:26:26.741908 (+  7893us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.833240 (+ 91332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.833273 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.837690 (+  4417us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.930659 (+ 92969us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:26.930718 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.930815 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.340038 (+1409223us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:28.347176 (+  7138us) raft_consensus.cc:1685] finished
0709 12:26:28.347197 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.347431 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52192,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"spinlock_wait_cycles":5134080,"thread_start_us":169813,"threads_started":2,"wal-append.queue_time_us":183915}
W20260709 12:26:28.352376 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.353322 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:28.353781 14423 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:28.355258 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:28.355620 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.360662 14422 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:28.368774 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.369984 13720 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:28.372175 13721 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:26:28.354156 25561 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:28.373970 25561 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:28.374352 14417 raft_consensus.cc:1240] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:28.374396 25561 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:28.375562 14424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.376804 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.380412 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.381203 25685 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.382414 25608 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.382548 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.383210 25661 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.390244 13794 raft_consensus.cc:1240] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:28.392077 13795 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:26:28.393762 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.394503 14419 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:28.398568 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.399542 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.400141 13723 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:28.402463 25642 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:28.404809 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.405423 13719 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:28.406507 13720 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:28.407603 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.412585 14422 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 39.
I20260709 12:26:28.412668 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:28.414053 14419 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 39: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:28.420578 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.425345 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.385749 25385 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.424532 14390 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.430054 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.431003 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.431702 13725 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:28.389772 25685 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.394131 25661 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.433743 14390 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:28.437719 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.438081 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.434623 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.441622 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.442564 13720 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:28.453279 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.427224 25642 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:28.456959 13723 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:28.464011 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.452598 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:28.460731 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.467476 13718 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 25: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:28.462664 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 37 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.469266 13615 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:28.442418 25472 raft_consensus.cc:2804] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:28.471794 25569 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:28.471935 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.472318 25569 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.472674 13796 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:28.472775 25569 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:28.474422 25651 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:28.475044 25651 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 39 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:28.476223 25651 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 37.26, Last appended by leader: 26, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:28.476930 25668 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.446s	user 0.003s	sys 0.000s
W20260709 12:26:28.478955 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.479979 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.480822 13795 raft_consensus.cc:2368] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:28.480916 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1868) took 1083 ms. Trace:
I20260709 12:26:28.481341 13798 rpcz_store.cc:276] 0709 12:26:27.397761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.416364 (+ 18603us) service_pool.cc:224] Handling call
0709 12:26:27.416739 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.748542 (+331803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.480624 (+732082us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54002cbfd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:28.480847 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730644736}
I20260709 12:26:28.482211 13796 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 38 index: 20. Preceding OpId from leader: term: 40 index: 21. (index mismatch)
W20260709 12:26:28.483189 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.478879 14421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1822) took 1004 ms. Trace:
I20260709 12:26:28.484931 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.486071 13793 raft_consensus.cc:2393] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 39: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:28.427841 25281 consensus_queue.cc:1059] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 38.26, Next index: 27, Last known committed idx: 26, Time since last communication: 0.029s
I20260709 12:26:28.452153 25565 raft_consensus.cc:2804] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:28.500427 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:28.454732 25641 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.500485 13799 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 36 index: 19. (index mismatch)
I20260709 12:26:28.484680 14421 rpcz_store.cc:276] 0709 12:26:27.474201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.486716 (+ 12515us) service_pool.cc:224] Handling call
0709 12:26:27.495732 (+  9016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.495987 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.496014 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.496626 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.784511 (+287885us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:27.784552 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.784642 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.476299 (+691657us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:28.478582 (+  2283us) raft_consensus.cc:1685] finished
0709 12:26:28.478606 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.478815 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25833728,"thread_start_us":287406,"threads_started":1,"wal-append.queue_time_us":471004}
I20260709 12:26:28.506896 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.508144 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.513484 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1866) took 1606 ms. Trace:
I20260709 12:26:28.524091 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.524945 13723 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
W20260709 12:26:28.518448 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 61690.3us
I20260709 12:26:28.528105 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.435468 25385 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.535427 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.536815 13793 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:28.538183 14420 raft_consensus.cc:1240] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 LEADER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 38. Current term is 39. Ops: []
I20260709 12:26:28.488029 25680 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.541126 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.465926 25529 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.067s
I20260709 12:26:28.547026 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:28.455821 25685 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.556509 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.558127 13723 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:28.554872 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.559026 13725 raft_consensus.cc:2393] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 37: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:28.559494 13723 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:28.560346 13725 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:28.561451 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.561937 13721 raft_consensus.cc:1240] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:28.562105 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.562575 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.536634 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:28.563223 13725 raft_consensus.cc:1240] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 36. Current term is 37. Ops: []
I20260709 12:26:28.518173 25472 raft_consensus.cc:697] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:28.569140 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:28.573424 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.574941 13720 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:28.536149 25557 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:28.583715 13718 raft_consensus.cc:1240] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:28.537714 25565 raft_consensus.cc:697] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:28.582858 13796 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:28.562796 13798 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:28.589816 13649 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60832 (request call id 1849) took 3040 ms. Trace:
I20260709 12:26:28.483366 25642 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:28.550810 25587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.007s	sys 0.032s
I20260709 12:26:28.593096 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.593801 13792 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:28.588376 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 255704us
I20260709 12:26:28.598361 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:28.590307 13649 rpcz_store.cc:276] 0709 12:26:25.550041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.689277 (+1139236us) service_pool.cc:224] Handling call
0709 12:26:26.689684 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.740665 (+ 50981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.740714 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.289892 (+549178us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b18002b27c0. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a
0709 12:26:27.291280 (+  1388us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b2c00027328. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:26:27.292021 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.331358 (+ 39337us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:26:27.373283 (+ 41925us) log.cc:844] Serialized 14 byte log entry
0709 12:26:27.438660 (+ 65377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.589482 (+1150822us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b5400095e54. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:28.589546 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.589597 (+    51us) raft_consensus.cc:1685] finished
0709 12:26:28.589613 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.589752 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78000,"prepare.queue_time_us":39160,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":80346,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":14811,"spinlock_wait_cycles":117202944,"thread_start_us":38880,"threads_started":1,"wal-append.queue_time_us":134148}
I20260709 12:26:28.528954 25641 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:28.605793 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.606624 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.586148 25682 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.594990 25587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.206s	user 0.007s	sys 0.033s
I20260709 12:26:28.580097 25501 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:26:28.643695 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:28.583499 25608 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 40.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.028s
I20260709 12:26:28.609869 25685 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.608269 25557 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.646013 13795 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
I20260709 12:26:28.555440 25680 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.607990 25535 consensus_queue.cc:1048] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.044s
I20260709 12:26:28.643359 25281 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:28.650717 25661 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.648130 25684 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:28.650570 25597 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.677914 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 172127us
I20260709 12:26:28.633873 25682 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.684180 13719 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
I20260709 12:26:28.727901 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.693710 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 41.
W20260709 12:26:28.729465 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 203097us
I20260709 12:26:28.723953 25530 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:28.785635 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 153390us
I20260709 12:26:28.724841 25608 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:28.734411 25551 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:28.709956 25661 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.676919 25472 consensus_queue.cc:237] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:28.788408 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60607.1us
I20260709 12:26:28.801265 13693 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:28.680457 25557 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.740550 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:28.710330 25684 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.722275 25281 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.694425 25565 consensus_queue.cc:237] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.836071 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:28.794184 25530 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.733740 25597 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.826484 25551 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:28.852464 25557 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.853336 25557 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.842209 25385 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.858682 25680 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:28.859261 25693 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:26:28.861306 25689 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:26:28.861724 25281 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.809356 25665 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.863991 25281 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:28.865051 25665 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.865543 25641 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.868772 13724 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:28.872058 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.874612 25642 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:28.877660 25561 consensus_meta.cc:287] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
I20260709 12:26:28.800275 25608 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.883663 13722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34464 (request call id 5108) took 1194 ms. Trace:
I20260709 12:26:28.882071 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.872052 25682 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:28.884049 13722 rpcz_store.cc:276] 0709 12:26:27.689175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.204810 (+515635us) service_pool.cc:224] Handling call
0709 12:26:28.205092 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.205299 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:26:28.513737 (+308438us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.883264 (+369527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:28.883324 (+    60us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:28.883377 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.883449 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.883597 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":453928,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2510,"thread_start_us":651487,"threads_started":2}
I20260709 12:26:28.897581 13796 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:26:28.898554 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 62598.2us
I20260709 12:26:28.899574 13693 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:28.899922 25692 log.cc:927] Time spent T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:26:28.880558 25608 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:28.905560 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.908905 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.894788 25680 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:28.853933 25557 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.912459 14418 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:28.912607 14423 raft_consensus.cc:1275] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:26:28.878232 25561 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.916145 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:28.911334 25641 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:28.918337 25651 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.928917 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:28.920506 25597 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:28.932669 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:28.933657 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:28.933501 13767 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:28.934738 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:28.937915 13767 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:28.938575 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:28.947434 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:28.948423 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:28.934762 25705 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.948002 25676 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:26:28.956224 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:28.925782 25651 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:28.946806 25682 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:28.952144 25641 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:26:28.970197 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 229741us
W20260709 12:26:28.972627 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 445336us
I20260709 12:26:28.961472 25661 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:28.966543 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:28.971412 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:28.976699 13767 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:28.978436 13693 leader_election.cc:400] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:28.978875 13693 leader_election.cc:403] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:26:28.980952 25665 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:28.977559 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:28.980093 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:28.975383 25569 consensus_meta.cc:287] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.005s
I20260709 12:26:28.956099 25705 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:28.979890 25680 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:29.001147 25676 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
W20260709 12:26:29.000694 13767 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.004920 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:29.005546 13693 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.005643 14419 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:29.006201 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:29.006327 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:29.006763 25714 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:29.008244 13693 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.007891 25641 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:29.008808 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:29.009405 25597 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:29.012795 25705 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:28.986128 25499 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.018s
I20260709 12:26:29.010130 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:29.063236 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 53190.1us
I20260709 12:26:29.064169 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.013693 25608 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:29.067619 14423 raft_consensus.cc:1275] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
W20260709 12:26:29.027355 25710 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:26:29.027187 25530 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:29.030356 25694 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.235s	user 0.003s	sys 0.000s
I20260709 12:26:28.999881 25569 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.077224 14425 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48736 (request call id 1823) took 1568 ms. Trace:
I20260709 12:26:29.031973 25591 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:29.035930 25674 raft_consensus.cc:3055] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 LEADER]: Stepping down as leader of term 38
I20260709 12:26:29.004678 25561 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.010890 25597 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:29.039784 25557 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:29.068418 25705 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:29.049130 14420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48720 (request call id 2141) took 1589 ms. Trace:
I20260709 12:26:29.049594 25665 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.051957 25651 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.092540 14420 rpcz_store.cc:276] 0709 12:26:27.459394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.467734 (+  8340us) service_pool.cc:224] Handling call
0709 12:26:27.468037 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.538706 (+1070669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.048854 (+510148us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400664254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:29.049062 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2460612480}
I20260709 12:26:29.052037 25498 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:29.053141 25641 consensus_queue.cc:1048] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:26:29.057156 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:29.097854 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.059566 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.100057 14418 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 41: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:26:29.060614 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.060957 13723 raft_consensus.cc:1240] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 29. Current term is 30. Ops: []
W20260709 12:26:29.102257 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.064854 13693 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:29.013888 13767 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.103718 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.104140 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.104929 13793 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:29.104857 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:29.068719 14419 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:29.070995 25091 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:29.077644 14425 rpcz_store.cc:276] 0709 12:26:27.509282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.509466 (+   184us) service_pool.cc:224] Handling call
0709 12:26:27.525509 (+ 16043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.525905 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.525937 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:28.274123 (+748186us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b2c000d57a8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53924eda11 00007f53924fd372 00007f5393c2fa01 00007f5393bf443a 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2
0709 12:26:28.274756 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:26:28.545254 (+270498us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:28.545295 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.545364 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.029574 (+484210us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:29.076970 (+ 47396us) raft_consensus.cc:1685] finished
0709 12:26:29.076995 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.077155 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1618,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":97,"spinlock_wait_cycles":71618688,"thread_start_us":269976,"threads_started":1,"wal-append.queue_time_us":410732}
I20260709 12:26:29.098994 14423 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:29.113837 25569 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.118541 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:29.122083 14416 raft_consensus.cc:1240] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:28.521349 13800 rpcz_store.cc:276] 0709 12:26:26.907379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.925687 (+ 18308us) service_pool.cc:224] Handling call
0709 12:26:26.937300 (+ 11613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.937543 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:26:27.121598 (+184055us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.330525 (+1208927us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:28.478971 (+148446us) spinlock_profiling.cc:243] Waited 9.47 ms on lock 0x7b1800434860. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f539379c4bc 00007f5390aea48b 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f539379423a
0709 12:26:28.513106 (+ 34135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:28.513193 (+    87us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:26:28.513244 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.513313 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.513427 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8899,"prepare.queue_time_us":263947,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":9108,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":20687,"spinlock_wait_cycles":39049344,"thread_start_us":352455,"threads_started":2,"wal-append.queue_time_us":265340}
I20260709 12:26:29.082059 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.047948 25666 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 LEADER]: Stepping down as leader of term 40
W20260709 12:26:29.108673 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 51608.2us
I20260709 12:26:29.107472 13615 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:29.087038 25674 raft_consensus.cc:740] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:29.126868 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.127354 13797 pending_rounds.cc:85] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 27
I20260709 12:26:29.131822 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.132035 13794 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:29.095731 25597 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:29.071060 25603 consensus_queue.cc:1048] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.039s
I20260709 12:26:29.131089 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.132171 13797 pending_rounds.cc:107] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 38.28
I20260709 12:26:29.160392 13797 raft_consensus.cc:2889] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:29.091876 25673 raft_consensus.cc:3055] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Stepping down as leader of term 34
W20260709 12:26:29.134645 13727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50270 (request call id 2455) took 1835 ms (client timeout 1709 ms). Trace:
W20260709 12:26:29.139008 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45237.8us
W20260709 12:26:29.126166 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44274.4us
I20260709 12:26:29.127732 25666 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:29.146054 14421 raft_consensus.cc:1240] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:26:29.147535 25674 consensus_queue.cc:260] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 38.26, Last appended by leader: 26, Current term: 38, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.116145 25608 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:29.121191 25572 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:29.183337 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.188017 13795 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
W20260709 12:26:29.185590 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 120759us
I20260709 12:26:29.096405 25501 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:26:29.133761 25498 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:29.211454 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.109841 25665 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:29.189970 13693 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.191340 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:29.219843 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:29.220156 14425 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:29.162685 25385 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:29.212525 13794 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:29.219010 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.091305 25591 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.160586 25499 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election won for term 35
W20260709 12:26:29.183290 13727 rpcz_store.cc:269] 0709 12:26:27.299501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.308936 (+  9435us) service_pool.cc:224] Handling call
0709 12:26:29.134370 (+1825434us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b54000f5854. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:29.134521 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144523648}
I20260709 12:26:29.249595 13800 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 41: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:29.163151 25673 raft_consensus.cc:740] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:29.264349 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74410.4us
I20260709 12:26:29.265018 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:29.265338 13693 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.265810 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:29.264052 13798 raft_consensus.cc:2468] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
W20260709 12:26:29.273191 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 269205us
I20260709 12:26:29.223261 25572 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:29.261795 25674 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:29.258867 25608 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:29.284854 13618 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:29.281901 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 91378.8us
I20260709 12:26:29.300009 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:29.300902 13794 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:29.307760 13693 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.308902 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.252064 25091 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 12:26:29.276314 25641 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:29.308538 25651 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.319327 25256 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:29.351266 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 220200us
W20260709 12:26:29.354689 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73277.4us
I20260709 12:26:29.296348 25557 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:26:29.325026 25499 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:29.366434 14425 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:29.334367 25608 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:29.359895 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:29.377434 25641 raft_consensus.cc:493] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:29.393533 25641 raft_consensus.cc:515] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.396148 25641 leader_election.cc:290] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.397411 14425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:29.400266 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:29.400918 13767 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.389298 25671 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:29.401866 13767 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.401793 25671 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:29.403053 13767 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.403496 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.404168 25671 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.406023 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.407256 25501 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:29.409428 25591 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.411468 13794 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:29.421082 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.389003 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:29.444525 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.362939 25256 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.450965 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 91146.4us
I20260709 12:26:29.343787 25712 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 12:26:29.386693 25559 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.297587 25666 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 40.21, Last appended by leader: 21, Current term: 40, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.360344 25651 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:29.327512 25673 consensus_queue.cc:260] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.443495 25641 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:29.502374 25499 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:29.490396 25728 log.cc:927] Time spent T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:26:29.493099 14420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.525419 25557 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:29.493538 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.471346 25737 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:29.442883 13724 raft_consensus.cc:2376] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30. Re-sending same reply.
I20260709 12:26:29.484828 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.481774 25723 consensus_queue.cc:1059] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.047s
I20260709 12:26:29.514242 14418 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 37 index: 26. Preceding OpId from leader: term: 39 index: 27. (index mismatch)
I20260709 12:26:29.457010 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:29.479756 25717 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:26:29.465257 25256 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:29.474918 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 71905.3us
I20260709 12:26:29.484416 25665 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:29.500115 25684 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:29.492764 13615 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:29.557072 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 64391.2us
W20260709 12:26:29.502874 25719 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:26:29.526113 25557 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:29.538398 13767 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.465068 25686 raft_consensus.cc:2764] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election decision vote started in defunct term 36: won
I20260709 12:26:29.528113 14390 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:29.574836 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46816.9us
W20260709 12:26:29.575249 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 274325us
I20260709 12:26:29.523286 13725 raft_consensus.cc:1275] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
W20260709 12:26:29.576288 13690 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.534072 25651 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:29.475970 25671 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:26:29.570948 13767 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.583958 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.544909 13801 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 32 index: 23. (term mismatch)
W20260709 12:26:29.572072 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 287263us
I20260709 12:26:29.536337 25666 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:29.567085 25712 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:29.529649 25723 raft_consensus.cc:3055] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:29.452622 25572 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.586490 13801 pending_rounds.cc:85] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 22
I20260709 12:26:29.597026 13801 pending_rounds.cc:107] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 31.23
I20260709 12:26:29.597440 13801 raft_consensus.cc:2889] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:29.586076 13767 leader_election.cc:336] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.586966 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.461495 25680 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:29.582553 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.530562 25708 consensus_queue.cc:1059] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.049s
I20260709 12:26:29.528334 25559 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:29.526623 25737 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:29.592654 25684 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:29.599931 13767 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.618901 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.564379 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.580536 25530 raft_consensus.cc:2804] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:26:29.623762 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 59519us
I20260709 12:26:29.627503 25530 raft_consensus.cc:697] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:29.627799 25559 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:29.611635 25712 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:29.628655 25530 consensus_queue.cc:237] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.594640 25723 raft_consensus.cc:740] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:29.621241 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.633131 13725 raft_consensus.cc:1275] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:29.514101 25641 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:29.588850 25626 consensus_queue.cc:1048] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.028s
W20260709 12:26:29.641426 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55427.3us
I20260709 12:26:29.642951 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:29.644009 13767 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.644495 13767 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:29.646967 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:29.617058 25737 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:29.647819 13766 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:29.648770 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 227769us
W20260709 12:26:29.647576 13767 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.651311 13719 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:29.624241 25741 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:29.643062 25567 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:29.643464 25498 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:26:29.646512 25070 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:29.651469 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:29.653774 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:29.629738 25712 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.642871 25559 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:29.642791 25685 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:29.650175 25684 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:26:29.661500 13767 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.664008 13767 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.645355 25565 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.015s
I20260709 12:26:29.614799 25557 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:29.693660 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.658248 25498 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:29.698347 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55818.9us
I20260709 12:26:29.702116 13618 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:29.700012 13718 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:26:29.700601 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.658403 25736 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:29.710052 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.665510 25551 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:29.705924 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44471.2us
I20260709 12:26:29.714135 13721 raft_consensus.cc:2410] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:26:29.658432 25603 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.040s
W20260709 12:26:29.718477 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 65607.3us
I20260709 12:26:29.699162 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:29.716176 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 102183us
I20260709 12:26:29.711110 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.698400 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.716579 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.719514 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 41 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.721673 13719 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:26:29.719699 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56003.1us
W20260709 12:26:29.702827 25597 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.002s
I20260709 12:26:29.657123 25651 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.045s
I20260709 12:26:29.723003 13719 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:26:29.721036 13718 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:29.696558 25665 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.705053 25498 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:29.701289 25256 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.724642 25557 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:29.722424 25551 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.705430 25741 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.731865 13725 raft_consensus.cc:2393] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
W20260709 12:26:29.537142 25722 consensus_queue.cc:1175] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a4188193a10c4d50a555d76ea85c60b7" responder_term: 40 status { last_received { term: 38 index: 20 } last_committed_idx: 20 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 38 index: 20. Preceding OpId from leader: term: 40 index: 21. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:29.657300 25723 consensus_queue.cc:260] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:29.779207 25735 log.cc:927] Time spent T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:26:29.718520 25684 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.714921 25591 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:29.672009 25559 consensus_queue.cc:1059] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 36.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.050s
I20260709 12:26:29.744251 25597 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:29.782192 25641 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:29.785660 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:29.697893 25282 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:29.708437 25091 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.718011 25748 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
W20260709 12:26:29.787597 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 85576.8us
I20260709 12:26:29.788244 25742 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:29.788864 25714 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:26:29.789610 13615 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.790112 25501 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.709630 25608 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:29.712211 25561 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.817366 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:29.663401 25385 consensus_queue.cc:1059] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 12:26:29.870601 25665 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.840955 25472 consensus_queue.cc:1048] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:26:29.857788 25070 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:29.874639 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:29.881381 13693 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:29.937574 13720 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
W20260709 12:26:29.937920 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 58552.7us
W20260709 12:26:29.909806 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124205us
I20260709 12:26:29.884748 25686 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.885443 25745 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:29.917421 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:29.891577 25666 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:29.840126 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 49770.3us
I20260709 12:26:29.944746 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:29.880730 25551 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:29.938757 13693 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:29.994742 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.994786 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56062.4us
I20260709 12:26:29.924106 25687 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:26:29.999233 13693 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.004199 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:30.005036 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60405.4us
I20260709 12:26:30.005792 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:30.010070 13618 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:30.007532 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.893349 25684 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.025249 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:29.932057 25723 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:29.992624 25591 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:30.026583 13618 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:29.916150 25282 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.047494 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.048465 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dcbdced6d0a46b394cae3d101fbbf40" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.049068 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.049793 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.050669 13796 raft_consensus.cc:2410] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for term 37 because replica has last-logged OpId of term: 36 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 33 index: 21.
I20260709 12:26:29.919189 25091 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.932552 25501 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:29.930403 25608 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.056632 25736 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:30.058362 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:30.060426 25582 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.062011 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:30.062674 25674 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:30.063027 25665 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:29.950383 25686 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.063848 25684 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.127400 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.128494 13718 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:30.064216 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:29.926023 25742 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:29.936501 25561 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.015249 25676 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.021724 25714 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:30.018601 25499 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:30.098521 13720 raft_consensus.cc:1275] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 36 index: 21. Preceding OpId from leader: term: 38 index: 22. (index mismatch)
I20260709 12:26:30.154277 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:29.993311 25745 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:30.119419 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 57509.7us
I20260709 12:26:30.089301 13726 raft_consensus.cc:2410] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 41 because replica has last-logged OpId of term: 40 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 38 index: 20.
I20260709 12:26:30.069543 14423 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:29.888240 25741 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.106791 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.029265 25687 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:30.035521 25651 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:29.992379 25666 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.040153 25591 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.150987 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 139841us
I20260709 12:26:30.068915 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.084093 25282 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.138614 25685 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:26:30.177146 14391 leader_election.cc:336] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.020913 25729 consensus_queue.cc:1059] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.122s
I20260709 12:26:30.095945 25673 raft_consensus.cc:3060] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:30.092468 25724 raft_consensus.cc:3055] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:30.066129 25567 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.155603 25714 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.095940 25748 raft_consensus.cc:3055] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:30.134600 25736 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.170848 13722 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 34.
W20260709 12:26:30.180912 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.161897 13618 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:30.158914 25499 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.168136 25651 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.202567 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:30.209497 25608 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.209616 25561 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.211659 25551 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.200630 25748 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:30.205039 13767 consensus_peers.cc:597] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.207000 25282 raft_consensus.cc:2804] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:30.202316 25714 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:26:30.216816 25759 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:26:30.179286 25708 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.038s
I20260709 12:26:30.141934 25582 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.179430 25591 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.184172 25724 raft_consensus.cc:740] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:30.265928 25724 consensus_queue.cc:260] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 36.19, Last appended by leader: 19, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.267733 25724 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:30.277350 25597 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.278615 25748 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.280280 25748 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:30.231616 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 517221us
I20260709 12:26:30.202325 25736 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:30.282285 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:30.282971 13767 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.220012 25499 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:30.254413 25687 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.255565 25576 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:30.237179 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 56487.7us
W20260709 12:26:30.295610 14391 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:30.296761 14391 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.297199 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:30.159665 25676 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.223637 25651 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:30.300396 25676 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:30.301587 25756 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:30.303277 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.303723 25501 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:30.304466 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:30.305094 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.305444 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 39 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:30.311249 25091 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.236496 25282 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:30.312270 25282 raft_consensus.cc:3060] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:30.314527 25756 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader election lost for term 39. Reason: could not achieve majority
I20260709 12:26:30.322620 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:30.321880 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:30.240579 25737 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.003s
W20260709 12:26:30.340476 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:30.241515 25760 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 12:26:30.247882 25666 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.232371 25559 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:30.258370 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.340983 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:30.306005 25741 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:30.343781 25732 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 12:26:30.344008 25737 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.357685 25737 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.358916 25745 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.363526 25686 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:30.371415 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:30.343070 25768 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.052s	user 0.001s	sys 0.003s
W20260709 12:26:30.342566 25752 log.cc:927] Time spent T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.412s	user 0.009s	sys 0.002s
W20260709 12:26:30.342795 25754 log.cc:927] Time spent T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.447s	user 0.007s	sys 0.004s
I20260709 12:26:30.353438 25499 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.360337 25742 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.398329 25684 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:30.398890 25684 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.400288 25499 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.381301 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:30.401980 13792 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:30.402539 25651 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.401059 25684 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:30.389173 25498 consensus_meta.cc:287] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
W20260709 12:26:30.392403 25764 log.cc:927] Time spent T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.208s	user 0.002s	sys 0.000s
I20260709 12:26:30.392601 14422 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:26:30.369663 25714 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.396819 25326 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.408309 25326 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.408622 25498 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.410842 25498 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.411424 25326 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:30.394853 25686 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:30.412549 25686 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.414870 25686 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:30.414788 25282 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.414635 25714 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.420324 25727 raft_consensus.cc:3055] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:30.420742 25727 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:30.421070 25282 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.417507 25498 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.005s
W20260709 12:26:30.394866 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.430794 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.431636 13794 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:26:30.432623 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:30.434084 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.434790 13799 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:30.435854 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:30.436866 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 489835us
I20260709 12:26:30.438444 25722 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.439013 25722 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.441222 25722 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.421691 25727 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.443907 25727 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:30.446652 25762 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:30.447261 25762 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.448909 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:30.449473 25762 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.450886 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:30.459321 13727 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:30.434986 13792 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 37 index: 26. Preceding OpId from leader: term: 39 index: 27. (index mismatch)
I20260709 12:26:30.456809 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:30.474011 14391 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.478204 13798 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:26:30.414031 25551 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.450011 25762 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:30.506039 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 506789us
I20260709 12:26:30.479507 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.474639 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:30.503190 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:30.514463 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.514727 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:30.404760 25651 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.526611 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.527137 13795 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 35 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
I20260709 12:26:30.520522 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.474740 25686 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:30.523562 13727 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:26:30.515446 13721 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
W20260709 12:26:30.526209 13693 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:30.564798 13693 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.565426 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:30.522794 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.567013 13719 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:30.530083 13797 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:30.527467 13798 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
W20260709 12:26:30.524757 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59183.3us
I20260709 12:26:30.477295 25326 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:26:30.525856 25624 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader election lost for term 41. Reason: could not achieve majority
I20260709 12:26:30.515234 25385 consensus_queue.cc:1048] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.046s
I20260709 12:26:30.509298 25722 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.626746 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.629783 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.559767 25729 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:30.630710 25729 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:30.630770 13725 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:30.631189 25729 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:30.562091 25762 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.636531 25762 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:30.639923 13677 heartbeater.cc:646] Failed to heartbeat to 127.12.45.126:44051 (7 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.12.45.126:44051 timed out after 15.000s (SENT)
I20260709 12:26:30.572408 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.479982 25608 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:30.573839 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:30.575263 13693 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.648984 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:30.649273 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 74079.2us
W20260709 12:26:30.649920 13693 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:30.650861 13693 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.651328 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:30.652454 13693 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.389246 25736 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:30.587060 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.584218 25686 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.523178 25626 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:30.571923 25674 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:30.606923 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:30.657269 13799 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:30.607445 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:30.608122 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:30.659049 13792 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:30.610621 25665 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:30.640654 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:30.660030 13795 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:30.642418 25673 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:30.555457 25591 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.647549 25582 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.649655 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 25.
W20260709 12:26:30.654376 13693 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.580514 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:30.657570 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:30.677302 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:30.578552 25326 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.677996 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:30.667907 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:30.672968 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 388916us
I20260709 12:26:30.637658 25755 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:30.712735 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:30.690526 13797 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:26:30.715073 13798 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:30.571280 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:30.716034 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 144844us
W20260709 12:26:30.717103 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.719910 13792 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:30.680017 13618 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:30.702777 13693 leader_election.cc:304] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:30.717654 14391 leader_election.cc:336] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.722061 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:30.657325 25651 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.723755 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:30.722970 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:30.721843 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.735836 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:30.764145 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:30.691879 25576 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:26:30.771843 13693 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:30.772541 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92585us
I20260709 12:26:30.769982 13799 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:30.714499 25674 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.664424 25762 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election lost for term 37. Reason: could not achieve majority
I20260709 12:26:30.722692 25682 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:30.723845 25687 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:30.755262 25665 raft_consensus.cc:2749] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:30.698401 25326 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:26:30.773638 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51891.1us
I20260709 12:26:30.773641 25651 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.785780 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:30.758994 25666 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.777292 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44271.8us
I20260709 12:26:30.666834 25608 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.675768 25673 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.724659 25748 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:30.666071 25626 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.692447 25498 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.083s
I20260709 12:26:30.723203 25282 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
W20260709 12:26:30.790359 13767 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 88 similar messages]
I20260709 12:26:30.733357 25705 raft_consensus.cc:3060] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:30.771440 25722 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:30.754451 25686 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.775163 25559 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:30.791359 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:30.774561 25603 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:30.778007 25582 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:26:30.810096 25724 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.542s	user 0.011s	sys 0.001s
I20260709 12:26:30.829471 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.827529 13618 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:30.806989 25737 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:30.787818 25687 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.784384 25682 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.834079 13795 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:30.806555 25736 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:30.792861 25741 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.026s
I20260709 12:26:30.836862 13722 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:26:30.865563 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:30.866394 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.868541 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.789440 25530 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:26:30.816195 25472 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:30.819693 25559 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:30.820644 25582 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.879093 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51661.7us
I20260709 12:26:30.886020 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:30.883275 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.887511 13720 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:30.889380 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:30.890166 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.890314 14423 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 39.
W20260709 12:26:30.891750 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 177635us
I20260709 12:26:30.893962 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:30.902561 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:30.903523 13720 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:30.903609 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:30.867403 25682 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.867172 25687 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:30.904493 13727 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:30.861780 25722 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:30.823439 25642 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
I20260709 12:26:30.817484 25748 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.922034 25736 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.922721 25736 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.923875 25748 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.867672 25561 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:30.801188 25666 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:30.926939 25736 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.889577 25559 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:30.889396 25582 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:30.933313 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:30.915207 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 34 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:30.933706 13767 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:30.916814 25499 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:30.919348 25608 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.889865 25714 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:30.909775 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:30.943102 13719 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:26:30.943271 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:30.910796 25742 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:30.948148 13727 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
W20260709 12:26:30.936762 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.872990 25091 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.041s
I20260709 12:26:30.899204 25651 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:30.937743 25499 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.967139 13793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.012s	sys 0.015s
W20260709 12:26:30.950531 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.970577 13720 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:30.942703 25714 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.949846 25742 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:30.970956 13793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.012s	sys 0.015s
I20260709 12:26:30.978935 25742 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:30.962177 25591 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:30.979753 25591 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.980304 25591 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:30.982409 25714 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:30.985102 13719 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
W20260709 12:26:30.983892 13693 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:30.968035 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 250380us
W20260709 12:26:30.995225 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:30.999723 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.967382 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 254535us
I20260709 12:26:30.994052 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:31.023070 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:30.973495 25499 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:31.030781 25626 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:31.028344 25722 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:31.052366 25666 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:31.064072 25763 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:31.067781 25673 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:31.085422 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 192826us
I20260709 12:26:31.049496 25472 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:26:31.046164 25729 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:31.103704 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 328178us
W20260709 12:26:31.099072 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 198314us
I20260709 12:26:31.095410 25591 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:31.105813 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.106678 13618 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.107213 13792 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:31.113446 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:31.114372 25782 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:31.114590 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:31.115415 14423 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 39: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:31.117172 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:31.125007 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:31.127719 25788 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:31.128832 25779 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:26:31.138279 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:31.155181 13723 raft_consensus.cc:1275] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
W20260709 12:26:31.145792 25777 log.cc:927] Time spent T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 12:26:31.148008 25775 log.cc:927] Time spent T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 12:26:31.184121 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77537.2us
I20260709 12:26:31.185087 13720 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
I20260709 12:26:31.147375 25674 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:31.147536 25748 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:26:31.167419 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 124569us
I20260709 12:26:31.186522 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:31.191229 13766 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.193881 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:31.156131 25091 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.195410 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:31.202770 13723 raft_consensus.cc:2393] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:31.205020 13792 raft_consensus.cc:1240] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 36. Current term is 37. Ops: [34.18-36.19]
I20260709 12:26:31.202503 13724 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:26:31.207518 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:31.147303 25687 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.208792 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.213882 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.214218 13792 raft_consensus.cc:2368] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:31.197183 25748 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.221940 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.196066 25736 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:31.156452 25326 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.197480 25796 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:31.226238 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:31.216425 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.230523 25736 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.235419 25729 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:31.196400 25624 consensus_queue.cc:1048] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.030s
I20260709 12:26:31.237912 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:31.217268 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 76112.5us
I20260709 12:26:31.240411 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.241470 13792 raft_consensus.cc:2393] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:31.242199 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:31.244804 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.246248 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.247854 13797 raft_consensus.cc:2393] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
W20260709 12:26:31.248468 13767 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.228470 25748 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:31.253443 13618 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.254767 25687 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:31.255052 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:31.255932 13618 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:31.256856 13618 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:31.259725 25795 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:31.260516 25795 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:31.262583 25651 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:31.262841 25591 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:31.263219 25651 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:31.263386 25591 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:31.264279 25651 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.269037 25748 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.264326 25591 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 34.18, Last appended by leader: 18, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.271042 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:31.270866 25498 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:31.291189 25498 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:31.291652 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:31.292945 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.274348 25785 consensus_queue.cc:1059] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 34.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:26:31.274847 13693 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.276366 25742 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.261957 25795 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:31.284353 13767 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:31.288368 25326 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:31.210111 25091 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:31.318600 13693 leader_election.cc:336] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.312260 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.319403 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.274120 25705 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:31.322034 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73581.8us
I20260709 12:26:31.233393 25736 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:31.324339 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:31.321640 13693 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.327332 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:31.326294 13767 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:31.329344 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:31.337400 13693 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:31.338335 13693 leader_election.cc:336] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.346246 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:31.347106 13724 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:31.302918 25498 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:31.350934 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:31.351410 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 42 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.352718 25742 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:31.352909 13798 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 41.
I20260709 12:26:31.356389 25714 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.324991 25797 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:31.326061 25729 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:31.361615 13617 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:31.339161 13693 leader_election.cc:336] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:31.372784 13693 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.326521 25682 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.258108 25559 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.369405 25674 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:31.327296 25795 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:31.371073 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 114303us
W20260709 12:26:31.377487 13767 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.383103 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:31.377502 13693 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.327500 25673 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:31.384320 13767 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.389606 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:31.393532 13721 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:31.390112 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:31.394904 13720 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:31.391616 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:31.406787 13801 raft_consensus.cc:2368] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29.
I20260709 12:26:31.409092 13693 leader_election.cc:400] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:31.409475 13693 leader_election.cc:403] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:31.380030 25651 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:31.425523 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:31.419245 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:31.421275 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:31.427764 13727 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:26:31.428999 13722 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:31.431923 13693 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.436267 13792 raft_consensus.cc:1240] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 33. Current term is 34. Ops: [33.19-33.19]
I20260709 12:26:31.437136 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:31.437808 13793 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:31.438992 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.446728 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:31.447461 13794 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:31.430346 13767 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.458199 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.459040 13798 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:31.463245 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.385545 25557 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:31.456058 13767 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:31.394418 25708 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.490057 25572 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Stepping down as leader of term 30
W20260709 12:26:31.534839 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 103037us
I20260709 12:26:31.493572 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.542402 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.404455 25501 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.419935 25742 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.503425 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:31.502801 13690 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:31.534737 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 71582.1us
I20260709 12:26:31.490706 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 42 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:31.379794 25582 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.407856 25687 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.442157 25737 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:31.549108 13794 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:31.556070 13720 raft_consensus.cc:2468] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
W20260709 12:26:31.573417 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 143136us
I20260709 12:26:31.500310 25784 raft_consensus.cc:3055] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:26:31.563277 14422 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 41.
I20260709 12:26:31.426445 25603 consensus_queue.cc:1059] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.119s
I20260709 12:26:31.469774 25666 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:31.619987 13719 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:31.475261 25790 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.104s
W20260709 12:26:31.689353 13767 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.541791 25608 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:31.534587 25572 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:31.501415 25557 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.551298 25561 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
W20260709 12:26:31.718768 13767 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.545629 25708 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.719635 13722 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
W20260709 12:26:31.784838 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 97597.2us
W20260709 12:26:31.921581 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 202843us
W20260709 12:26:31.870859 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 368572us
W20260709 12:26:31.871287 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 296630us
W20260709 12:26:31.914026 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 371713us
I20260709 12:26:31.931730 13693 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:31.923816 13767 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.927462 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:31.939178 13766 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.560052 25651 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:31.937529 13767 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:31.947988 13793 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
W20260709 12:26:31.948333 13727 consensus_meta.cc:287] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.002s
I20260709 12:26:31.953543 13793 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:31.953640 13727 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:26:31.954501 13767 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.955118 13767 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:31.956106 25379 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:31.956933 25379 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:26:31.958886 13767 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:31.959431 13767 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:31.585316 25736 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:31.686784 25722 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:26:31.961759 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 471694us
W20260709 12:26:31.820077 25779 log.cc:927] Time spent T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:26:31.767872 25809 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.170s	user 0.002s	sys 0.000s
W20260709 12:26:31.750031 25799 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.355s	user 0.000s	sys 0.000s
I20260709 12:26:31.950359 25686 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:31.982515 12469 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:31.584842 25742 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:31.964444 14419 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.001s	sys 0.019s
I20260709 12:26:31.685429 25784 raft_consensus.cc:740] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:31.752770 25714 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:31.851646 25673 consensus_queue.cc:1059] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 32.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.152s
W20260709 12:26:32.013967 14419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.001s	sys 0.019s
I20260709 12:26:31.728686 25737 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:31.756662 25091 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:31.757484 25682 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:31.739631 25748 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:31.720331 25805 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:31.933638 25803 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:31.950524 25729 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:31.962936 13693 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:31.949143 25559 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:31.949321 25811 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:31.961625 13618 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:31.577906 25501 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:31.685741 25687 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:32.028416 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 87820us
I20260709 12:26:31.561312 25795 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:31.972822 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:32.028875 25814 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:32.106711 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 137127us
W20260709 12:26:32.010099 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51237.5us
I20260709 12:26:32.052613 25608 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:32.015220 25724 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:32.011417 25686 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:32.063140 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 100299us
I20260709 12:26:32.120445 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:32.066794 25790 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.947285 25561 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.090520 13766 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:32.060199 25714 raft_consensus.cc:3055] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:26:32.132140 25714 raft_consensus.cc:740] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:32.132370 25742 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.133069 25714 consensus_queue.cc:260] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.19, Last appended by leader: 19, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.117586 25820 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:32.134828 25714 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:32.135341 25790 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.078903 25803 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.103180 25796 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.138514 25796 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.109946 25811 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.142134 25708 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.115649 25814 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.143652 25687 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.144809 25557 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.077351 25651 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.113008 13794 raft_consensus.cc:1240] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 30. Current term is 31. Ops: []
W20260709 12:26:32.079742 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 118196us
I20260709 12:26:32.156845 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.157862 13793 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:26:32.159322 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.160599 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.161295 13793 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:26:32.090349 25671 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:32.124547 25686 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:32.130383 25501 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.055956 25572 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.130631 25561 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:32.116015 13767 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:32.166954 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51124.1us
W20260709 12:26:32.164589 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74221.7us
I20260709 12:26:32.081063 25818 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:32.162483 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:32.172346 13794 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 29: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:32.120942 25385 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.178678 13798 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 37 index: 19. (index mismatch)
I20260709 12:26:32.178350 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.178719 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.184463 13618 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:32.195135 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 74802.2us
I20260709 12:26:32.198660 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:32.068202 25805 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.164247 25559 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.164813 25820 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:32.216624 25820 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.167886 25708 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.219030 25708 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.205060 25723 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.168336 25608 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:32.221856 25708 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.179319 25682 raft_consensus.cc:2804] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:32.213688 25818 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:32.207293 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:32.196869 25763 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:32.194214 25794 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.248682 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:32.238305 25561 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:32.275490 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:32.198664 25385 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.191913 25565 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.058s
I20260709 12:26:32.268277 25682 raft_consensus.cc:697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:32.222024 25723 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.249531 25379 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.342604 25379 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:32.335016 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 86462.6us
I20260709 12:26:32.269375 25608 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:32.342454 25682 consensus_queue.cc:237] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.345937 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:32.346604 25723 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.237352 25559 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.347114 25608 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 41 FOLLOWER]: Leader pre-election won for term 42
I20260709 12:26:32.347702 25608 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 41 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.350353 13693 leader_election.cc:400] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29.
I20260709 12:26:32.351013 13693 leader_election.cc:403] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:32.348286 25608 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:32.362553 25559 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.363292 25826 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.364323 25836 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:32.365192 25836 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.367923 25741 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:32.368407 25741 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.368768 25741 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:32.362751 25821 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:26:32.371759 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:32.276401 25091 consensus_queue.cc:1059] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.076s
I20260709 12:26:32.373286 13617 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:32.373811 25091 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:32.374249 25091 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.374711 25091 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:32.373415 25608 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.319159 25673 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.376956 25673 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.379062 25673 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.374585 25708 raft_consensus.cc:3055] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:32.379585 25673 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:32.380067 25708 raft_consensus.cc:740] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:32.279681 25501 consensus_queue.cc:1059] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.080s
W20260709 12:26:32.353317 25830 log.cc:927] Time spent T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:26:32.362639 25818 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:32.414671 25385 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:32.419399 25708 consensus_queue.cc:260] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:32.428510 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 153107us
I20260709 12:26:32.422441 25814 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:32.428169 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.363867 25826 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.398584 25737 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.365686 25836 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.423676 25091 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:32.434216 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 121649us
I20260709 12:26:32.426844 25795 raft_consensus.cc:3055] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:32.397668 25608 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 42 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.440402 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.442350 25826 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.442741 25790 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:32.443038 25499 consensus_meta.cc:287] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.378s	user 0.015s	sys 0.000s
I20260709 12:26:32.448082 25836 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.481153 25841 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.481765 25841 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.482211 25499 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.449805 25379 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:32.450901 25091 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.436460 25501 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.015s
W20260709 12:26:32.431154 25603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.024s	sys 0.037s
I20260709 12:26:32.460809 25741 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.467063 25805 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.428644 25818 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.476869 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 42 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:32.278980 25794 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.478389 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:32.448753 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.485953 25708 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:32.486575 25499 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.486723 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.453851 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.490545 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.491972 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:32.435475 25803 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.493366 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 38 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:32.494742 14418 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:32.478189 25795 raft_consensus.cc:740] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:32.492569 25603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.025s	sys 0.038s
I20260709 12:26:32.509717 25724 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:32.510232 25686 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.511482 13801 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:32.513754 25686 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:32.514469 13798 raft_consensus.cc:1240] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 29. Current term is 30. Ops: []
I20260709 12:26:32.497596 25839 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.515301 25795 consensus_queue.cc:260] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.515686 25839 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.516418 25795 raft_consensus.cc:3060] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:32.505036 14416 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:32.499258 25737 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.508379 25841 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.496282 25608 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:32.494750 25561 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:32.530591 14418 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 42.
I20260709 12:26:32.530555 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:32.534962 13797 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:32.529700 13801 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 40.
I20260709 12:26:32.532802 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:32.523557 25807 log.cc:927] Time spent T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.754s	user 0.003s	sys 0.000s
I20260709 12:26:32.538611 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.539675 13795 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:32.522290 25741 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.523836 25794 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.522969 25737 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.537580 25686 raft_consensus.cc:493] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.550536 25686 raft_consensus.cc:515] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.551375 25737 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.518217 25839 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:32.552641 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.559796 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.592267 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 81311.1us
I20260709 12:26:32.563835 25784 consensus_queue.cc:260] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.639276 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.631402 14416 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 40.
I20260709 12:26:32.634027 25671 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.644131 13615 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:32.648540 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:32.650224 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.653067 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.653872 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 40 candidate_status { last_received { term: 39 index: 28 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.655737 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 42 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:32.655977 13726 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:26:32.656188 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.656348 13795 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:32.535018 25608 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.665730 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.637766 25818 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.673528 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.676350 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.680738 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.656978 25379 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.642199 25565 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.686131 13797 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:32.655669 25671 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:32.696599 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:32.698097 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:32.707659 13767 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:32.692552 13795 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 42 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 42.
I20260709 12:26:32.659894 25820 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:32.656541 25785 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:32.708761 25820 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 40 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:32.709107 13767 leader_election.cc:336] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:32.709297 25785 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.709762 25820 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 38.26, Last appended by leader: 26, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.712019 25785 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.656669 25737 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:32.712473 13767 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:32.713034 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:32.712767 25737 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.662585 25784 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:32.693229 25818 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.716631 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.717681 13797 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:32.718991 13794 raft_consensus.cc:1240] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:26:32.714290 25603 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.699059 25814 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.704198 25708 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:32.698661 25565 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.698945 25379 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:32.739038 25812 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 1.135s	user 0.009s	sys 0.002s
I20260709 12:26:32.742453 25839 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.753681 25839 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.754588 25843 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.755302 25843 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.721599 25741 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.757782 25843 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.758030 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.757550 25741 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.752148 25379 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.761124 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.763166 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.694651 25561 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.766817 13792 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5218) took 1322 ms. Trace:
I20260709 12:26:32.769870 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.719455 25818 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.773010 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.767397 13792 rpcz_store.cc:276] 0709 12:26:31.444441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.444567 (+   126us) service_pool.cc:224] Handling call
0709 12:26:31.444887 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:31.475328 (+ 30441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.475369 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:31.488791 (+ 13422us) log.cc:844] Serialized 26 byte log entry
0709 12:26:31.604909 (+116118us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:32.035560 (+430651us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.060488 (+ 24928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.060571 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.738171 (+677600us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:32.761571 (+ 23400us) raft_consensus.cc:1685] finished
0709 12:26:32.766551 (+  4980us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.766748 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69147,"prepare.queue_time_us":544411,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":4652,"raft.queue_time_us":90961,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":52071,"spinlock_wait_cycles":184176000,"thread_start_us":540526,"threads_started":2,"wal-append.queue_time_us":115092}
I20260709 12:26:32.773468 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:32.728919 25682 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:26:32.775198 13793 raft_consensus.cc:1240] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 34. Current term is 35. Ops: []
I20260709 12:26:32.725590 25603 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.758579 25839 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.731871 25814 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.770773 25843 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:32.779295 25686 leader_election.cc:290] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.817339 25565 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.817929 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:32.821871 13618 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:32.832741 25824 log.cc:927] Time spent T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.543s	user 0.001s	sys 0.000s
W20260709 12:26:32.845103 25848 log.cc:927] Time spent T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:32.849854 25785 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:32.850613 25785 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:32.851398 25829 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.575s	user 0.002s	sys 0.000s
I20260709 12:26:32.853888 13723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50270 (request call id 2501) took 1254 ms. Trace:
I20260709 12:26:32.854307 13723 rpcz_store.cc:276] 0709 12:26:31.599942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.600176 (+   234us) service_pool.cc:224] Handling call
0709 12:26:31.600675 (+   499us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:31.625569 (+ 24894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.625621 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:31.717969 (+ 92348us) log.cc:844] Serialized 52 byte log entry
0709 12:26:32.155533 (+437564us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:26:32.227379 (+ 71846us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.228023 (+   644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.228203 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.850555 (+622352us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f539378f86e 00007f539378ebb6 00007f539378e7d8 00007f5393798221 00007f53937981d9 00007f5393798169
0709 12:26:32.853549 (+  2994us) raft_consensus.cc:1685] finished
0709 12:26:32.853665 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.853820 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24322,"prepare.queue_time_us":45773,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":1763,"raft.queue_time_us":162693,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":7419,"spinlock_wait_cycles":8356096,"thread_start_us":355930,"threads_started":1,"wal-append.queue_time_us":510850}
I20260709 12:26:32.856684 25785 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:32.849959 25642 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:32.857885 25642 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:32.858853 25642 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.864768 25582 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.865502 25821 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 28. Current term is 29.
I20260709 12:26:32.869480 25741 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.871726 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:32.872434 13721 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:26:32.873368 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.873950 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.875605 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.877094 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.877766 13719 raft_consensus.cc:2468] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:32.876201 13722 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:32.877797 13792 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
W20260709 12:26:32.881253 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.874177 13725 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 25.
I20260709 12:26:32.823942 25673 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:32.887179 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.890228 13725 raft_consensus.cc:2468] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:32.887713 13723 raft_consensus.cc:3055] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Stepping down as leader of term 36
I20260709 12:26:32.891222 13723 raft_consensus.cc:740] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:32.892226 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.892877 13719 raft_consensus.cc:2468] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:32.893100 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.893756 13725 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:32.876317 13726 raft_consensus.cc:2468] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
I20260709 12:26:32.892099 13723 consensus_queue.cc:260] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 36.19, Last appended by leader: 19, Current term: 36, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.896792 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.897341 13723 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:32.897496 13727 raft_consensus.cc:2468] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
W20260709 12:26:32.899366 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.896978 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:32.901328 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:32.893770 25673 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.905069 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:32.905838 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.804733 25843 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:32.885272 25821 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.914499 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.915174 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.915841 13724 raft_consensus.cc:2468] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:32.915271 13725 raft_consensus.cc:2468] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:32.924072 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:32.924959 13727 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:32.909356 25673 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:32.897742 25839 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:32.930605 14388 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.907711 25789 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:32.928454 13721 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:32.937171 13723 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 37 index: 19. (term mismatch)
I20260709 12:26:32.939380 13723 pending_rounds.cc:85] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 18
I20260709 12:26:32.939711 13723 pending_rounds.cc:107] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 36.19
I20260709 12:26:32.942066 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.937463 25836 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:32.951529 25836 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:32.955374 25849 log.cc:927] Time spent T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:26:32.958478 25813 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:32.959573 25813 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:32.962356 25846 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.231s	user 0.000s	sys 0.002s
I20260709 12:26:32.970130 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.971158 25813 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:32.978979 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.979164 25814 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.982360 25843 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.983258 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:32.984107 13796 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 27: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:32.922969 25821 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:32.944314 25737 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:32.987596 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:32.988266 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.940049 13723 raft_consensus.cc:2889] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.995070 13725 raft_consensus.cc:2410] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 for term 33 because replica has last-logged OpId of term: 31 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 18.
I20260709 12:26:32.994513 13726 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:32.992465 13724 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 18. (term mismatch)
I20260709 12:26:32.996433 13724 pending_rounds.cc:85] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Aborting all ops after (but not including) 17
I20260709 12:26:32.991195 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:32.989782 13793 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:26:32.996860 13724 pending_rounds.cc:107] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:26:32.997851 13724 raft_consensus.cc:2889] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.994094 14419 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
W20260709 12:26:32.989266 12469 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:32.999220 13721 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:33.001180 13792 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:33.009609 13793 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:33.012080 13727 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:33.015635 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.016403 13722 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:33.017998 13721 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 26.
W20260709 12:26:33.015209 25666 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 37 current term: 37 is higher.
I20260709 12:26:33.025241 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:32.989421 25821 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:32.939930 25839 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.013816 25813 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:33.014580 25805 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:26:33.046976 13800 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5216) took 1610 ms. Trace:
I20260709 12:26:33.047796 13800 rpcz_store.cc:276] 0709 12:26:31.436730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.442596 (+  5866us) service_pool.cc:224] Handling call
0709 12:26:31.443082 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.125072 (+681990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.125138 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.125893 (+   755us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.272655 (+146762us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:26:32.272722 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.046447 (+773725us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54000d2fd4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:33.046511 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.046575 (+    64us) raft_consensus.cc:1685] finished
0709 12:26:33.046599 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.046903 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14257,"raft.queue_time_us":14289,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":18106,"spinlock_wait_cycles":1538631424,"thread_start_us":146321,"threads_started":1,"wal-append.queue_time_us":300672}
I20260709 12:26:33.027462 25714 consensus_queue.cc:1059] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 12:26:33.035997 25836 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.042044 25651 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 12:26:33.043454 25813 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.072438 25714 raft_consensus.cc:3055] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:26:33.080729 25714 raft_consensus.cc:740] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:33.086781 25714 consensus_queue.cc:260] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.081609 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:33.082983 25811 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:33.076471 25845 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.356s	user 0.002s	sys 0.000s
I20260709 12:26:33.090442 25682 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:33.090941 25682 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:33.091673 25682 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:33.096567 13693 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.097128 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.100519 25821 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:33.101047 25821 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.102001 25737 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:33.111778 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:33.114598 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:33.115501 13727 raft_consensus.cc:2393] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:33.115958 25811 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.116505 25811 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:33.119467 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.120254 13693 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.074558 25839 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.132764 25714 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:33.138999 25821 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.140226 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.140962 13727 raft_consensus.cc:2468] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 27.
I20260709 12:26:33.135339 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:33.136193 25686 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:33.142858 25686 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:33.059760 25795 consensus_meta.cc:287] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.008s
I20260709 12:26:33.043198 25671 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.146853 13799 raft_consensus.cc:1240] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 30. Current term is 31. Ops: []
I20260709 12:26:33.148805 25671 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.149489 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:33.138193 25811 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.152118 13721 raft_consensus.cc:1240] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 30. Current term is 31. Ops: [30.15-30.15]
I20260709 12:26:33.151315 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:33.172377 25806 log.cc:927] Time spent T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 1.559s	user 0.006s	sys 0.014s
I20260709 12:26:33.064905 25741 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.109s
I20260709 12:26:33.181798 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:33.182618 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.182904 13727 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 29: Already voted for candidate 4dafe08a4f404502822a0732fe570de4 in this term.
I20260709 12:26:33.186965 25572 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:33.194481 25741 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.201491 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.205711 13727 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:33.208678 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.209719 13727 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:33.213717 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.214730 13727 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
W20260709 12:26:33.219668 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.290196 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.290987 13727 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:26:33.306136 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.307974 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:33.308329 13616 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.310837 13767 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.316663 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.317996 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.319559 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.320796 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:33.211309 25741 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.058504 25642 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.230s
W20260709 12:26:33.320410 13767 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:33.327993 13767 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:33.329888 13767 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.327405 25742 raft_consensus.cc:2804] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:33.331683 13767 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.330005 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.323092 25561 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.323664 25844 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:33.355948 13767 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.357303 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ce599ec23ec4ac0b358f4e47922bc3f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.367369 25796 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:33.371846 13767 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.353173 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.355006 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 38 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.339440 25742 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:33.346395 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.391325 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.399070 13792 raft_consensus.cc:2393] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 34: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:33.343995 25671 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:33.358016 25818 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.357158 25708 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.357725 25794 raft_consensus.cc:2804] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:26:33.402689 13767 leader_election.cc:336] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.413230 13767 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:33.417251 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 71044us
I20260709 12:26:33.418443 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.418730 14423 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 38 index: 26. Preceding OpId from leader: term: 40 index: 27. (index mismatch)
I20260709 12:26:33.401949 25742 raft_consensus.cc:3060] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:33.423072 13767 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.425899 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.359974 25561 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.427670 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.414880 25794 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.441542 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.420352 25805 raft_consensus.cc:2804] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:33.419871 25671 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:33.420184 25722 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:33.463661 13693 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 42 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.452793 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:33.471472 13767 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.445529 25582 raft_consensus.cc:2804] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:33.437947 25642 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.454495 25673 consensus_meta.cc:287] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.003s
I20260709 12:26:33.416826 25708 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.452322 25794 raft_consensus.cc:3060] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:33.491297 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.462396 25805 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:33.467638 25811 raft_consensus.cc:2804] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:33.450681 25796 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.044s
I20260709 12:26:33.484727 25582 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:33.494457 25682 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.509259 25834 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:26:33.514844 25857 log.cc:927] Time spent T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:33.505023 25811 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.493731 25498 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:33.493898 25642 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.526202 25708 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.527393 13798 raft_consensus.cc:1240] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 35. Current term is 36. Ops: []
I20260709 12:26:33.504652 25805 raft_consensus.cc:3060] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:33.495046 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.525147 25582 raft_consensus.cc:3060] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:33.527457 25561 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.486708 25722 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.490501 25673 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:33.558288 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 63416.6us
I20260709 12:26:33.568677 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.544651 25811 raft_consensus.cc:3060] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:33.570201 13617 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.545239 25498 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:33.550498 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 73369us
W20260709 12:26:33.553886 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.528046 25741 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.577731 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:33.579334 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.580024 13797 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:33.545357 25818 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:33.565330 25834 raft_consensus.cc:2804] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:33.583851 13617 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:33.587188 25861 log.cc:927] Time spent T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:33.587670 25796 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:33.588619 25796 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:33.590009 25796 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.589017 25741 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:33.593122 25608 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.593304 25741 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:33.594345 25741 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.585536 25834 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:33.596022 25834 raft_consensus.cc:3060] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:33.579023 25797 raft_consensus.cc:2804] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:33.598918 25797 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:33.599398 25797 raft_consensus.cc:3060] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:33.599886 25708 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:33.568452 25603 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.601794 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:33.577821 25498 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:33.582557 25866 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:33.582334 25561 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.604878 25844 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.006s
I20260709 12:26:33.634130 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:33.631505 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 131736us
I20260709 12:26:33.610330 13617 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.645375 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:33.651221 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.619565 25642 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.656894 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:33.611831 25722 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.647872 25795 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:33.669796 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 59625.6us
I20260709 12:26:33.602440 25818 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.648041 25561 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:33.684850 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104544us
I20260709 12:26:33.672523 25741 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:33.686975 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:33.686751 13766 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:33.700373 13693 leader_election.cc:336] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.700910 13693 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.701196 13794 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5237) took 1211 ms. Trace:
I20260709 12:26:33.701633 13794 rpcz_store.cc:276] 0709 12:26:32.489639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.489813 (+   174us) service_pool.cc:224] Handling call
0709 12:26:32.490194 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.719466 (+229272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.700916 (+981450us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54000e60d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:33.701120 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525418368}
I20260709 12:26:33.707800 25722 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:26:33.708725 25872 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:33.711354 25795 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:33.715371 25818 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.717170 25561 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.718158 25673 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.718408 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:33.703203 25789 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.720747 13617 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.721016 25789 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.722373 14416 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:33.723595 25789 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.725127 25742 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.726145 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.727285 13721 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 34.
I20260709 12:26:33.727975 25742 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:33.745337 25572 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.007s
I20260709 12:26:33.741576 25866 consensus_queue.cc:1059] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
W20260709 12:26:33.749380 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:33.750571 13693 leader_election.cc:336] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.751713 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.699445 25741 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:33.759516 14390 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:33.751000 13693 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:33.763373 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 261) took 18346 ms (client timeout 15000 ms). Trace:
I20260709 12:26:33.765280 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.766130 13797 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 35.
I20260709 12:26:33.746994 25804 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.776034 13767 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.777431 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.778640 25796 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.780028 13801 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:33.780086 25796 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.781333 25642 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.781826 25642 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.782651 25796 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.783875 25642 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.778739 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:33.784919 25871 raft_consensus.cc:2804] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:33.785363 25871 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.785671 13796 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:33.785704 25871 raft_consensus.cc:3060] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:33.786226 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.807925 13800 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:33.815047 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.763110 25722 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:33.816190 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83932.3us
I20260709 12:26:33.816433 13799 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1914) took 1037 ms. Trace:
I20260709 12:26:33.765435 25561 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:33.818353 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:33.825037 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:33.824769 13799 rpcz_store.cc:276] 0709 12:26:32.778902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.791763 (+ 12861us) service_pool.cc:224] Handling call
0709 12:26:32.800530 (+  8767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.147272 (+346742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.816149 (+668877us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54000d34d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:33.816371 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796015360}
I20260709 12:26:33.825611 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:33.826467 13800 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:33.770234 25870 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:33.806815 25804 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:33.839545 14418 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:33.841909 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:33.842406 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.843708 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:33.843097 13792 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 34.
I20260709 12:26:33.829466 25870 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.851646 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:33.853953 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:33.859244 25844 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:33.862248 25557 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:26:33.864004 13767 leader_election.cc:336] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.866014 25805 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:33.871569 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92942.8us
I20260709 12:26:33.874378 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:33.867144 25671 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:33.875591 25671 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:33.876029 25671 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:33.876266 25805 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:33.883981 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.884469 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.870736 13792 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
I20260709 12:26:33.885080 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.887044 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:33.888545 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:33.896775 13723 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:26:33.899353 13724 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:33.842703 25603 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.843519 25866 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:33.903568 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 36 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:33.904788 13725 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:33.907788 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.908042 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:33.908767 13726 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 33.
I20260709 12:26:33.909461 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.901705 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:33.914606 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.915995 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:33.917095 25603 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.920176 25603 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.911448 25834 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:33.926518 25834 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:33.937166 13725 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
I20260709 12:26:33.833076 25582 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:33.763862 13547 rpcz_store.cc:269] 0709 12:26:15.345766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.346281 (+   515us) service_pool.cc:224] Handling call
0709 12:26:33.691961 (+18345680us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:26.014788 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:26.015568 (+   780us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:26.015635 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:26:26.015679 (+    44us) tablet.cc:662] Decoding operations
0709 12:26:26.018154 (+  2475us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:26.037109 (+ 18955us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:26.039071 (+  1962us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:26:26.048262 (+  9191us) tablet.cc:701] Row locks acquired
0709 12:26:26.048325 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:26:26.048555 (+   230us) write_op.cc:270] Start()
0709 12:26:26.048739 (+   184us) write_op.cc:276] Timestamp: P: 1783599986048463 usec, L: 0
0709 12:26:26.048781 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:26:26.115561 (+ 66780us) log.cc:844] Serialized 11264 byte log entry
0709 12:26:26.919181 (+803620us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390a2dc85 00007f5390a2e6a2 00007f53937a01db 00007f539379baaf 00007f5390aea0ac 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b
0709 12:26:27.731376 (+812195us) op_driver.cc:464] REPLICATION: finished
0709 12:26:28.164509 (+433133us) write_op.cc:301] APPLY: starting
0709 12:26:28.194323 (+ 29814us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:28.376719 (+182396us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:28.376763 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:29.500715 (+1123952us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:29.509744 (+  9029us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=0,mrs_lookups=0
0709 12:26:29.509828 (+    84us) write_op.cc:312] APPLY: finished
0709 12:26:29.572814 (+ 62986us) log.cc:844] Serialized 252 byte log entry
0709 12:26:29.792771 (+219957us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:29.821508 (+ 28737us) write_op.cc:454] Released schema lock
0709 12:26:29.885875 (+ 64367us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:29.936309 (+ 50434us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:29.940077 (+  3768us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":190,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":368894,"apply.run_cpu_time_us":67622,"apply.run_wall_time_us":1777485,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125579,"num_ops":29,"prepare.queue_time_us":319486,"prepare.run_cpu_time_us":36911,"prepare.run_wall_time_us":1119627,"raft.queue_time_us":495366,"raft.run_cpu_time_us":12274,"raft.run_wall_time_us":350357,"replication_time_us":1636819,"spinlock_wait_cycles":32263040,"thread_start_us":1793464,"threads_started":5,"wal-append.queue_time_us":1279705,"wal-append.run_cpu_time_us":90489,"wal-append.run_wall_time_us":2346503}]]}
I20260709 12:26:33.943667 13721 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:26:33.946954 13793 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 38 index: 26. Preceding OpId from leader: term: 40 index: 27. (index mismatch)
I20260709 12:26:33.947507 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:33.949123 25796 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:33.905148 25875 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:33.891278 25870 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:33.953433 14418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48720 (request call id 2174) took 1190 ms. Trace:
I20260709 12:26:33.910790 25642 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:33.910588 25724 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:33.987179 13800 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 38.
W20260709 12:26:33.995421 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:34.007722 13767 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:33.998950 12469 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:34.000998 14418 rpcz_store.cc:276] 0709 12:26:32.762730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.762902 (+   172us) service_pool.cc:224] Handling call
0709 12:26:33.953209 (+1190307us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400600254. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf82fa 00007f5398bb9b17 00007f53921346d4 00007f5392134662 00007f5392134591 00007f539213450c 00007f53921337c2 00007f5391f2f48c 00007f5391f2e8c6 00007f5391f2e26f
0709 12:26:33.953339 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466544896}
W20260709 12:26:34.003641 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 113944us
I20260709 12:26:34.009811 13766 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:33.904942 25794 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:34.008554 13767 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:33.849364 25741 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.011076 13618 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:34.024120 13766 leader_election.cc:304] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.027275 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:33.992333 25796 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.014945 25642 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.000452 25875 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:34.005723 25877 raft_consensus.cc:2764] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election decision vote started in defunct term 37: won
I20260709 12:26:34.015218 25724 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:34.058995 25878 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:34.038730 25796 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:34.064144 25498 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.066747 25736 raft_consensus.cc:2749] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:34.067575 25736 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:34.067973 25736 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:34.068356 25736 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:34.026285 25326 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:34.070286 25686 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.071058 25724 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:34.049729 25642 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:34.078248 25813 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:34.080581 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.089488 25872 raft_consensus.cc:3055] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:34.090065 25872 raft_consensus.cc:740] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:34.090971 25872 consensus_queue.cc:260] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.15, Last appended by leader: 15, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.040738 25834 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader election won for term 38
W20260709 12:26:34.106146 13693 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:34.018146 25868 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 27, Last known committed idx: 26, Time since last communication: 0.062s
I20260709 12:26:34.112810 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.049630 25875 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:34.098345 25603 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.114219 25862 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:26:34.126662 25722 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:34.125290 25834 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 38 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:34.214852 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 68337us
I20260709 12:26:34.140909 25603 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.229683 25794 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:34.285008 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 235814us
I20260709 12:26:34.147934 13690 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:34.395516 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 371288us
I20260709 12:26:34.456892 25498 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.403208 25818 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.462942 25818 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.372908 25722 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.387413 25603 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:34.308007 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.328366 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 108316us
W20260709 12:26:34.305575 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 278208us
W20260709 12:26:34.419322 25883 log.cc:927] Time spent T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 12:26:34.419891 25871 consensus_meta.cc:287] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.006s
I20260709 12:26:34.476475 25818 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:34.423275 25887 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 12:26:34.424871 25888 log.cc:927] Time spent T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:26:34.409755 25561 raft_consensus.cc:493] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:34.481498 25561 raft_consensus.cc:515] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.485343 25561 leader_election.cc:290] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.486310 25682 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.447481 25714 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.488014 25714 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.448628 13798 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60492 (request call id 5254) took 1269 ms. Trace:
I20260709 12:26:34.453673 25794 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:34.475960 25871 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.239481 25834 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 36.19, Last appended by leader: 19, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.510481 25722 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.512311 25871 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.494381 13798 rpcz_store.cc:276] 0709 12:26:33.178670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.180563 (+  1893us) service_pool.cc:224] Handling call
0709 12:26:33.181026 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.549376 (+368350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.447862 (+898486us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54003c0354. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:34.448300 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795925376}
I20260709 12:26:34.527868 25565 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:34.509655 25794 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.540220 25871 raft_consensus.cc:493] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.551688 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:34.513990 25722 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.565599 25603 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:26:34.566308 25603 raft_consensus.cc:493] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:34.566773 25603 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:34.567387 25565 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.568015 25871 raft_consensus.cc:515] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:34.570326 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 354883us
I20260709 12:26:34.573937 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:34.575469 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:34.555543 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
W20260709 12:26:34.579785 25811 consensus_meta.cc:287] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.000s	user 0.010s	sys 0.017s
I20260709 12:26:34.580447 25811 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.584690 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.585542 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.586161 13798 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:34.587026 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.584730 25722 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:34.588976 25565 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:34.589251 13799 raft_consensus.cc:2393] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:34.589921 25813 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:26:34.591024 25813 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.574306 25871 leader_election.cc:290] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.617661 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:34.619611 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 44594.3us
I20260709 12:26:34.626533 25714 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.636282 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:34.637389 25642 consensus_meta.cc:287] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.562s	user 0.017s	sys 0.008s
I20260709 12:26:34.617717 25804 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.638016 25642 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.645777 25642 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.650470 25818 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:34.651574 25818 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.657452 25872 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:34.658092 13800 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:34.662406 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 42 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.663269 13799 raft_consensus.cc:2376] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 42 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 42. Re-sending same reply.
I20260709 12:26:34.664196 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.664378 13793 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:26:34.665755 25724 consensus_meta.cc:287] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.594s	user 0.016s	sys 0.015s
I20260709 12:26:34.666314 25724 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:34.671015 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.678515 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:34.680250 25582 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:34.682619 25796 consensus_meta.cc:287] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 0.574s	user 0.005s	sys 0.022s
I20260709 12:26:34.683188 25796 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:34.667989 25869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.003s	sys 0.019s
I20260709 12:26:34.641338 25811 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.712544 25722 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:34.713277 25722 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:34.714433 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55700.1us
I20260709 12:26:34.717005 25813 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:34.718389 25870 consensus_meta.cc:287] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent flushing consensus metadata: real 0.701s	user 0.014s	sys 0.010s
I20260709 12:26:34.713986 25722 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:34.718973 25870 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.721783 25900 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:34.722688 25900 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:34.724381 25736 consensus_meta.cc:287] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.655s	user 0.006s	sys 0.022s
I20260709 12:26:34.724933 25736 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.697523 25818 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.726070 13794 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:34.728050 25724 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:34.688011 25603 raft_consensus.cc:515] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.686125 25796 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.730883 25603 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:34.733793 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 60921us
W20260709 12:26:34.734498 25875 consensus_meta.cc:287] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.543s	user 0.015s	sys 0.010s
I20260709 12:26:34.734723 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.735455 13693 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.736792 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.737696 13793 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:34.725970 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.740449 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.740628 25897 consensus_queue.cc:1059] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:34.742208 25785 raft_consensus.cc:3055] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:26:34.742820 25785 raft_consensus.cc:740] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:34.744225 25785 consensus_queue.cc:260] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.744654 25642 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:34.745867 25642 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.746477 13764 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.748112 25818 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:34.748279 13764 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.748739 25642 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.749053 25714 consensus_queue.cc:1059] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
W20260709 12:26:34.755541 25671 consensus_meta.cc:287] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: Time spent flushing consensus metadata: real 0.879s	user 0.017s	sys 0.009s
I20260709 12:26:34.735190 25875 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.725219 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.756103 25671 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:34.746696 25797 consensus_meta.cc:287] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: Time spent flushing consensus metadata: real 1.147s	user 0.014s	sys 0.015s
I20260709 12:26:34.749289 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:34.757459 13795 raft_consensus.cc:1240] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Rejecting Update request from peer b708d8f5f6ac4aabb1eb5cb821ace195 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:34.751308 25815 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:34.757417 25797 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.758359 25875 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:34.755299 25877 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:34.756640 25714 raft_consensus.cc:3055] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:26:34.752913 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:34.760507 25714 raft_consensus.cc:740] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:34.752651 25557 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:34.761791 25714 consensus_queue.cc:260] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.763504 25714 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:34.765682 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.766286 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:34.767212 14419 raft_consensus.cc:2368] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 35. Current term is 36.
I20260709 12:26:34.770174 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.770998 13798 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:26:34.771584 13795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:44068 (request call id 1916) took 1020 ms. Trace:
I20260709 12:26:34.772006 13795 rpcz_store.cc:276] 0709 12:26:33.751499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.752253 (+   754us) service_pool.cc:224] Handling call
0709 12:26:34.073373 (+321120us) spinlock_profiling.cc:243] Waited 898 us on lock 0x7b5800004298. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390acfeee 000000000036eb39 00007f5398b431e0 00007f5398c0c0c4 00007f5398c0c131 00007f5398bcdfc0 00007f5398bb8929 00007f5398bb83d2 00007f539212fbd4
0709 12:26:34.073558 (+   185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.758074 (+684516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.771241 (+ 13167us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b540012b454. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf4e01 00007f5393bf383e 00007f5398bb8419 00007f539212fbd4 00007f539212fb62 00007f539212fa91 00007f539212fa0c 00007f539212ecc2 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:34.771487 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1573998208}
I20260709 12:26:34.773422 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.766455 25724 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:26:34.775228 25794 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.775962 13794 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:34.757714 14423 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:34.778143 13617 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.779079 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:34.779713 13690 leader_election.cc:400] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 35. Current term is 36.
I20260709 12:26:34.801455 13690 leader_election.cc:403] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 election: Cancelling election due to peer responding with higher term
I20260709 12:26:34.802145 13617 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.802495 25603 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:34.802778 25818 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.803550 25818 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.804214 25642 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:34.780057 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.805083 25642 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:26:34.779775 13543 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 25 to 27, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 27 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:34.783977 25870 leader_election.cc:290] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.784425 25900 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.754969 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.781891 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:34.807401 13617 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:34.808169 13796 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:34.808856 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 54261.9us
I20260709 12:26:34.808734 13543 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 20 to 25, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 25 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.810774 25642 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:34.811759 25900 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:34.760025 25797 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.792549 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.781461 25736 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:34.816100 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 50584.8us
I20260709 12:26:34.759837 25671 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:34.796751 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.818470 25870 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.818687 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:34.819802 25671 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:34.819427 25870 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.803881 25785 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:34.801349 25557 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:34.825802 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:34.826944 13718 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:34.827667 25797 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:34.829157 25603 raft_consensus.cc:2749] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election lost for term 35. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 35. Current term is 36.
I20260709 12:26:34.829311 25582 consensus_queue.cc:1059] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 27.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:34.830507 25582 raft_consensus.cc:3055] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:26:34.830920 25582 raft_consensus.cc:740] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:34.832090 25582 consensus_queue.cc:260] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.833398 13617 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:34.833604 25582 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:34.839788 25797 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:34.842761 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:34.844573 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:34.845256 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:34.851094 25818 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.795138 25722 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.852402 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:34.854527 13793 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:34.857612 25722 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.859295 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:34.865828 13618 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.869053 25797 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:34.870105 25797 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:34.871714 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:34.872942 13727 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:34.874130 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 43 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:34.875849 13718 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 42.
I20260709 12:26:34.882612 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:34.883061 25714 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:34.885818 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 42 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:34.886543 25722 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:34.831770 13543 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 27 to 29, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 29 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:34.907840 13796 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
W20260709 12:26:34.691144 25869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.004s	sys 0.019s
I20260709 12:26:34.914309 14423 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:34.917474 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ff310d454634aa0bbc38f2b233ccb8e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:34.931176 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.945438 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e68c4d38b4465937dee955f980255" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 40 candidate_status { last_received { term: 37 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:34.945909 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:34.934932 13618 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:34.948992 25870 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 43 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:34.954015 25671 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:34.954892 25671 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:34.955019 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:34.955456 25501 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:34.955996 25501 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:34.956056 25671 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:34.953418 13725 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:34.943773 13798 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:34.932663 25722 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.962524 13618 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:34.949014 25818 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 FOLLOWER]: Leader election won for term 42
I20260709 12:26:34.970163 13793 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 30 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
W20260709 12:26:34.971969 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 407443us
I20260709 12:26:34.973151 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:34.964144 25722 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:34.975502 13718 raft_consensus.cc:1275] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:26:34.975476 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:34.978130 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.980020 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 199140us
I20260709 12:26:34.957150 25501 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:34.980361 25818 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:34.982686 25557 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.006929 25674 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:26:35.007696 25674 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:35.002571 25844 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 41 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.014551 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 128816us
I20260709 12:26:35.014187 25603 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.017158 12469 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:35.017409 25603 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.019815 25603 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.020998 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:35.029284 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.030130 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.032081 13543 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 18 to 24, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 24 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:35.033929 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.036031 25603 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:35.016155 25844 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.048573 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f8f1b91eeaf4fc28fa45d4b45b907cd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.050748 25603 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.052362 25603 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:35.007332 25557 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.053648 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:35.058408 14390 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.055779 13793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.034163 25818 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 40.21, Last appended by leader: 21, Current term: 42, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.062891 13793 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 28: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:35.063457 13799 raft_consensus.cc:2393] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 35: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:35.063342 13615 leader_election.cc:304] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.067385 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.067687 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.064476 25902 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:35.070956 25902 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.073587 25902 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.073601 25498 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:35.076100 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 42 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.078167 13794 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 42 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 42: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:35.078912 13615 leader_election.cc:304] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.083436 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.084748 13725 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 28: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:35.085883 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.099633 13615 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.074518 25498 consensus_queue.cc:1243] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Peer a4188193a10c4d50a555d76ea85c60b7 log is divergent from this leader: its last log entry 30.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:26:35.099829 14391 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.074644 25674 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 33.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.110615 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.112946 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.096789 25869 raft_consensus.cc:2749] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:35.108613 25642 raft_consensus.cc:2749] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:35.098584 25902 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:35.120831 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.130999 25902 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.113152 25811 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:35.134971 13720 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34450 (request call id 4818) took 1112 ms. Trace:
I20260709 12:26:35.137729 13720 rpcz_store.cc:276] 0709 12:26:33.884451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.886247 (+  1796us) service_pool.cc:224] Handling call
0709 12:26:33.886535 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.886911 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.886939 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.887444 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.444437 (+556993us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b18003f9160. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ae9e07 00007f53999449a9 00007f5390b0669b 00007f5390b0493f 00007f5390b06c02 00007f539378e20b 00007f5393793d23 00007f53937940d1 00007f5393bafa0f
0709 12:26:34.618558 (+174121us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:26:34.980580 (+362022us) log.cc:844] Serialized 14 byte log entry
0709 12:26:34.983361 (+  2781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.990335 (+  6974us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.990402 (+    67us) raft_consensus.cc:1685] finished
0709 12:26:34.993316 (+  2914us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.996408 (+  3092us) inbound_call.cc:177] Queueing success response
0709 12:26:35.058952 (+ 62544us) spinlock_profiling.cc:243] Waited 9.45 ms on lock 0x7b48000b22d8. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5398b1daa1 00007f5398b14b37 00007f5398b149ca 00007f5398c0c26f 00007f5398c0bf37 00007f5398c2055c 00007f5398c20508 00007f5398c20488 00007f5398c20443 00007f5398c1f6ec 00007f5397b49a96
Metrics: {"mutex_wait_us":4057,"prepare.queue_time_us":474791,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":1384,"raft.queue_time_us":2687,"raft.run_cpu_time_us":3139,"raft.run_wall_time_us":85971,"spinlock_wait_cycles":55700224,"thread_start_us":422189,"threads_started":2,"wal-append.queue_time_us":843566}
W20260709 12:26:35.143038 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 77314.1us
W20260709 12:26:35.145423 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.136930 25844 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:35.146147 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.148754 25870 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:35.153707 13799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89717fd329da4af68d87ae1ef1b104fb" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.154521 13799 raft_consensus.cc:3060] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:26:35.155778 13691 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.158247 13766 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.158044 25736 consensus_queue.cc:1048] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:35.159390 25872 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:35.134169 25902 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.170509 25785 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:35.179482 25557 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.186602 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 43 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.191344 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 42 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:35.214504 14422 raft_consensus.cc:2393] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 42: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:35.216065 13799 raft_consensus.cc:2468] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 36.
I20260709 12:26:35.213182 13543 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:35.220264 13615 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.224159 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.236694 25866 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:26:35.239089 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 157617us
I20260709 12:26:35.183050 25722 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.242537 25722 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.243245 25900 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:35.243916 25900 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.220597 13543 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 28 to 29, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 29 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:35.248812 13723 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 31: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:35.232090 13794 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 42 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 42.
I20260709 12:26:35.257441 25682 raft_consensus.cc:493] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
W20260709 12:26:35.274369 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 54105.3us
W20260709 12:26:35.280623 13693 leader_election.cc:336] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.281280 13693 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.282456 25818 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:35.246143 25900 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:35.284718 13693 leader_election.cc:336] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.285727 13693 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:35.288794 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 55122.7us
W20260709 12:26:35.291390 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 115746us
I20260709 12:26:35.290892 25797 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 41 FOLLOWER]: Advancing to term 42
I20260709 12:26:35.276255 25682 raft_consensus.cc:515] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.310693 13801 pending_rounds.cc:85] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Aborting all ops after (but not including) 14
I20260709 12:26:35.311177 13801 pending_rounds.cc:107] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: Aborting uncommitted NO_OP operation due to leader change: 30.15
I20260709 12:26:35.311486 13801 raft_consensus.cc:2889] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.311765 14390 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:35.312152 25870 consensus_queue.cc:1059] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:35.312673 25897 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:35.313100 25897 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.313443 25897 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:35.314951 13797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.315752 13797 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 30.
I20260709 12:26:35.320102 14391 leader_election.cc:304] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:35.321861 25785 raft_consensus.cc:2804] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:26:35.322436 25785 raft_consensus.cc:697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:35.323447 25785 consensus_queue.cc:237] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.325073 25897 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.313411 25870 raft_consensus.cc:3055] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:35.286450 25603 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.304019 25900 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:35.358290 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 73636.1us
W20260709 12:26:35.359290 13693 leader_election.cc:336] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.359783 13693 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:35.361176 13693 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.362478 25818 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:35.334600 25870 raft_consensus.cc:740] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:35.367936 25682 leader_election.cc:290] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:35.368537 25870 consensus_queue.cc:260] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.370971 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:35.373193 13693 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.391124 13725 raft_consensus.cc:2368] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32.
I20260709 12:26:35.391355 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.390556 25818 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:35.395116 25870 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:35.395421 25818 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.399791 25897 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:35.406857 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44560.5us
I20260709 12:26:35.406937 25722 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:35.407840 13693 leader_election.cc:336] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.408326 13693 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:35.409678 13693 leader_election.cc:336] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.410493 25902 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:35.410092 13693 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.413775 25603 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:35.427408 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 104140us
I20260709 12:26:35.428491 14390 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 43 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:35.443419 13765 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.450358 13693 leader_election.cc:336] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.450928 13693 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.430179 25902 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:35.462531 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:35.466892 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45dc2f34593649778ec4b80d29638572" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.473682 13799 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 38 index: 20. (index mismatch)
W20260709 12:26:35.477108 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 48742.2us
I20260709 12:26:35.456187 25818 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.464720 25603 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:35.480252 25868 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.480849 25868 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.484040 25868 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:35.490353 25924 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Leader pre-election won for term 43
I20260709 12:26:35.490980 25924 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.491349 25924 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:35.493348 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:35.491359 13693 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.476799 25582 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:35.497138 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:35.495256 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.496215 25582 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.500247 25582 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:35.502740 25902 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:26:35.512501 14422 raft_consensus.cc:1240] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Rejecting Update request from peer a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:26:35.522487 25920 consensus_queue.cc:1059] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:35.523787 25920 raft_consensus.cc:3055] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:26:35.524165 25920 raft_consensus.cc:740] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:35.525014 25920 consensus_queue.cc:260] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:35.529397 13693 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.535939 13693 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.539170 25818 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:35.542196 25797 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 42 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:26:35.543004 25582 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:35.545101 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.545763 25582 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:35.561318 13767 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.562021 13767 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:35.567564 25926 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:26:35.569748 13767 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.532766 25920 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:35.578624 13767 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:35.579309 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 248862us
W20260709 12:26:35.579396 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 65132.9us
I20260709 12:26:35.597018 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.601451 13766 leader_election.cc:304] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.603600 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.604277 13794 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:35.592402 13767 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.610081 25866 raft_consensus.cc:2749] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.615703 25868 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:35.606248 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:35.632805 25902 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:26:35.634456 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.626567 25922 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:35.653568 25932 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:35.654275 25932 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.656729 13720 raft_consensus.cc:1240] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Rejecting Update request from peer 4dafe08a4f404502822a0732fe570de4 for earlier term 27. Current term is 28. Ops: []
I20260709 12:26:35.626899 25924 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 43 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:35.649061 14391 leader_election.cc:336] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.640817 25866 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:35.665047 25932 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.658566 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:35.685700 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 84331.9us
I20260709 12:26:35.707190 13766 leader_election.cc:400] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 election: Vote denied by peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841) with higher term. Message: Invalid argument: T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 for earlier term 31. Current term is 32.
I20260709 12:26:35.698303 13794 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
W20260709 12:26:35.709694 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 69015.8us
W20260709 12:26:35.710659 14391 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.713613 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:35.713936 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.715154 13720 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 31: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:35.716522 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.687425 25919 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:35.726503 25932 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:35.727520 25834 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.728188 25834 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:35.732290 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.709694 25722 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:35.737083 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.740612 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.740741 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:35.734302 13618 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:35.746666 25642 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:35.747239 25642 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:35.748210 25642 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.751705 25834 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:35.764675 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:35.773870 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.774732 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.775847 13722 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:26:35.778003 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.779508 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 35 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.780259 13727 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:26:35.788192 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 61920.2us
I20260709 12:26:35.790879 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:35.793752 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.793895 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.794582 13718 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:35.796828 14418 raft_consensus.cc:2393] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:35.796859 25902 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.797669 25902 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:35.807722 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.816298 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:35.820192 13767 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 142 similar messages]
W20260709 12:26:35.828015 14391 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 43 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.833114 13801 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:35.832981 13726 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:26:35.838318 25927 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:35.848130 25924 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 43 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.848616 25902 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:35.858399 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.860085 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 43 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.860548 13727 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:35.861009 13801 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 42 FOLLOWER]: Advancing to term 43
I20260709 12:26:35.862432 25742 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:35.863948 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:35.864565 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:35.870671 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:35.879210 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.880165 13723 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:35.881215 13727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.882383 13727 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:26:35.884727 13798 raft_consensus.cc:2410] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 for term 32 because replica has last-logged OpId of term: 31 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:26:35.892870 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.894219 13794 raft_consensus.cc:2368] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:35.897692 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.898676 13795 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:35.879338 25932 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:35.903182 13720 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:26:35.904613 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 43 candidate_status { last_received { term: 40 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:35.908916 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.910420 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.905954 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 42 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:35.929111 13718 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 32.
I20260709 12:26:35.938609 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 38 candidate_status { last_received { term: 37 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:35.947640 13801 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 43 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 43.
I20260709 12:26:35.952582 13727 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 31.
W20260709 12:26:35.969463 13617 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.970181 13796 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:35.970192 13720 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:35.967034 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:35.974639 13794 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 43 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 42. Current term is 43. Ops: []
I20260709 12:26:35.989486 13693 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:35.991101 25722 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:35.985400 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:35.961814 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:35.996076 13718 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 36 index: 19. Preceding OpId from leader: term: 38 index: 20. (index mismatch)
I20260709 12:26:35.997701 13693 leader_election.cc:400] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:35.998745 13693 leader_election.cc:403] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:35.998934 13617 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:36.011195 25868 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:26:36.012499 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.013903 25815 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:26:36.014968 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.015911 25815 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.017298 13617 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.018190 25815 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:26:36.018461 12469 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:36.019470 13617 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.020745 25815 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:26:36.022059 25942 log.cc:927] Time spent T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 12:26:36.022693 13617 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.024413 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.030619 25794 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:36.023609 25815 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:26:36.054929 13693 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:36.025118 25722 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: Vote denied by peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415) with higher term. Message: Invalid argument: T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 30. Current term is 31.
I20260709 12:26:36.061172 25815 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:36.063362 13617 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.064304 25815 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:26:36.079900 13693 leader_election.cc:336] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.080696 13693 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:36.092070 13693 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.092873 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.089102 25818 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.098006 25818 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:36.099464 25603 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:36.057657 25722 consensus_queue.cc:1059] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.019s
I20260709 12:26:36.088975 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:36.077702 25902 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:26:36.140157 25902 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:36.143369 25722 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:36.144052 25722 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.120622 25818 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:36.182924 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 92225.5us
I20260709 12:26:36.140268 25900 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 LEADER]: Stepping down as leader of term 42
I20260709 12:26:36.189914 25900 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:36.191226 25722 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:36.192401 25834 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:26:36.228520 25934 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.229287 25934 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.220834 25714 raft_consensus.cc:493] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.231240 25714 raft_consensus.cc:515] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:36.232915 25934 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.233520 25714 leader_election.cc:290] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.238747 25900 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 42.22, Last appended by leader: 22, Current term: 42, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.240861 25900 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 42 FOLLOWER]: Advancing to term 43
W20260709 12:26:36.267586 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 175521us
W20260709 12:26:36.269371 13693 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.270752 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:36.271538 13722 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 30.
W20260709 12:26:36.275389 25956 log.cc:927] Time spent T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.087s	user 0.003s	sys 0.000s
I20260709 12:26:36.275642 14390 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:36.279173 25934 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:36.279623 25934 raft_consensus.cc:493] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.279953 25934 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:36.269979 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.284653 25582 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.285269 25582 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:36.287797 25582 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:36.294970 25949 log.cc:927] Time spent T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 12:26:36.304421 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.305142 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:36.310861 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 228446us
I20260709 12:26:36.312122 13617 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:36.315074 13693 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.315796 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:36.316615 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:36.325207 13693 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.326030 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.320564 25818 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:36.345268 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.359985 13617 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:36.318922 25955 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.361810 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.324079 25722 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:36.325713 25815 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:36.374452 13617 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:36.375917 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.347337 25902 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:36.379459 13693 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.379583 13794 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:26:36.380911 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.384164 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.385234 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.385854 13794 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:36.393365 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.394061 13801 raft_consensus.cc:2393] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 31: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:36.396677 13796 raft_consensus.cc:1240] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 43 FOLLOWER]: Rejecting Update request from peer b4f6bfcda63148e4965ca02c639336f1 for earlier term 42. Current term is 43. Ops: [40.21-42.22]
I20260709 12:26:36.361904 25797 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:26:36.342994 25957 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:26:36.383942 25818 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:26:36.386529 25962 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:26:36.410501 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 462697us
I20260709 12:26:36.418859 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.430897 25963 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:36.432866 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.433385 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:36.435830 13726 raft_consensus.cc:1275] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 34 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:26:36.414983 25714 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:36.440969 25928 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:36.441650 25928 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.443375 25785 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:36.443861 25928 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.452463 13801 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 31.
I20260709 12:26:36.452291 25897 consensus_queue.cc:1048] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:26:36.452490 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:36.463706 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 56084.7us
I20260709 12:26:36.464365 25897 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:26:36.476330 14391 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 43 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.478860 25797 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:26:36.482383 25794 raft_consensus.cc:493] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.482972 25794 raft_consensus.cc:515] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:36.479835 25957 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:36.485087 25794 leader_election.cc:290] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.491037 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 43 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.492002 25785 raft_consensus.cc:2749] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 43 FOLLOWER]: Leader election lost for term 43. Reason: could not achieve majority
I20260709 12:26:36.492195 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.496838 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.498876 13796 raft_consensus.cc:2393] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 36: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:36.503297 25868 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:36.508299 25742 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.516767 13724 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:26:36.517244 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
I20260709 12:26:36.518437 25868 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:36.518868 25868 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.519244 25868 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:36.509047 25742 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.524585 25742 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:36.473301 25934 raft_consensus.cc:515] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.534303 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e71fd62c156c4baf990c9e212f69ecc2" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:36.537537 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:36.541419 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48612be3c56f482c81533c30abebe60b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:36.538249 13723 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 28.
I20260709 12:26:36.544062 14419 raft_consensus.cc:2368] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:36.551390 13690 leader_election.cc:400] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:36.551785 13690 leader_election.cc:403] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:36.554109 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:36.575781 13693 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.576364 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:36.580197 13693 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.580804 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.591742 25934 leader_election.cc:290] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.592494 25902 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:36.593524 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:36.594203 13801 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:36.596554 13795 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
I20260709 12:26:36.591796 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:36.603229 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:36.608624 13794 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 36.
W20260709 12:26:36.607532 13693 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.584425 25900 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:36.577749 25818 raft_consensus.cc:3060] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:26:36.599071 14419 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 35.
I20260709 12:26:36.609704 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.610881 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85fee8a18ff4429cb870f59aa46b3892" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:36.611750 13801 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
I20260709 12:26:36.621582 14391 leader_election.cc:304] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:36.626505 13693 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:36.627625 13693 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.626622 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:36.628068 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:36.626780 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:36.631349 25722 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.633919 25582 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:36.639828 13693 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:36.640341 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:36.637108 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:36.641240 25815 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
I20260709 12:26:36.626066 25797 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:26:36.636234 25934 raft_consensus.cc:2804] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:36.670332 25934 raft_consensus.cc:697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:36.671648 25934 consensus_queue.cc:237] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.641383 25722 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:36.673911 25902 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:36.638542 25868 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:36.698941 25965 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:26:36.636932 25582 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.703866 25582 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:26:36.715581 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:36.716396 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:36.647058 13693 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:36.717950 13543 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 238) took 46489 ms (client timeout 15000 ms). Trace:
I20260709 12:26:36.719556 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:36.720773 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:36.721485 13801 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:26:36.727641 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80617.2us
I20260709 12:26:36.732344 13801 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
W20260709 12:26:36.734546 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 104215us
W20260709 12:26:36.742182 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:36.690091 25868 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.753585 13618 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7; no voters: 
W20260709 12:26:36.757349 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:36.741890 25900 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:26:36.773738 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 136666us
I20260709 12:26:36.776060 13617 leader_election.cc:304] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.772770 25794 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.790320 13542 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 26 to 29. New cstate: current_term: 29 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:36.794634 25954 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:36.786430 25811 raft_consensus.cc:2749] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:36.804337 25954 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
W20260709 12:26:36.719558 13543 rpcz_store.cc:269] 0709 12:25:50.228706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.441875 (+9213169us) service_pool.cc:224] Handling call
0709 12:26:01.083139 (+1641264us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:26:02.853081 (+1769942us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393bfcf3a 00007f53999b2e9c 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:36.717821 (+33864740us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:35.676727 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:35.778339 (+101612us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:35.778397 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:26:35.778431 (+    34us) tablet.cc:662] Decoding operations
0709 12:26:35.779334 (+   903us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:35.779434 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:35.779461 (+    27us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:26:35.780111 (+   650us) tablet.cc:701] Row locks acquired
0709 12:26:35.780158 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:26:35.780351 (+   193us) write_op.cc:270] Start()
0709 12:26:35.780513 (+   162us) write_op.cc:276] Timestamp: P: 1783599995780281 usec, L: 0
0709 12:26:35.780569 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 12:26:35.781413 (+   844us) log.cc:844] Serialized 2439 byte log entry
0709 12:26:36.184857 (+403444us) op_driver.cc:464] REPLICATION: finished
0709 12:26:36.285917 (+101060us) write_op.cc:301] APPLY: starting
0709 12:26:36.286047 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:36.301096 (+ 15049us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:36.301422 (+   326us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:36.404643 (+103221us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:36.405006 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 12:26:36.405067 (+    61us) write_op.cc:312] APPLY: finished
0709 12:26:36.405431 (+   364us) log.cc:844] Serialized 65 byte log entry
0709 12:26:36.407728 (+  2297us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:36.411488 (+  3760us) write_op.cc:454] Released schema lock
0709 12:26:36.411573 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:36.448135 (+ 36562us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:36.448185 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":826,"reported_tablets":50,"spinlock_wait_cycles":879883392,"child_traces":[["op",{"apply.queue_time_us":100716,"apply.run_cpu_time_us":13612,"apply.run_wall_time_us":129085,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29255,"num_ops":6,"prepare.queue_time_us":278007,"prepare.run_cpu_time_us":16042,"prepare.run_wall_time_us":206367,"raft.queue_time_us":160985,"raft.run_cpu_time_us":3789,"raft.run_wall_time_us":36049,"replication_time_us":404085,"thread_start_us":417755,"threads_started":4,"wal-append.queue_time_us":90686}]]}
I20260709 12:26:36.800560 25794 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:36.884393 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96451.1us
I20260709 12:26:36.967157 25818 raft_consensus.cc:2749] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 31. Current term is 32.
I20260709 12:26:36.967628 13617 leader_election.cc:304] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:36.968364 25794 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:36.969236 25844 raft_consensus.cc:2749] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:36.912187 25968 log.cc:927] Time spent T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 12:26:36.913679 25971 log.cc:927] Time spent T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:26:36.804693 25736 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.977001 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 147375us
I20260709 12:26:36.975819 25671 raft_consensus.cc:493] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:36.981197 25671 raft_consensus.cc:515] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.984601 25671 leader_election.cc:290] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:36.969776 13542 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 25 to 30. New cstate: current_term: 30 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.992741 25582 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:36.993077 13542 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 24 to 28, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 28 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:36.994428 25974 log.cc:927] Time spent T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.209s	user 0.000s	sys 0.007s
I20260709 12:26:36.995103 25736 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.012620 13720 raft_consensus.cc:2468] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 32.
W20260709 12:26:37.021821 12469 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:37.022764 13719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.034413 25582 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.036048 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "068a8cc7309648e3a661d4d056f11b7a" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.045502 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.046602 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:37.047271 13798 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:37.061556 13723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.061043 25603 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:37.067873 25954 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:37.061219 13542 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 21 to 26. New cstate: current_term: 26 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:37.068506 25603 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.069819 13542 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 26 to 28, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:37.071019 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:37.071709 13726 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:37.071816 13542 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 27 to 30. New cstate: current_term: 30 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:26:37.061632 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:37.101812 13615 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 51258.9us
W20260709 12:26:37.107071 25975 log.cc:927] Time spent T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.320s	user 0.003s	sys 0.000s
I20260709 12:26:37.113262 25978 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.113881 25982 log.cc:927] Time spent T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:37.114032 25978 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:37.117295 25980 log.cc:927] Time spent T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:26:37.130437 13542 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 32, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:37.142715 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 37 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:37.143573 14419 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 37
I20260709 12:26:37.158890 13798 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
W20260709 12:26:37.175817 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 161217us
I20260709 12:26:37.145623 13542 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:26:37.182221 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 99096.8us
W20260709 12:26:37.184036 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 74397.3us
I20260709 12:26:37.201189 13794 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:37.206640 13726 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 29.
I20260709 12:26:37.210461 13542 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:37.220371 13766 leader_election.cc:304] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:37.223090 25989 raft_consensus.cc:2749] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:37.212512 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.224561 14418 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:37.230635 14419 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dafe08a4f404502822a0732fe570de4 in term 37.
I20260709 12:26:37.232029 13615 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:37.237234 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.249996 25992 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:37.250721 25992 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Becoming Leader. State: Replica: 4dafe08a4f404502822a0732fe570de4, State: Running, Role: LEADER
I20260709 12:26:37.250936 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.251729 13724 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:37.251765 25992 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.252921 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.253610 13726 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:37.254606 25603 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.255048 25984 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:37.256115 13542 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 31 to 35. New cstate: current_term: 35 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:37.250845 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:37.261619 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.262303 14419 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:37.262826 25714 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:37.207441 25736 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.269120 25978 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.270753 13792 raft_consensus.cc:2393] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:37.273343 13766 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:37.274453 25682 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:37.274900 25682 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.275278 25682 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.258194 13542 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 32. New cstate: current_term: 32 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:37.278744 13766 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:37.279954 13542 catalog_manager.cc:5697] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 23 to 31, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 31 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
W20260709 12:26:37.295248 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 114260us
W20260709 12:26:37.300047 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 89386.9us
W20260709 12:26:37.302917 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:37.310636 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.311412 14418 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:26:37.323118 25674 raft_consensus.cc:493] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:37.328553 25674 raft_consensus.cc:515] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:37.333618 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:37.321578 13542 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 24 to 34, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 34 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:37.306953 25990 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:37.337986 25990 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.342324 25990 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.349211 13724 raft_consensus.cc:1275] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:37.368745 25785 consensus_queue.cc:1048] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:26:37.393409 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 78630.2us
I20260709 12:26:37.399051 13724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:37.417666 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:37.426885 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:37.431170 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 94307.6us
I20260709 12:26:37.437547 13690 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:37.439393 25818 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:37.440006 25818 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader b708d8f5f6ac4aabb1eb5cb821ace195)
I20260709 12:26:37.440507 25818 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:37.469962 25674 leader_election.cc:290] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.434722 25984 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:37.430955 25682 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.477725 25682 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.500895 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:37.501688 14422 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.506960 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
W20260709 12:26:37.527135 14391 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.12.45.126:44051, user_credentials={real_user=slave}} blocked reactor thread for 68075.2us
I20260709 12:26:37.527448 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdd4114edc6d43c08d41462fd399480c" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.553076 25603 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 43 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.559935 25603 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 43 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:37.553670 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:37.565248 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:37.584091 13615 leader_election.cc:304] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:37.596874 14422 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:37.600042 13764 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:37.603681 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 44 candidate_status { last_received { term: 42 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.607396 13792 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 43 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 43.
I20260709 12:26:37.608024 25736 raft_consensus.cc:2804] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:37.613622 25736 raft_consensus.cc:697] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:37.614732 25736 consensus_queue.cc:237] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.617650 25999 raft_consensus.cc:493] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:37.599030 25868 raft_consensus.cc:2749] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:37.601064 25963 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.630182 25603 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 44 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.618432 25999 raft_consensus.cc:515] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:37.638041 25995 log.cc:927] Time spent T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.205s	user 0.003s	sys 0.000s
I20260709 12:26:37.639981 25999 leader_election.cc:290] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:37.628227 13693 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 44 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:37.635608 25963 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:37.642772 25963 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:37.648618 25818 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.651226 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 44 candidate_status { last_received { term: 42 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.656246 14422 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 43 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 43.
I20260709 12:26:37.657655 13690 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 44 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:37.657898 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.658227 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.663053 25955 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 43 FOLLOWER]: Leader pre-election won for term 44
I20260709 12:26:37.663725 25955 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 43 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.664186 25955 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:26:37.693783 25990 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:37.697522 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:37.697716 25990 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.699118 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:37.699807 14422 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.707525 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1866ffc09d7a4cd3b2f2883e9baf96fe" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:37.708261 13726 raft_consensus.cc:3060] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:37.710232 13721 raft_consensus.cc:3060] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:26:37.711179 13618 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 309582us
W20260709 12:26:37.708249 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:37.720779 14422 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:37.722049 13764 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:37.722978 25990 raft_consensus.cc:2804] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:37.723572 25990 raft_consensus.cc:697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:37.724267 25964 raft_consensus.cc:493] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.724880 25990 consensus_queue.cc:237] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.724898 25964 raft_consensus.cc:515] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:37.730204 25964 leader_election.cc:290] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:37.735728 25818 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.734719 13721 raft_consensus.cc:2468] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:37.736518 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:37.741827 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.755050 13726 raft_consensus.cc:2468] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 33.
I20260709 12:26:37.759553 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e96f5f06294edba36ea19efe454394" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.759898 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ff03c62343d402fa469da8080d33a6d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.761849 13795 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:26:37.762140 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:37.762926 14422 raft_consensus.cc:2368] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:26:37.763732 13796 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:37.765210 13690 leader_election.cc:400] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:26:37.765559 13690 leader_election.cc:403] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:26:37.765833 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "684ee7a163da446aba11bf15d10d691a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.787557 25866 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.790303 25866 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:37.792630 25866 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:37.795900 25957 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.796548 25957 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.799034 25957 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.803299 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:37.806914 13725 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:26:37.808434 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e52c8b752b54bd9b959b2363df455a4" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.811599 13617 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:37.816094 25498 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:37.816672 13618 leader_election.cc:304] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:37.820436 25498 raft_consensus.cc:2749] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:37.818504 25868 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:37.821208 13618 leader_election.cc:304] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:37.813482 25797 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:37.823817 26001 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:37.831599 13618 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:37.833483 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.834504 13726 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
W20260709 12:26:37.835830 13767 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:37.845881 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:37.788362 25818 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965) with higher term. Message: Invalid argument: T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 for earlier term 32. Current term is 33.
I20260709 12:26:37.842990 26001 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:37.869377 25682 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:37.851796 25998 raft_consensus.cc:2749] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:37.875445 25682 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.881276 25682 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:37.887440 14422 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Refusing update from remote peer 4dafe08a4f404502822a0732fe570de4: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:37.889379 25844 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:26:37.901796 25955 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:37.924736 25955 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 44 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:37.930920 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 44 candidate_status { last_received { term: 42 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:37.931679 14422 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 43 FOLLOWER]: Advancing to term 44
I20260709 12:26:37.930920 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 44 candidate_status { last_received { term: 42 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:37.933111 13794 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 43 FOLLOWER]: Advancing to term 44
W20260709 12:26:37.969904 13693 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 44 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:37.972033 14422 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 44.
I20260709 12:26:37.990016 25957 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.990960 26016 log.cc:927] Time spent T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:37.990741 25957 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:37.994120 25957 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:37.995007 13794 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 44 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 44.
I20260709 12:26:37.995800 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:37.996559 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54361b2742ab4613b55749cbb699113b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:37.996727 13693 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 44 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:26:37.994158 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:38.000299 13617 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:38.003968 25999 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:38.000867 25955 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 FOLLOWER]: Leader election won for term 44
I20260709 12:26:38.011343 25999 raft_consensus.cc:493] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.014894 25955 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:38.022075 25999 raft_consensus.cc:515] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:38.027558 12469 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 13677)
I20260709 12:26:38.028200 25955 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 42.22, Last appended by leader: 22, Current term: 44, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:38.034578 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:38.038599 25682 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:38.049446 25682 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:38.050551 25999 leader_election.cc:290] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:38.050707 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.051542 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5c65e93a0c1420ca348d0fc678031ca" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:38.052006 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:38.054838 13718 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:26:38.056293 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:38.058763 13617 leader_election.cc:304] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:38.060153 26010 raft_consensus.cc:2749] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:26:38.063194 13718 raft_consensus.cc:2468] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:38.063843 13726 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
W20260709 12:26:38.066084 13767 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:26:38.069708 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:38.071022 26014 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:38.087991 13766 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:38.089259 25922 raft_consensus.cc:2804] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:38.089779 25922 raft_consensus.cc:697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:38.090718 25922 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:38.092525 25724 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:38.106317 26010 raft_consensus.cc:493] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.107270 26010 raft_consensus.cc:515] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:38.117381 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:38.119812 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2dcf256ec7a4ea6be04f95be4459f59" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.124552 13618 leader_election.cc:304] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:38.114203 26010 leader_election.cc:290] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:38.127599 26010 raft_consensus.cc:2749] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:38.161614 14422 raft_consensus.cc:1275] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:38.164750 25989 consensus_queue.cc:1048] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:38.192817 14419 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:38.196223 25989 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:26:38.214748 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 62707.3us
I20260709 12:26:38.229432 13726 raft_consensus.cc:1275] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:26:38.231227 25989 consensus_queue.cc:1048] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:26:38.266928 13542 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 239) took 21543 ms (client timeout 15000 ms). Trace:
W20260709 12:26:38.274420 13542 rpcz_store.cc:269] 0709 12:26:16.723992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.485955 (+5761963us) service_pool.cc:224] Handling call
0709 12:26:38.266768 (+15780813us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:37.546188 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:37.547052 (+   864us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:37.547115 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:26:37.547134 (+    19us) tablet.cc:662] Decoding operations
0709 12:26:37.565471 (+ 18337us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:37.565569 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:37.565626 (+    57us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:26:37.566843 (+  1217us) tablet.cc:701] Row locks acquired
0709 12:26:37.566863 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:26:37.567024 (+   161us) write_op.cc:270] Start()
0709 12:26:37.567177 (+   153us) write_op.cc:276] Timestamp: P: 1783599997566977 usec, L: 0
0709 12:26:37.567201 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:26:37.568011 (+   810us) log.cc:844] Serialized 5120 byte log entry
0709 12:26:37.686628 (+118617us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b18000df720. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390ad52a6 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393befe28 00007f539762a581 00007f5397629d58
0709 12:26:37.699913 (+ 13285us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b18000df720. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f5390b073f3 00007f5390b097c1 00007f5390b09779 00007f5390b09709 00007f5390b096d1 00007f5390b089cd 0000000000388634 0000000000388479 00007f5390aea83a
0709 12:26:37.900551 (+200638us) op_driver.cc:464] REPLICATION: finished
0709 12:26:37.938509 (+ 37958us) write_op.cc:301] APPLY: starting
0709 12:26:37.938642 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:37.991364 (+ 52722us) spinlock_profiling.cc:243] Waited 9.11 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393bf132a 00007f5393b76f14 00007f5393b76ec3 00007f5393b76e38 00007f5393b76df3 00007f5393b760dc 00007f5393b9fe16 00007f5393b87cbb 00007f5393b7239d 00007f5393b75d75
0709 12:26:38.027090 (+ 35726us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:38.027122 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:38.052860 (+ 25738us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:38.070557 (+ 17697us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:26:38.070622 (+    65us) write_op.cc:312] APPLY: finished
0709 12:26:38.134362 (+ 63740us) log.cc:844] Serialized 121 byte log entry
0709 12:26:38.141744 (+  7382us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:38.161288 (+ 19544us) write_op.cc:454] Released schema lock
0709 12:26:38.164259 (+  2971us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:38.164787 (+   528us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:38.164818 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":79800,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":37564,"apply.run_cpu_time_us":22675,"apply.run_wall_time_us":226254,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58571,"num_ops":13,"prepare.queue_time_us":113974,"prepare.run_cpu_time_us":72366,"prepare.run_wall_time_us":140793,"raft.queue_time_us":171693,"raft.run_cpu_time_us":11728,"raft.run_wall_time_us":108055,"replication_time_us":328220,"spinlock_wait_cycles":55437568,"thread_start_us":305502,"threads_started":4,"wal-append.queue_time_us":91209}]]}
W20260709 12:26:38.280841 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:38.287171 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:38.287909 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:38.296551 26010 raft_consensus.cc:493] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.297263 26010 raft_consensus.cc:515] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:38.299698 26010 leader_election.cc:290] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:38.301370 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:38.303805 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63b331138ada4d5e96b6136417e740f7" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.319582 13617 leader_election.cc:304] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:38.324187 25973 raft_consensus.cc:2749] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:26:38.364392 13544 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 157) took 136409 ms (client timeout 15000 ms). Trace:
W20260709 12:26:38.364998 13544 rpcz_store.cc:269] 0709 12:24:21.960050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.987100 (+ 27050us) service_pool.cc:224] Handling call
0709 12:26:38.364072 (+136376972us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80334,"reported_tablets":28}
I20260709 12:26:38.367928 26010 raft_consensus.cc:493] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.368799 26010 raft_consensus.cc:515] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:38.383414 26010 leader_election.cc:290] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:38.385179 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:38.385582 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30d1d057032c4ae0ad5ae21d374a2baa" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.387545 13617 leader_election.cc:304] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:38.391245 26010 raft_consensus.cc:2749] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:26:38.464068 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:38.486326 13545 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 237) took 247804 ms (client timeout 15000 ms). Trace:
W20260709 12:26:38.486828 13545 rpcz_store.cc:269] 0709 12:22:30.682014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:22:30.682424 (+   410us) service_pool.cc:224] Handling call
0709 12:22:31.136326 (+453902us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5e090 00007f5393bfcf3a 00007f53999b2e9c 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:38.486081 (+247349755us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7037,"reported_tablets":50,"spinlock_wait_cycles":28914816}
I20260709 12:26:38.504403 13541 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 20 to 22, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 22 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:38.510558 13693 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:38.533674 25998 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.534334 25998 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:38.539427 14419 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 44 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 40 index: 21. Preceding OpId from leader: term: 44 index: 23. (index mismatch)
I20260709 12:26:38.541764 25955 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:38.547751 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.548671 13725 raft_consensus.cc:2393] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4dafe08a4f404502822a0732fe570de4 in current term 32: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:38.550115 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:38.553165 13617 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:38.554134 25973 raft_consensus.cc:3060] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:38.543561 13541 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.564775 25998 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:38.577230 25973 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:38.599303 13767 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:38.600435 25973 raft_consensus.cc:493] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.601068 25973 raft_consensus.cc:515] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:38.604398 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.606276 13796 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 44 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 38 index: 20. Preceding OpId from leader: term: 44 index: 23. (index mismatch)
I20260709 12:26:38.607066 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac3728586e264940a93f76650edf9ad3" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:38.608388 25818 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:38.611793 13615 leader_election.cc:304] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:38.612943 26010 raft_consensus.cc:2749] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:38.619448 13718 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:26:38.631891 25922 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:38.637243 25973 leader_election.cc:290] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:38.704916 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:38.735006 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:38.736215 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:38.772809 26010 raft_consensus.cc:493] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.773543 26010 raft_consensus.cc:515] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:38.783007 13544 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.787962 13544 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 26, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 26 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.788522 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:38.791993 13544 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 34 to 37. New cstate: current_term: 37 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.793560 13544 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 31. New cstate: current_term: 31 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.794991 13544 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 26 to 31, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 31 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.800014 26010 leader_election.cc:290] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:38.802374 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cb6d086379f4190b73fc7f692fb490a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:38.806284 13615 leader_election.cc:304] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:38.810969 25960 raft_consensus.cc:2749] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:26:38.823736 13541 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 243) took 78818 ms (client timeout 15000 ms). Trace:
I20260709 12:26:38.814067 13544 catalog_manager.cc:5697] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 31 to 35. New cstate: current_term: 35 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:38.830499 13544 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 28 to 31, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 31 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.832410 13544 catalog_manager.cc:5697] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 37, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 37 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:38.833724 13544 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 38 to 40. New cstate: current_term: 40 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.836138 13544 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 23 to 24, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 24 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.838416 13544 catalog_manager.cc:5697] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:38.839898 13544 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 29, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 29 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:38.824651 13541 rpcz_store.cc:269] 0709 12:25:19.995074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.651931 (+11656857us) service_pool.cc:224] Handling call
0709 12:26:38.813011 (+67161080us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:38.602473 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:38.603142 (+   669us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:38.603189 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:26:38.603211 (+    22us) tablet.cc:662] Decoding operations
0709 12:26:38.603748 (+   537us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:38.603853 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:38.603888 (+    35us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:26:38.604248 (+   360us) tablet.cc:701] Row locks acquired
0709 12:26:38.604270 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:26:38.604437 (+   167us) write_op.cc:270] Start()
0709 12:26:38.604563 (+   126us) write_op.cc:276] Timestamp: P: 1783599998604380 usec, L: 0
0709 12:26:38.604591 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:26:38.623108 (+ 18517us) log.cc:844] Serialized 923 byte log entry
0709 12:26:38.626963 (+  3855us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b54003282d4. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f5393b5d571 00007f5393befe28 00007f539762a581 00007f5397629d58 00007f539762e9f1 00007f539762e9a9 00007f539762e939 00007f539762e901 00007f539762db2d 0000000000388634 0000000000388479
0709 12:26:38.631228 (+  4265us) op_driver.cc:464] REPLICATION: finished
0709 12:26:38.631677 (+   449us) write_op.cc:301] APPLY: starting
0709 12:26:38.631949 (+   272us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:38.634315 (+  2366us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:38.634337 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:38.643262 (+  8925us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:38.643506 (+   244us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:26:38.643571 (+    65us) write_op.cc:312] APPLY: finished
0709 12:26:38.643985 (+   414us) log.cc:844] Serialized 33 byte log entry
0709 12:26:38.644170 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:38.655556 (+ 11386us) write_op.cc:454] Released schema lock
0709 12:26:38.658216 (+  2660us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:38.673722 (+ 15506us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:38.676228 (+  2506us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6127,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":120,"apply.run_cpu_time_us":5432,"apply.run_wall_time_us":38384,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8376,"num_ops":2,"prepare.queue_time_us":16528,"prepare.run_cpu_time_us":6140,"prepare.run_wall_time_us":24886,"replication_time_us":26442,"spinlock_wait_cycles":40721280,"thread_start_us":16691,"threads_started":1}]]}
I20260709 12:26:38.841780 13544 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 37 to 39. New cstate: current_term: 39 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.852061 13544 catalog_manager.cc:5697] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 36 to 37, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 37 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:38.854205 13544 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 34 to 37, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 37 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.856774 13544 catalog_manager.cc:5697] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 28 to 29, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 29 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:38.890949 13544 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 33 to 36, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 36 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.915139 13544 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.916889 13544 catalog_manager.cc:5697] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:38.918128 13544 catalog_manager.cc:5697] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:38.922340 13544 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 27 to 28, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 28 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.924335 13544 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 18 to 29. New cstate: current_term: 29 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:38.926251 13544 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 27 to 30, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 30 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:38.946352 13544 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:38.953517 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 124239us
I20260709 12:26:38.904544 25960 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:38.958676 25960 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:39.019404 25960 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:39.028131 12469 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 13677)
W20260709 12:26:39.039281 13617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 126659us
I20260709 12:26:39.052805 26010 raft_consensus.cc:493] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.053433 26010 raft_consensus.cc:515] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:39.055778 26010 leader_election.cc:290] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:39.061057 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:39.070300 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:39.083178 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.108633 26010 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.109349 26010 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:39.112089 26010 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:39.112360 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:39.123811 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9c30191f356495b9732659f925abc9f" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:39.129805 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8debd745c407406c88fc03ec3a21f4db" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:39.136276 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a0048f5294f4012ad1519c0bc8e9e5a" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 39 candidate_status { last_received { term: 36 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:39.148020 13615 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:39.150100 26010 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:39.153690 26040 raft_consensus.cc:493] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.154438 26040 raft_consensus.cc:515] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:39.156750 26040 leader_election.cc:290] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:39.157430 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:39.160320 26037 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.161098 26037 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:39.166379 26037 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:39.168052 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:39.186579 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.189441 13618 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:39.200657 13544 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45090 (request call id 262) took 3893 ms. Trace:
I20260709 12:26:39.204185 26040 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:39.206845 13544 rpcz_store.cc:276] 0709 12:26:35.310429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.389670 (+3079241us) service_pool.cc:224] Handling call
0709 12:26:39.200545 (+810875us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:38.971535 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:38.972215 (+   680us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:38.972262 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:26:38.972282 (+    20us) tablet.cc:662] Decoding operations
0709 12:26:38.974202 (+  1920us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:38.974319 (+   117us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:38.974348 (+    29us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:26:38.979732 (+  5384us) tablet.cc:701] Row locks acquired
0709 12:26:38.979753 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:26:38.979909 (+   156us) write_op.cc:270] Start()
0709 12:26:38.980045 (+   136us) write_op.cc:276] Timestamp: P: 1783599998979860 usec, L: 0
0709 12:26:38.980069 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:26:38.980904 (+   835us) log.cc:844] Serialized 9345 byte log entry
0709 12:26:38.990229 (+  9325us) op_driver.cc:464] REPLICATION: finished
0709 12:26:38.991091 (+   862us) write_op.cc:301] APPLY: starting
0709 12:26:38.991181 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:39.023818 (+ 32637us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b1800446260. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f53999cfab1 00007f53999dc281 00007f53939330ec 00007f539397fc0a 00007f539397f4f7 00007f539397f490 00007f5393936f48 00007f539764b1bb 00007f539764b532 00007f5397692b93 00007f539753fde3
0709 12:26:39.085525 (+ 61707us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:39.085563 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:39.140640 (+ 55077us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:39.140949 (+   309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:26:39.140992 (+    43us) write_op.cc:312] APPLY: finished
0709 12:26:39.141498 (+   506us) log.cc:844] Serialized 212 byte log entry
0709 12:26:39.141685 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:39.141920 (+   235us) write_op.cc:454] Released schema lock
0709 12:26:39.141992 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:39.169455 (+ 27463us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:39.169492 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21588,"reported_tablets":50,"spinlock_wait_cycles":78080,"child_traces":[["op",{"apply.queue_time_us":562,"apply.run_cpu_time_us":37383,"apply.run_wall_time_us":178327,"cfile_cache_hit":96,"cfile_cache_hit_bytes":100512,"num_ops":24,"prepare.queue_time_us":1214,"prepare.run_cpu_time_us":6511,"prepare.run_wall_time_us":39363,"replication_time_us":9931,"spinlock_wait_cycles":2762624}]]}
I20260709 12:26:39.225018 26037 raft_consensus.cc:493] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.235886 26037 raft_consensus.cc:515] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:39.237811 13618 leader_election.cc:304] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:39.238704 26037 leader_election.cc:290] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:39.239475 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:39.254961 25973 raft_consensus.cc:2749] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:39.274868 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de32481a3be147a5b854251468fc667b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 31 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:39.276844 13615 leader_election.cc:304] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:39.282332 26037 raft_consensus.cc:2749] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:26:39.335107 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 53179.9us
W20260709 12:26:39.344326 13548 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 155) took 186433 ms (client timeout 15000 ms). Trace:
W20260709 12:26:39.345074 13546 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 235) took 136133 ms (client timeout 15000 ms). Trace:
W20260709 12:26:39.338294 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:39.345747 13546 rpcz_store.cc:269] 0709 12:24:23.211503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.613156 (+401653us) service_pool.cc:224] Handling call
0709 12:24:24.174455 (+561299us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390ad0640 00007f53999a7846 00007f5399ab190e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc 00007f5397f71392 00007f5391f2f48c 00007f5391f2e8c6
0709 12:26:39.342277 (+135167822us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22277,"reported_tablets":50,"spinlock_wait_cycles":45637888}
W20260709 12:26:39.346944 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:39.348137 13548 rpcz_store.cc:269] 0709 12:23:32.910630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:32.910923 (+   293us) service_pool.cc:224] Handling call
0709 12:26:39.344102 (+186433179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76607,"reported_tablets":28}
W20260709 12:26:39.348754 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:39.357314 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.364931 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79b849c107424cc79f4cbd223feeeb47" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:39.366443 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:39.376699 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:26:39.378580 12469 tablet_replica.cc:333] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.380271 12469 raft_consensus.cc:2243] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:39.381036 12469 raft_consensus.cc:2272] T e71fd62c156c4baf990c9e212f69ecc2 P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:39.404217 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:39.417600 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.432230 13618 leader_election.cc:304] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:39.439316 25960 raft_consensus.cc:2749] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:39.439851 13618 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:39.455742 25960 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:39.443401 12469 tablet_replica.cc:333] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.461072 12469 raft_consensus.cc:2243] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:26:39.481359 12469 raft_consensus.cc:2272] T 749228d3459a444a9382bc38bbfcbbee P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.519837 12469 tablet_replica.cc:333] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.519711 26034 raft_consensus.cc:493] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.583877 12469 raft_consensus.cc:2243] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:26:39.595391 12469 raft_consensus.cc:2272] T 0b7a0a5ab87d4adbb517d125624bc8ad P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.555464 26034 raft_consensus.cc:515] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:39.615624 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 83767.9us
I20260709 12:26:39.647755 12469 tablet_replica.cc:333] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.650367 12469 raft_consensus.cc:2243] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:39.663076 12469 pending_rounds.cc:70] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Trying to abort 1 pending ops.
I20260709 12:26:39.663457 12469 pending_rounds.cc:77] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4: Aborting op as it isn't in flight: id { term: 31 index: 18 } timestamp: 7305625551094239232 op_type: NO_OP noop_request { }
I20260709 12:26:39.663913 12469 raft_consensus.cc:2889] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:39.664290 12469 raft_consensus.cc:2272] T 63b331138ada4d5e96b6136417e740f7 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.668025 12469 tablet_replica.cc:333] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.669250 12469 raft_consensus.cc:2243] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:39.670105 12469 raft_consensus.cc:2272] T 684ee7a163da446aba11bf15d10d691a P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.717480 12469 tablet_replica.cc:333] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.721207 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:39.727991 13549 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 240) took 174746 ms (client timeout 15000 ms). Trace:
W20260709 12:26:39.728542 13549 rpcz_store.cc:269] 0709 12:23:44.981180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:44.983627 (+  2447us) service_pool.cc:224] Handling call
0709 12:26:39.727752 (+174744125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":616927,"reported_tablets":50}
I20260709 12:26:39.738824 26034 leader_election.cc:290] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:39.748118 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 123918us
I20260709 12:26:39.753508 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6966d62b938845429a9aa8feb8fe2b9b" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 35 index: 26 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:39.757809 26034 raft_consensus.cc:493] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:26:39.758498 26034 raft_consensus.cc:515] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:39.760807 26034 leader_election.cc:290] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:39.766664 12469 raft_consensus.cc:2243] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:39.767988 12469 raft_consensus.cc:2272] T b5c65e93a0c1420ca348d0fc678031ca P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.767833 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:39.769500 13615 leader_election.cc:304] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:39.773937 12469 tablet_replica.cc:333] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.783947 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8916aaed46c543b59ce09ffca942c33d" candidate_uuid: "4dafe08a4f404502822a0732fe570de4" candidate_term: 38 candidate_status { last_received { term: 37 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:39.786592 26047 raft_consensus.cc:2749] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:39.791627 13618 leader_election.cc:304] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4dafe08a4f404502822a0732fe570de4; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:39.792671 26010 raft_consensus.cc:2749] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:26:39.814786 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.820387 12469 raft_consensus.cc:2243] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:39.821532 12469 raft_consensus.cc:2272] T 3ff310d454634aa0bbc38f2b233ccb8e P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.871666 12469 tablet_replica.cc:333] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
W20260709 12:26:39.890982 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.885471 12469 raft_consensus.cc:2243] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:26:39.899281 13547 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 27 to 29, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:39.904395 12469 raft_consensus.cc:2272] T 7d21a9fba1484f4985407f4cde83b77c P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:39.909547 13550 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 154) took 209768 ms (client timeout 15000 ms). Trace:
W20260709 12:26:39.910276 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:39.922618 13550 rpcz_store.cc:269] 0709 12:23:10.141309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:10.142359 (+  1050us) service_pool.cc:224] Handling call
0709 12:26:39.909314 (+209766955us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":550444,"reported_tablets":28}
I20260709 12:26:39.910624 13547 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 24 to 28, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 28 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:39.930580 13547 catalog_manager.cc:5697] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.949623 12469 tablet_replica.cc:333] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:39.951050 12469 raft_consensus.cc:2243] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:26:39.942909 13547 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 33 to 38. New cstate: current_term: 38 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:39.961529 13547 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 24 to 29. New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:39.952092 12469 raft_consensus.cc:2272] T 3a8b916ed93d478c990f668818f43952 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:39.981359 12469 tablet_replica.cc:333] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
W20260709 12:26:39.992412 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:39.998805 12469 raft_consensus.cc:2243] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:39.966068 13547 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 31 to 37, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 37 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.001065 12469 raft_consensus.cc:2272] T 6966d62b938845429a9aa8feb8fe2b9b P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.001601 13547 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 27 to 30, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 30 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:40.024235 13547 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 21 to 28, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 28 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.025422 12469 tablet_replica.cc:333] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.026793 12469 raft_consensus.cc:2243] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.027465 12469 raft_consensus.cc:2272] T 990ed59e36454f16b81d57d0e4f59c4e P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.030776 12469 tablet_replica.cc:333] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.031956 12469 raft_consensus.cc:2243] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.032665 12469 raft_consensus.cc:2272] T 3cb6d086379f4190b73fc7f692fb490a P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.036084 12469 tablet_replica.cc:333] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.037034 12469 raft_consensus.cc:2243] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.037772 12469 raft_consensus.cc:2272] T 8ab361e956dd4342b80045629d159a2b P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.054126 12469 tablet_replica.cc:333] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.055677 12469 raft_consensus.cc:2243] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.057117 12469 raft_consensus.cc:2272] T 20dc442f4fae4cf49e336a3d350b7760 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.106407 12469 tablet_replica.cc:333] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.107546 12469 raft_consensus.cc:2243] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.108771 12469 raft_consensus.cc:2272] T cd4fd9484f184d2c994c269eb10d6ec1 P 4dafe08a4f404502822a0732fe570de4 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.118491 12469 tablet_replica.cc:333] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.119645 12469 raft_consensus.cc:2243] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.120894 12469 raft_consensus.cc:2272] T e2603f7eea4f49ab89f0ebb1304ee239 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.124125 12469 tablet_replica.cc:333] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.125308 12469 raft_consensus.cc:2243] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.126014 12469 raft_consensus.cc:2272] T 0a0048f5294f4012ad1519c0bc8e9e5a P 4dafe08a4f404502822a0732fe570de4 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.132283 12469 tablet_replica.cc:333] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.133366 12469 raft_consensus.cc:2243] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Raft consensus shutting down.
W20260709 12:26:40.137329 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.140511 12469 raft_consensus.cc:2272] T 255f10165e844f2cb50d654996d639cf P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.147228 12469 tablet_replica.cc:333] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.148320 12469 raft_consensus.cc:2243] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.149158 12469 raft_consensus.cc:2272] T 54361b2742ab4613b55749cbb699113b P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.161855 12469 tablet_replica.cc:333] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.162914 12469 raft_consensus.cc:2243] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.165067 12469 raft_consensus.cc:2272] T f22eab7d8372432799f894a161194acd P 4dafe08a4f404502822a0732fe570de4 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.173849 12469 tablet_replica.cc:333] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.175063 12469 raft_consensus.cc:2243] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.175889 12469 raft_consensus.cc:2272] T 0d7e68c4d38b4465937dee955f980255 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.180378 12469 tablet_replica.cc:333] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.181612 12469 raft_consensus.cc:2243] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.182747 12469 raft_consensus.cc:2272] T bdd4114edc6d43c08d41462fd399480c P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.238801 12469 tablet_replica.cc:333] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.243755 12469 raft_consensus.cc:2243] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.245355 12469 raft_consensus.cc:2272] T 52052a89b7534d0488f41d7f8e905a14 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.263878 12469 tablet_replica.cc:333] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.270783 12469 raft_consensus.cc:2243] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 42 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.271746 12469 raft_consensus.cc:2272] T 56f25c6fe9f245d99ae467bdd3948582 P 4dafe08a4f404502822a0732fe570de4 [term 42 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.284158 12469 tablet_replica.cc:333] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.285065 12469 raft_consensus.cc:2243] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.285768 12469 pending_rounds.cc:70] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Trying to abort 1 pending ops.
I20260709 12:26:40.286043 12469 pending_rounds.cc:77] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4: Aborting op as it isn't in flight: id { term: 35 index: 22 } timestamp: 7305625528549998592 op_type: NO_OP noop_request { }
I20260709 12:26:40.286415 12469 raft_consensus.cc:2889] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:40.286772 12469 raft_consensus.cc:2272] T 8debd745c407406c88fc03ec3a21f4db P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.303035 12469 tablet_replica.cc:333] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.304244 12469 raft_consensus.cc:2243] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.316082 13543 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 27 to 28, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 28 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.317343 12469 raft_consensus.cc:2272] T 46d57087aee64ae9a498d3d60b92ab92 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.321363 12469 tablet_replica.cc:333] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.334859 12469 raft_consensus.cc:2243] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.336356 12469 raft_consensus.cc:2272] T f136fb5f4e574b6db1336e9434176d83 P 4dafe08a4f404502822a0732fe570de4 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.346215 12469 tablet_replica.cc:333] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.354511 12469 raft_consensus.cc:2243] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.355571 12469 pending_rounds.cc:70] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Trying to abort 1 pending ops.
I20260709 12:26:40.355886 12469 pending_rounds.cc:77] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4: Aborting op as it isn't in flight: id { term: 34 index: 23 } timestamp: 7305625546442903552 op_type: NO_OP noop_request { }
I20260709 12:26:40.356276 12469 raft_consensus.cc:2889] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:26:40.356640 12469 raft_consensus.cc:2272] T ac3728586e264940a93f76650edf9ad3 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:40.374430 13547 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 245) took 19477 ms (client timeout 15000 ms). Trace:
I20260709 12:26:40.374691 12469 tablet_replica.cc:333] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.375902 12469 raft_consensus.cc:2243] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.377184 12469 raft_consensus.cc:2272] T e1d1b82fa8024303a24cb00ab8e7f845 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.380369 12469 tablet_replica.cc:333] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.381357 12469 raft_consensus.cc:2243] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.382019 12469 raft_consensus.cc:2272] T e9c30191f356495b9732659f925abc9f P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:40.375021 13547 rpcz_store.cc:269] 0709 12:26:20.897177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.069087 (+13171910us) service_pool.cc:224] Handling call
0709 12:26:40.374310 (+6305223us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:40.058870 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:40.059473 (+   603us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:40.059507 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:26:40.059527 (+    20us) tablet.cc:662] Decoding operations
0709 12:26:40.060469 (+   942us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:40.060547 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:40.060582 (+    35us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:26:40.061319 (+   737us) tablet.cc:701] Row locks acquired
0709 12:26:40.061355 (+    36us) write_op.cc:260] PREPARE: finished
0709 12:26:40.061522 (+   167us) write_op.cc:270] Start()
0709 12:26:40.061653 (+   131us) write_op.cc:276] Timestamp: P: 1783600000061465 usec, L: 0
0709 12:26:40.061677 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:26:40.070543 (+  8866us) log.cc:844] Serialized 3271 byte log entry
0709 12:26:40.138620 (+ 68077us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.185527 (+ 46907us) write_op.cc:301] APPLY: starting
0709 12:26:40.186904 (+  1377us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:40.201880 (+ 14976us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:40.214470 (+ 12590us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:40.243629 (+ 29159us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:40.243934 (+   305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:26:40.243976 (+    42us) write_op.cc:312] APPLY: finished
0709 12:26:40.244265 (+   289us) log.cc:844] Serialized 81 byte log entry
0709 12:26:40.266645 (+ 22380us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.266813 (+   168us) write_op.cc:454] Released schema lock
0709 12:26:40.266877 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:40.267388 (+   511us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.267415 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11145,"reported_tablets":50,"spinlock_wait_cycles":15104,"child_traces":[["op",{"apply.queue_time_us":45919,"apply.run_cpu_time_us":14369,"apply.run_wall_time_us":81855,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37631,"num_ops":8,"prepare.queue_time_us":27924,"prepare.run_cpu_time_us":4067,"prepare.run_wall_time_us":12063,"replication_time_us":76748,"thread_start_us":40249,"threads_started":2}]]}
I20260709 12:26:40.392917 12469 tablet_replica.cc:333] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.394011 12469 raft_consensus.cc:2243] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.395277 12469 raft_consensus.cc:2272] T 6c6e2c2ada9f4af79430ba4cd2c6558b P 4dafe08a4f404502822a0732fe570de4 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.398589 12469 tablet_replica.cc:333] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
W20260709 12:26:40.408136 13543 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 162) took 18165 ms (client timeout 15000 ms). Trace:
I20260709 12:26:40.410883 12469 raft_consensus.cc:2243] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.411664 12469 raft_consensus.cc:2272] T 7ff03c62343d402fa469da8080d33a6d P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:40.408797 13543 rpcz_store.cc:269] 0709 12:26:22.242731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.975328 (+14732597us) service_pool.cc:224] Handling call
0709 12:26:37.083046 (+107718us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b5400002210. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539098fdb1 00007f5390acfb71 00007f5390ad00b2 00007f5399a49759 00007f53999e1a76 00007f53999b2d58 00007f5399ab104e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:26:40.408013 (+3324967us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:40.343857 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:40.344506 (+   649us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:40.344547 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:26:40.344568 (+    21us) tablet.cc:662] Decoding operations
0709 12:26:40.344956 (+   388us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:40.345023 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:40.345065 (+    42us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:26:40.345285 (+   220us) tablet.cc:701] Row locks acquired
0709 12:26:40.345305 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:26:40.345471 (+   166us) write_op.cc:270] Start()
0709 12:26:40.345611 (+   140us) write_op.cc:276] Timestamp: P: 1783600000345421 usec, L: 0
0709 12:26:40.345636 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:26:40.346475 (+   839us) log.cc:844] Serialized 533 byte log entry
0709 12:26:40.349421 (+  2946us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.349718 (+   297us) write_op.cc:301] APPLY: starting
0709 12:26:40.349817 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:40.350754 (+   937us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:40.350851 (+    97us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:40.351588 (+   737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:40.351773 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:26:40.351834 (+    61us) write_op.cc:312] APPLY: finished
0709 12:26:40.352049 (+   215us) log.cc:844] Serialized 25 byte log entry
0709 12:26:40.352236 (+   187us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.352326 (+    90us) write_op.cc:454] Released schema lock
0709 12:26:40.352373 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:40.353893 (+  1520us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.353925 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26588,"reported_tablets":28,"spinlock_wait_cycles":74424064,"child_traces":[["op",{"apply.queue_time_us":106,"apply.run_cpu_time_us":3069,"apply.run_wall_time_us":4104,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4188,"num_ops":1,"prepare.queue_time_us":170,"prepare.run_cpu_time_us":2999,"prepare.run_wall_time_us":2999,"replication_time_us":3632}]]}
I20260709 12:26:40.415174 12469 tablet_replica.cc:333] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.416152 12469 raft_consensus.cc:2243] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.417315 12469 raft_consensus.cc:2272] T 4613215991134c66b270c6024bec9d73 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.420511 12469 tablet_replica.cc:333] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.421588 12469 raft_consensus.cc:2243] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Raft consensus shutting down.
W20260709 12:26:40.425402 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.418840 13542 catalog_manager.cc:5697] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 31, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 31 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.432487 13542 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.438885 12469 raft_consensus.cc:2272] T 80eb1d112d05410a917177e781e466ed P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.442459 12469 tablet_replica.cc:333] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.443536 12469 raft_consensus.cc:2243] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.444252 12469 raft_consensus.cc:2272] T a2dcf256ec7a4ea6be04f95be4459f59 P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.447232 12469 tablet_replica.cc:333] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.448064 12469 raft_consensus.cc:2243] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.448698 12469 raft_consensus.cc:2272] T 79b849c107424cc79f4cbd223feeeb47 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.434127 13542 catalog_manager.cc:5697] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 26 to 34. New cstate: current_term: 34 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.455430 13542 catalog_manager.cc:5697] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 28 to 29, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.456935 13542 catalog_manager.cc:5697] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 25 to 30, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 30 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.458837 13542 catalog_manager.cc:5697] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.467799 12469 tablet_replica.cc:333] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.471098 12469 raft_consensus.cc:2243] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.472359 12469 raft_consensus.cc:2272] T 3934ae8ea567487da7a989adece218c2 P 4dafe08a4f404502822a0732fe570de4 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.460315 13542 catalog_manager.cc:5697] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.475500 12469 tablet_replica.cc:333] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.476415 12469 raft_consensus.cc:2243] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.475986 13542 catalog_manager.cc:5697] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 36 to 38, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 38 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.477061 12469 raft_consensus.cc:2272] T 48612be3c56f482c81533c30abebe60b P 4dafe08a4f404502822a0732fe570de4 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.477934 13542 catalog_manager.cc:5697] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 33 to 34, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 34 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.486680 13542 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 26 to 30, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 30 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.491052 13542 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.492082 12469 tablet_replica.cc:333] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.493006 12469 raft_consensus.cc:2243] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.492435 13542 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 28 to 30, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 30 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.493795 12469 raft_consensus.cc:2272] T 8916aaed46c543b59ce09ffca942c33d P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.496902 12469 tablet_replica.cc:333] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.497781 12469 raft_consensus.cc:2243] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.499118 12469 raft_consensus.cc:2272] T b1d64b0184dc44a1afc1b5885669d80d P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.493937 13542 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 30 to 33, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 33 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.513679 13542 catalog_manager.cc:5697] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.515741 13542 catalog_manager.cc:5697] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 32 to 33, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 33 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.516516 12469 tablet_replica.cc:333] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.517531 12469 raft_consensus.cc:2243] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.519152 12469 raft_consensus.cc:2272] T 15d97b0b20fd480e8f728a8d38f52e03 P 4dafe08a4f404502822a0732fe570de4 [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:26:40.528343 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.530674 12469 tablet_replica.cc:333] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.531674 12469 raft_consensus.cc:2243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.531401 13542 catalog_manager.cc:5697] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 30 to 31, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 31 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.532963 12469 raft_consensus.cc:2272] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P 4dafe08a4f404502822a0732fe570de4 [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.532974 13542 catalog_manager.cc:5697] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 22 to 32, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 32 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.536098 13542 catalog_manager.cc:5697] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.537351 13542 catalog_manager.cc:5697] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 22 to 29. New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.541754 12469 tablet_replica.cc:333] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.542716 12469 raft_consensus.cc:2243] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.544271 12469 raft_consensus.cc:2272] T af56607c7c564ce4bda1487902e25c7f P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.544803 26061 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.545423 26061 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:40.548182 26061 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:40.552896 12469 tablet_replica.cc:333] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.539506 13542 catalog_manager.cc:5697] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 35 to 37. New cstate: current_term: 37 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:40.552664 13693 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.550527 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:40.553802 13542 catalog_manager.cc:5697] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 35 to 36, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 36 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.554932 13693 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:40.555878 26061 raft_consensus.cc:2749] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election lost for term 39. Reason: could not achieve majority
I20260709 12:26:40.555336 13542 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 32 to 34. New cstate: current_term: 34 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:40.559417 13542 catalog_manager.cc:5697] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:40.566953 12469 raft_consensus.cc:2243] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.568216 12469 raft_consensus.cc:2272] T 28452187c3924de9940235e9b965477d P 4dafe08a4f404502822a0732fe570de4 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.571228 12469 tablet_replica.cc:333] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.572043 12469 raft_consensus.cc:2243] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.580114 12469 raft_consensus.cc:2272] T 0a27ddca262a4e0e8c46331cd36be4d1 P 4dafe08a4f404502822a0732fe570de4 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.590353 12469 tablet_replica.cc:333] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.591351 12469 raft_consensus.cc:2243] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.591987 12469 raft_consensus.cc:2272] T 3e52c8b752b54bd9b959b2363df455a4 P 4dafe08a4f404502822a0732fe570de4 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.617431 12469 tablet_replica.cc:333] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.618533 12469 raft_consensus.cc:2243] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 12:26:40.621064 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.626809 26061 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.638939 26061 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:40.641352 26061 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:40.643236 12469 raft_consensus.cc:2272] T a9e96f5f06294edba36ea19efe454394 P 4dafe08a4f404502822a0732fe570de4 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.643590 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:40.645857 13693 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.653268 26055 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.653965 26055 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:40.656380 26055 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:40.663987 12469 tablet_replica.cc:333] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.664634 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:40.666869 13718 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
W20260709 12:26:40.668782 13767 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.670048 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:40.670617 12469 raft_consensus.cc:2243] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.672542 25922 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:26:40.674990 25922 raft_consensus.cc:493] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.675403 25922 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:40.676163 12469 raft_consensus.cc:2272] T 30d1d057032c4ae0ad5ae21d374a2baa P 4dafe08a4f404502822a0732fe570de4 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.665902 13693 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:40.690819 26045 raft_consensus.cc:2749] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:26:40.701786 12469 tablet_replica.cc:333] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.702809 12469 raft_consensus.cc:2243] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:26:40.704149 12469 raft_consensus.cc:2272] T 77fe434d99534108934a1cdcf44e68ac P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.707330 12469 tablet_replica.cc:333] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4: stopping tablet replica
I20260709 12:26:40.708060 12469 raft_consensus.cc:2243] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:26:40.708911 12469 raft_consensus.cc:2272] T de32481a3be147a5b854251468fc667b P 4dafe08a4f404502822a0732fe570de4 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:26:40.703856 25922 raft_consensus.cc:515] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:40.726387 13767 leader_election.cc:336] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.732493 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b7a0a5ab87d4adbb517d125624bc8ad" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:40.733209 13718 raft_consensus.cc:3060] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:26:40.745515 13718 raft_consensus.cc:2468] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:40.747023 13766 leader_election.cc:304] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:40.748041 26055 raft_consensus.cc:2804] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:26:40.748901 26055 raft_consensus.cc:697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:40.749966 26055 consensus_queue.cc:237] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:40.762399 25922 leader_election.cc:290] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:40.788062 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:40.790984 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:40.807533 13541 catalog_manager.cc:5697] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 34 to 36. New cstate: current_term: 36 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:40.821869 14391 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 245 similar messages]
W20260709 12:26:40.824692 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.827306 13541 catalog_manager.cc:5697] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 37 to 39. New cstate: current_term: 39 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.829736 13541 catalog_manager.cc:5697] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:40.850013 26055 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.850850 26055 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:40.853590 26055 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:40.858284 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:40.860524 13767 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.864625 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:40.865495 13718 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:40.867626 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:40.869599 26053 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:40.870179 26053 raft_consensus.cc:493] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.870682 26053 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:40.939188 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.947907 26053 raft_consensus.cc:515] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:40.954909 13767 leader_election.cc:336] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.949173 13542 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45092 (request call id 240) took 6662 ms. Trace:
I20260709 12:26:40.955761 13542 rpcz_store.cc:276] 0709 12:26:34.462711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.288478 (+3825767us) service_pool.cc:224] Handling call
0709 12:26:40.597898 (+2309420us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5400328128. stack: 00007f5390ad5343 00007f5390ad5240 00007f53904641d4 00007f539a1a2304 00007f539a1a2299 00007f539755e161 00007f53975e83af 00007f53975e9b11 00007f5399ae9fa1 00007f5399aea7ff 00007f53999a9a1b 00007f5399ab190e 00007f5397f722a4 00007f5397f72232 00007f5397f72161 00007f5397f720dc
0709 12:26:40.949092 (+351194us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:40.598839 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:40.599405 (+   566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:40.599441 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:26:40.599461 (+    20us) tablet.cc:662] Decoding operations
0709 12:26:40.601279 (+  1818us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:40.601354 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:40.601395 (+    41us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:26:40.603591 (+  2196us) tablet.cc:701] Row locks acquired
0709 12:26:40.603617 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:26:40.603808 (+   191us) write_op.cc:270] Start()
0709 12:26:40.604017 (+   209us) write_op.cc:276] Timestamp: P: 1783600000603751 usec, L: 0
0709 12:26:40.604040 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:26:40.604940 (+   900us) log.cc:844] Serialized 8935 byte log entry
0709 12:26:40.619131 (+ 14191us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.619662 (+   531us) write_op.cc:301] APPLY: starting
0709 12:26:40.619749 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:40.632233 (+ 12484us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:40.632340 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:40.724694 (+ 92354us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:40.725177 (+   483us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=0,mrs_lookups=0
0709 12:26:40.725236 (+    59us) write_op.cc:312] APPLY: finished
0709 12:26:40.725700 (+   464us) log.cc:844] Serialized 204 byte log entry
0709 12:26:40.728031 (+  2331us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.728281 (+   250us) write_op.cc:454] Released schema lock
0709 12:26:40.728346 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:40.751080 (+ 22734us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.751117 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25197,"reported_tablets":50,"spinlock_wait_cycles":4569472,"child_traces":[["op",{"apply.queue_time_us":296,"apply.run_cpu_time_us":28618,"apply.run_wall_time_us":131362,"cfile_cache_hit":92,"cfile_cache_hit_bytes":96324,"num_ops":23,"prepare.queue_time_us":143,"prepare.run_cpu_time_us":6193,"prepare.run_wall_time_us":6425,"replication_time_us":14962}]]}
I20260709 12:26:40.967947 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "749228d3459a444a9382bc38bbfcbbee" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 38 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:40.973994 13718 raft_consensus.cc:3060] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Advancing to term 39
I20260709 12:26:40.977550 13541 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43972 (request call id 163) took 3155 ms. Trace:
I20260709 12:26:40.978624 26063 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.972440 26045 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:40.984277 26045 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:40.988613 26045 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:40.989876 13718 raft_consensus.cc:2468] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 39.
I20260709 12:26:40.984161 26063 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:40.993008 13693 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.968576 26053 leader_election.cc:290] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:40.995950 13767 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:26:40.996892 26063 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.003049 13766 leader_election.cc:304] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.004616 26063 raft_consensus.cc:2804] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:41.005065 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.005592 26063 raft_consensus.cc:697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:40.977986 13541 rpcz_store.cc:276] 0709 12:26:37.821779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.894449 (+1072670us) service_pool.cc:224] Handling call
0709 12:26:40.977436 (+2082987us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:40.928802 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:40.929640 (+   838us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:40.929686 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:26:40.929707 (+    21us) tablet.cc:662] Decoding operations
0709 12:26:40.930342 (+   635us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:40.930405 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:40.930438 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:26:40.930806 (+   368us) tablet.cc:701] Row locks acquired
0709 12:26:40.930828 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:26:40.931032 (+   204us) write_op.cc:270] Start()
0709 12:26:40.931165 (+   133us) write_op.cc:276] Timestamp: P: 1783600000930972 usec, L: 0
0709 12:26:40.931190 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:26:40.931917 (+   727us) log.cc:844] Serialized 1289 byte log entry
0709 12:26:40.933912 (+  1995us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.934401 (+   489us) write_op.cc:301] APPLY: starting
0709 12:26:40.934494 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:40.936629 (+  2135us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:40.936730 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:40.939556 (+  2826us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:40.939773 (+   217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:26:40.939817 (+    44us) write_op.cc:312] APPLY: finished
0709 12:26:40.940163 (+   346us) log.cc:844] Serialized 41 byte log entry
0709 12:26:40.941110 (+   947us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.941212 (+   102us) write_op.cc:454] Released schema lock
0709 12:26:40.941255 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:40.941639 (+   384us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:40.941673 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":33317,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":277,"apply.run_cpu_time_us":6374,"apply.run_wall_time_us":7210,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12564,"num_ops":3,"prepare.queue_time_us":534,"prepare.run_cpu_time_us":3455,"prepare.run_wall_time_us":3455,"replication_time_us":2594}]]}
I20260709 12:26:41.006748 26063 consensus_queue.cc:237] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 38.20, Last appended by leader: 20, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.007754 13693 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:41.008961 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.010056 26045 raft_consensus.cc:2749] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:41.010632 13766 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:41.011865 26053 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:41.045463 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:40.982712 13545 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.079473 13545 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.094435 13545 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.122299 13545 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.139884 26045 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.142762 13726 raft_consensus.cc:1275] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:26:41.143383 13545 catalog_manager.cc:5697] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 29 to 32. New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.144892 26045 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.148408 13545 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 36 to 38, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 38 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.153331 26063 consensus_queue.cc:1048] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:41.153180 13545 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 23 to 27, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to 4dafe08a4f404502822a0732fe570de4 (127.12.45.65). New cstate: current_term: 27 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.157536 26061 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.159356 26061 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.159888 26045 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.161605 26061 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:41.202746 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45581.3us
I20260709 12:26:41.227950 13542 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 33 to 37. New cstate: current_term: 37 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:41.247751 13542 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 38 to 44, leader changed from a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 44 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:41.251436 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.251302 13542 catalog_manager.cc:5697] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 31 to 33, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 33 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:41.252118 13796 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 26.
I20260709 12:26:41.252805 13542 catalog_manager.cc:5697] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 28 to 33, leader changed from b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 33 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:41.254210 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:41.256596 13693 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:41.257236 26068 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:41.258594 26064 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.261019 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.262074 13796 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:41.264341 26064 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.255399 25922 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.260754 26068 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.285732 26068 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:41.286878 26072 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.287261 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.287549 26072 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.288339 26069 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:41.288821 26069 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.289247 26069 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:41.289969 26072 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.292821 13545 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45106 (request call id 246) took 3797 ms. Trace:
I20260709 12:26:41.293300 13545 rpcz_store.cc:276] 0709 12:26:37.495218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.487819 (+992601us) service_pool.cc:224] Handling call
0709 12:26:41.292652 (+2804833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:41.165206 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:41.165810 (+   604us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:41.165848 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:26:41.165867 (+    19us) tablet.cc:662] Decoding operations
0709 12:26:41.166761 (+   894us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:41.166828 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:41.166863 (+    35us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:26:41.167485 (+   622us) tablet.cc:701] Row locks acquired
0709 12:26:41.167505 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:26:41.167658 (+   153us) write_op.cc:270] Start()
0709 12:26:41.167786 (+   128us) write_op.cc:276] Timestamp: P: 1783600001167608 usec, L: 0
0709 12:26:41.167812 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:26:41.168539 (+   727us) log.cc:844] Serialized 2849 byte log entry
0709 12:26:41.194528 (+ 25989us) op_driver.cc:464] REPLICATION: finished
0709 12:26:41.195542 (+  1014us) write_op.cc:301] APPLY: starting
0709 12:26:41.195672 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:41.207357 (+ 11685us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:41.207462 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:41.215722 (+  8260us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:41.215937 (+   215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:26:41.215987 (+    50us) write_op.cc:312] APPLY: finished
0709 12:26:41.216277 (+   290us) log.cc:844] Serialized 73 byte log entry
0709 12:26:41.216459 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:41.216590 (+   131us) write_op.cc:454] Released schema lock
0709 12:26:41.216638 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:41.217045 (+   407us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:41.217074 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1726,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":618,"apply.run_cpu_time_us":11175,"apply.run_wall_time_us":21483,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29316,"num_ops":7,"prepare.queue_time_us":810,"prepare.run_cpu_time_us":3841,"prepare.run_wall_time_us":15727,"replication_time_us":13984}]]}
W20260709 12:26:41.308660 13693 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.309979 13693 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.299131 25922 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.329365 13721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.331048 25922 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.308331 26068 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.333601 26068 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.339679 26064 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.341168 26045 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.341769 26045 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.345115 26045 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.351298 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:41.352170 13796 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:26:41.353749 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:41.354594 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:41.355165 13767 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:41.358453 26064 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.359527 26064 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.361958 26064 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.364322 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.364322 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:41.367823 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:41.375118 13725 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 31.
I20260709 12:26:41.375849 13796 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:41.376567 13726 raft_consensus.cc:2393] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 27: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:41.379659 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:41.386121 13767 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.369159 13693 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.386754 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:41.393502 13693 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.394824 26065 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.399309 26065 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.399662 26069 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.398675 26068 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:26:41.401770 26065 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.402151 26068 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:41.403357 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.404278 13794 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
I20260709 12:26:41.405071 26076 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.405763 26076 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.407850 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:41.407660 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:41.408207 26063 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:26:41.408655 13767 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.409569 13767 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.410115 13767 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:41.395337 13541 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "4dafe08a4f404502822a0732fe570de4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.412355 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:41.412587 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.413192 13796 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:41.414557 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:41.415628 13693 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.414956 13541 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 32. New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } }
I20260709 12:26:41.413210 14422 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:41.401763 26069 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.408241 26076 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.412079 26064 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:41.424129 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.423259 26031 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:41.425148 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d21a9fba1484f4985407f4cde83b77c" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:41.403244 26068 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.438442 26069 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:41.439002 26069 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.439461 26069 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:41.428059 26055 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:26:41.428735 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.432193 13796 raft_consensus.cc:3060] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:41.423767 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:41.431499 26064 raft_consensus.cc:493] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.449468 26064 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:41.452050 26031 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.452265 13796 raft_consensus.cc:2468] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:26:41.452628 26031 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
W20260709 12:26:41.453585 13693 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.457432 26045 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.458070 26045 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.459225 26065 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.459803 26065 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.462386 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:41.463039 26045 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.464289 26065 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:41.467643 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 43973.9us
I20260709 12:26:41.467749 26076 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
W20260709 12:26:41.436823 13767 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.468454 26076 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.468997 26076 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:41.470760 26055 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.471379 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.471287 26055 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:41.474596 14391 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.476054 26055 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.477011 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:41.477698 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:41.479455 13693 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.484818 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:41.487305 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50530.5us
I20260709 12:26:41.495322 26031 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.497747 26031 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.490959 26065 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
I20260709 12:26:41.491619 26073 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:26:41.493251 26063 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.532397 26045 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.533861 26045 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.536692 26045 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.523690 26068 raft_consensus.cc:2804] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:41.527611 26063 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.544004 26073 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.544656 26073 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:41.547479 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 64369.9us
W20260709 12:26:41.558672 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 77411.8us
I20260709 12:26:41.561232 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:41.531850 26074 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.562480 13542 catalog_manager.cc:5697] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 28 to 29, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 29 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.550016 26061 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.549444 26068 raft_consensus.cc:697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:41.565475 13542 catalog_manager.cc:5697] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 38 to 39, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 39 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:41.557145 26072 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:41.576570 13767 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.577505 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:41.579635 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.583738 13725 raft_consensus.cc:1275] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 38 index: 20. Preceding OpId from leader: term: 39 index: 21. (index mismatch)
I20260709 12:26:41.588186 26073 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.590792 26055 consensus_queue.cc:1048] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:41.592116 26069 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.578490 26063 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.568670 26061 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.581197 26072 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.583473 26045 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.564996 26074 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.598695 26076 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.580694 26068 consensus_queue.cc:237] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.603271 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
W20260709 12:26:41.614113 13767 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.614874 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:26:41.619120 13767 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.619994 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:41.617249 13726 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:41.622637 25922 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:41.623572 26061 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.624620 26088 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.625437 26088 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.608368 26075 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.629069 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.629524 13725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.631044 13796 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 38: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
W20260709 12:26:41.632126 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 152801us
I20260709 12:26:41.612090 26045 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.633035 26075 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.629906 26064 raft_consensus.cc:515] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.635248 26072 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.635391 26088 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.642685 13726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46704
I20260709 12:26:41.664247 13726 raft_consensus.cc:606] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Received request to transfer leadership
W20260709 12:26:41.646062 13693 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.652150 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.626075 26074 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.665272 13693 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:41.665560 13796 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
I20260709 12:26:41.680792 13725 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:41.682206 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.682832 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.683465 13794 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:41.683642 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:41.662603 26063 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:41.690866 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.688220 26074 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.695952 26074 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.684670 26061 raft_consensus.cc:2749] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:41.683306 25922 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.688829 26063 raft_consensus.cc:493] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.692142 26085 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.721755 26063 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:41.722556 13726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46704
I20260709 12:26:41.723470 13726 raft_consensus.cc:606] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Received request to transfer leadership
I20260709 12:26:41.700801 26065 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.740120 26065 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.742800 26074 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.708038 26055 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election won for term 37
W20260709 12:26:41.756472 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 81827.9us
I20260709 12:26:41.712636 25922 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.728482 26068 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.759446 26055 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.756630 26061 raft_consensus.cc:993] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: : Instructing follower b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:26:41.761031 26091 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.770226 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:41.772464 14422 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 40.
I20260709 12:26:41.767529 26074 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.777284 26095 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.774681 26069 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.780071 26064 leader_election.cc:290] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.769098 26055 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:41.780582 26075 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.782819 26045 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.774076 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:41.778218 13693 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.787716 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 96954.5us
I20260709 12:26:41.746706 26085 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.791523 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8b916ed93d478c990f668818f43952" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:41.793718 13718 raft_consensus.cc:3060] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:41.791816 26064 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.793874 26045 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.795392 26064 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.795785 26045 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.797621 26045 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.797700 26064 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.776875 26065 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.806097 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "255f10165e844f2cb50d654996d639cf" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:41.806840 13792 raft_consensus.cc:3060] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:41.778942 26074 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.815416 13792 raft_consensus.cc:2468] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:41.800204 25922 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.774963 26099 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.819350 26085 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.786023 26076 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.819557 26099 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.788479 26100 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:41.822286 26100 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.822705 26100 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:41.822814 26089 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.823488 26089 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.825197 26089 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.779102 26095 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.827741 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.828212 26095 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:41.829598 13767 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.769444 26068 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.830978 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.831648 13726 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:41.831844 26068 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.790561 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.834232 26068 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:26:41.834602 26068 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.816097 26074 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:41.835443 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45023us
I20260709 12:26:41.842397 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.845952 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.788738 26097 raft_consensus.cc:1081] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Signalling peer b708d8f5f6ac4aabb1eb5cb821ace195 to start an election
I20260709 12:26:41.830089 13767 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:41.849889 26075 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:41.850432 26075 raft_consensus.cc:493] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.850999 26075 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:41.850862 26063 raft_consensus.cc:515] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.852157 13796 raft_consensus.cc:2393] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:41.834985 26068 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:41.847497 13726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582"
dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46704
I20260709 12:26:41.854993 13726 raft_consensus.cc:606] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Received request to transfer leadership
I20260709 12:26:41.855974 26063 leader_election.cc:290] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.859010 13693 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.859860 13718 raft_consensus.cc:2468] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 32.
I20260709 12:26:41.822310 26099 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.865151 13766 leader_election.cc:304] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.862594 26045 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:41.868526 26045 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.868954 26045 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:41.866899 26064 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:41.866734 26088 raft_consensus.cc:2804] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:41.874410 26072 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.875090 26072 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.877430 26072 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:26:41.880187 13767 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.880951 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:41.881233 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
W20260709 12:26:41.880962 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:41.881953 14419 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 39.
W20260709 12:26:41.882365 13767 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.882792 13767 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:41.884240 14422 raft_consensus.cc:2393] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 38: Already voted for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in this term.
I20260709 12:26:41.881295 25922 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.881386 26088 raft_consensus.cc:697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:41.885535 25922 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.886353 26088 consensus_queue.cc:237] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.887850 25922 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.888541 25922 raft_consensus.cc:2804] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:41.889243 25922 raft_consensus.cc:697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
W20260709 12:26:41.889354 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.889919 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:41.891212 13767 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.892117 13767 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.892483 13767 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.892853 13767 leader_election.cc:336] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:41.893461 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:41.893764 13767 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.894129 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:41.777036 26091 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.900516 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.899257 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 38 candidate_status { last_received { term: 37 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.899771 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:26:41.898813 13767 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.900629 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52052a89b7534d0488f41d7f8e905a14" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:41.904170 13718 raft_consensus.cc:3060] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:41.906435 26103 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:41.890481 25922 consensus_queue.cc:237] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.908670 26091 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.911708 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.912065 13542 catalog_manager.cc:5697] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 31 to 32, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 32 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:41.913524 13542 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 26 to 27, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 27 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:41.916682 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.919518 26100 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.920316 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:41.920936 14418 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 40.
I20260709 12:26:41.921792 26100 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:41.912648 13794 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 35.
I20260709 12:26:41.910149 26053 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:26:41.894372 26097 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.926561 26075 raft_consensus.cc:515] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.931154 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.926759 26097 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.934494 13800 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:41.927791 13792 raft_consensus.cc:2393] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 38: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:41.927138 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:41.936201 14418 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 35.
I20260709 12:26:41.928805 26075 leader_election.cc:290] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.930797 26103 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.931808 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:41.934545 26055 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.937664 26103 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:41.896324 26076 raft_consensus.cc:2749] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:26:41.942677 13796 raft_consensus.cc:2393] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 37: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:41.896929 26063 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.945019 26063 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.946188 13795 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 29.
I20260709 12:26:41.947432 26063 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:41.949771 13720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1d1b82fa8024303a24cb00ab8e7f845" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 38 candidate_status { last_received { term: 37 index: 19 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:41.950596 13720 raft_consensus.cc:3060] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:41.952039 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 37 candidate_status { last_received { term: 36 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:41.952955 13722 raft_consensus.cc:3060] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:26:41.955829 26099 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.956444 26099 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:41.957226 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:41.957710 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d64b0184dc44a1afc1b5885669d80d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:41.958535 14418 raft_consensus.cc:3060] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:41.958760 26099 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.962620 13801 raft_consensus.cc:2393] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 41: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:41.966656 26103 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.968067 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.968937 13726 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 29.
I20260709 12:26:41.969144 26063 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader pre-election won for term 40
I20260709 12:26:41.969692 26063 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:41.968950 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:41.976897 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 64222.5us
I20260709 12:26:41.979509 13718 raft_consensus.cc:2468] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 30.
I20260709 12:26:41.979560 13720 raft_consensus.cc:2468] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
W20260709 12:26:41.981161 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 76212us
I20260709 12:26:41.983721 13722 raft_consensus.cc:2468] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 37.
I20260709 12:26:41.985586 26097 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:41.971844 13766 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:41.981546 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:41.990192 13693 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:41.990939 13693 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:41.970131 26063 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Advancing to term 40
I20260709 12:26:41.992228 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:42.005015 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:42.003475 14422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582"
dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
 from {username='slave'} at 127.0.0.1:48712
W20260709 12:26:42.005637 13767 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.007726 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:42.005978 14422 raft_consensus.cc:493] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 44 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:42.009723 14422 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 44 FOLLOWER]: Advancing to term 45
I20260709 12:26:41.986779 26100 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election lost for term 38. Reason: could not achieve majority
I20260709 12:26:42.009459 26073 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader pre-election won for term 41
I20260709 12:26:42.016903 26073 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.017473 26073 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Advancing to term 41
I20260709 12:26:41.994923 26069 raft_consensus.cc:2749] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:26:42.024089 26045 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.026429 26045 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.028770 26068 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.031324 26068 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:42.012162 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.039039 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:42.013445 26075 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Leader pre-election won for term 36
W20260709 12:26:42.043054 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 48781.1us
I20260709 12:26:42.012485 13766 leader_election.cc:304] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.014046 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:26:41.942689 26055 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:42.039134 14418 raft_consensus.cc:2468] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 41.
I20260709 12:26:42.009289 26053 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:26:42.043656 13693 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.044181 13726 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 24.
I20260709 12:26:42.044318 13766 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.044940 26075 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.055202 26075 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:42.046058 26053 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.046058 26100 raft_consensus.cc:2804] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:42.058990 26100 raft_consensus.cc:697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.059942 26100 consensus_queue.cc:237] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:42.053967 13693 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.044440 13767 leader_election.cc:336] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.055958 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:42.056773 26055 raft_consensus.cc:2804] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:42.052845 14422 raft_consensus.cc:515] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 45 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.061956 26055 raft_consensus.cc:697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.062024 26083 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:42.062736 26083 raft_consensus.cc:493] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.063133 26083 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:42.063163 26055 consensus_queue.cc:237] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 37.19, Last appended by leader: 19, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.057942 26063 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.056354 13766 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.067094 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 45 candidate_status { last_received { term: 44 index: 23 } } ignore_live_leader: true dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:42.071770 13726 raft_consensus.cc:3055] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Stepping down as leader of term 44
I20260709 12:26:42.068085 14422 leader_election.cc:290] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 45 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.061111 26053 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:42.072117 13726 raft_consensus.cc:740] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 LEADER]: Becoming Follower/Learner. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.070676 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.073681 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.052316 26105 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:42.076066 25922 raft_consensus.cc:2804] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:26:42.081252 26099 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:42.076085 26088 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:26:42.078431 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:42.083010 13801 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 27.
I20260709 12:26:42.081543 25922 raft_consensus.cc:697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 37 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.082194 26099 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.084301 26099 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:42.082314 26088 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.085335 26088 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:42.084501 25922 consensus_queue.cc:237] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 36.23, Last appended by leader: 23, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.076968 13726 consensus_queue.cc:260] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 44.23, Last appended by leader: 23, Current term: 44, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.092628 13726 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 44 FOLLOWER]: Advancing to term 45
W20260709 12:26:42.097956 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57951.4us
I20260709 12:26:42.100636 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:42.104480 26073 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:42.105131 13692 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.107197 26073 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:42.121671 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:42.099757 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.119800 13726 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 45 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 45.
I20260709 12:26:42.124524 13795 raft_consensus.cc:2393] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 41: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:42.125368 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.130002 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:42.131294 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2603f7eea4f49ab89f0ebb1304ee239" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.132318 13792 raft_consensus.cc:3060] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:42.125633 26045 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:26:42.133759 26045 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.134366 26045 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:42.135735 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4613215991134c66b270c6024bec9d73" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.136348 13800 raft_consensus.cc:3060] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:26:42.130658 26098 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:26:42.139647 26098 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.140069 26098 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:42.130765 13796 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:42.155225 26063 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:42.159224 13795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:42.159930 13795 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 37.
W20260709 12:26:42.160123 13693 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.160637 13693 leader_election.cc:304] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:42.162446 26045 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.169061 13542 catalog_manager.cc:5697] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 31 to 32, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:42.172451 13693 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.172724 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.174537 13800 raft_consensus.cc:2468] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 33.
W20260709 12:26:42.175038 12469 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:42.175969 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.174778 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.172508 13792 raft_consensus.cc:2468] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
W20260709 12:26:42.177207 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.176903 14391 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.179679 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.183545 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56f25c6fe9f245d99ae467bdd3948582" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 45 candidate_status { last_received { term: 44 index: 23 } } ignore_live_leader: true dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:42.183791 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:42.184285 13796 raft_consensus.cc:3060] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 44 FOLLOWER]: Advancing to term 45
I20260709 12:26:42.172278 26097 raft_consensus.cc:2749] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
W20260709 12:26:42.184993 13767 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.189230 13767 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.189932 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:42.191533 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:42.192215 13767 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.194994 13792 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:26:42.195406 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88a3d1d336b2495ea9a3a7ae1d7d37a2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 41 candidate_status { last_received { term: 40 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:42.194023 26103 raft_consensus.cc:3060] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Advancing to term 41
W20260709 12:26:42.196239 13767 leader_election.cc:336] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.196367 26069 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
W20260709 12:26:42.197082 13767 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.196974 26069 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:42.197866 13767 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.206547 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f136fb5f4e574b6db1336e9434176d83" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 40 candidate_status { last_received { term: 39 index: 27 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:42.207365 14419 raft_consensus.cc:3060] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Advancing to term 40
W20260709 12:26:42.208690 13693 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.209177 14391 leader_election.cc:336] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 45 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.210668 13764 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:26:42.211431 13693 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.211781 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:42.203430 26063 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:26:42.214607 26045 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.223304 26069 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.224305 26099 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.229240 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
W20260709 12:26:42.236527 13767 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.239166 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:42.239789 13767 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.241315 26109 raft_consensus.cc:2804] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:42.243182 26109 raft_consensus.cc:697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 41 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.244220 26109 consensus_queue.cc:237] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 40.27, Last appended by leader: 27, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.241698 26069 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:26:42.248173 26099 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.252116 26105 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Leader pre-election won for term 38
I20260709 12:26:42.252660 26105 raft_consensus.cc:493] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.253087 26105 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:42.241496 13796 raft_consensus.cc:2468] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 45 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 45.
I20260709 12:26:42.241621 14419 raft_consensus.cc:2468] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 40.
I20260709 12:26:42.227592 26083 raft_consensus.cc:515] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:42.263551 13598 auto_rebalancer.cc:893] Removing replace marker failed: Replica b4f6bfcda63148e4965ca02c639336f1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 45 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:42.264155 13598 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 56f25c6fe9f245d99ae467bdd3948582 already in progress
W20260709 12:26:42.264470 13598 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 56f25c6fe9f245d99ae467bdd3948582 already in progress
I20260709 12:26:42.265285 26083 leader_election.cc:290] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:42.226583 26053 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.270996 26053 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:42.225718 26075 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.272461 13726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:42.272769 13801 raft_consensus.cc:2393] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b708d8f5f6ac4aabb1eb5cb821ace195 in current term 36: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:42.273294 13726 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a4188193a10c4d50a555d76ea85c60b7 in current term 30: Already voted for candidate b4f6bfcda63148e4965ca02c639336f1 in this term.
I20260709 12:26:42.224864 26088 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.275714 13722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6e2c2ada9f4af79430ba4cd2c6558b" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1"
I20260709 12:26:42.276122 26088 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.276394 13722 raft_consensus.cc:3060] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:42.277261 26098 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.279577 26098 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.287803 26075 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:42.268680 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.294520 13764 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 40 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:42.289043 13693 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:42.296554 13693 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.297458 26099 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:26:42.297871 26099 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.298251 26099 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:42.298446 26100 raft_consensus.cc:2804] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Leader election won for term 40
I20260709 12:26:42.299141 26100 raft_consensus.cc:697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 40 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.250015 26103 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:26:42.300356 26100 consensus_queue.cc:237] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 39.27, Last appended by leader: 27, Current term: 40, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.308670 26069 raft_consensus.cc:2804] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:26:42.309440 26069 raft_consensus.cc:697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 33 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.310150 14418 raft_consensus.cc:2468] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 41.
I20260709 12:26:42.311100 26045 raft_consensus.cc:2804] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:42.311674 26045 raft_consensus.cc:697] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.311730 26088 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:42.312753 26045 consensus_queue.cc:237] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.314607 26069 consensus_queue.cc:237] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.317642 14391 leader_election.cc:304] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 45 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:42.285966 13722 raft_consensus.cc:2468] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 25.
I20260709 12:26:42.325719 26065 raft_consensus.cc:2804] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 45 FOLLOWER]: Leader election won for term 45
I20260709 12:26:42.326327 26065 raft_consensus.cc:697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 45 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
W20260709 12:26:42.322847 13767 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.327382 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:26:42.327152 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.327086 14390 leader_election.cc:304] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
W20260709 12:26:42.328655 13767 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.330045 14419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d97b0b20fd480e8f728a8d38f52e03" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:42.330852 14419 raft_consensus.cc:3060] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:42.331236 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:26:42.334034 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:42.334756 13792 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:42.334779 13801 raft_consensus.cc:2393] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate b4f6bfcda63148e4965ca02c639336f1 in current term 30: Already voted for candidate a4188193a10c4d50a555d76ea85c60b7 in this term.
I20260709 12:26:42.337533 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46d57087aee64ae9a498d3d60b92ab92" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.338132 13792 raft_consensus.cc:3060] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:26:42.339664 13800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb1d112d05410a917177e781e466ed" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 36 candidate_status { last_received { term: 35 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.340468 13800 raft_consensus.cc:3060] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:26:42.342348 26104 raft_consensus.cc:2804] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:42.348665 13792 raft_consensus.cc:2468] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:42.348537 13542 catalog_manager.cc:5697] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 30, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 30 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:42.351670 14391 leader_election.cc:336] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.362072 13796 raft_consensus.cc:1275] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:26:42.340394 14419 raft_consensus.cc:2468] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 36.
I20260709 12:26:42.370375 13800 raft_consensus.cc:2468] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 36.
I20260709 12:26:42.380765 13764 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 41 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:42.352644 26053 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:26:42.383323 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.383606 13764 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:42.296584 26105 raft_consensus.cc:515] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.371899 26095 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:26:42.395867 26099 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.398399 26099 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:42.387852 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.403963 13693 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.404968 13693 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.406663 13693 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.391304 26104 raft_consensus.cc:697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:42.387483 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.409660 13796 raft_consensus.cc:3060] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:26:42.416042 13726 raft_consensus.cc:1275] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:42.417778 13541 catalog_manager.cc:5697] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 29 to 30, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 30 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:42.419091 26072 consensus_queue.cc:1048] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:42.419610 13541 catalog_manager.cc:5697] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 37 to 38, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 38 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:42.421226 13796 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 28.
I20260709 12:26:42.403514 26086 raft_consensus.cc:2804] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Leader election won for term 41
I20260709 12:26:42.421897 13541 catalog_manager.cc:5697] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 36 to 37, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 37 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:42.425774 13541 catalog_manager.cc:5697] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 40 to 41, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 41 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.423380 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.430991 26105 leader_election.cc:290] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.433727 26045 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:42.434190 26045 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4dafe08a4f404502822a0732fe570de4)
I20260709 12:26:42.434564 26045 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:42.406575 26065 consensus_queue.cc:237] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 44.23, Last appended by leader: 23, Current term: 45, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.442637 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a27ddca262a4e0e8c46331cd36be4d1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.443356 13796 raft_consensus.cc:3060] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Advancing to term 38
I20260709 12:26:42.428575 26086 raft_consensus.cc:697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 41 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.444940 26086 consensus_queue.cc:237] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 40.27, Last appended by leader: 27, Current term: 41, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.423451 26096 raft_consensus.cc:2804] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader election won for term 36
W20260709 12:26:42.450691 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.457356 13796 raft_consensus.cc:2468] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 38.
I20260709 12:26:42.458546 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
W20260709 12:26:42.461297 14391 leader_election.cc:336] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.463632 14391 leader_election.cc:304] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 38 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:42.457710 26096 raft_consensus.cc:697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:42.465296 26096 consensus_queue.cc:237] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.469065 26099 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:26:42.473088 26045 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.460224 13693 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.478668 26095 raft_consensus.cc:2804] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader election won for term 38
I20260709 12:26:42.479075 26099 raft_consensus.cc:2804] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:26:42.479300 26095 raft_consensus.cc:697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 LEADER]: Becoming Leader. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Running, Role: LEADER
I20260709 12:26:42.479705 26099 raft_consensus.cc:697] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.480348 26095 consensus_queue.cc:237] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 37.24, Last appended by leader: 24, Current term: 38, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:42.480664 26099 consensus_queue.cc:237] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.501156 13722 raft_consensus.cc:1275] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:26:42.504472 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.505622 26100 consensus_queue.cc:1048] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:42.509327 13693 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.512993 13693 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.521410 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:42.522307 13792 raft_consensus.cc:3060] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:26:42.532706 13792 raft_consensus.cc:2468] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 32.
W20260709 12:26:42.540066 13693 consensus_peers.cc:597] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:42.540910 13693 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.533805 26098 raft_consensus.cc:2804] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Leader election won for term 36
W20260709 12:26:42.541757 13693 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.542994 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:42.543864 13693 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.544658 13693 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.545827 26098 raft_consensus.cc:697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.534199 26088 consensus_queue.cc:1048] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:26:42.540067 26068 raft_consensus.cc:2804] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:26:42.554435 26068 raft_consensus.cc:697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.555444 26068 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.549592 26045 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:42.560182 13722 raft_consensus.cc:1275] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 37 index: 19. Preceding OpId from leader: term: 38 index: 20. (index mismatch)
I20260709 12:26:42.560681 13542 catalog_manager.cc:5697] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 44 to 45, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 45 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:42.562767 26098 consensus_queue.cc:237] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 35.20, Last appended by leader: 20, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.572141 13792 raft_consensus.cc:1275] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:26:42.569805 13693 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
W20260709 12:26:42.576678 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:42.585624 26088 raft_consensus.cc:2804] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:26:42.586494 26088 raft_consensus.cc:697] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.587652 26088 consensus_queue.cc:237] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:42.589957 26053 consensus_queue.cc:1048] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.008s
W20260709 12:26:42.606504 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.550457 26104 consensus_queue.cc:237] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:42.625860 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 56145.5us
I20260709 12:26:42.628847 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77fe434d99534108934a1cdcf44e68ac" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
W20260709 12:26:42.629673 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:42.629767 13792 raft_consensus.cc:3060] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:26:42.630800 13693 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:42.654508 13792 raft_consensus.cc:2468] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:42.659838 13693 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 
I20260709 12:26:42.638856 26098 consensus_queue.cc:1048] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:26:42.672627 26088 raft_consensus.cc:2804] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:26:42.690666 26121 log.cc:927] Time spent T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:42.696102 13722 raft_consensus.cc:1275] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 36 index: 23. Preceding OpId from leader: term: 37 index: 24. (index mismatch)
I20260709 12:26:42.697789 26075 consensus_queue.cc:1048] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
W20260709 12:26:42.713712 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.714960 26088 raft_consensus.cc:697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:42.718705 26088 consensus_queue.cc:237] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:42.722524 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:42.723147 13693 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:42.725217 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:42.746043 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.826398 13541 catalog_manager.cc:5697] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 40 to 41, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 41 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.829237 13541 catalog_manager.cc:5697] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 39 to 40, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 40 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.831720 13541 catalog_manager.cc:5697] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 35 to 36, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 36 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.848812 13800 raft_consensus.cc:1275] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:26:42.868834 26088 consensus_queue.cc:1048] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
W20260709 12:26:42.886083 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.939563 13725 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 45 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 44 index: 23. Preceding OpId from leader: term: 45 index: 24. (index mismatch)
W20260709 12:26:42.941449 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.943449 13545 catalog_manager.cc:5697] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 32 to 33, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 33 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.946079 13545 catalog_manager.cc:5697] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 30, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 30 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.947608 13545 catalog_manager.cc:5697] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 35 to 36, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 36 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.948829 13545 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 27 to 28, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 28 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:42.951745 26104 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.006s
W20260709 12:26:42.954250 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.956054 14418 raft_consensus.cc:1275] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 40 index: 27. Preceding OpId from leader: term: 41 index: 28. (index mismatch)
I20260709 12:26:42.958865 26076 consensus_queue.cc:1048] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20260709 12:26:42.969004 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.975975 14419 raft_consensus.cc:1275] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
W20260709 12:26:42.977187 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.977903 26096 consensus_queue.cc:1048] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
W20260709 12:26:42.979074 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:42.949973 13545 catalog_manager.cc:5697] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 31 to 32, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 32 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:43.015638 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.018733 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:43.025929 13541 catalog_manager.cc:5697] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 24 to 25, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 25 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:43.036089 13800 raft_consensus.cc:1275] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:26:43.035375 13541 catalog_manager.cc:5697] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: term changed from 37 to 38, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68). New cstate: current_term: 38 leader_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:43.036880 14422 raft_consensus.cc:1275] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 39 index: 27. Preceding OpId from leader: term: 40 index: 28. (index mismatch)
I20260709 12:26:43.043998 13792 raft_consensus.cc:1275] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:26:43.046020 26068 consensus_queue.cc:1048] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:26:43.049084 26068 consensus_queue.cc:1048] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:43.051486 26072 consensus_queue.cc:1048] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20260709 12:26:43.056607 13800 raft_consensus.cc:1275] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 37 index: 24. Preceding OpId from leader: term: 38 index: 25. (index mismatch)
I20260709 12:26:43.060783 13800 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:26:43.062500 14418 raft_consensus.cc:1275] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 40 index: 27. Preceding OpId from leader: term: 41 index: 28. (index mismatch)
I20260709 12:26:43.089856 26103 consensus_queue.cc:1048] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:26:43.067896 26068 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 12:26:43.105321 13795 raft_consensus.cc:1275] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 45 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 44 index: 23. Preceding OpId from leader: term: 45 index: 24. (index mismatch)
I20260709 12:26:43.088353 26145 consensus_queue.cc:1048] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.039s
I20260709 12:26:43.121716 26103 consensus_queue.cc:1048] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
W20260709 12:26:43.123345 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.176970 12469 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13673)
I20260709 12:26:43.177745 13798 raft_consensus.cc:1275] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 35 index: 20. Preceding OpId from leader: term: 36 index: 21. (index mismatch)
W20260709 12:26:43.179057 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:43.182967 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:43.194350 13542 catalog_manager.cc:5697] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 30 to 31, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 31 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.196934 13542 catalog_manager.cc:5697] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 31 to 32, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 32 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.202579 13721 raft_consensus.cc:1275] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Refusing update from remote peer b708d8f5f6ac4aabb1eb5cb821ace195: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:26:43.224655 26142 consensus_queue.cc:1048] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.010s
I20260709 12:26:43.216084 26083 consensus_queue.cc:1048] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.016s
W20260709 12:26:43.231935 26149 log.cc:927] Time spent T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 12:26:43.268231 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.272589 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51924.7us
I20260709 12:26:43.296453 13792 raft_consensus.cc:1275] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:26:43.299289 13792 raft_consensus.cc:1275] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:26:43.313862 26099 consensus_queue.cc:1048] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:43.316082 26099 consensus_queue.cc:1048] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:26:43.318450 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.361749 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.376621 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:43.413501 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.442745 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:43.493263 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:43.510689 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:43.511439 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:43.544021 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50169.9us
I20260709 12:26:43.567508 26063 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.568320 26063 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:43.577358 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:43.596416 26063 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:43.598034 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:26:43.598851 14418 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 38.
I20260709 12:26:43.600450 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:26:43.605247 26100 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Leader pre-election won for term 39
I20260709 12:26:43.605741 26100 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.606534 26100 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:43.621904 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:43.633056 13767 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:43.657487 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:43.658450 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:43.679224 26100 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:43.697386 13719 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:43.707908 26100 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:26:43.709173 14418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20dc442f4fae4cf49e336a3d350b7760" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 39 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:26:43.709856 14418 raft_consensus.cc:3060] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Advancing to term 39
W20260709 12:26:43.713750 13767 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:43.716678 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:26:43.725425 14418 raft_consensus.cc:2468] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 39.
I20260709 12:26:43.732641 13792 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:26:43.734817 13690 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28452187c3924de9940235e9b965477d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:43.735942 26069 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:43.735709 13764 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 39 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:43.737793 26100 raft_consensus.cc:2804] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Leader election won for term 39
I20260709 12:26:43.739260 26100 raft_consensus.cc:697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:26:43.740307 26100 consensus_queue.cc:237] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 37.25, Last appended by leader: 25, Current term: 39, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:43.743407 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:43.751967 26068 raft_consensus.cc:2955] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:26:43.755168 13792 raft_consensus.cc:2955] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index -1 to 18, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
W20260709 12:26:43.776333 13767 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:43.793678 13542 catalog_manager.cc:5697] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 37 to 39, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 39 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:43.815663 13719 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:26:43.834038 13690 consensus_peers.cc:597] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd4fd9484f184d2c994c269eb10d6ec1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:43.840739 13801 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:26:43.846769 26088 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:26:43.897455 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 52933.3us
W20260709 12:26:43.907388 13693 consensus_peers.cc:597] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:43.907842 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:43.911772 13541 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 18, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 28 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:43.992051 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:44.051656 13792 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Committing config change with OpId 27.20: config changed from index -1 to 20, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:26:44.044746 26088 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Committing config change with OpId 27.20: config changed from index -1 to 20, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
W20260709 12:26:44.063583 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:44.138509 13542 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index -1 to 20, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 27 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:44.164803 26088 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.165709 26088 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:44.168619 26088 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:44.171765 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:44.172413 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:44.174695 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:44.177487 12469 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 13673)
I20260709 12:26:44.179625 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:26:44.180433 13801 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 30.
I20260709 12:26:44.186908 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:44.188009 26088 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:26:44.188452 26088 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.188846 26088 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:44.204901 26088 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:26:44.240617 26088 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Requested vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:26:44.246974 13792 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:26:44.252715 13693 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:44.255383 13801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af56607c7c564ce4bda1487902e25c7f" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7"
I20260709 12:26:44.256112 13801 raft_consensus.cc:3060] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:26:44.267059 14418 raft_consensus.cc:1275] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 37 index: 25. Preceding OpId from leader: term: 39 index: 26. (index mismatch)
I20260709 12:26:44.269685 25922 consensus_queue.cc:1048] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.001s
I20260709 12:26:44.268188 13801 raft_consensus.cc:2468] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4f6bfcda63148e4965ca02c639336f1 in term 31.
I20260709 12:26:44.284523 13539 maintenance_manager.cc:419] P d5e54c6d2c954fbf93d5136276b7b78c: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033363
I20260709 12:26:44.299124 13693 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4
I20260709 12:26:44.300242 26045 raft_consensus.cc:2804] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:26:44.300796 26045 raft_consensus.cc:697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Becoming Leader. State: Replica: b4f6bfcda63148e4965ca02c639336f1, State: Running, Role: LEADER
I20260709 12:26:44.301704 26045 consensus_queue.cc:237] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
W20260709 12:26:44.317394 13764 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ff03c62343d402fa469da8080d33a6d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:44.320480 13718 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:26:44.324707 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:44.333151 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:44.337441 25922 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:44.366305 13542 catalog_manager.cc:5697] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: term changed from 29 to 31, leader changed from 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) to b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66). New cstate: current_term: 31 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:44.369859 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:44.394022 13538 maintenance_manager.cc:643] P d5e54c6d2c954fbf93d5136276b7b78c: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.108s	user 0.038s	sys 0.010s Metrics: {"bytes_written":7664,"delete_count":0,"lbm_write_time_us":1958,"lbm_writes_lt_1ms":7,"reinsert_count":0,"update_count":307}
I20260709 12:26:44.398196 13539 maintenance_manager.cc:419] P d5e54c6d2c954fbf93d5136276b7b78c: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.337368
W20260709 12:26:44.444103 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:44.461696 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:44.490929 26179 ts_tablet_manager.cc:933] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:26:44.498211 26181 ts_tablet_manager.cc:933] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:26:44.498760 26178 log.cc:927] Time spent T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 12:26:44.528215 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 44587.1us
I20260709 12:26:44.603854 26181 tablet_copy_client.cc:323] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
W20260709 12:26:44.632041 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:44.637473 26179 tablet_copy_client.cc:323] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.66:41841
I20260709 12:26:44.639907 26100 raft_consensus.cc:2955] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:26:44.790627 13737 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet 28452187c3924de9940235e9b965477d from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:26:44.792011 13737 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet 28452187c3924de9940235e9b965477d from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-28452187c3924de9940235e9b965477d
I20260709 12:26:44.818042 13736 tablet_copy_service.cc:140] P b4f6bfcda63148e4965ca02c639336f1: Received BeginTabletCopySession request for tablet cd4fd9484f184d2c994c269eb10d6ec1 from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:50270)
I20260709 12:26:44.819310 13736 tablet_copy_service.cc:161] P b4f6bfcda63148e4965ca02c639336f1: Beginning new tablet copy session on tablet cd4fd9484f184d2c994c269eb10d6ec1 from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:50270: session id = b708d8f5f6ac4aabb1eb5cb821ace195-cd4fd9484f184d2c994c269eb10d6ec1
I20260709 12:26:44.870870 13800 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:44.867758 13718 raft_consensus.cc:2955] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Committing config change with OpId 27.16: config changed from index -1 to 16, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:26:44.879177 13737 tablet_copy_source_session.cc:215] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:26:44.881098 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:44.895542 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:44.894272 13736 tablet_copy_source_session.cc:215] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:26:44.933926 26181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28452187c3924de9940235e9b965477d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:44.980312 13542 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 16, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:44.983297 26179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd4fd9484f184d2c994c269eb10d6ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:44.987197 26200 ts_tablet_manager.cc:933] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:44.992326 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:44.993040 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.031982 13767 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:45.044809 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:45.048899 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:45.062935 13722 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
W20260709 12:26:45.072806 13764 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965): Couldn't send request to peer b708d8f5f6ac4aabb1eb5cb821ace195. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 990ed59e36454f16b81d57d0e4f59c4e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:45.081840 26200 tablet_copy_client.cc:323] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
I20260709 12:26:45.093838 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 7ff03c62343d402fa469da8080d33a6d from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:26:45.094602 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 7ff03c62343d402fa469da8080d33a6d from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-7ff03c62343d402fa469da8080d33a6d
I20260709 12:26:45.095234 13800 raft_consensus.cc:1275] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
W20260709 12:26:45.102982 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.120292 26142 consensus_queue.cc:1048] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:26:45.130502 13811 tablet_copy_source_session.cc:215] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:26:45.142582 26181 tablet_copy_client.cc:806] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:26:45.143795 26181 tablet_copy_client.cc:670] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:26:45.151378 26200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ff03c62343d402fa469da8080d33a6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:26:45.157814 26181 tablet_copy_client.cc:538] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:26:45.165308 26198 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:26:45.178304 12469 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 13673)
I20260709 12:26:45.201300 26181 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
W20260709 12:26:45.232390 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.233330 26198 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Committing config change with OpId 32.18: config changed from index -1 to 18, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:26:45.305889 26179 tablet_copy_client.cc:806] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:26:45.306666 26179 tablet_copy_client.cc:670] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
W20260709 12:26:45.308749 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.271493 13726 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Committing config change with OpId 32.18: config changed from index -1 to 18, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } } }
I20260709 12:26:45.323726 26179 tablet_copy_client.cc:538] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:26:45.331314 26200 tablet_copy_client.cc:806] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:26:45.332330 26200 tablet_copy_client.cc:670] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
W20260709 12:26:45.353003 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.372061 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:45.385248 26200 tablet_copy_client.cc:538] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:26:45.399746 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.411513 13542 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index -1 to 18, NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) added. New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:45.415343 26200 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
W20260709 12:26:45.420272 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:45.468317 26179 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
W20260709 12:26:45.588790 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 53413us
I20260709 12:26:45.587755 13538 maintenance_manager.cc:643] P d5e54c6d2c954fbf93d5136276b7b78c: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 1.188s	user 0.112s	sys 0.031s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":27823,"cfile_init":5,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":39766,"dirs.run_cpu_time_us":668,"dirs.run_wall_time_us":2342,"lbm_read_time_us":4858,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":27,"lbm_write_time_us":19594,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":11,"peak_mem_usage":308819,"reinsert_count":0,"thread_start_us":87902,"threads_started":2,"update_count":307}
W20260709 12:26:45.590768 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 128243us
I20260709 12:26:45.628394 26219 ts_tablet_manager.cc:933] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: Initiating tablet copy from peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:26:45.648401 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 47587.9us
W20260709 12:26:45.700635 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.732671 26219 tablet_copy_client.cc:323] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Beginning tablet copy session from remote peer at address 127.12.45.67:40415
W20260709 12:26:45.743355 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:45.756660 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.759289 13811 tablet_copy_service.cc:140] P a4188193a10c4d50a555d76ea85c60b7: Received BeginTabletCopySession request for tablet 990ed59e36454f16b81d57d0e4f59c4e from peer b708d8f5f6ac4aabb1eb5cb821ace195 ({username='slave'} at 127.0.0.1:44068)
I20260709 12:26:45.759914 13811 tablet_copy_service.cc:161] P a4188193a10c4d50a555d76ea85c60b7: Beginning new tablet copy session on tablet 990ed59e36454f16b81d57d0e4f59c4e from peer b708d8f5f6ac4aabb1eb5cb821ace195 at {username='slave'} at 127.0.0.1:44068: session id = b708d8f5f6ac4aabb1eb5cb821ace195-990ed59e36454f16b81d57d0e4f59c4e
W20260709 12:26:45.778910 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.808993 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.809799 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:45.817378 13811 tablet_copy_source_session.cc:215] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:26:45.813990 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.849210 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:45.858906 26219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990ed59e36454f16b81d57d0e4f59c4e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:26:45.872802 13693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111) [suppressed 458 similar messages]
W20260709 12:26:45.875061 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.884130 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.883419 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.915216 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.931576 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:45.967355 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:45.975591 26200 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:45.977291 26200 tablet_bootstrap.cc:492] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:26:45.982839 26179 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=22 overwritten=2 applied=20 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:45.987068 26200 ts_tablet_manager.cc:1403] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.572s	user 0.055s	sys 0.012s
I20260709 12:26:45.988160 26179 tablet_bootstrap.cc:492] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:26:45.991263 26200 raft_consensus.cc:359] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:45.995422 26179 ts_tablet_manager.cc:1403] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.527s	user 0.064s	sys 0.015s
W20260709 12:26:45.995200 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:45.985054 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:45.995713 26200 raft_consensus.cc:740] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:26:46.002434 26200 consensus_queue.cc:260] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:26:46.009433 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:46.012132 26179 raft_consensus.cc:359] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:46.020685 26200 ts_tablet_manager.cc:1434] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 12:26:46.030490 26179 raft_consensus.cc:740] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
W20260709 12:26:46.030161 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:46.031316 26179 consensus_queue.cc:260] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:46.033340 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-7ff03c62343d402fa469da8080d33a6d received from {username='slave'} at 127.0.0.1:44068
I20260709 12:26:46.033888 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-7ff03c62343d402fa469da8080d33a6d on tablet 7ff03c62343d402fa469da8080d33a6d with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:46.039629 14422 raft_consensus.cc:1217] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.15->[27.16-27.16]   Dedup: 27.16->[]
I20260709 12:26:46.063275 26179 ts_tablet_manager.cc:1434] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.002s
I20260709 12:26:46.071866 13736 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-cd4fd9484f184d2c994c269eb10d6ec1 received from {username='slave'} at 127.0.0.1:50270
I20260709 12:26:46.072448 13736 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-cd4fd9484f184d2c994c269eb10d6ec1 on tablet cd4fd9484f184d2c994c269eb10d6ec1 with peer b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:26:46.075155 26181 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=2 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:46.104885 26181 tablet_bootstrap.cc:492] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:26:46.131875 26219 tablet_copy_client.cc:806] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 0 data blocks...
I20260709 12:26:46.135907 26181 ts_tablet_manager.cc:1403] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.935s	user 0.077s	sys 0.023s
I20260709 12:26:46.154237 26219 tablet_copy_client.cc:670] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Starting download of 1 WAL segments...
I20260709 12:26:46.166389 26219 tablet_copy_client.cc:538] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:26:46.173236 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:46.178937 12469 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 13673)
I20260709 12:26:46.178059 26181 raft_consensus.cc:359] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:46.180449 26181 raft_consensus.cc:740] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:26:46.181125 26181 consensus_queue.cc:260] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:46.184916 26181 ts_tablet_manager.cc:1434] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.003s
I20260709 12:26:46.195398 13736 tablet_copy_service.cc:342] P b4f6bfcda63148e4965ca02c639336f1: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-28452187c3924de9940235e9b965477d received from {username='slave'} at 127.0.0.1:50270
I20260709 12:26:46.195911 13736 tablet_copy_service.cc:434] P b4f6bfcda63148e4965ca02c639336f1: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-28452187c3924de9940235e9b965477d on tablet 28452187c3924de9940235e9b965477d with peer b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:46.233631 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:46.237097 14422 raft_consensus.cc:1217] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.19->[27.20-27.20]   Dedup: 27.20->[]
W20260709 12:26:46.246500 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:26:46.232800 26219 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap starting.
W20260709 12:26:46.316592 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:46.427789 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:46.531613 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:46.565629 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:46.603683 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:46.609738 26221 raft_consensus.cc:1064] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:26:46.613595 14422 raft_consensus.cc:1217] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEARNER]: Deduplicated request from leader. Original: 28.17->[28.18-28.18]   Dedup: 28.18->[]
I20260709 12:26:46.623310 26221 consensus_queue.cc:237] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 14, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:46.643337 26219 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:46.644506 26219 tablet_bootstrap.cc:492] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: Bootstrap complete.
I20260709 12:26:46.645372 26219 ts_tablet_manager.cc:1403] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent bootstrapping tablet: real 0.413s	user 0.063s	sys 0.032s
W20260709 12:26:46.647006 13693 consensus_peers.cc:597] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:26:46.672750 14422 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:46.673568 13726 raft_consensus.cc:1275] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:26:46.674492 26219 raft_consensus.cc:359] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
I20260709 12:26:46.676112 26053 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:46.676573 26219 raft_consensus.cc:740] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEARNER]: Becoming Follower/Learner. State: Replica: b708d8f5f6ac4aabb1eb5cb821ace195, State: Initialized, Role: LEARNER
I20260709 12:26:46.677644 26219 consensus_queue.cc:260] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: NON_VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: true } }
W20260709 12:26:46.683703 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:46.687913 26219 ts_tablet_manager.cc:1434] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.000s
I20260709 12:26:46.691268 26053 consensus_queue.cc:1048] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:26:46.704018 13811 tablet_copy_service.cc:342] P a4188193a10c4d50a555d76ea85c60b7: Request end of tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-990ed59e36454f16b81d57d0e4f59c4e received from {username='slave'} at 127.0.0.1:44068
I20260709 12:26:46.704576 13811 tablet_copy_service.cc:434] P a4188193a10c4d50a555d76ea85c60b7: ending tablet copy session b708d8f5f6ac4aabb1eb5cb821ace195-990ed59e36454f16b81d57d0e4f59c4e on tablet 990ed59e36454f16b81d57d0e4f59c4e with peer b708d8f5f6ac4aabb1eb5cb821ace195
W20260709 12:26:46.718693 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:46.729195 26196 raft_consensus.cc:2955] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Committing config change with OpId 27.17: config changed from index 16 to 17, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:46.746764 13719 raft_consensus.cc:2955] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:46.756639 26217 raft_consensus.cc:1064] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:26:46.760110 26217 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 18, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:46.791610 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:46.801388 13541 catalog_manager.cc:5697] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 16 to 17, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:46.824043 14422 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:26:46.826642 26232 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:26:46.827266 13801 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 12:26:46.830997 13693 consensus_peers.cc:597] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:46.832090 26217 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:26:46.845495 14418 raft_consensus.cc:2955] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index 16 to 17, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:46.880729 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:46.896656 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:46.897221 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:46.944346 14418 raft_consensus.cc:1217] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEARNER]: Deduplicated request from leader. Original: 32.17->[32.18-32.18]   Dedup: 32.18->[]
I20260709 12:26:46.961876 26045 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Committing config change with OpId 27.21: config changed from index 20 to 21, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:46.979029 14422 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Committing config change with OpId 27.21: config changed from index 20 to 21, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:47.007347 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.008114 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:47.025141 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:47.059317 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 50814.8us
I20260709 12:26:47.062362 13795 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Committing config change with OpId 27.21: config changed from index 20 to 21, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.106534 26045 raft_consensus.cc:1064] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:26:47.110180 26045 consensus_queue.cc:237] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 16, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:47.126924 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:47.130926 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:47.135492 14422 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 LEARNER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:47.135483 13795 raft_consensus.cc:1275] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:26:47.137082 26232 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:47.147166 26232 consensus_queue.cc:1048] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:26:47.158329 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:26:47.179790 12469 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 13673)
I20260709 12:26:47.195497 13542 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 20 to 21, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.324113 13726 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:47.345285 26232 raft_consensus.cc:2955] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Committing config change with OpId 28.19: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:47.362396 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:47.375995 13800 raft_consensus.cc:2955] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:47.380954 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:47.390399 14422 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 27 index: 22. (index mismatch)
I20260709 12:26:47.392151 26045 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:47.392685 14422 raft_consensus.cc:2955] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Committing config change with OpId 28.19: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.396441 13795 raft_consensus.cc:1275] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Refusing update from remote peer b4f6bfcda63148e4965ca02c639336f1: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 27 index: 22. (index mismatch)
I20260709 12:26:47.421350 26045 consensus_queue.cc:1048] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:26:47.429530 26217 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Committing config change with OpId 27.22: config changed from index 21 to 22, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.435690 13541 catalog_manager.cc:5697] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.440693 14418 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Committing config change with OpId 27.22: config changed from index 21 to 22, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:47.466346 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:47.505234 13536 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cd4fd9484f184d2c994c269eb10d6ec1 with cas_config_opid_index 21: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:26:47.519765 13541 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 reported cstate change: config changed from index 21 to 22, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 27 leader_uuid: "b4f6bfcda63148e4965ca02c639336f1" committed_config { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.527877 13801 raft_consensus.cc:2955] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Committing config change with OpId 27.22: config changed from index 21 to 22, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.544917 26196 raft_consensus.cc:1064] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: attempting to promote NON_VOTER b708d8f5f6ac4aabb1eb5cb821ace195 to VOTER
I20260709 12:26:47.551551 26196 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 16, Current term: 32, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:26:47.619549 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:47.680903 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cd4fd9484f184d2c994c269eb10d6ec1: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:26:47.681380 14418 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEARNER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:47.690750 26221 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:26:47.703380 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.707890 13767 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:47.728976 13726 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:26:47.748642 26250 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:26:47.779251 26250 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Committing config change with OpId 32.19: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:47.791085 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:47.792166 13726 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Committing config change with OpId 32.19: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.852111 13545 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 reported cstate change: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
I20260709 12:26:47.861506 14418 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Committing config change with OpId 32.19: config changed from index 18 to 19, b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68) changed from NON_VOTER to VOTER. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } attrs { replace: true } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:47.877439 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.926329 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.981760 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.982506 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.994118 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:47.996682 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.020257 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:48.031278 13800 consensus_queue.cc:237] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:26:48.055457 14418 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:26:48.063401 26250 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:26:48.078521 26193 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Committing config change with OpId 32.20: config changed from index 19 to 20, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:48.080947 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:48.087950 13722 raft_consensus.cc:1275] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Refusing update from remote peer a4188193a10c4d50a555d76ea85c60b7: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
W20260709 12:26:48.094506 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:48.095917 26250 consensus_queue.cc:1048] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.002s
W20260709 12:26:48.097115 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:26:48.100678 14422 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Committing config change with OpId 32.20: config changed from index 19 to 20, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:48.108107 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:48.125577 13537 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 990ed59e36454f16b81d57d0e4f59c4e with cas_config_opid_index 19: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:26:48.128089 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:26:48.134780 13541 catalog_manager.cc:5697] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: config changed from index 19 to 20, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New cstate: current_term: 32 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:48.154436 13722 raft_consensus.cc:2955] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Committing config change with OpId 32.20: config changed from index 19 to 20, VOTER 4dafe08a4f404502822a0732fe570de4 (127.12.45.65) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } } }
W20260709 12:26:48.180383 12469 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:48.191869 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.231418 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:48.291034 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.394629 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:48.412652 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.437300 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:48.464995 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:48.486722 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:48.568804 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.571781 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:48.596210 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:48.638883 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.670359 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:48.951215 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:48.963562 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 990ed59e36454f16b81d57d0e4f59c4e: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:49.049443 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 45293.3us
W20260709 12:26:49.109764 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.110661 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:49.181149 12469 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:49.193468 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:49.236079 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.240432 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:49.257875 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.266366 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.279340 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.290948 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:49.364185 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:49.401203 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:49.495362 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.523653 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.813436 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:26:49.872031 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 263869us
W20260709 12:26:49.925216 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.966912 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:49.997227 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cd4fd9484f184d2c994c269eb10d6ec1: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:50.011524 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.062026 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:50.155980 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.181734 12469 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:50.203886 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.298424 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.341041 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:50.397930 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.405287 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.441143 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.475136 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.487679 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:50.497131 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.502763 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.516569 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.647081 13748 debug-util.cc:398] Leaking SignalData structure 0x7b080038ac60 after lost signal to thread 12472
W20260709 12:26:50.648725 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08001e7ae0 after lost signal to thread 13584
W20260709 12:26:50.650070 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.650588 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800102ca0 after lost signal to thread 13751
W20260709 12:26:50.652055 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08002fcae0 after lost signal to thread 13825
W20260709 12:26:50.653577 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08003adb60 after lost signal to thread 14449
W20260709 12:26:50.667075 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:50.783453 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:50.851600 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.873961 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.876071 13767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111) [suppressed 505 similar messages]
W20260709 12:26:50.940338 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:50.971158 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 57746.1us
W20260709 12:26:50.990962 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.048246 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:51.066170 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:51.108314 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.129071 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.182336 12469 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:51.187355 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.210171 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.248575 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.250506 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:51.311331 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.413025 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80446.3us
W20260709 12:26:51.484081 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:51.534883 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.553128 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.572036 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:51.755396 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.796514 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.834975 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:51.841957 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:51.861181 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.877772 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:51.931524 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.985592 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:51.986202 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:52.083529 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:52.166546 13537 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46412.5us
W20260709 12:26:52.169957 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cd4fd9484f184d2c994c269eb10d6ec1: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:52.182992 12469 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:52.273046 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:52.279557 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:26:52.389321 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:52.412887 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:52.492491 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:52.612360 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:52.695685 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:52.722579 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:52.772993 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 50484.7us
W20260709 12:26:52.771040 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:52.862860 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:52.909895 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:52.956885 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.001840 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.028756 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.129354 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.183632 12469 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:53.196331 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.230823 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.358029 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 130047us
W20260709 12:26:53.406304 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 46577.1us
W20260709 12:26:53.441570 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.456382 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:53.512305 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.525672 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.528988 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:53.529690 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:53.527506 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.531812 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:53.646770 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.678987 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.704885 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:53.742097 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:53.754602 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:53.779587 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.792052 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:53.845857 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:53.917800 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:54.129911 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 72851.5us
W20260709 12:26:54.137153 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 82284.6us
W20260709 12:26:54.159986 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:54.184330 12469 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:54.194712 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:54.239593 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.279883 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:54.280647 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.317394 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.318091 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:54.418866 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:54.437438 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.489481 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:54.505882 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:54.633742 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 59703.7us
W20260709 12:26:54.657662 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:54.676749 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.686714 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:26:54.727393 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:54.740042 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.815461 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:54.853631 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:55.056372 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 70887.3us
W20260709 12:26:55.067997 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:55.082127 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:55.133265 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51195.6us
W20260709 12:26:55.182423 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 51709.7us
W20260709 12:26:55.185200 12469 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:55.318979 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:55.323487 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:55.504531 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.534341 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.611387 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.613281 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82993.7us
W20260709 12:26:55.630245 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.655859 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.672451 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.675797 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.735157 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.758137 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.883010 13693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111) [suppressed 468 similar messages]
W20260709 12:26:55.906181 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:55.965452 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:55.966771 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.018218 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.021005 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.044945 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.140959 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:56.185817 12469 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:56.274382 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.290297 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.295308 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.342298 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.378441 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.383445 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cd4fd9484f184d2c994c269eb10d6ec1: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:26:56.394093 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.417408 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:56.462749 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:56.500808 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:56.596629 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:56.674191 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.675076 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.824505 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 52582.2us
W20260709 12:26:56.832664 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:56.930693 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:56.933439 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 92538.8us
W20260709 12:26:56.951434 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:56.952183 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:56.978884 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:56.983621 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:57.016983 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:57.034180 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.048640 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:57.070567 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.134419 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.186436 12469 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:57.186318 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.268728 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:26:57.402354 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:26:57.431046 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:57.433547 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.466034 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.514096 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.882161 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800132400 after lost signal to thread 12472
W20260709 12:26:57.883652 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8ea0 after lost signal to thread 13584
W20260709 12:26:57.885459 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800232660 after lost signal to thread 13751
W20260709 12:26:57.887321 13822 debug-util.cc:398] Leaking SignalData structure 0x7b0800237200 after lost signal to thread 13825
W20260709 12:26:57.891427 13822 debug-util.cc:398] Leaking SignalData structure 0x7b08003c34a0 after lost signal to thread 14449
W20260709 12:26:57.922869 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:57.951463 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:57.982225 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:58.133452 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.187970 12469 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:58.196851 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.204245 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.234102 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.265075 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.332526 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.399921 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.414170 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.490694 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:58.496826 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.565318 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.648488 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:58.655864 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.682132 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:58.713464 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.753258 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800173820 after lost signal to thread 12472
W20260709 12:26:58.754820 13581 debug-util.cc:398] Leaking SignalData structure 0x7b0800292fc0 after lost signal to thread 13584
W20260709 12:26:58.756505 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080030f0a0 after lost signal to thread 13751
W20260709 12:26:58.757893 13581 debug-util.cc:398] Leaking SignalData structure 0x7b08003a4f40 after lost signal to thread 13825
W20260709 12:26:58.759409 13581 debug-util.cc:398] Leaking SignalData structure 0x7b080021a880 after lost signal to thread 14449
W20260709 12:26:58.822466 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.856655 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:58.906445 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:58.985823 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.032673 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.058351 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:59.058465 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:59.113657 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:59.180225 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.183084 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:59.188624 12469 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 13673)
W20260709 12:26:59.220175 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:59.332748 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:59.368110 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:59.415351 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:59.469633 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.477759 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:59.540673 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.549815 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:59.563315 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:26:59.603529 13766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 46171.7us
W20260709 12:26:59.685127 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:59.687296 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:26:59.713399 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.714190 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.722790 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:26:59.724311 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:26:59.757587 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:26:59.847930 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:00.189150 12469 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:00.251739 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:00.294745 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.357295 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:00.481252 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:00.486434 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.492614 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:00.502355 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.569700 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.600840 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.619051 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.684211 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:00.713930 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.721503 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:00.829142 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.888763 13693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111) [suppressed 471 similar messages]
W20260709 12:27:00.917763 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:00.949290 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.125428 14390 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:41841, user_credentials={real_user=slave}} blocked reactor thread for 44390.6us
W20260709 12:27:01.145692 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.190593 12469 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:01.196583 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:01.294528 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.306932 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:01.311235 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.327529 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:01.330322 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.361182 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.365921 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:01.382372 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:01.435774 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.605165 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:01.640069 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:01.702327 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:01.767467 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:01.770766 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:02.191253 12469 thread.cc:527] Waited for 21000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:02.622977 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 730539us
W20260709 12:27:02.627849 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:02.629418 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:02.630695 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:02.632347 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:02.676709 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44396.6us
W20260709 12:27:02.718995 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:02.738457 26373 raft_consensus.cc:493] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:02.739712 26373 raft_consensus.cc:515] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:02.753345 26373 leader_election.cc:290] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:02.756129 13693 leader_election.cc:336] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:02.787181 26373 raft_consensus.cc:493] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:02.787788 26373 raft_consensus.cc:515] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:02.790055 26373 leader_election.cc:290] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
I20260709 12:27:02.790787 26393 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:02.791396 26393 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:02.787221 26378 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:02.798087 26378 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:02.798460 26373 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:02.799078 26373 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:02.800746 26393 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:27:02.801692 26373 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
W20260709 12:27:02.802433 13693 leader_election.cc:336] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:02.811262 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:02.809894 26378 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:27:02.812975 14422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:27:02.813689 14416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:27:02.817763 13798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f22eab7d8372432799f894a161194acd" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 38 candidate_status { last_received { term: 37 index: 24 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:27:02.817975 13796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ab361e956dd4342b80045629d159a2b" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:27:02.818538 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "435988495fcf4cf49f65293e95b5ceb5" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:27:02.821421 13718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3934ae8ea567487da7a989adece218c2" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "b4f6bfcda63148e4965ca02c639336f1" is_pre_election: true
I20260709 12:27:02.818794 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990ed59e36454f16b81d57d0e4f59c4e" candidate_uuid: "b4f6bfcda63148e4965ca02c639336f1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:27:02.828539 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 46159.1us
W20260709 12:27:02.842510 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:02.843217 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:02.843716 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:02.844763 13767 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:02.845302 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:27:02.848332 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:02.848891 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:02.849419 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:02.849766 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:27:02.853470 13766 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:02.854770 26395 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:02.858526 13693 leader_election.cc:304] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:02.859946 26378 raft_consensus.cc:2749] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:27:02.861193 13693 leader_election.cc:304] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:02.862509 26378 raft_consensus.cc:2749] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:02.864619 13693 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:27:02.866432 26378 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:02.869505 13693 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4f6bfcda63148e4965ca02c639336f1; no voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:27:02.871843 26378 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:03.084556 12474 debug-util.cc:398] Leaking SignalData structure 0x7b080000f920 after lost signal to thread 14391
W20260709 12:27:03.193092 12469 thread.cc:527] Waited for 22000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:03.199703 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:03.464280 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:03.470016 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.473067 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 268803us
W20260709 12:27:03.511436 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 44627.5us
W20260709 12:27:03.530968 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.569849 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:03.577661 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08003541c0 after lost signal to thread 12472
W20260709 12:27:03.579156 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08003773a0 after lost signal to thread 13584
W20260709 12:27:03.582609 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6b00 after lost signal to thread 13751
W20260709 12:27:03.586376 14446 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6280 after lost signal to thread 13825
W20260709 12:27:03.591848 14446 debug-util.cc:398] Leaking SignalData structure 0x7b0800392f20 after lost signal to thread 14449
W20260709 12:27:03.610682 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:03.652925 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.653478 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:03.665859 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:03.668601 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:03.709674 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.786062 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.868829 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.872244 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:03.910032 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.012166 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:04.052490 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.163036 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.194005 12469 thread.cc:527] Waited for 23000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:04.315924 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08000f49c0 after lost signal to thread 12472
W20260709 12:27:04.371433 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800299ae0 after lost signal to thread 13584
W20260709 12:27:04.373339 13748 debug-util.cc:398] Leaking SignalData structure 0x7b0800340640 after lost signal to thread 13751
W20260709 12:27:04.386030 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08000a84e0 after lost signal to thread 13825
W20260709 12:27:04.391613 13748 debug-util.cc:398] Leaking SignalData structure 0x7b08002d2ec0 after lost signal to thread 14449
W20260709 12:27:04.392343 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:04.401017 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.495019 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.513818 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.551988 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.599495 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:04.635491 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 80711.7us
W20260709 12:27:04.673651 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cd4fd9484f184d2c994c269eb10d6ec1: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:04.729693 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:04.808174 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:04.954243 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:05.051262 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.051839 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.084225 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.162737 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 63546us
W20260709 12:27:05.163594 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.177009 13764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 77206.8us
W20260709 12:27:05.219534 12469 thread.cc:527] Waited for 24000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:05.234997 13690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:42965, user_credentials={real_user=slave}} blocked reactor thread for 69532.2us
W20260709 12:27:05.320232 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.323331 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:05.403503 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:05.474978 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.478238 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.497314 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:05.503312 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:05.526868 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:05.531953 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:27:05.542073 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:05.560693 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:05.572561 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.639199 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:05.683651 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:05.684763 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:05.685357 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:05.693563 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:05.747269 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:05.752907 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:05.888962 13767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111) [suppressed 395 similar messages]
W20260709 12:27:06.011731 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:06.054399 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.083455 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:06.085166 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.092370 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.180475 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:06.220824 12469 thread.cc:527] Waited for 25000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:06.241048 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.271567 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.290417 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.323555 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:06.350771 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:06.353417 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.582420 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.790593 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:06.806342 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:06.934520 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.050439 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:07.087205 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.120231 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.221527 12469 thread.cc:527] Waited for 26000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:07.244257 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:07.309324 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.324321 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:07.366991 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:07.371994 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:07.455133 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:07.457955 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.541378 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.569463 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:07.606184 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.639456 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:07.647795 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:07.699716 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:07.711254 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 43908.4us
W20260709 12:27:07.800850 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:07.826906 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:07.915100 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:08.073305 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:08.076804 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:08.080075 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:08.173872 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.176817 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:08.179138 13693 consensus_peers.cc:597] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:08.195184 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:08.222195 12469 thread.cc:527] Waited for 27000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:08.240962 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:27:08.257658 13693 consensus_peers.cc:597] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:08.250608 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:27:08.325518 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 83485.5us
W20260709 12:27:08.292490 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 51581.5us
W20260709 12:27:08.369144 13693 consensus_peers.cc:597] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:08.383905 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:08.402829 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:08.553267 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.560570 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.593721 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.607232 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.633867 13693 consensus_peers.cc:597] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.717406 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 12:27:08.719295 13693 consensus_peers.cc:597] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.750322 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:08.951658 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.007426 14391 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 55009.4us
W20260709 12:27:09.017671 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.023671 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 45086.5us
W20260709 12:27:09.142880 13693 consensus_peers.cc:597] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:09.161836 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.218853 13767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33075, user_credentials={real_user=slave}} blocked reactor thread for 57077.2us
W20260709 12:27:09.222832 12469 thread.cc:527] Waited for 28000ms trying to join with diag-logger (tid 13673)
W20260709 12:27:09.230415 13693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:40415, user_credentials={real_user=slave}} blocked reactor thread for 82426.8us
W20260709 12:27:09.242596 13693 consensus_peers.cc:597] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:09.258391 13693 consensus_peers.cc:597] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:09.300354 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.525177 13693 consensus_peers.cc:597] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:09.569473 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:09.660429 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.691485 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:09.796260 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:09.846395 13693 consensus_peers.cc:597] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:09.939766 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.955997 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:09.956748 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:09.960680 13693 consensus_peers.cc:597] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:10.007803 13767 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:10.019934 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:10.056811 13693 consensus_peers.cc:597] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:10.080214 13767 consensus_peers.cc:597] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:10.095387 13767 consensus_peers.cc:597] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:10.207093 13693 consensus_peers.cc:597] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:10.234409 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 12:27:10.289568 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
W20260709 12:27:10.335075 13767 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:10.337112 13767 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:10.344802 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:27:10.435063 13767 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:10.438592 13767 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:27:10.489140 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
W20260709 12:27:10.499396 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:10.525098 14390 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.533923 14390 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.535054 14390 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.548305 13766 consensus_peers.cc:597] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.549683 13766 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.561241 14390 consensus_peers.cc:597] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.567487 14390 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.565501 13766 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.569298 14390 consensus_peers.cc:597] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.600708 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:10.604437 12469 tablet_replica.cc:333] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.605760 12469 raft_consensus.cc:2243] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:10.606065 14390 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.606984 12469 raft_consensus.cc:2272] T 4ce599ec23ec4ac0b358f4e47922bc3f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.609905 12469 tablet_replica.cc:333] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.610787 12469 raft_consensus.cc:2243] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.611493 12469 raft_consensus.cc:2272] T e71fd62c156c4baf990c9e212f69ecc2 P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.614012 12469 tablet_replica.cc:333] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.614918 12469 raft_consensus.cc:2243] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.615569 12469 raft_consensus.cc:2272] T 435988495fcf4cf49f65293e95b5ceb5 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.618963 12469 tablet_replica.cc:333] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
W20260709 12:27:10.619728 13767 consensus_peers.cc:597] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:27:10.619972 12469 raft_consensus.cc:2243] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.620803 12469 raft_consensus.cc:2272] T 63b331138ada4d5e96b6136417e740f7 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.623006 13766 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.625108 12469 tablet_replica.cc:333] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.626754 12469 raft_consensus.cc:2243] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.628558 12469 raft_consensus.cc:2272] T 684ee7a163da446aba11bf15d10d691a P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.631886 12469 tablet_replica.cc:333] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.633009 12469 raft_consensus.cc:2243] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.633813 12469 raft_consensus.cc:2272] T 0b7a0a5ab87d4adbb517d125624bc8ad P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.637635 12469 tablet_replica.cc:333] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.638825 12469 raft_consensus.cc:2243] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.639712 12469 raft_consensus.cc:2272] T 749228d3459a444a9382bc38bbfcbbee P b4f6bfcda63148e4965ca02c639336f1 [term 39 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.642835 13766 consensus_peers.cc:597] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.642941 12469 tablet_replica.cc:333] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.644176 12469 raft_consensus.cc:2243] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.645035 12469 raft_consensus.cc:2272] T b5c65e93a0c1420ca348d0fc678031ca P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.647891 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:10.648958 12469 tablet_replica.cc:333] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.649996 12469 raft_consensus.cc:2243] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.651866 12469 raft_consensus.cc:2272] T 7d21a9fba1484f4985407f4cde83b77c P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.655282 12469 tablet_replica.cc:333] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.656380 12469 raft_consensus.cc:2243] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.657289 12469 raft_consensus.cc:2272] T 3a8b916ed93d478c990f668818f43952 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.660919 12469 tablet_replica.cc:333] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.661871 12469 raft_consensus.cc:2243] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.662791 12469 raft_consensus.cc:2272] T 89717fd329da4af68d87ae1ef1b104fb P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.663368 13766 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.666085 12469 tablet_replica.cc:333] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.667900 12469 raft_consensus.cc:2243] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.668958 12469 raft_consensus.cc:2272] T 6966d62b938845429a9aa8feb8fe2b9b P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.672691 13767 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 12:27:10.672691 12469 tablet_replica.cc:333] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.674225 12469 raft_consensus.cc:2243] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.676236 12469 raft_consensus.cc:2272] T 990ed59e36454f16b81d57d0e4f59c4e P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.679500 12469 tablet_replica.cc:333] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.680527 12469 raft_consensus.cc:2243] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.681742 12469 raft_consensus.cc:2272] T 45dc2f34593649778ec4b80d29638572 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.685343 12469 tablet_replica.cc:333] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
W20260709 12:27:10.686002 13766 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.686338 12469 raft_consensus.cc:2243] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.687312 12469 raft_consensus.cc:2272] T 3cb6d086379f4190b73fc7f692fb490a P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.693691 12469 tablet_replica.cc:333] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
W20260709 12:27:10.694000 14390 consensus_peers.cc:597] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.694998 12469 raft_consensus.cc:2243] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.695901 12469 raft_consensus.cc:2272] T 8ab361e956dd4342b80045629d159a2b P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.696868 13766 consensus_peers.cc:597] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.699165 12469 tablet_replica.cc:333] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.700189 12469 raft_consensus.cc:2243] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.700937 12469 raft_consensus.cc:2272] T 068a8cc7309648e3a661d4d056f11b7a P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.704331 12469 tablet_replica.cc:333] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.705212 12469 raft_consensus.cc:2243] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.705991 12469 raft_consensus.cc:2272] T 2dcbdced6d0a46b394cae3d101fbbf40 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.708860 12469 tablet_replica.cc:333] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.709723 12469 raft_consensus.cc:2243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.710682 12469 raft_consensus.cc:2272] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.713541 12469 tablet_replica.cc:333] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
W20260709 12:27:10.714193 13766 consensus_peers.cc:597] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.714461 12469 raft_consensus.cc:2243] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.715485 12469 raft_consensus.cc:2272] T e1d1b82fa8024303a24cb00ab8e7f845 P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.718489 12469 tablet_replica.cc:333] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.719370 12469 raft_consensus.cc:2243] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.720898 12469 raft_consensus.cc:2272] T e2603f7eea4f49ab89f0ebb1304ee239 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.723960 12469 tablet_replica.cc:333] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.724792 12469 raft_consensus.cc:2243] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:27:10.725512 13766 consensus_peers.cc:597] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.725972 12469 raft_consensus.cc:2272] T cd4fd9484f184d2c994c269eb10d6ec1 P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.726106 13766 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.729328 12469 tablet_replica.cc:333] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.730476 12469 raft_consensus.cc:2243] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.731911 12469 raft_consensus.cc:2272] T 0a0048f5294f4012ad1519c0bc8e9e5a P b4f6bfcda63148e4965ca02c639336f1 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.735363 12469 tablet_replica.cc:333] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.736265 12469 raft_consensus.cc:2243] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.737589 12469 raft_consensus.cc:2272] T 255f10165e844f2cb50d654996d639cf P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.740922 12469 tablet_replica.cc:333] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.741849 12469 raft_consensus.cc:2243] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.743427 12469 raft_consensus.cc:2272] T 54361b2742ab4613b55749cbb699113b P b4f6bfcda63148e4965ca02c639336f1 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.746637 12469 tablet_replica.cc:333] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.747464 12469 raft_consensus.cc:2243] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.748241 12469 raft_consensus.cc:2272] T f22eab7d8372432799f894a161194acd P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.751458 12469 tablet_replica.cc:333] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.752280 12469 raft_consensus.cc:2243] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.753093 12469 raft_consensus.cc:2272] T bdd4114edc6d43c08d41462fd399480c P b4f6bfcda63148e4965ca02c639336f1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.756512 12469 tablet_replica.cc:333] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.757469 12469 raft_consensus.cc:2243] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.758584 12469 raft_consensus.cc:2272] T 52052a89b7534d0488f41d7f8e905a14 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.760901 14390 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.761916 12469 tablet_replica.cc:333] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.763073 12469 raft_consensus.cc:2243] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.764029 12469 raft_consensus.cc:2272] T 85fee8a18ff4429cb870f59aa46b3892 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.765713 14390 consensus_peers.cc:597] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.767372 12469 tablet_replica.cc:333] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.768389 12469 raft_consensus.cc:2243] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 45 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.770356 12469 raft_consensus.cc:2272] T 56f25c6fe9f245d99ae467bdd3948582 P b4f6bfcda63148e4965ca02c639336f1 [term 45 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.771055 13767 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:10.772962 13767 consensus_peers.cc:597] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 12:27:10.773639 12469 tablet_replica.cc:333] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.774621 12469 raft_consensus.cc:2243] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.776046 12469 raft_consensus.cc:2272] T e9c30191f356495b9732659f925abc9f P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.779208 12469 tablet_replica.cc:333] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.780150 12469 raft_consensus.cc:2243] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.780834 12469 raft_consensus.cc:2272] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b4f6bfcda63148e4965ca02c639336f1 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.782299 13766 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.784173 12469 tablet_replica.cc:333] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.785254 12469 raft_consensus.cc:2243] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.786134 12469 raft_consensus.cc:2272] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.789687 12469 tablet_replica.cc:333] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.790563 12469 raft_consensus.cc:2243] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.791884 12469 raft_consensus.cc:2272] T 46d57087aee64ae9a498d3d60b92ab92 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.795023 12469 tablet_replica.cc:333] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.795938 12469 raft_consensus.cc:2243] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.796667 12469 raft_consensus.cc:2272] T ac3728586e264940a93f76650edf9ad3 P b4f6bfcda63148e4965ca02c639336f1 [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.797284 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:27:10.799667 12469 tablet_replica.cc:333] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.800678 12469 raft_consensus.cc:2243] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:10.801594 14390 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.801721 12469 raft_consensus.cc:2272] T 7ff03c62343d402fa469da8080d33a6d P b4f6bfcda63148e4965ca02c639336f1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.805069 12469 tablet_replica.cc:333] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.805930 12469 raft_consensus.cc:2243] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.807670 12469 raft_consensus.cc:2272] T 4613215991134c66b270c6024bec9d73 P b4f6bfcda63148e4965ca02c639336f1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.810854 12469 tablet_replica.cc:333] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.811743 12469 raft_consensus.cc:2243] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 36 LEADER]: Raft consensus shutting down.
W20260709 12:27:10.811815 13766 consensus_peers.cc:597] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.813321 12469 raft_consensus.cc:2272] T 80eb1d112d05410a917177e781e466ed P b4f6bfcda63148e4965ca02c639336f1 [term 36 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.814261 13766 consensus_peers.cc:597] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.815145 13766 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.816425 12469 tablet_replica.cc:333] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.817301 12469 raft_consensus.cc:2243] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.818483 12469 raft_consensus.cc:2272] T 48612be3c56f482c81533c30abebe60b P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.821501 12469 tablet_replica.cc:333] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.822377 12469 raft_consensus.cc:2243] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.823720 12469 raft_consensus.cc:2272] T 3934ae8ea567487da7a989adece218c2 P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.826499 12469 tablet_replica.cc:333] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.827232 12469 raft_consensus.cc:2243] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.829058 12469 raft_consensus.cc:2272] T af56607c7c564ce4bda1487902e25c7f P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.829454 14390 consensus_peers.cc:597] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.832401 12469 tablet_replica.cc:333] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.833393 12469 raft_consensus.cc:2243] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.834693 12469 raft_consensus.cc:2272] T a2dcf256ec7a4ea6be04f95be4459f59 P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.837749 12469 tablet_replica.cc:333] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.838573 12469 raft_consensus.cc:2243] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.839411 12469 raft_consensus.cc:2272] T 79b849c107424cc79f4cbd223feeeb47 P b4f6bfcda63148e4965ca02c639336f1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.842615 12469 tablet_replica.cc:333] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.843427 12469 raft_consensus.cc:2243] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.844481 12469 raft_consensus.cc:2272] T 8916aaed46c543b59ce09ffca942c33d P b4f6bfcda63148e4965ca02c639336f1 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.847833 12469 tablet_replica.cc:333] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.848707 12469 raft_consensus.cc:2243] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.850605 12469 raft_consensus.cc:2272] T 28452187c3924de9940235e9b965477d P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.853828 12469 tablet_replica.cc:333] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.854681 12469 raft_consensus.cc:2243] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.856112 12469 raft_consensus.cc:2272] T 3e52c8b752b54bd9b959b2363df455a4 P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.859786 12469 tablet_replica.cc:333] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.860666 12469 raft_consensus.cc:2243] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.861655 12469 raft_consensus.cc:2272] T a9e96f5f06294edba36ea19efe454394 P b4f6bfcda63148e4965ca02c639336f1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:10.864964 12469 tablet_replica.cc:333] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.865770 12469 raft_consensus.cc:2243] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.866864 12469 raft_consensus.cc:2272] T 30d1d057032c4ae0ad5ae21d374a2baa P b4f6bfcda63148e4965ca02c639336f1 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.867277 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
I20260709 12:27:10.870169 12469 tablet_replica.cc:333] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.871049 12469 raft_consensus.cc:2243] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:27:10.872668 12469 raft_consensus.cc:2272] T 77fe434d99534108934a1cdcf44e68ac P b4f6bfcda63148e4965ca02c639336f1 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.874636 13766 consensus_peers.cc:597] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:10.875793 12469 tablet_replica.cc:333] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1: stopping tablet replica
I20260709 12:27:10.876636 12469 raft_consensus.cc:2243] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:10.877410 12469 raft_consensus.cc:2272] T de32481a3be147a5b854251468fc667b P b4f6bfcda63148e4965ca02c639336f1 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:10.889503 13766 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 500 similar messages]
W20260709 12:27:10.892511 13766 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.921375 14390 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.929162 13766 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.940956 13766 consensus_peers.cc:597] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.953192 13767 consensus_peers.cc:597] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:10.970168 14390 consensus_peers.cc:597] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:10.991770 13766 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:11.015295 13766 consensus_peers.cc:597] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:11.029664 13766 consensus_peers.cc:597] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:11.070410 13767 consensus_peers.cc:597] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:11.132547 13767 consensus_peers.cc:597] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:11.150068 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:11.150195 13767 consensus_peers.cc:597] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:11.186344 13767 consensus_peers.cc:597] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:11.422724 13767 consensus_peers.cc:597] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:11.458973 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20260709 12:27:11.582773 13767 consensus_peers.cc:597] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 12:27:11.780790 26401 raft_consensus.cc:493] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.781466 26401 raft_consensus.cc:515] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:11.784192 26401 leader_election.cc:290] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:11.789942 13766 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:11.798017 13767 leader_election.cc:336] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:11.799196 13767 leader_election.cc:304] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:11.800019 26401 raft_consensus.cc:2749] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:11.803117 26425 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.803714 26425 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:11.806237 26425 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:11.812340 14390 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:11.814313 13794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
I20260709 12:27:11.819998 14391 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:11.821513 26425 raft_consensus.cc:2749] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:11.839107 13767 consensus_peers.cc:597] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 12:27:11.870656 26438 raft_consensus.cc:493] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.871304 26438 raft_consensus.cc:515] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:11.873627 26438 leader_election.cc:290] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:11.879060 13766 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:11.880892 13767 leader_election.cc:336] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:11.882591 13767 leader_election.cc:304] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:11.883459 26438 raft_consensus.cc:2749] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:11.905764 13767 consensus_peers.cc:597] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 12:27:11.952873 26401 raft_consensus.cc:493] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.953547 26401 raft_consensus.cc:515] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:11.956154 26401 leader_election.cc:290] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:11.960290 13767 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:11.969781 26401 raft_consensus.cc:493] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.970997 26401 raft_consensus.cc:515] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:11.973392 26401 leader_election.cc:290] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:11.974325 13766 leader_election.cc:336] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:11.981961 26401 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.982687 26401 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:11.975090 13766 leader_election.cc:304] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:11.986469 26438 raft_consensus.cc:2749] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:11.991050 26438 raft_consensus.cc:493] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.991240 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:27:11.992060 14423 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 27.
I20260709 12:27:11.991587 26438 raft_consensus.cc:515] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:11.993659 13764 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:27:11.994755 26426 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:11.995193 26426 raft_consensus.cc:493] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:11.995558 26426 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:12.006697 26438 leader_election.cc:290] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:27:12.008703 26401 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:27:12.010444 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:27:12.014391 26436 raft_consensus.cc:493] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.012365 26426 raft_consensus.cc:515] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:12.015072 26436 raft_consensus.cc:515] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:12.017508 26436 leader_election.cc:290] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:27:12.021301 26438 raft_consensus.cc:493] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.021975 26438 raft_consensus.cc:515] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:12.022979 26426 leader_election.cc:290] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Requested vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:27:12.024211 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd4fd9484f184d2c994c269eb10d6ec1" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 28 candidate_status { last_received { term: 27 index: 22 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195"
I20260709 12:27:12.024897 14423 raft_consensus.cc:3060] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:12.025317 13767 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:12.037915 13766 leader_election.cc:336] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.038573 13766 leader_election.cc:304] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:27:12.039543 13766 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.042506 26426 raft_consensus.cc:2749] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:12.044943 14423 raft_consensus.cc:2468] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4188193a10c4d50a555d76ea85c60b7 in term 28.
W20260709 12:27:12.045507 13766 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:12.046278 13766 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.046452 13764 leader_election.cc:304] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7, b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 
I20260709 12:27:12.047308 26426 raft_consensus.cc:2804] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:12.047878 26426 raft_consensus.cc:697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Becoming Leader. State: Replica: a4188193a10c4d50a555d76ea85c60b7, State: Running, Role: LEADER
I20260709 12:27:12.048959 26426 consensus_queue.cc:237] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 27.22, Last appended by leader: 22, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:12.055168 26438 leader_election.cc:290] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
I20260709 12:27:12.055869 26401 raft_consensus.cc:493] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.056488 26401 raft_consensus.cc:515] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:12.058796 26401 leader_election.cc:290] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:12.059597 13766 leader_election.cc:336] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:12.062574 13767 leader_election.cc:336] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:12.063153 13767 leader_election.cc:304] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.064121 26421 raft_consensus.cc:2749] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:12.065814 13767 leader_election.cc:336] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:12.067432 13767 leader_election.cc:304] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.076504 26438 raft_consensus.cc:2749] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:12.081084 13766 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:12.081908 13766 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.086071 26426 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
W20260709 12:27:12.086786 13767 leader_election.cc:336] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:12.086663 26426 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:12.087229 13767 leader_election.cc:304] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.087973 26438 raft_consensus.cc:2749] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:12.089864 26438 raft_consensus.cc:493] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.088961 13541 catalog_manager.cc:5697] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 reported cstate change: term changed from 27 to 28, leader changed from b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66) to a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67). New cstate: current_term: 28 leader_uuid: "a4188193a10c4d50a555d76ea85c60b7" committed_config { opid_index: 22 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:12.090845 26426 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), b708d8f5f6ac4aabb1eb5cb821ace195 (127.12.45.68:42965)
I20260709 12:27:12.091476 14423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "a4188193a10c4d50a555d76ea85c60b7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" is_pre_election: true
I20260709 12:27:12.092552 26437 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.093132 26437 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:12.090457 26438 raft_consensus.cc:515] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:12.098346 26438 leader_election.cc:290] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:12.101788 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:12.102749 13767 leader_election.cc:336] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:12.107542 13767 leader_election.cc:304] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
W20260709 12:27:12.108335 14390 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.108956 14390 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.103107 26437 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:12.104734 13766 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.110323 26424 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:12.113147 13766 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: b4f6bfcda63148e4965ca02c639336f1, b708d8f5f6ac4aabb1eb5cb821ace195
I20260709 12:27:12.110718 26426 raft_consensus.cc:2749] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:12.114346 26426 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:12.119704 13766 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.123942 26438 raft_consensus.cc:493] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.124663 26438 raft_consensus.cc:515] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } }
I20260709 12:27:12.127187 26438 leader_election.cc:290] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:12.140740 13767 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:12.144061 13767 leader_election.cc:336] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:12.142980 13766 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.144584 13767 leader_election.cc:304] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.153649 26438 raft_consensus.cc:2749] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:12.158241 13767 leader_election.cc:336] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:12.158866 13767 leader_election.cc:304] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a4188193a10c4d50a555d76ea85c60b7; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.160063 26438 raft_consensus.cc:2749] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:27:12.176394 13767 consensus_peers.cc:597] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:27:12.213809 26437 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.214656 26437 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:12.224452 26437 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:12.228305 13767 consensus_peers.cc:597] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:12.229694 14390 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.230075 13792 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28452187c3924de9940235e9b965477d" candidate_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a4188193a10c4d50a555d76ea85c60b7" is_pre_election: true
W20260709 12:27:12.231818 14391 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:12.232334 14391 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.232431 13792 raft_consensus.cc:2468] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b708d8f5f6ac4aabb1eb5cb821ace195 in term 28.
I20260709 12:27:12.233834 26437 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:12.242007 14391 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 12:27:12.251693 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
I20260709 12:27:12.259954 26437 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:12.261469 26437 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:12.265014 26437 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:12.272372 14391 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:12.276747 14390 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:12.277405 14390 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:12.279414 26424 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:12.298748 13767 consensus_peers.cc:597] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 12:27:12.343796 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:12.347668 12469 tablet_replica.cc:333] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.348914 12469 raft_consensus.cc:2243] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.350533 12469 raft_consensus.cc:2272] T e71fd62c156c4baf990c9e212f69ecc2 P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.353981 12469 tablet_replica.cc:333] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.355120 12469 raft_consensus.cc:2243] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.355829 12469 raft_consensus.cc:2272] T 4ce599ec23ec4ac0b358f4e47922bc3f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.359046 12469 tablet_replica.cc:333] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.359931 12469 raft_consensus.cc:2243] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.361048 12469 raft_consensus.cc:2272] T 684ee7a163da446aba11bf15d10d691a P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.363996 12469 tablet_replica.cc:333] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.365012 12469 raft_consensus.cc:2243] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.367326 12469 raft_consensus.cc:2272] T 435988495fcf4cf49f65293e95b5ceb5 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:12.369971 13767 consensus_peers.cc:597] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:27:12.370430 12469 tablet_replica.cc:333] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.371515 12469 raft_consensus.cc:2243] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.372794 12469 raft_consensus.cc:2272] T 0b7a0a5ab87d4adbb517d125624bc8ad P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.376287 12469 tablet_replica.cc:333] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.377182 12469 raft_consensus.cc:2243] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.378364 12469 raft_consensus.cc:2272] T 749228d3459a444a9382bc38bbfcbbee P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:12.379717 14391 consensus_peers.cc:597] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:12.381629 12469 tablet_replica.cc:333] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.382822 12469 raft_consensus.cc:2243] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.383993 12469 raft_consensus.cc:2272] T 63b331138ada4d5e96b6136417e740f7 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.387099 12469 tablet_replica.cc:333] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.387936 12469 raft_consensus.cc:2243] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.389158 12469 raft_consensus.cc:2272] T b5c65e93a0c1420ca348d0fc678031ca P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.392562 12469 tablet_replica.cc:333] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.393472 12469 raft_consensus.cc:2243] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.394191 12469 raft_consensus.cc:2272] T 3ff310d454634aa0bbc38f2b233ccb8e P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.397293 12469 tablet_replica.cc:333] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.398321 12469 raft_consensus.cc:2243] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.398972 12469 raft_consensus.cc:2272] T 2dcbdced6d0a46b394cae3d101fbbf40 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.402295 12469 tablet_replica.cc:333] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.403173 12469 raft_consensus.cc:2243] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.403949 12469 raft_consensus.cc:2272] T 068a8cc7309648e3a661d4d056f11b7a P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.407003 12469 tablet_replica.cc:333] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.407819 12469 raft_consensus.cc:2243] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.408721 12469 raft_consensus.cc:2272] T 89717fd329da4af68d87ae1ef1b104fb P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.411865 12469 tablet_replica.cc:333] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.412779 12469 raft_consensus.cc:2243] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.414039 12469 raft_consensus.cc:2272] T 3a8b916ed93d478c990f668818f43952 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.417719 12469 tablet_replica.cc:333] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.418637 12469 raft_consensus.cc:2243] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.419271 12469 raft_consensus.cc:2272] T 7d21a9fba1484f4985407f4cde83b77c P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.422166 12469 tablet_replica.cc:333] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.423194 12469 raft_consensus.cc:2243] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 29 LEADER]: Raft consensus shutting down.
W20260709 12:27:12.423269 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 12:27:12.424405 12469 raft_consensus.cc:2272] T 8ab361e956dd4342b80045629d159a2b P a4188193a10c4d50a555d76ea85c60b7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.427546 12469 tablet_replica.cc:333] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.428448 12469 raft_consensus.cc:2243] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.429598 12469 raft_consensus.cc:2272] T 990ed59e36454f16b81d57d0e4f59c4e P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.433075 12469 tablet_replica.cc:333] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.434346 12469 raft_consensus.cc:2243] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.434979 12469 raft_consensus.cc:2272] T 45dc2f34593649778ec4b80d29638572 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.437973 12469 tablet_replica.cc:333] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.438889 12469 raft_consensus.cc:2243] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.440770 12469 raft_consensus.cc:2272] T e1d1b82fa8024303a24cb00ab8e7f845 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.443970 12469 tablet_replica.cc:333] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.444943 12469 raft_consensus.cc:2243] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.446310 12469 raft_consensus.cc:2272] T 20dc442f4fae4cf49e336a3d350b7760 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.449323 12469 tablet_replica.cc:333] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.450150 12469 raft_consensus.cc:2243] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.451550 12469 pending_rounds.cc:70] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Trying to abort 1 pending ops.
I20260709 12:27:12.451843 12469 pending_rounds.cc:77] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7: Aborting op as it isn't in flight: id { term: 28 index: 23 } timestamp: 7305625731284111360 op_type: NO_OP noop_request { }
I20260709 12:27:12.452279 12469 raft_consensus.cc:2889] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:27:12.452625 12469 raft_consensus.cc:2272] T cd4fd9484f184d2c994c269eb10d6ec1 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.456161 12469 tablet_replica.cc:333] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.457226 12469 raft_consensus.cc:2243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.458928 12469 raft_consensus.cc:2272] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.461951 12469 tablet_replica.cc:333] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.462790 12469 raft_consensus.cc:2243] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.463613 12469 raft_consensus.cc:2272] T e2603f7eea4f49ab89f0ebb1304ee239 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.466490 12469 tablet_replica.cc:333] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.467342 12469 raft_consensus.cc:2243] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.468066 12469 raft_consensus.cc:2272] T 0a27ddca262a4e0e8c46331cd36be4d1 P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.471158 12469 tablet_replica.cc:333] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.471933 12469 raft_consensus.cc:2243] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.473110 12469 raft_consensus.cc:2272] T a9e96f5f06294edba36ea19efe454394 P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.476126 12469 tablet_replica.cc:333] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.477032 12469 raft_consensus.cc:2243] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.477861 12469 raft_consensus.cc:2272] T 80eb1d112d05410a917177e781e466ed P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.481038 12469 tablet_replica.cc:333] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.481897 12469 raft_consensus.cc:2243] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.482558 12469 raft_consensus.cc:2272] T 4613215991134c66b270c6024bec9d73 P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.486169 12469 tablet_replica.cc:333] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.487185 12469 raft_consensus.cc:2243] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.487851 12469 raft_consensus.cc:2272] T 54361b2742ab4613b55749cbb699113b P a4188193a10c4d50a555d76ea85c60b7 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.490727 12469 tablet_replica.cc:333] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.491518 12469 raft_consensus.cc:2243] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.492784 12469 raft_consensus.cc:2272] T f22eab7d8372432799f894a161194acd P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.495857 12469 tablet_replica.cc:333] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.496724 12469 raft_consensus.cc:2243] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.497625 12469 raft_consensus.cc:2272] T 0d7e68c4d38b4465937dee955f980255 P a4188193a10c4d50a555d76ea85c60b7 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.501389 12469 tablet_replica.cc:333] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.502205 12469 raft_consensus.cc:2243] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 33 LEADER]: Raft consensus shutting down.
W20260709 12:27:12.502753 14391 consensus_peers.cc:597] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.503517 14391 consensus_peers.cc:597] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:12.503551 12469 raft_consensus.cc:2272] T 1866ffc09d7a4cd3b2f2883e9baf96fe P a4188193a10c4d50a555d76ea85c60b7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.506865 12469 tablet_replica.cc:333] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.507735 12469 raft_consensus.cc:2243] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.509028 12469 raft_consensus.cc:2272] T 52052a89b7534d0488f41d7f8e905a14 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.511838 12469 tablet_replica.cc:333] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.512923 12469 raft_consensus.cc:2243] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.513736 12469 raft_consensus.cc:2272] T 46d57087aee64ae9a498d3d60b92ab92 P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.516942 12469 tablet_replica.cc:333] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.517798 12469 raft_consensus.cc:2243] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 40 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.519040 12469 raft_consensus.cc:2272] T f136fb5f4e574b6db1336e9434176d83 P a4188193a10c4d50a555d76ea85c60b7 [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.522223 12469 tablet_replica.cc:333] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.523078 12469 raft_consensus.cc:2243] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.523736 12469 raft_consensus.cc:2272] T 85fee8a18ff4429cb870f59aa46b3892 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.526716 12469 tablet_replica.cc:333] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.527493 12469 raft_consensus.cc:2243] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 45 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.528216 12469 raft_consensus.cc:2272] T 56f25c6fe9f245d99ae467bdd3948582 P a4188193a10c4d50a555d76ea85c60b7 [term 45 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.530915 12469 tablet_replica.cc:333] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.531677 12469 raft_consensus.cc:2243] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.532809 12469 raft_consensus.cc:2272] T 8debd745c407406c88fc03ec3a21f4db P a4188193a10c4d50a555d76ea85c60b7 [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:12.534504 14391 consensus_peers.cc:597] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:12.535619 12469 tablet_replica.cc:333] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.536471 12469 raft_consensus.cc:2243] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.537220 12469 raft_consensus.cc:2272] T 255f10165e844f2cb50d654996d639cf P a4188193a10c4d50a555d76ea85c60b7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.539981 12469 tablet_replica.cc:333] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.540766 12469 raft_consensus.cc:2243] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.542006 12469 raft_consensus.cc:2272] T 0a0048f5294f4012ad1519c0bc8e9e5a P a4188193a10c4d50a555d76ea85c60b7 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.545323 12469 tablet_replica.cc:333] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.546108 12469 raft_consensus.cc:2243] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.547322 12469 raft_consensus.cc:2272] T 48612be3c56f482c81533c30abebe60b P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.550658 12469 tablet_replica.cc:333] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.551468 12469 raft_consensus.cc:2243] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.552048 12469 raft_consensus.cc:2272] T 3934ae8ea567487da7a989adece218c2 P a4188193a10c4d50a555d76ea85c60b7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.554805 12469 tablet_replica.cc:333] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.555589 12469 raft_consensus.cc:2243] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.558377 12469 raft_consensus.cc:2272] T 7ff03c62343d402fa469da8080d33a6d P a4188193a10c4d50a555d76ea85c60b7 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:12.561619 14391 consensus_peers.cc:597] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:12.564249 12469 tablet_replica.cc:333] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.565722 12469 raft_consensus.cc:2243] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.566982 12469 raft_consensus.cc:2272] T a2dcf256ec7a4ea6be04f95be4459f59 P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.570606 12469 tablet_replica.cc:333] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.571224 12469 raft_consensus.cc:2243] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.572152 12469 raft_consensus.cc:2272] T 79b849c107424cc79f4cbd223feeeb47 P a4188193a10c4d50a555d76ea85c60b7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.574555 12469 tablet_replica.cc:333] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.575162 12469 raft_consensus.cc:2243] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.576084 12469 raft_consensus.cc:2272] T 8916aaed46c543b59ce09ffca942c33d P a4188193a10c4d50a555d76ea85c60b7 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.578222 12469 tablet_replica.cc:333] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.578903 12469 raft_consensus.cc:2243] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.579962 12469 raft_consensus.cc:2272] T b1d64b0184dc44a1afc1b5885669d80d P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.582365 12469 tablet_replica.cc:333] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.582937 12469 raft_consensus.cc:2243] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.584120 12469 raft_consensus.cc:2272] T 28452187c3924de9940235e9b965477d P a4188193a10c4d50a555d76ea85c60b7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.586552 12469 tablet_replica.cc:333] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.587098 12469 raft_consensus.cc:2243] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.588372 12469 raft_consensus.cc:2272] T 15d97b0b20fd480e8f728a8d38f52e03 P a4188193a10c4d50a555d76ea85c60b7 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.591655 12469 tablet_replica.cc:333] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.592370 12469 raft_consensus.cc:2243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 41 LEADER]: Raft consensus shutting down.
I20260709 12:27:12.593660 12469 raft_consensus.cc:2272] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P a4188193a10c4d50a555d76ea85c60b7 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.596273 12469 tablet_replica.cc:333] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.596848 12469 raft_consensus.cc:2243] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.597421 12469 raft_consensus.cc:2272] T af56607c7c564ce4bda1487902e25c7f P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:12.599828 12469 tablet_replica.cc:333] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7: stopping tablet replica
I20260709 12:27:12.600353 12469 raft_consensus.cc:2243] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:12.600889 12469 raft_consensus.cc:2272] T 77fe434d99534108934a1cdcf44e68ac P a4188193a10c4d50a555d76ea85c60b7 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:12.640645 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.649464 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.670914 14391 consensus_peers.cc:597] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.677186 14391 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:12.707806 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.755545 14391 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:12.816201 14391 consensus_peers.cc:597] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Couldn't send request to peer a4188193a10c4d50a555d76ea85c60b7. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:12.847743 14390 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:12.929575 14391 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:12.966012 14391 consensus_peers.cc:597] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:12.988950 14390 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:12.996168 14390 consensus_peers.cc:597] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.018311 14390 consensus_peers.cc:597] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.084839 14390 consensus_peers.cc:597] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.092661 14390 consensus_peers.cc:597] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.116901 14390 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.147691 14390 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.212203 14391 consensus_peers.cc:597] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:13.230669 14390 consensus_peers.cc:597] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.274199 14391 consensus_peers.cc:597] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:27:13.404701 14390 consensus_peers.cc:597] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.412736 14390 consensus_peers.cc:597] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:13.422078 14390 consensus_peers.cc:597] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:13.455683 26424 raft_consensus.cc:493] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.456259 26424 raft_consensus.cc:515] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.458328 26424 leader_election.cc:290] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.463301 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:13.467242 14391 leader_election.cc:336] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.467816 14391 leader_election.cc:304] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 40 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:13.469359 26424 raft_consensus.cc:2749] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Leader pre-election lost for term 40. Reason: could not achieve majority
W20260709 12:27:13.471838 14390 consensus_peers.cc:597] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:13.521073 26424 raft_consensus.cc:493] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.522078 26424 raft_consensus.cc:515] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.524317 26424 leader_election.cc:290] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.532150 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:13.532967 14391 leader_election.cc:336] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.533422 14391 leader_election.cc:304] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:13.535423 26437 raft_consensus.cc:2749] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
W20260709 12:27:13.545210 14391 consensus_peers.cc:597] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20260709 12:27:13.557375 14391 consensus_peers.cc:597] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
I20260709 12:27:13.564474 26424 raft_consensus.cc:493] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.565037 26424 raft_consensus.cc:515] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.567221 26424 leader_election.cc:290] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.571241 14390 leader_election.cc:336] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:13.572808 14391 leader_election.cc:336] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.573241 14391 leader_election.cc:304] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:13.573971 26424 raft_consensus.cc:2749] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:13.645993 26424 raft_consensus.cc:493] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.646602 26424 raft_consensus.cc:515] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.648726 26424 leader_election.cc:290] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.656081 14390 leader_election.cc:336] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:13.658982 26424 raft_consensus.cc:493] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.659556 26424 raft_consensus.cc:515] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } attrs { replace: true } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
W20260709 12:27:13.661478 14391 leader_election.cc:336] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.662652 26424 leader_election.cc:290] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.663316 14390 consensus_peers.cc:597] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Couldn't send request to peer b4f6bfcda63148e4965ca02c639336f1. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:13.663266 14391 leader_election.cc:304] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:13.664753 26424 raft_consensus.cc:2749] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:13.671201 14390 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:13.673210 14391 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:13.673713 14391 leader_election.cc:304] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:13.674479 26424 raft_consensus.cc:2749] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:13.674629 14391 leader_election.cc:336] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.701372 26424 raft_consensus.cc:493] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.701946 26424 raft_consensus.cc:515] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.704723 26424 leader_election.cc:290] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.710683 14391 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:13.711395 14391 leader_election.cc:336] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.711781 14391 leader_election.cc:304] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 41 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:13.712711 26424 raft_consensus.cc:2749] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Leader pre-election lost for term 41. Reason: could not achieve majority
I20260709 12:27:13.742022 26424 raft_consensus.cc:493] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.742672 26424 raft_consensus.cc:515] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.746219 26424 leader_election.cc:290] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.751374 14391 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
W20260709 12:27:13.751880 14390 leader_election.cc:336] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:13.752329 14390 leader_election.cc:304] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:13.753106 26424 raft_consensus.cc:2749] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:13.774660 26424 raft_consensus.cc:493] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.775249 26424 raft_consensus.cc:515] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.777355 26424 leader_election.cc:290] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.783186 14391 leader_election.cc:336] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:13.784118 14391 leader_election.cc:336] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.784559 14391 leader_election.cc:304] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:13.785342 26424 raft_consensus.cc:2749] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:27:13.791702 14391 consensus_peers.cc:597] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 -> Peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Couldn't send request to peer 4dafe08a4f404502822a0732fe570de4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
I20260709 12:27:13.828634 26424 raft_consensus.cc:493] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.829202 26424 raft_consensus.cc:515] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.831740 26424 leader_election.cc:290] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.838531 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:13.839268 14391 leader_election.cc:336] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.839695 14391 leader_election.cc:304] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 42 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:13.840627 26424 raft_consensus.cc:2749] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Leader pre-election lost for term 42. Reason: could not achieve majority
I20260709 12:27:13.871680 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:27:13.888635 26424 raft_consensus.cc:493] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.889354 26424 raft_consensus.cc:515] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.891629 26424 leader_election.cc:290] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.898093 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
W20260709 12:27:13.898904 14391 leader_election.cc:336] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:13.899323 14391 leader_election.cc:304] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, a4188193a10c4d50a555d76ea85c60b7
I20260709 12:27:13.900146 26424 raft_consensus.cc:2749] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:13.912281 26424 raft_consensus.cc:493] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader a4188193a10c4d50a555d76ea85c60b7)
I20260709 12:27:13.912900 26424 raft_consensus.cc:515] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.915616 26424 leader_election.cc:290] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:13.921525 14391 leader_election.cc:336] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
W20260709 12:27:13.922454 14390 leader_election.cc:336] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:13.922950 14390 leader_election.cc:304] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: a4188193a10c4d50a555d76ea85c60b7, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:13.923822 26424 raft_consensus.cc:2749] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:27:13.961262 26424 raft_consensus.cc:493] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:13.963713 26424 raft_consensus.cc:515] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.968348 26424 leader_election.cc:290] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:13.974560 14390 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:13.977236 14391 leader_election.cc:336] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:13.979139 14391 leader_election.cc:304] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:13.979993 26424 raft_consensus.cc:2749] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:13.985155 26424 raft_consensus.cc:493] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:13.985750 26424 raft_consensus.cc:515] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:13.988662 26424 leader_election.cc:290] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841)
W20260709 12:27:13.996088 14390 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
I20260709 12:27:14.001626 12469 tablet_server.cc:179] TabletServer@127.12.45.68:0 shutting down...
W20260709 12:27:14.003793 14391 leader_election.cc:336] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:14.006302 14391 leader_election.cc:304] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:14.007328 26424 raft_consensus.cc:2749] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:14.033634 26437 raft_consensus.cc:493] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader b4f6bfcda63148e4965ca02c639336f1)
I20260709 12:27:14.034224 26437 raft_consensus.cc:515] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4dafe08a4f404502822a0732fe570de4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33075 } } peers { permanent_uuid: "b4f6bfcda63148e4965ca02c639336f1" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 41841 } } peers { permanent_uuid: "a4188193a10c4d50a555d76ea85c60b7" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 40415 } attrs { replace: true } } peers { permanent_uuid: "b708d8f5f6ac4aabb1eb5cb821ace195" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 42965 } attrs { promote: false } }
I20260709 12:27:14.037519 26437 leader_election.cc:290] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075), b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841), a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415)
W20260709 12:27:14.047443 14390 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer b4f6bfcda63148e4965ca02c639336f1 (127.12.45.66:41841): Network error: Client connection negotiation failed: client connection to 127.12.45.66:41841: connect: Connection refused (error 111)
W20260709 12:27:14.049889 14391 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:14.050496 14391 leader_election.cc:304] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: b708d8f5f6ac4aabb1eb5cb821ace195; no voters: 4dafe08a4f404502822a0732fe570de4, b4f6bfcda63148e4965ca02c639336f1
I20260709 12:27:14.051437 26437 raft_consensus.cc:2749] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:14.051590 14391 leader_election.cc:336] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a4188193a10c4d50a555d76ea85c60b7 (127.12.45.67:40415): Network error: Client connection negotiation failed: client connection to 127.12.45.67:40415: connect: Connection refused (error 111)
I20260709 12:27:14.080869 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:14.081921 12469 tablet_replica.cc:333] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.083822 12469 raft_consensus.cc:2243] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.084584 12469 raft_consensus.cc:2272] T 7ff03c62343d402fa469da8080d33a6d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.087970 12469 tablet_replica.cc:333] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.088936 12469 raft_consensus.cc:2243] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.089754 12469 raft_consensus.cc:2272] T cd4fd9484f184d2c994c269eb10d6ec1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.092765 12469 tablet_replica.cc:333] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.093662 12469 raft_consensus.cc:2243] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.094342 12469 raft_consensus.cc:2272] T 28452187c3924de9940235e9b965477d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.096880 12469 tablet_replica.cc:333] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.097494 12469 raft_consensus.cc:2243] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.098089 12469 raft_consensus.cc:2272] T 88a3d1d336b2495ea9a3a7ae1d7d37a2 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.100330 12469 tablet_replica.cc:333] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.101022 12469 raft_consensus.cc:2243] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.101625 12469 raft_consensus.cc:2272] T e9c30191f356495b9732659f925abc9f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.104539 12469 tablet_replica.cc:333] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.105340 12469 raft_consensus.cc:2243] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.105907 12469 raft_consensus.cc:2272] T 2f8f1b91eeaf4fc28fa45d4b45b907cd P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.108110 12469 tablet_replica.cc:333] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.109010 12469 raft_consensus.cc:2243] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.110176 12469 raft_consensus.cc:2272] T 6966d62b938845429a9aa8feb8fe2b9b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.112262 12469 tablet_replica.cc:333] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.112782 12469 raft_consensus.cc:2243] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.113487 12469 raft_consensus.cc:2272] T 20dc442f4fae4cf49e336a3d350b7760 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.115650 12469 tablet_replica.cc:333] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.116230 12469 raft_consensus.cc:2243] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.117094 12469 raft_consensus.cc:2272] T bdd4114edc6d43c08d41462fd399480c P b708d8f5f6ac4aabb1eb5cb821ace195 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.119588 12469 tablet_replica.cc:333] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.120283 12469 raft_consensus.cc:2243] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.121181 12469 raft_consensus.cc:2272] T 0d7e68c4d38b4465937dee955f980255 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 39 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.123865 12469 tablet_replica.cc:333] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.124496 12469 raft_consensus.cc:2243] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.125432 12469 raft_consensus.cc:2272] T 3ff310d454634aa0bbc38f2b233ccb8e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.127547 12469 tablet_replica.cc:333] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.128113 12469 raft_consensus.cc:2243] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.128932 12469 raft_consensus.cc:2272] T 4ce599ec23ec4ac0b358f4e47922bc3f P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.131021 12469 tablet_replica.cc:333] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.131567 12469 raft_consensus.cc:2243] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.132507 12469 raft_consensus.cc:2272] T 2dcbdced6d0a46b394cae3d101fbbf40 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.134720 12469 tablet_replica.cc:333] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.135313 12469 raft_consensus.cc:2243] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.136102 12469 raft_consensus.cc:2272] T 89717fd329da4af68d87ae1ef1b104fb P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.138128 12469 tablet_replica.cc:333] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.138967 12469 raft_consensus.cc:2243] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.139920 12469 raft_consensus.cc:2272] T b1d64b0184dc44a1afc1b5885669d80d P b708d8f5f6ac4aabb1eb5cb821ace195 [term 41 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.142966 12469 tablet_replica.cc:333] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.144033 12469 raft_consensus.cc:2243] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.145342 12469 raft_consensus.cc:2272] T 990ed59e36454f16b81d57d0e4f59c4e P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.148934 12469 tablet_replica.cc:333] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.149956 12469 raft_consensus.cc:2243] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.151448 12469 raft_consensus.cc:2272] T 30d1d057032c4ae0ad5ae21d374a2baa P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.154994 12469 tablet_replica.cc:333] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.155905 12469 raft_consensus.cc:2243] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.156555 12469 raft_consensus.cc:2272] T 435988495fcf4cf49f65293e95b5ceb5 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.159178 12469 tablet_replica.cc:333] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.159875 12469 raft_consensus.cc:2243] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.161294 12469 raft_consensus.cc:2272] T 85fee8a18ff4429cb870f59aa46b3892 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.164608 12469 tablet_replica.cc:333] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.165544 12469 raft_consensus.cc:2243] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.166527 12469 raft_consensus.cc:2272] T f136fb5f4e574b6db1336e9434176d83 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 40 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.169767 12469 tablet_replica.cc:333] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.170665 12469 raft_consensus.cc:2243] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.172015 12469 raft_consensus.cc:2272] T 0a27ddca262a4e0e8c46331cd36be4d1 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 38 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.174986 12469 tablet_replica.cc:333] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.175781 12469 raft_consensus.cc:2243] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.176450 12469 raft_consensus.cc:2272] T 3e52c8b752b54bd9b959b2363df455a4 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.179927 12469 tablet_replica.cc:333] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.180855 12469 raft_consensus.cc:2243] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.182377 12469 raft_consensus.cc:2272] T 6c6e2c2ada9f4af79430ba4cd2c6558b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.185843 12469 tablet_replica.cc:333] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.186893 12469 raft_consensus.cc:2243] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.187594 12469 raft_consensus.cc:2272] T 1866ffc09d7a4cd3b2f2883e9baf96fe P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.190508 12469 tablet_replica.cc:333] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.191788 12469 raft_consensus.cc:2243] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.193166 12469 raft_consensus.cc:2272] T de32481a3be147a5b854251468fc667b P b708d8f5f6ac4aabb1eb5cb821ace195 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.196976 12469 tablet_replica.cc:333] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.198132 12469 raft_consensus.cc:2243] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.199635 12469 raft_consensus.cc:2272] T 45dc2f34593649778ec4b80d29638572 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.203181 12469 tablet_replica.cc:333] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.204110 12469 raft_consensus.cc:2243] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.206130 12469 raft_consensus.cc:2272] T ac3728586e264940a93f76650edf9ad3 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.209780 12469 tablet_replica.cc:333] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.210733 12469 raft_consensus.cc:2243] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.211999 12469 raft_consensus.cc:2272] T 068a8cc7309648e3a661d4d056f11b7a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.214872 12469 tablet_replica.cc:333] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.215376 12469 raft_consensus.cc:2243] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 45 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.216482 12469 raft_consensus.cc:2272] T 56f25c6fe9f245d99ae467bdd3948582 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 45 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.218750 12469 tablet_replica.cc:333] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.219197 12469 raft_consensus.cc:2243] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.219781 12469 raft_consensus.cc:2272] T 8debd745c407406c88fc03ec3a21f4db P b708d8f5f6ac4aabb1eb5cb821ace195 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.222208 12469 tablet_replica.cc:333] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.223004 12469 raft_consensus.cc:2243] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:14.223562 12469 raft_consensus.cc:2272] T 15d97b0b20fd480e8f728a8d38f52e03 P b708d8f5f6ac4aabb1eb5cb821ace195 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.226048 12469 tablet_replica.cc:333] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195: stopping tablet replica
I20260709 12:27:14.226655 12469 raft_consensus.cc:2243] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:27:14.227541 12469 raft_consensus.cc:2272] T 3cb6d086379f4190b73fc7f692fb490a P b708d8f5f6ac4aabb1eb5cb821ace195 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:14.603197 12469 tablet_server.cc:196] TabletServer@127.12.45.68:0 shutdown complete.
I20260709 12:27:14.671422 12469 master.cc:562] Master@127.12.45.126:44051 shutting down...
W20260709 12:27:14.747124 13598 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:27:14.747658 13598 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:14.748018 13598 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:14.748787 13598 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:27:14.749006 13598 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:14.749262 13598 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:27:21.102873 13537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111) [suppressed 405 similar messages]
W20260709 12:27:21.107523 13537 catalog_manager.cc:4755] TS 4dafe08a4f404502822a0732fe570de4 (127.12.45.65:33075): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet cd4fd9484f184d2c994c269eb10d6ec1: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33075: connect: Connection refused (error 111)
I20260709 12:27:21.748049 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:21.749305 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:21.749768 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P d5e54c6d2c954fbf93d5136276b7b78c: stopping tablet replica
I20260709 12:27:22.062616 12469 master.cc:584] Master@127.12.45.126:44051 shutdown complete.
I20260709 12:27:22.128733 12469 test_util.cc:182] -----------------------------------------------
I20260709 12:27:22.129027 12469 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599574270161-12469-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (406265 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:27:22.132886 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:41947
I20260709 12:27:22.134142 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:22.141315 26456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.141770 26455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.143241 26458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:22.143749 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:22.148331 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:22.148551 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:22.148669 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600042148657 us; error 0 us; skew 500 ppm
I20260709 12:27:22.149212 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:22.152181 12469 webserver.cc:533] Webserver started at http://127.12.45.126:45611/ using document root <none> and password file <none>
I20260709 12:27:22.152709 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:22.152885 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:22.153244 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:22.154693 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "8a50c935c236466db6d2ca6b4a823a7c"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.160004 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:27:22.163977 26463 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.164988 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:27:22.165344 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "8a50c935c236466db6d2ca6b4a823a7c"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.165673 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:22.184175 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:22.185544 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:22.231228 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:41947
I20260709 12:27:22.231429 26514 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:41947 every 8 connection(s)
I20260709 12:27:22.235885 26515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:22.249450 26515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c: Bootstrap starting.
I20260709 12:27:22.255622 26515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:22.260866 26515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c: No bootstrap required, opened a new log
I20260709 12:27:22.263630 26515 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a50c935c236466db6d2ca6b4a823a7c" member_type: VOTER }
I20260709 12:27:22.264134 26515 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:22.264420 26515 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a50c935c236466db6d2ca6b4a823a7c, State: Initialized, Role: FOLLOWER
I20260709 12:27:22.265164 26515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a50c935c236466db6d2ca6b4a823a7c" member_type: VOTER }
I20260709 12:27:22.265661 26515 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:27:22.265898 26515 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:27:22.266227 26515 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:22.273059 26515 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a50c935c236466db6d2ca6b4a823a7c" member_type: VOTER }
I20260709 12:27:22.273708 26515 leader_election.cc:304] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a50c935c236466db6d2ca6b4a823a7c; no voters: 
I20260709 12:27:22.275115 26515 leader_election.cc:290] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:27:22.275650 26518 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:22.277402 26518 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 1 LEADER]: Becoming Leader. State: Replica: 8a50c935c236466db6d2ca6b4a823a7c, State: Running, Role: LEADER
I20260709 12:27:22.278142 26518 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a50c935c236466db6d2ca6b4a823a7c" member_type: VOTER }
I20260709 12:27:22.278690 26515 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:27:22.287043 26519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a50c935c236466db6d2ca6b4a823a7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a50c935c236466db6d2ca6b4a823a7c" member_type: VOTER } }
I20260709 12:27:22.286906 26520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a50c935c236466db6d2ca6b4a823a7c. Latest consensus state: current_term: 1 leader_uuid: "8a50c935c236466db6d2ca6b4a823a7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a50c935c236466db6d2ca6b4a823a7c" member_type: VOTER } }
I20260709 12:27:22.288108 26519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [sys.catalog]: This master's current role is: LEADER
I20260709 12:27:22.288322 26520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [sys.catalog]: This master's current role is: LEADER
I20260709 12:27:22.290300 26528 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:27:22.295840 26528 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:27:22.297191 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:27:22.305584 26528 catalog_manager.cc:1383] Generated new cluster ID: 57e686ea00564b6791d8a49804dd8dcc
I20260709 12:27:22.305994 26528 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:27:22.330610 26528 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:27:22.332114 26528 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:27:22.345513 26528 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c: Generated new TSK 0
I20260709 12:27:22.346489 26528 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:27:22.364526 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:22.371719 26536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.373389 26537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.374557 26539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:22.376334 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:22.380888 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:22.381116 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:22.381310 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600042381294 us; error 0 us; skew 500 ppm
I20260709 12:27:22.382005 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:22.384733 12469 webserver.cc:533] Webserver started at http://127.12.45.65:40879/ using document root <none> and password file <none>
I20260709 12:27:22.385242 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:22.385435 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:22.385753 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:22.387102 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.392120 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:27:22.395984 26544 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.397042 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:27:22.397541 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.398027 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:22.418370 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:22.419700 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:22.421770 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:22.424634 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:22.424887 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.425184 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:22.425374 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.479347 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:37295
I20260709 12:27:22.479439 26606 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:37295 every 8 connection(s)
I20260709 12:27:22.485777 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:22.494020 26611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.495090 26612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.498449 26614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:22.500066 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:22.505129 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:22.505513 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:22.505715 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600042505680 us; error 0 us; skew 500 ppm
I20260709 12:27:22.506465 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:22.506973 26607 heartbeater.cc:344] Connected to a master server at 127.12.45.126:41947
I20260709 12:27:22.507472 26607 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:22.508569 26607 heartbeater.cc:507] Master 127.12.45.126:41947 requested a full tablet report, sending...
I20260709 12:27:22.509609 12469 webserver.cc:533] Webserver started at http://127.12.45.66:42321/ using document root <none> and password file <none>
I20260709 12:27:22.510397 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:22.510648 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:22.510983 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:22.511101 26480 ts_manager.cc:194] Registered new tserver with Master: cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:22.512624 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "643057f6f77241529979f572a42ed9b0"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.513324 26480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43886
I20260709 12:27:22.518587 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.005s
I20260709 12:27:22.522740 26619 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.523712 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:27:22.523998 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "643057f6f77241529979f572a42ed9b0"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.524255 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:22.534760 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:22.536352 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:22.537990 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:22.540596 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:22.540793 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.541100 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:22.541261 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.593835 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:43883
I20260709 12:27:22.593943 26681 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:43883 every 8 connection(s)
I20260709 12:27:22.599670 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:22.609486 26685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.610594 26686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:22.613551 26682 heartbeater.cc:344] Connected to a master server at 127.12.45.126:41947
I20260709 12:27:22.614143 26682 heartbeater.cc:461] Registering TS with master...
W20260709 12:27:22.614468 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:22.614749 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:22.615729 26682 heartbeater.cc:507] Master 127.12.45.126:41947 requested a full tablet report, sending...
I20260709 12:27:22.616125 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:22.616428 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:22.616647 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600042616626 us; error 0 us; skew 500 ppm
I20260709 12:27:22.617544 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:22.618676 26480 ts_manager.cc:194] Registered new tserver with Master: 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:22.620662 12469 webserver.cc:533] Webserver started at http://127.12.45.67:38349/ using document root <none> and password file <none>
I20260709 12:27:22.620705 26480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43892
I20260709 12:27:22.621428 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:22.621717 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:22.622099 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:22.624053 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "f57551729f2f4037a4c769ee1df81853"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.629056 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:27:22.633595 26693 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.634671 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:27:22.635108 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "f57551729f2f4037a4c769ee1df81853"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.635468 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:22.648051 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:22.649540 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:22.651322 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:22.653844 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:22.654088 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.654372 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:22.654558 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.707926 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:39937
I20260709 12:27:22.708070 26755 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:39937 every 8 connection(s)
I20260709 12:27:22.713729 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:22.723489 26759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.723714 26760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:22.727059 26762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:22.728896 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:22.730314 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:22.730612 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:22.730724 26756 heartbeater.cc:344] Connected to a master server at 127.12.45.126:41947
I20260709 12:27:22.730882 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600042730857 us; error 0 us; skew 500 ppm
I20260709 12:27:22.731245 26756 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:22.732038 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:22.732496 26756 heartbeater.cc:507] Master 127.12.45.126:41947 requested a full tablet report, sending...
I20260709 12:27:22.735447 12469 webserver.cc:533] Webserver started at http://127.12.45.68:36785/ using document root <none> and password file <none>
I20260709 12:27:22.735844 26480 ts_manager.cc:194] Registered new tserver with Master: f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:22.736271 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:22.736598 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:22.736971 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:22.737980 26480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43906
I20260709 12:27:22.738842 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-3-root/instance:
uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.744858 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:27:22.749270 26767 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.750618 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 12:27:22.751011 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-3-root
uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:22.751402 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:22.769256 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:22.770949 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:22.772459 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:22.775136 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:22.775396 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.775713 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:22.775910 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:22.831830 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.68:45475
I20260709 12:27:22.831959 26829 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.68:45475 every 8 connection(s)
I20260709 12:27:22.849116 26830 heartbeater.cc:344] Connected to a master server at 127.12.45.126:41947
I20260709 12:27:22.849960 26830 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:22.851022 26830 heartbeater.cc:507] Master 127.12.45.126:41947 requested a full tablet report, sending...
I20260709 12:27:22.853518 26480 ts_manager.cc:194] Registered new tserver with Master: 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:22.854352 12469 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.018895693s
I20260709 12:27:22.856138 26480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43916
I20260709 12:27:22.886503 26480 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43928:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:27:23.148624 26795 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc5d1308d484c77941c0fc3a0322157 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:27:23.150362 26795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc5d1308d484c77941c0fc3a0322157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.153342 26794 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee02a1415f74accb6c585fd3556fa3b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:27:23.155146 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee02a1415f74accb6c585fd3556fa3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.159399 26793 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2217c6605e4f229a55e993de7a8696 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:27:23.161090 26793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2217c6605e4f229a55e993de7a8696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.166581 26792 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf8b638cc7b4d68a88deeacea18f36d (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:27:23.167738 26791 tablet_service.cc:1511] Processing CreateTablet for tablet 6147d09bb6fe4ec5be65fa9239d10536 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:27:23.168186 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf8b638cc7b4d68a88deeacea18f36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.171309 26791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6147d09bb6fe4ec5be65fa9239d10536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.176538 26790 tablet_service.cc:1511] Processing CreateTablet for tablet 413d45f35e654998908cd4c1c6b8a0f2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:27:23.179670 26789 tablet_service.cc:1511] Processing CreateTablet for tablet cf0134ff709c4ab3991a5bb56a32f3a1 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:27:23.184696 26788 tablet_service.cc:1511] Processing CreateTablet for tablet 559dcf990edd4760b228f7ecff46a4b7 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:27:23.187155 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413d45f35e654998908cd4c1c6b8a0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.191862 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559dcf990edd4760b228f7ecff46a4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.191368 26787 tablet_service.cc:1511] Processing CreateTablet for tablet a60b354295114347b828b0acc3386536 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:27:23.193392 26787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60b354295114347b828b0acc3386536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.193764 26849 tablet_bootstrap.cc:492] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:23.194151 26789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf0134ff709c4ab3991a5bb56a32f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.203532 26793 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0fc2016a9b4f2390b55edc0d8289f0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:27:23.203836 26786 tablet_service.cc:1511] Processing CreateTablet for tablet 518c064d1a7740bbbb35c46cfa79a3ff (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:27:23.205852 26786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 518c064d1a7740bbbb35c46cfa79a3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.211377 26793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0fc2016a9b4f2390b55edc0d8289f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.215639 26646 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee02a1415f74accb6c585fd3556fa3b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:27:23.217329 26646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee02a1415f74accb6c585fd3556fa3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.211735 26647 tablet_service.cc:1511] Processing CreateTablet for tablet ff4761da8e934968ae1169a27136254a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:27:23.219544 26645 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2217c6605e4f229a55e993de7a8696 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:27:23.227094 26644 tablet_service.cc:1511] Processing CreateTablet for tablet 46ee7ee8e9664d66823351436762d6b2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:27:23.223675 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2217c6605e4f229a55e993de7a8696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.223855 26643 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf8b638cc7b4d68a88deeacea18f36d (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:27:23.243240 26638 tablet_service.cc:1511] Processing CreateTablet for tablet fdc9b665b7de4e61ae5821f49d7459ec (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:27:23.245994 26639 tablet_service.cc:1511] Processing CreateTablet for tablet a60b354295114347b828b0acc3386536 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:27:23.236177 26642 tablet_service.cc:1511] Processing CreateTablet for tablet 413d45f35e654998908cd4c1c6b8a0f2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:27:23.238804 26641 tablet_service.cc:1511] Processing CreateTablet for tablet faab3b25212c4a0085b5907de0936703 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:27:23.241036 26640 tablet_service.cc:1511] Processing CreateTablet for tablet cf0134ff709c4ab3991a5bb56a32f3a1 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:27:23.252640 26639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60b354295114347b828b0acc3386536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.254098 26647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4761da8e934968ae1169a27136254a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.255465 26642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413d45f35e654998908cd4c1c6b8a0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.258704 26643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf8b638cc7b4d68a88deeacea18f36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.264065 26640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf0134ff709c4ab3991a5bb56a32f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.266731 26638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc9b665b7de4e61ae5821f49d7459ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.268357 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ee7ee8e9664d66823351436762d6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.274753 26641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faab3b25212c4a0085b5907de0936703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.291672 26849 tablet_bootstrap.cc:654] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:23.292227 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:23.300836 26794 tablet_service.cc:1511] Processing CreateTablet for tablet bba92ba095b54d48914151dcac5033d1 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:27:23.314564 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bba92ba095b54d48914151dcac5033d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.350391 26791 tablet_service.cc:1511] Processing CreateTablet for tablet b647e01f42114926b7c04cd1ebf9e23f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:27:23.356290 26791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b647e01f42114926b7c04cd1ebf9e23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.363395 26572 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc5d1308d484c77941c0fc3a0322157 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:27:23.365130 26572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc5d1308d484c77941c0fc3a0322157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.367125 26571 tablet_service.cc:1511] Processing CreateTablet for tablet ff4761da8e934968ae1169a27136254a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:27:23.369184 26570 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee02a1415f74accb6c585fd3556fa3b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:27:23.371166 26569 tablet_service.cc:1511] Processing CreateTablet for tablet 46ee7ee8e9664d66823351436762d6b2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:27:23.373001 26568 tablet_service.cc:1511] Processing CreateTablet for tablet 3cf8b638cc7b4d68a88deeacea18f36d (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:27:23.363269 26788 tablet_service.cc:1511] Processing CreateTablet for tablet df9b6736d82e4e18ba0068b834c114d0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:27:23.374861 26567 tablet_service.cc:1511] Processing CreateTablet for tablet 6147d09bb6fe4ec5be65fa9239d10536 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:27:23.388041 26565 tablet_service.cc:1511] Processing CreateTablet for tablet cf0134ff709c4ab3991a5bb56a32f3a1 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:27:23.389590 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9b6736d82e4e18ba0068b834c114d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.384608 26564 tablet_service.cc:1511] Processing CreateTablet for tablet 559dcf990edd4760b228f7ecff46a4b7 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:27:23.394979 26564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559dcf990edd4760b228f7ecff46a4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.386399 26563 tablet_service.cc:1511] Processing CreateTablet for tablet fdc9b665b7de4e61ae5821f49d7459ec (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:27:23.380470 26566 tablet_service.cc:1511] Processing CreateTablet for tablet faab3b25212c4a0085b5907de0936703 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:27:23.399696 26563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc9b665b7de4e61ae5821f49d7459ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.407508 26571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4761da8e934968ae1169a27136254a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.421587 26641 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0fc2016a9b4f2390b55edc0d8289f0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:27:23.427382 26647 tablet_service.cc:1511] Processing CreateTablet for tablet bba92ba095b54d48914151dcac5033d1 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:27:23.430358 26641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0fc2016a9b4f2390b55edc0d8289f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:23.438000 26465 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 55743.2us
I20260709 12:27:23.438484 26570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee02a1415f74accb6c585fd3556fa3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.438961 26850 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:23.440806 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf0134ff709c4ab3991a5bb56a32f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.445362 26567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6147d09bb6fe4ec5be65fa9239d10536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.446795 26566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faab3b25212c4a0085b5907de0936703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.450120 26569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ee7ee8e9664d66823351436762d6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.451462 26568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cf8b638cc7b4d68a88deeacea18f36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.443751 26792 tablet_service.cc:1511] Processing CreateTablet for tablet 7bd8a7d163624a52837a57273d4a7892 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:27:23.455272 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd8a7d163624a52837a57273d4a7892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.460433 26647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bba92ba095b54d48914151dcac5033d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.462010 26646 tablet_service.cc:1511] Processing CreateTablet for tablet 744dd7e87b4540ad930b1de1885f334a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:27:23.463511 26794 tablet_service.cc:1511] Processing CreateTablet for tablet f3ca1a807fb8465785df25e6f3ba9931 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:27:23.466833 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ca1a807fb8465785df25e6f3ba9931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.470171 26646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744dd7e87b4540ad930b1de1885f334a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.482185 26850 tablet_bootstrap.cc:654] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:23.496397 26788 tablet_service.cc:1511] Processing CreateTablet for tablet b1d43a3098bd4f5d80095bc30638c181 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:27:23.498064 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d43a3098bd4f5d80095bc30638c181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.499756 26642 tablet_service.cc:1511] Processing CreateTablet for tablet f3ca1a807fb8465785df25e6f3ba9931 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:27:23.501324 26642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ca1a807fb8465785df25e6f3ba9931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.502754 26643 tablet_service.cc:1511] Processing CreateTablet for tablet b647e01f42114926b7c04cd1ebf9e23f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:27:23.514824 26786 tablet_service.cc:1511] Processing CreateTablet for tablet cbf182133ad942e582a03b114c25daf3 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:27:23.526320 26607 heartbeater.cc:499] Master 127.12.45.126:41947 was elected leader, sending a full tablet report...
I20260709 12:27:23.526671 26643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b647e01f42114926b7c04cd1ebf9e23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.527863 26786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf182133ad942e582a03b114c25daf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.529501 26849 tablet_bootstrap.cc:492] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:23.530155 26849 ts_tablet_manager.cc:1403] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.337s	user 0.013s	sys 0.031s
I20260709 12:27:23.541411 26849 raft_consensus.cc:359] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:23.543525 26644 tablet_service.cc:1511] Processing CreateTablet for tablet 86583bdc7b4a453f9d4215b64a0d9a86 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:27:23.550560 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86583bdc7b4a453f9d4215b64a0d9a86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.555511 26850 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:23.561844 26721 tablet_service.cc:1511] Processing CreateTablet for tablet 9dc5d1308d484c77941c0fc3a0322157 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:27:23.563565 26721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9dc5d1308d484c77941c0fc3a0322157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.563196 26718 tablet_service.cc:1511] Processing CreateTablet for tablet 46ee7ee8e9664d66823351436762d6b2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:27:23.563454 26717 tablet_service.cc:1511] Processing CreateTablet for tablet 6147d09bb6fe4ec5be65fa9239d10536 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:27:23.561854 26719 tablet_service.cc:1511] Processing CreateTablet for tablet 4b2217c6605e4f229a55e993de7a8696 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:27:23.565871 26716 tablet_service.cc:1511] Processing CreateTablet for tablet 413d45f35e654998908cd4c1c6b8a0f2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:27:23.566718 26720 tablet_service.cc:1511] Processing CreateTablet for tablet ff4761da8e934968ae1169a27136254a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:27:23.568066 26564 tablet_service.cc:1511] Processing CreateTablet for tablet 518c064d1a7740bbbb35c46cfa79a3ff (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:27:23.568184 26715 tablet_service.cc:1511] Processing CreateTablet for tablet faab3b25212c4a0085b5907de0936703 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:27:23.570390 26714 tablet_service.cc:1511] Processing CreateTablet for tablet 559dcf990edd4760b228f7ecff46a4b7 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:27:23.572996 26713 tablet_service.cc:1511] Processing CreateTablet for tablet a60b354295114347b828b0acc3386536 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:27:23.575057 26718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46ee7ee8e9664d66823351436762d6b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.556144 26849 raft_consensus.cc:385] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:23.575595 26849 raft_consensus.cc:740] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:23.575480 26790 tablet_service.cc:1511] Processing CreateTablet for tablet 5150550259ec4e3fac2af6e10b6ca9b5 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:27:23.577033 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5150550259ec4e3fac2af6e10b6ca9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.576944 26849 consensus_queue.cc:260] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:23.584394 26712 tablet_service.cc:1511] Processing CreateTablet for tablet fdc9b665b7de4e61ae5821f49d7459ec (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:27:23.585567 26565 tablet_service.cc:1511] Processing CreateTablet for tablet df9b6736d82e4e18ba0068b834c114d0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:27:23.557824 26850 ts_tablet_manager.cc:1403] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.010s
I20260709 12:27:23.587159 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9b6736d82e4e18ba0068b834c114d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.588665 26647 tablet_service.cc:1511] Processing CreateTablet for tablet 7bd8a7d163624a52837a57273d4a7892 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:27:23.593714 26563 tablet_service.cc:1511] Processing CreateTablet for tablet 744dd7e87b4540ad930b1de1885f334a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:27:23.595224 26647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd8a7d163624a52837a57273d4a7892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.592793 26713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a60b354295114347b828b0acc3386536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.600332 26720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4761da8e934968ae1169a27136254a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.602675 26719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b2217c6605e4f229a55e993de7a8696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.595300 26563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744dd7e87b4540ad930b1de1885f334a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.604490 26714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 559dcf990edd4760b228f7ecff46a4b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.605777 26712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdc9b665b7de4e61ae5821f49d7459ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.607733 26717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6147d09bb6fe4ec5be65fa9239d10536. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.610630 26716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413d45f35e654998908cd4c1c6b8a0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.591121 26569 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0fc2016a9b4f2390b55edc0d8289f0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:27:23.598960 26850 raft_consensus.cc:359] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:23.617595 26850 raft_consensus.cc:385] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:23.618964 26850 raft_consensus.cc:740] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:23.624214 26682 heartbeater.cc:499] Master 127.12.45.126:41947 was elected leader, sending a full tablet report...
I20260709 12:27:23.632606 26641 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfa515c10414832964dc85f6c82d88b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:27:23.633066 26794 tablet_service.cc:1511] Processing CreateTablet for tablet 95b062d361f340cd9a9202852a196241 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:27:23.634454 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b062d361f340cd9a9202852a196241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.621286 26569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0fc2016a9b4f2390b55edc0d8289f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.634147 26641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfa515c10414832964dc85f6c82d88b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.651604 26572 tablet_service.cc:1511] Processing CreateTablet for tablet b647e01f42114926b7c04cd1ebf9e23f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:27:23.652658 26850 consensus_queue.cc:260] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:23.656610 26849 ts_tablet_manager.cc:1434] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.118s	user 0.012s	sys 0.003s
I20260709 12:27:23.673759 26645 tablet_service.cc:1511] Processing CreateTablet for tablet 5150550259ec4e3fac2af6e10b6ca9b5 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:27:23.675108 26639 tablet_service.cc:1511] Processing CreateTablet for tablet b1d43a3098bd4f5d80095bc30638c181 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:27:23.666237 26715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faab3b25212c4a0085b5907de0936703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.679876 26639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d43a3098bd4f5d80095bc30638c181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.678457 26564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 518c064d1a7740bbbb35c46cfa79a3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.685141 26570 tablet_service.cc:1511] Processing CreateTablet for tablet 86583bdc7b4a453f9d4215b64a0d9a86 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:27:23.686614 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5150550259ec4e3fac2af6e10b6ca9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.688091 26571 tablet_service.cc:1511] Processing CreateTablet for tablet 7bd8a7d163624a52837a57273d4a7892 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:27:23.688656 26830 heartbeater.cc:499] Master 127.12.45.126:41947 was elected leader, sending a full tablet report...
I20260709 12:27:23.689767 26571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bd8a7d163624a52837a57273d4a7892. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.690951 26849 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:23.692233 26853 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:23.708576 26857 raft_consensus.cc:493] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.719105 26713 tablet_service.cc:1511] Processing CreateTablet for tablet df9b6736d82e4e18ba0068b834c114d0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:27:23.723941 26644 tablet_service.cc:1511] Processing CreateTablet for tablet 6cb4745cc61545b0a28693865420f43f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:27:23.731500 26568 tablet_service.cc:1511] Processing CreateTablet for tablet cbf182133ad942e582a03b114c25daf3 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:27:23.735724 26788 tablet_service.cc:1511] Processing CreateTablet for tablet 6cb4745cc61545b0a28693865420f43f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:27:23.741626 26756 heartbeater.cc:499] Master 127.12.45.126:41947 was elected leader, sending a full tablet report...
I20260709 12:27:23.742774 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb4745cc61545b0a28693865420f43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.751958 26567 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfa515c10414832964dc85f6c82d88b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:27:23.754940 26720 tablet_service.cc:1511] Processing CreateTablet for tablet 518c064d1a7740bbbb35c46cfa79a3ff (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:27:23.756493 26720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 518c064d1a7740bbbb35c46cfa79a3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.756188 26640 tablet_service.cc:1511] Processing CreateTablet for tablet 83ac1836c91d4280be584799f8bb0c2a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:27:23.757747 26640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac1836c91d4280be584799f8bb0c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.767777 26854 raft_consensus.cc:493] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.768227 26567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfa515c10414832964dc85f6c82d88b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.768612 26854 raft_consensus.cc:515] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:23.786459 26568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf182133ad942e582a03b114c25daf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.787145 26849 tablet_bootstrap.cc:654] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:23.789834 26850 ts_tablet_manager.cc:1434] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.203s	user 0.013s	sys 0.002s
I20260709 12:27:23.810901 26850 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:23.814666 26713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df9b6736d82e4e18ba0068b834c114d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.801769 26570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86583bdc7b4a453f9d4215b64a0d9a86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.817548 26714 tablet_service.cc:1511] Processing CreateTablet for tablet 744dd7e87b4540ad930b1de1885f334a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:27:23.819676 26714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 744dd7e87b4540ad930b1de1885f334a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.818554 26856 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:23.823218 26721 tablet_service.cc:1511] Processing CreateTablet for tablet bba92ba095b54d48914151dcac5033d1 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:27:23.824767 26721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bba92ba095b54d48914151dcac5033d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.791610 26857 raft_consensus.cc:515] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:23.828886 26857 leader_election.cc:290] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:23.832711 26563 tablet_service.cc:1511] Processing CreateTablet for tablet 5150550259ec4e3fac2af6e10b6ca9b5 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
W20260709 12:27:23.837345 26831 tablet.cc:2406] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:27:23.838801 26566 tablet_service.cc:1511] Processing CreateTablet for tablet 95b062d361f340cd9a9202852a196241 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:27:23.840365 26563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5150550259ec4e3fac2af6e10b6ca9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.843453 26854 leader_election.cc:290] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:23.800552 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb4745cc61545b0a28693865420f43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.857213 26572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b647e01f42114926b7c04cd1ebf9e23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.866011 26856 tablet_bootstrap.cc:654] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:23.873203 26794 tablet_service.cc:1511] Processing CreateTablet for tablet 8bd7e0d139cb41beb4076b3fe3672c73 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:27:23.875976 26720 tablet_service.cc:1511] Processing CreateTablet for tablet 86583bdc7b4a453f9d4215b64a0d9a86 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:27:23.877475 26717 tablet_service.cc:1511] Processing CreateTablet for tablet cbf182133ad942e582a03b114c25daf3 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:27:23.883232 26718 tablet_service.cc:1511] Processing CreateTablet for tablet f3ca1a807fb8465785df25e6f3ba9931 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:27:23.894076 26720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86583bdc7b4a453f9d4215b64a0d9a86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.896162 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bd7e0d139cb41beb4076b3fe3672c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.903364 26718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ca1a807fb8465785df25e6f3ba9931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.905965 26569 tablet_service.cc:1511] Processing CreateTablet for tablet 83ac1836c91d4280be584799f8bb0c2a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:27:23.907223 26717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf182133ad942e582a03b114c25daf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.887633 26853 tablet_bootstrap.cc:654] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:23.917659 26566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b062d361f340cd9a9202852a196241. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:23.927840 26465 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 54850.7us
I20260709 12:27:23.931865 26849 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:23.932659 26849 ts_tablet_manager.cc:1403] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.242s	user 0.018s	sys 0.007s
I20260709 12:27:23.958858 26569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac1836c91d4280be584799f8bb0c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.961052 26565 tablet_service.cc:1511] Processing CreateTablet for tablet 8bd7e0d139cb41beb4076b3fe3672c73 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:27:23.963430 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bd7e0d139cb41beb4076b3fe3672c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.936817 26849 raft_consensus.cc:359] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:23.977186 26849 raft_consensus.cc:385] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:23.977669 26849 raft_consensus.cc:740] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:23.978718 26849 consensus_queue.cc:260] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:23.995731 26640 tablet_service.cc:1511] Processing CreateTablet for tablet 8bd7e0d139cb41beb4076b3fe3672c73 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:27:23.997619 26640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bd7e0d139cb41beb4076b3fe3672c73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.006841 26788 tablet_service.cc:1511] Processing CreateTablet for tablet 51a778606ffa467f9faaadb8412a4e8c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:27:24.008415 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a778606ffa467f9faaadb8412a4e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.009641 26714 tablet_service.cc:1511] Processing CreateTablet for tablet 7dfa515c10414832964dc85f6c82d88b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:27:24.011418 26714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dfa515c10414832964dc85f6c82d88b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:23.942476 26719 tablet_service.cc:1511] Processing CreateTablet for tablet b1d43a3098bd4f5d80095bc30638c181 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:27:24.022713 26850 tablet_bootstrap.cc:654] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.039327 26849 ts_tablet_manager.cc:1434] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.106s	user 0.004s	sys 0.004s
I20260709 12:27:24.043974 26853 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:24.046476 26853 ts_tablet_manager.cc:1403] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.379s	user 0.024s	sys 0.000s
I20260709 12:27:24.049367 26716 tablet_service.cc:1511] Processing CreateTablet for tablet 6cb4745cc61545b0a28693865420f43f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:27:24.046586 26719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1d43a3098bd4f5d80095bc30638c181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.051548 26712 tablet_service.cc:1511] Processing CreateTablet for tablet 95b062d361f340cd9a9202852a196241 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:27:24.052980 26712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95b062d361f340cd9a9202852a196241. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.052728 26853 raft_consensus.cc:359] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.053929 26853 raft_consensus.cc:385] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.054400 26853 raft_consensus.cc:740] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.055202 26716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb4745cc61545b0a28693865420f43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.055753 26853 consensus_queue.cc:260] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.063122 26570 tablet_service.cc:1511] Processing CreateTablet for tablet 51a778606ffa467f9faaadb8412a4e8c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:27:24.069777 26570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a778606ffa467f9faaadb8412a4e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.071941 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:24.079049 26768 leader_election.cc:343] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.062950 26715 tablet_service.cc:1511] Processing CreateTablet for tablet 83ac1836c91d4280be584799f8bb0c2a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:27:24.081630 26715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac1836c91d4280be584799f8bb0c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.082732 26849 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:24.083921 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:24.087440 26790 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb970d065d54c94be1a04a340a5821c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
W20260709 12:27:24.095686 26768 leader_election.cc:343] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.096383 26768 leader_election.cc:304] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:24.097239 26854 raft_consensus.cc:2749] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:24.102908 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb970d065d54c94be1a04a340a5821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.105237 26641 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb970d065d54c94be1a04a340a5821c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:27:24.106925 26641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb970d065d54c94be1a04a340a5821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.109467 26568 tablet_service.cc:1511] Processing CreateTablet for tablet 3a4f3d968244475e805596ce58c213e7 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:27:24.139609 26645 tablet_service.cc:1511] Processing CreateTablet for tablet 3a4f3d968244475e805596ce58c213e7 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:27:24.141304 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4f3d968244475e805596ce58c213e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.117739 26713 tablet_service.cc:1511] Processing CreateTablet for tablet 51a778606ffa467f9faaadb8412a4e8c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:27:24.157055 26713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51a778606ffa467f9faaadb8412a4e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.159493 26853 ts_tablet_manager.cc:1434] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.110s	user 0.017s	sys 0.000s
I20260709 12:27:24.161044 26853 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:24.171736 26854 raft_consensus.cc:493] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.172418 26854 raft_consensus.cc:515] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.175135 26854 leader_election.cc:290] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:24.177018 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:24.177773 26657 raft_consensus.cc:2468] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:24.179443 26768 leader_election.cc:304] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:24.180436 26854 raft_consensus.cc:2804] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:24.180814 26854 raft_consensus.cc:493] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.181136 26854 raft_consensus.cc:3060] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:24.184579 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:24.188095 26621 leader_election.cc:343] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.191079 26721 tablet_service.cc:1511] Processing CreateTablet for tablet 2cb970d065d54c94be1a04a340a5821c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:27:24.192799 26721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cb970d065d54c94be1a04a340a5821c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.199712 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:24.212077 26867 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.212901 26867 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.213462 26849 tablet_bootstrap.cc:654] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.213707 26568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4f3d968244475e805596ce58c213e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.234092 26856 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:24.234874 26856 ts_tablet_manager.cc:1403] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.427s	user 0.045s	sys 0.027s
I20260709 12:27:24.235461 26867 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:24.240950 26853 tablet_bootstrap.cc:654] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.229547 26854 raft_consensus.cc:515] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.254076 26850 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:24.254855 26850 ts_tablet_manager.cc:1403] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.444s	user 0.031s	sys 0.035s
I20260709 12:27:24.256727 26794 tablet_service.cc:1511] Processing CreateTablet for tablet d89d07b1f88649de8fe8c262f31a0ac6 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:27:24.258602 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d89d07b1f88649de8fe8c262f31a0ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.266153 26719 tablet_service.cc:1511] Processing CreateTablet for tablet 3a4f3d968244475e805596ce58c213e7 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:27:24.268236 26719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a4f3d968244475e805596ce58c213e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.272713 26805 raft_consensus.cc:2393] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:24.278061 26621 leader_election.cc:304] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:24.279800 26857 raft_consensus.cc:3060] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:24.280454 26854 leader_election.cc:290] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:24.300102 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:24.301973 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:24.304148 26856 raft_consensus.cc:359] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.304966 26856 raft_consensus.cc:385] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.305255 26856 raft_consensus.cc:740] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.307494 26850 raft_consensus.cc:359] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.308336 26850 raft_consensus.cc:385] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.308660 26850 raft_consensus.cc:740] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.309648 26850 consensus_queue.cc:260] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.312052 26850 ts_tablet_manager.cc:1434] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.002s
I20260709 12:27:24.319868 26857 raft_consensus.cc:2749] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:24.320873 26640 tablet_service.cc:1511] Processing CreateTablet for tablet d89d07b1f88649de8fe8c262f31a0ac6 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:27:24.323938 26792 tablet_service.cc:1511] Processing CreateTablet for tablet 6cde6c3890534a20956dd0622e8cb1dd (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:27:24.325500 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:24.326273 26714 tablet_service.cc:1511] Processing CreateTablet for tablet 6cde6c3890534a20956dd0622e8cb1dd (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:27:24.327083 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:24.327894 26714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cde6c3890534a20956dd0622e8cb1dd. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:24.330593 26769 leader_election.cc:343] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:24.331720 26769 leader_election.cc:343] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.325500 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cde6c3890534a20956dd0622e8cb1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.306046 26856 consensus_queue.cc:260] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.340914 26645 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6beba6a6d747ebbcd858a8b61ad68b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:27:24.353900 26640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d89d07b1f88649de8fe8c262f31a0ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.357723 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6beba6a6d747ebbcd858a8b61ad68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.364946 26712 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6beba6a6d747ebbcd858a8b61ad68b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:27:24.369612 26712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6beba6a6d747ebbcd858a8b61ad68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.383332 26853 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:24.384038 26853 ts_tablet_manager.cc:1403] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.223s	user 0.018s	sys 0.009s
I20260709 12:27:24.386629 26850 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:24.387954 26849 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:24.387598 26853 raft_consensus.cc:359] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.388526 26853 raft_consensus.cc:385] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.388892 26853 raft_consensus.cc:740] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.390604 26849 ts_tablet_manager.cc:1403] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.308s	user 0.031s	sys 0.016s
I20260709 12:27:24.395294 26569 tablet_service.cc:1511] Processing CreateTablet for tablet d89d07b1f88649de8fe8c262f31a0ac6 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:27:24.396797 26569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d89d07b1f88649de8fe8c262f31a0ac6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.394234 26849 raft_consensus.cc:359] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.402418 26849 raft_consensus.cc:385] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.402737 26849 raft_consensus.cc:740] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.408669 26788 tablet_service.cc:1511] Processing CreateTablet for tablet 6e6beba6a6d747ebbcd858a8b61ad68b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:27:24.412395 26849 consensus_queue.cc:260] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.420814 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e6beba6a6d747ebbcd858a8b61ad68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.415834 26570 tablet_service.cc:1511] Processing CreateTablet for tablet 6cde6c3890534a20956dd0622e8cb1dd (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:27:24.426397 26856 ts_tablet_manager.cc:1434] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.191s	user 0.014s	sys 0.013s
I20260709 12:27:24.427282 26570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cde6c3890534a20956dd0622e8cb1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.421272 26565 tablet_service.cc:1511] Processing CreateTablet for tablet 737ddf5e609d480ea8a119e82a057898 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:27:24.429008 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737ddf5e609d480ea8a119e82a057898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.429104 26641 tablet_service.cc:1511] Processing CreateTablet for tablet 737ddf5e609d480ea8a119e82a057898 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:27:24.412850 26657 raft_consensus.cc:2468] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:24.424875 26849 ts_tablet_manager.cc:1434] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:27:24.435650 26849 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:24.434397 26641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737ddf5e609d480ea8a119e82a057898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.437690 26857 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.438335 26857 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.440538 26850 tablet_bootstrap.cc:654] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.441686 26716 tablet_service.cc:1511] Processing CreateTablet for tablet 737ddf5e609d480ea8a119e82a057898 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:27:24.442588 26768 leader_election.cc:304] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:24.443162 26716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737ddf5e609d480ea8a119e82a057898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.443504 26854 raft_consensus.cc:2804] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:24.452098 26792 tablet_service.cc:1511] Processing CreateTablet for tablet 37ea7dd7d5f4407abb57b9027b2bf586 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:27:24.453922 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:24.454610 26856 tablet_bootstrap.cc:492] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
W20260709 12:27:24.455956 26621 leader_election.cc:343] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.445439 26857 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:24.460382 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:24.461817 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ea7dd7d5f4407abb57b9027b2bf586. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:24.463431 26545 leader_election.cc:343] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.449177 26853 consensus_queue.cc:260] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.484314 26853 ts_tablet_manager.cc:1434] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.100s	user 0.009s	sys 0.001s
I20260709 12:27:24.506047 26566 tablet_service.cc:1511] Processing CreateTablet for tablet 37ea7dd7d5f4407abb57b9027b2bf586 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:27:24.508338 26566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ea7dd7d5f4407abb57b9027b2bf586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.525343 26853 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:24.532238 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:24.533059 26805 raft_consensus.cc:2468] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:24.533277 26644 tablet_service.cc:1511] Processing CreateTablet for tablet 37ea7dd7d5f4407abb57b9027b2bf586 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:27:24.534786 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37ea7dd7d5f4407abb57b9027b2bf586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.534883 26546 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:24.538431 26867 raft_consensus.cc:2804] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:24.538844 26867 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.539176 26867 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:24.549602 26712 tablet_service.cc:1511] Processing CreateTablet for tablet 5bf5c3113a22464e81d4cb8462e8ac67 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:27:24.552953 26712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf5c3113a22464e81d4cb8462e8ac67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.561621 26849 tablet_bootstrap.cc:654] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.562695 26856 tablet_bootstrap.cc:654] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.566718 26853 tablet_bootstrap.cc:654] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.569347 26854 raft_consensus.cc:697] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:24.570557 26854 consensus_queue.cc:237] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.572546 26850 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:24.575032 26867 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.576979 26794 tablet_service.cc:1511] Processing CreateTablet for tablet 5bf5c3113a22464e81d4cb8462e8ac67 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:27:24.578220 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:24.580338 26567 tablet_service.cc:1511] Processing CreateTablet for tablet 5bf5c3113a22464e81d4cb8462e8ac67 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 12:27:24.581477 26545 leader_election.cc:343] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.581846 26567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf5c3113a22464e81d4cb8462e8ac67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.580338 26642 tablet_service.cc:1511] Processing CreateTablet for tablet dc276f88554143c3bd0a5b4f74143b2f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:27:24.573145 26850 ts_tablet_manager.cc:1403] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.196s	user 0.013s	sys 0.021s
I20260709 12:27:24.583915 26642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc276f88554143c3bd0a5b4f74143b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.584647 26794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bf5c3113a22464e81d4cb8462e8ac67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.589872 26867 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:24.592924 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:24.593798 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:24.604187 26714 tablet_service.cc:1511] Processing CreateTablet for tablet dc276f88554143c3bd0a5b4f74143b2f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:27:24.605676 26714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc276f88554143c3bd0a5b4f74143b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.605813 26856 tablet_bootstrap.cc:492] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
W20260709 12:27:24.605897 26620 leader_election.cc:343] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.606592 26856 ts_tablet_manager.cc:1403] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.152s	user 0.017s	sys 0.006s
I20260709 12:27:24.606544 26620 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:24.608822 26857 raft_consensus.cc:2749] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:24.595067 26790 tablet_service.cc:1511] Processing CreateTablet for tablet dc276f88554143c3bd0a5b4f74143b2f (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:27:24.611183 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc276f88554143c3bd0a5b4f74143b2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.598711 26805 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:24.622198 26640 tablet_service.cc:1511] Processing CreateTablet for tablet 47d7e2ee9fa8426f9ce60485d6e4bb7a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:27:24.623948 26640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47d7e2ee9fa8426f9ce60485d6e4bb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.614187 26867 raft_consensus.cc:493] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.626605 26867 raft_consensus.cc:515] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.638873 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:24.639942 26849 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:24.640584 26849 ts_tablet_manager.cc:1403] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.205s	user 0.009s	sys 0.009s
W20260709 12:27:24.641113 26545 leader_election.cc:343] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.644358 26856 raft_consensus.cc:359] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:24.645443 26856 raft_consensus.cc:385] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.645893 26856 raft_consensus.cc:740] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.645664 26849 raft_consensus.cc:359] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.647001 26849 raft_consensus.cc:385] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.647368 26849 raft_consensus.cc:740] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.647152 26856 consensus_queue.cc:260] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:24.648362 26849 consensus_queue.cc:260] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.655228 26856 ts_tablet_manager.cc:1434] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.005s
I20260709 12:27:24.656746 26856 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:24.658766 26877 raft_consensus.cc:493] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.659343 26877 raft_consensus.cc:515] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.660843 26853 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:24.661636 26853 ts_tablet_manager.cc:1403] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.176s	user 0.013s	sys 0.006s
I20260709 12:27:24.663323 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:24.663584 26877 leader_election.cc:290] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:24.664163 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:24.665161 26769 leader_election.cc:343] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:24.665782 26768 leader_election.cc:343] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.666451 26768 leader_election.cc:304] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:24.673061 26719 tablet_service.cc:1511] Processing CreateTablet for tablet 47d7e2ee9fa8426f9ce60485d6e4bb7a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:27:24.675228 26719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47d7e2ee9fa8426f9ce60485d6e4bb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.597723 26850 raft_consensus.cc:359] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.681036 26850 raft_consensus.cc:385] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.681542 26850 raft_consensus.cc:740] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.686460 26877 raft_consensus.cc:2749] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:24.683594 26850 consensus_queue.cc:260] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.690109 26875 raft_consensus.cc:493] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.690776 26875 raft_consensus.cc:515] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.693567 26875 leader_election.cc:290] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:24.693939 26850 ts_tablet_manager.cc:1434] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.111s	user 0.005s	sys 0.005s
I20260709 12:27:24.695547 26850 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:24.643003 26867 leader_election.cc:290] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:24.700712 26849 ts_tablet_manager.cc:1434] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.003s
I20260709 12:27:24.701028 26853 raft_consensus.cc:359] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.701838 26853 raft_consensus.cc:385] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:24.702009 26849 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:24.709714 26568 tablet_service.cc:1511] Processing CreateTablet for tablet 3f37c0d19f1249cdaa3b380b1c0b01d0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:27:24.711683 26568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f37c0d19f1249cdaa3b380b1c0b01d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.716020 26805 raft_consensus.cc:2468] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:24.702200 26853 raft_consensus.cc:740] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:24.733250 26480 catalog_manager.cc:5697] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.722443 26786 tablet_service.cc:1511] Processing CreateTablet for tablet 293707805bab498a9449cd938d3b1b6b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:27:24.736676 26546 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:24.739212 26786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293707805bab498a9449cd938d3b1b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.739734 26867 raft_consensus.cc:2804] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:24.749809 26856 tablet_bootstrap.cc:654] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.727497 26569 tablet_service.cc:1511] Processing CreateTablet for tablet 47d7e2ee9fa8426f9ce60485d6e4bb7a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:27:24.751536 26569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47d7e2ee9fa8426f9ce60485d6e4bb7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.758708 26716 tablet_service.cc:1511] Processing CreateTablet for tablet 293707805bab498a9449cd938d3b1b6b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:27:24.758882 26644 tablet_service.cc:1511] Processing CreateTablet for tablet 3f37c0d19f1249cdaa3b380b1c0b01d0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:27:24.760257 26716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293707805bab498a9449cd938d3b1b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.760787 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f37c0d19f1249cdaa3b380b1c0b01d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.763196 26792 tablet_service.cc:1511] Processing CreateTablet for tablet 3f37c0d19f1249cdaa3b380b1c0b01d0 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:27:24.764622 26792 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f37c0d19f1249cdaa3b380b1c0b01d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.769302 26642 tablet_service.cc:1511] Processing CreateTablet for tablet 646472eb33fc46e88f978e2aa3fe8ff8 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:27:24.771346 26642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 646472eb33fc46e88f978e2aa3fe8ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.770951 26853 consensus_queue.cc:260] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.779002 26567 tablet_service.cc:1511] Processing CreateTablet for tablet 293707805bab498a9449cd938d3b1b6b (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:27:24.781299 26867 raft_consensus.cc:697] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:24.784282 26867 consensus_queue.cc:237] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:24.779382 26712 tablet_service.cc:1511] Processing CreateTablet for tablet 646472eb33fc46e88f978e2aa3fe8ff8 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:27:24.791978 26712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 646472eb33fc46e88f978e2aa3fe8ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.793432 26857 raft_consensus.cc:493] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.793653 26850 tablet_bootstrap.cc:654] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.793300 26789 tablet_service.cc:1511] Processing CreateTablet for tablet 646472eb33fc46e88f978e2aa3fe8ff8 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:27:24.793994 26857 raft_consensus.cc:515] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:24.794827 26567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293707805bab498a9449cd938d3b1b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.794828 26789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 646472eb33fc46e88f978e2aa3fe8ff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.796490 26857 leader_election.cc:290] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:24.798785 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:24.801476 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:24.802040 26621 leader_election.cc:343] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:24.803489 26620 leader_election.cc:343] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.803987 26620 leader_election.cc:304] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:24.804976 26857 raft_consensus.cc:2749] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:24.833175 26875 raft_consensus.cc:493] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.842434 26565 tablet_service.cc:1511] Processing CreateTablet for tablet 6f355a7f12b14432844a7a4508117120 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:27:24.844681 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f355a7f12b14432844a7a4508117120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.842494 26875 raft_consensus.cc:515] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:24.853529 26849 tablet_bootstrap.cc:700] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent opening tablet: real 0.119s	user 0.003s	sys 0.001s
I20260709 12:27:24.854498 26875 leader_election.cc:290] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:24.878069 26849 tablet_bootstrap.cc:654] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:24.891898 26788 tablet_service.cc:1511] Processing CreateTablet for tablet 8437e96f55e2448e883f6f8bcd057be2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:27:24.893658 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8437e96f55e2448e883f6f8bcd057be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.899495 26570 tablet_service.cc:1511] Processing CreateTablet for tablet 8437e96f55e2448e883f6f8bcd057be2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:27:24.899691 26566 tablet_service.cc:1511] Processing CreateTablet for tablet 03c01181f6bd4a92915740e17605f4b9 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:27:24.901185 26570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8437e96f55e2448e883f6f8bcd057be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.904186 26790 tablet_service.cc:1511] Processing CreateTablet for tablet 03c01181f6bd4a92915740e17605f4b9 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:27:24.905789 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03c01181f6bd4a92915740e17605f4b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.923444 26853 ts_tablet_manager.cc:1434] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.261s	user 0.016s	sys 0.017s
I20260709 12:27:24.923986 26791 tablet_service.cc:1511] Processing CreateTablet for tablet 2ceb8aabf0a94f9385121858268a3169 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:27:24.924876 26853 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:24.925730 26791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ceb8aabf0a94f9385121858268a3169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.920011 26566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03c01181f6bd4a92915740e17605f4b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.939011 26640 tablet_service.cc:1511] Processing CreateTablet for tablet 6f355a7f12b14432844a7a4508117120 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:27:24.940712 26640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f355a7f12b14432844a7a4508117120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.943399 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:24.944164 26713 tablet_service.cc:1511] Processing CreateTablet for tablet 6f355a7f12b14432844a7a4508117120 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:27:24.945670 26713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f355a7f12b14432844a7a4508117120. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.944347 26645 tablet_service.cc:1511] Processing CreateTablet for tablet 8437e96f55e2448e883f6f8bcd057be2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:27:24.946358 26641 tablet_service.cc:1511] Processing CreateTablet for tablet 2ceb8aabf0a94f9385121858268a3169 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:27:24.947113 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8437e96f55e2448e883f6f8bcd057be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.948369 26641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ceb8aabf0a94f9385121858268a3169. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:24.948166 26545 leader_election.cc:343] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:24.981590 26639 tablet_service.cc:1511] Processing CreateTablet for tablet e62bc644ef15433785589f1e01b29a89 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:27:24.994710 26639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62bc644ef15433785589f1e01b29a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:24.998183 26786 tablet_service.cc:1511] Processing CreateTablet for tablet 2564769c4e85479fbfb15bd8a05a5807 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:27:25.005825 26786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2564769c4e85479fbfb15bd8a05a5807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.013316 26647 tablet_service.cc:1511] Processing CreateTablet for tablet 2564769c4e85479fbfb15bd8a05a5807 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:27:24.948946 26545 leader_election.cc:304] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:25.015170 26647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2564769c4e85479fbfb15bd8a05a5807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.016059 26886 raft_consensus.cc:2749] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:25.016086 26545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 67981us
I20260709 12:27:25.017776 26850 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:25.018392 26850 ts_tablet_manager.cc:1403] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.323s	user 0.032s	sys 0.024s
I20260709 12:27:25.020506 26565 tablet_service.cc:1511] Processing CreateTablet for tablet e62bc644ef15433785589f1e01b29a89 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:27:25.020674 26571 tablet_service.cc:1511] Processing CreateTablet for tablet 2564769c4e85479fbfb15bd8a05a5807 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:27:25.004686 26853 tablet_bootstrap.cc:654] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.032480 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62bc644ef15433785589f1e01b29a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.043331 26715 tablet_service.cc:1511] Processing CreateTablet for tablet e62bc644ef15433785589f1e01b29a89 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:27:25.045382 26721 tablet_service.cc:1511] Processing CreateTablet for tablet 2ceb8aabf0a94f9385121858268a3169 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:27:25.053067 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60b354295114347b828b0acc3386536" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:25.052536 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:25.052485 26712 tablet_service.cc:1511] Processing CreateTablet for tablet 03c01181f6bd4a92915740e17605f4b9 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
W20260709 12:27:25.060150 26695 leader_election.cc:343] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:25.060997 26695 leader_election.cc:343] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.062598 26877 consensus_queue.cc:1048] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:25.066771 26571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2564769c4e85479fbfb15bd8a05a5807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.045567 26715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62bc644ef15433785589f1e01b29a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.083873 26721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ceb8aabf0a94f9385121858268a3169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.084811 26478 catalog_manager.cc:5697] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.100276 26867 raft_consensus.cc:493] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.100903 26867 raft_consensus.cc:515] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.101292 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:25.103440 26568 tablet_service.cc:1511] Processing CreateTablet for tablet dc7a25d1e78a4401b1347ed2f07e858d (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 12:27:25.104616 26769 consensus_peers.cc:597] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.105504 26568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc7a25d1e78a4401b1347ed2f07e858d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:25.105855 26695 leader_election.cc:343] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.106366 26695 leader_election.cc:304] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:25.107779 26875 raft_consensus.cc:2749] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.109050 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:25.109534 26790 tablet_service.cc:1511] Processing CreateTablet for tablet dc7a25d1e78a4401b1347ed2f07e858d (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 12:27:25.111241 26545 leader_election.cc:343] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.112439 26867 leader_election.cc:290] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:25.113140 26718 tablet_service.cc:1511] Processing CreateTablet for tablet d187e1b239c246cea5ae376f42a72d6a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:27:25.113708 26849 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:25.114069 26856 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:25.114540 26849 ts_tablet_manager.cc:1403] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.413s	user 0.024s	sys 0.009s
I20260709 12:27:25.114689 26856 ts_tablet_manager.cc:1403] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.458s	user 0.020s	sys 0.016s
I20260709 12:27:25.115293 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:25.117061 26545 leader_election.cc:343] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.117563 26545 leader_election.cc:304] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:25.118882 26867 raft_consensus.cc:2749] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.083189 26850 raft_consensus.cc:359] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.120177 26850 raft_consensus.cc:385] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.120532 26850 raft_consensus.cc:740] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.121399 26850 consensus_queue.cc:260] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.124006 26850 ts_tablet_manager.cc:1434] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.105s	user 0.006s	sys 0.001s
I20260709 12:27:25.125170 26850 tablet_bootstrap.cc:492] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:25.126573 26719 tablet_service.cc:1511] Processing CreateTablet for tablet dc7a25d1e78a4401b1347ed2f07e858d (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:27:25.127948 26712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03c01181f6bd4a92915740e17605f4b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.129980 26565 tablet_service.cc:1511] Processing CreateTablet for tablet c629cb845a5247089d3810e037fe2035 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:27:25.131479 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c629cb845a5247089d3810e037fe2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.131680 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60b354295114347b828b0acc3386536" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:25.133846 26694 leader_election.cc:343] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.118098 26849 raft_consensus.cc:359] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.134603 26694 leader_election.cc:304] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:25.135003 26849 raft_consensus.cc:385] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.135391 26849 raft_consensus.cc:740] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.136066 26875 raft_consensus.cc:2749] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.139336 26791 tablet_service.cc:1511] Processing CreateTablet for tablet d187e1b239c246cea5ae376f42a72d6a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:27:25.140836 26791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d187e1b239c246cea5ae376f42a72d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.141105 26644 tablet_service.cc:1511] Processing CreateTablet for tablet c629cb845a5247089d3810e037fe2035 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:27:25.142776 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c629cb845a5247089d3810e037fe2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.144397 26642 tablet_service.cc:1511] Processing CreateTablet for tablet d187e1b239c246cea5ae376f42a72d6a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:27:25.144959 26717 tablet_service.cc:1511] Processing CreateTablet for tablet c629cb845a5247089d3810e037fe2035 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:27:25.146113 26642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d187e1b239c246cea5ae376f42a72d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.147086 26717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c629cb845a5247089d3810e037fe2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.137519 26849 consensus_queue.cc:260] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.156215 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc7a25d1e78a4401b1347ed2f07e858d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.156256 26718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d187e1b239c246cea5ae376f42a72d6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.173789 26719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc7a25d1e78a4401b1347ed2f07e858d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.139863 26850 tablet_bootstrap.cc:654] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:25.234246 26464 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 83774.4us
I20260709 12:27:25.237788 26789 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4e1d3b6a6a41bc81db8ef63b2cc50a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:27:25.239415 26789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4e1d3b6a6a41bc81db8ef63b2cc50a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.143402 26856 raft_consensus.cc:359] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.244736 26566 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4e1d3b6a6a41bc81db8ef63b2cc50a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:27:25.248921 26566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4e1d3b6a6a41bc81db8ef63b2cc50a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.244815 26856 raft_consensus.cc:385] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.253122 26856 raft_consensus.cc:740] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.254153 26856 consensus_queue.cc:260] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.255961 26639 tablet_service.cc:1511] Processing CreateTablet for tablet 3d4e1d3b6a6a41bc81db8ef63b2cc50a (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:27:25.257458 26639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d4e1d3b6a6a41bc81db8ef63b2cc50a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.259658 26856 ts_tablet_manager.cc:1434] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.145s	user 0.018s	sys 0.000s
I20260709 12:27:25.261147 26856 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:25.265024 26853 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:25.265780 26853 ts_tablet_manager.cc:1403] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.341s	user 0.017s	sys 0.000s
I20260709 12:27:25.246134 26849 ts_tablet_manager.cc:1434] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.131s	user 0.011s	sys 0.005s
I20260709 12:27:25.276894 26875 raft_consensus.cc:493] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.277554 26875 raft_consensus.cc:515] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.280093 26875 leader_election.cc:290] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:25.281850 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:25.283635 26788 tablet_service.cc:1511] Processing CreateTablet for tablet 2bcd7e1d18334a01bc14e417fbbeaff2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 12:27:25.284252 26695 leader_election.cc:343] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.290509 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:25.290242 26853 raft_consensus.cc:359] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.291311 26853 raft_consensus.cc:385] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.291684 26853 raft_consensus.cc:740] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.292425 26867 consensus_queue.cc:1048] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:25.291316 26657 raft_consensus.cc:2468] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:25.295948 26694 leader_election.cc:304] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:25.297503 26875 raft_consensus.cc:2804] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:25.297911 26875 raft_consensus.cc:493] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.298437 26875 raft_consensus.cc:3060] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:25.297933 26788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bcd7e1d18334a01bc14e417fbbeaff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.315769 26849 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
W20260709 12:27:25.317068 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:25.315769 26713 tablet_service.cc:1511] Processing CreateTablet for tablet 2bcd7e1d18334a01bc14e417fbbeaff2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:27:25.319162 26713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bcd7e1d18334a01bc14e417fbbeaff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.331570 26875 raft_consensus.cc:515] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.335245 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:25.336097 26657 raft_consensus.cc:3060] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:25.343868 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 136344us
I20260709 12:27:25.373431 26875 leader_election.cc:290] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:25.347174 26853 consensus_queue.cc:260] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.382246 26571 tablet_service.cc:1511] Processing CreateTablet for tablet 2bcd7e1d18334a01bc14e417fbbeaff2 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:27:25.383213 26645 tablet_service.cc:1511] Processing CreateTablet for tablet f229d58a8c304ca1b723a4954aed76f4 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:27:25.384356 26571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bcd7e1d18334a01bc14e417fbbeaff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.384810 26645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f229d58a8c304ca1b723a4954aed76f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.385097 26853 ts_tablet_manager.cc:1434] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.119s	user 0.008s	sys 0.004s
I20260709 12:27:25.395546 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:25.399145 26877 raft_consensus.cc:493] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.402046 26657 raft_consensus.cc:2468] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:25.407529 26849 tablet_bootstrap.cc:654] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.386188 26853 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:25.399767 26877 raft_consensus.cc:515] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.417476 26856 tablet_bootstrap.cc:654] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:25.410089 26695 leader_election.cc:343] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.456251 26850 tablet_bootstrap.cc:492] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:25.457201 26850 ts_tablet_manager.cc:1403] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.332s	user 0.018s	sys 0.021s
W20260709 12:27:25.463793 26545 consensus_peers.cc:597] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:25.483616 26694 leader_election.cc:304] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:25.485086 26875 raft_consensus.cc:2804] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:25.486136 26867 raft_consensus.cc:493] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.487624 26867 raft_consensus.cc:515] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.491852 26877 leader_election.cc:290] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:25.493500 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:25.495503 26768 leader_election.cc:343] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.499812 26721 tablet_service.cc:1511] Processing CreateTablet for tablet f229d58a8c304ca1b723a4954aed76f4 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:27:25.501777 26721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f229d58a8c304ca1b723a4954aed76f4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:25.502736 26893 log.cc:927] Time spent T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:25.503671 26867 leader_election.cc:290] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:25.525966 26875 raft_consensus.cc:697] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:25.531824 26875 consensus_queue.cc:237] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.536496 26850 raft_consensus.cc:359] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:25.537806 26850 raft_consensus.cc:385] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.581207 26565 tablet_service.cc:1511] Processing CreateTablet for tablet 673d53dde2dc42e1b7feb31f4e28bb3c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:27:25.589958 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:25.590727 26644 tablet_service.cc:1511] Processing CreateTablet for tablet 673d53dde2dc42e1b7feb31f4e28bb3c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:27:25.599529 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:25.605276 26717 tablet_service.cc:1511] Processing CreateTablet for tablet 673d53dde2dc42e1b7feb31f4e28bb3c (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:27:25.613334 26642 tablet_service.cc:1511] Processing CreateTablet for tablet b2e5dc8624444d368676b1a1b3fa878e (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:27:25.663475 26717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673d53dde2dc42e1b7feb31f4e28bb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.664043 26642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2e5dc8624444d368676b1a1b3fa878e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.621318 26910 raft_consensus.cc:493] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.624068 26571 tablet_service.cc:1511] Processing CreateTablet for tablet b2e5dc8624444d368676b1a1b3fa878e (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:27:25.623710 26568 tablet_service.cc:1511] Processing CreateTablet for tablet 473c572574d044c0adc6366487b343fa (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:27:25.630638 26718 tablet_service.cc:1511] Processing CreateTablet for tablet 473c572574d044c0adc6366487b343fa (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:27:25.654062 26853 tablet_bootstrap.cc:700] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent opening tablet: real 0.119s	user 0.000s	sys 0.002s
I20260709 12:27:25.671018 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:25.677070 26856 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:25.655616 26850 raft_consensus.cc:740] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.679563 26850 consensus_queue.cc:260] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:25.681080 26849 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
W20260709 12:27:25.684393 26545 leader_election.cc:343] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.686021 26718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 473c572574d044c0adc6366487b343fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.686368 26786 tablet_service.cc:1511] Processing CreateTablet for tablet f229d58a8c304ca1b723a4954aed76f4 (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:27:25.696455 26644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673d53dde2dc42e1b7feb31f4e28bb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.694883 26565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673d53dde2dc42e1b7feb31f4e28bb3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.699668 26786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f229d58a8c304ca1b723a4954aed76f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.699947 26850 ts_tablet_manager.cc:1434] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.242s	user 0.008s	sys 0.006s
I20260709 12:27:25.691609 26856 ts_tablet_manager.cc:1403] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.431s	user 0.016s	sys 0.003s
I20260709 12:27:25.700628 26790 tablet_service.cc:1511] Processing CreateTablet for tablet b2e5dc8624444d368676b1a1b3fa878e (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:27:25.706100 26853 tablet_bootstrap.cc:654] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.707666 26790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2e5dc8624444d368676b1a1b3fa878e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.695289 26910 raft_consensus.cc:515] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:25.703781 26849 ts_tablet_manager.cc:1403] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.388s	user 0.010s	sys 0.011s
I20260709 12:27:25.710667 26900 raft_consensus.cc:493] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.711424 26900 raft_consensus.cc:515] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:25.717854 26850 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:25.718569 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:25.720871 26768 leader_election.cc:343] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.721577 26571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2e5dc8624444d368676b1a1b3fa878e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.722615 26900 leader_election.cc:290] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:25.725481 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:25.727767 26546 leader_election.cc:343] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.728404 26546 leader_election.cc:304] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
W20260709 12:27:25.728935 26769 leader_election.cc:343] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.728725 26791 tablet_service.cc:1511] Processing CreateTablet for tablet 473c572574d044c0adc6366487b343fa (DEFAULT_TABLE table=test-workload [id=79cd3582622b40f392ea1e98b9420950]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 12:27:25.729585 26768 leader_election.cc:343] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.730089 26768 leader_election.cc:304] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:25.726634 26568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 473c572574d044c0adc6366487b343fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.731415 26877 raft_consensus.cc:2749] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.739738 26857 raft_consensus.cc:493] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.740731 26857 raft_consensus.cc:515] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:25.729549 26769 leader_election.cc:304] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:25.747696 26877 raft_consensus.cc:2749] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:25.775893 26789 consensus_meta.cc:287] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.015s
I20260709 12:27:25.780409 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:25.782639 26621 leader_election.cc:343] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.794713 26867 raft_consensus.cc:2749] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.796481 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:25.798516 26621 leader_election.cc:343] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.799060 26621 leader_election.cc:304] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:25.785832 26856 raft_consensus.cc:359] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:25.825323 26856 raft_consensus.cc:385] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.816895 26791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 473c572574d044c0adc6366487b343fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:25.826639 26856 raft_consensus.cc:740] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.827617 26478 catalog_manager.cc:5697] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:25.833669 26857 leader_election.cc:290] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:25.834584 26857 raft_consensus.cc:2749] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.828588 26856 consensus_queue.cc:260] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:25.819572 26849 raft_consensus.cc:359] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:25.779054 26769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 44794.9us
W20260709 12:27:25.844131 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 45659.7us
I20260709 12:27:25.846592 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413d45f35e654998908cd4c1c6b8a0f2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:25.847107 26546 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 119578us
I20260709 12:27:25.804610 26910 leader_election.cc:290] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:25.842862 26849 raft_consensus.cc:385] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:25.852705 26620 leader_election.cc:343] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.871845 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413d45f35e654998908cd4c1c6b8a0f2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:25.875329 26856 ts_tablet_manager.cc:1434] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.174s	user 0.010s	sys 0.001s
I20260709 12:27:25.877069 26856 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:25.861804 26849 raft_consensus.cc:740] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
W20260709 12:27:25.864533 26465 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 44505.7us
I20260709 12:27:25.879397 26849 consensus_queue.cc:260] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:25.880228 26621 leader_election.cc:343] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:25.880873 26621 leader_election.cc:304] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:25.888376 26856 tablet_bootstrap.cc:654] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.889828 26850 tablet_bootstrap.cc:654] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.894143 26910 raft_consensus.cc:2749] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:25.864344 26853 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:25.899389 26853 ts_tablet_manager.cc:1403] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.513s	user 0.015s	sys 0.023s
I20260709 12:27:25.910955 26849 ts_tablet_manager.cc:1434] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.201s	user 0.016s	sys 0.003s
I20260709 12:27:25.912575 26849 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:25.919242 26853 raft_consensus.cc:359] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:25.920266 26853 raft_consensus.cc:385] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.920589 26853 raft_consensus.cc:740] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.942010 26850 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:25.942793 26850 ts_tablet_manager.cc:1403] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.225s	user 0.026s	sys 0.015s
I20260709 12:27:25.937496 26856 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:25.947916 26856 ts_tablet_manager.cc:1403] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.012s
I20260709 12:27:25.947310 26850 raft_consensus.cc:359] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.948423 26850 raft_consensus.cc:385] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:25.948798 26850 raft_consensus.cc:740] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:25.950169 26850 consensus_queue.cc:260] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:25.922672 26853 consensus_queue.cc:260] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:25.958760 26850 ts_tablet_manager.cc:1434] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:27:25.960685 26850 tablet_bootstrap.cc:492] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:25.962764 26853 ts_tablet_manager.cc:1434] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.063s	user 0.002s	sys 0.007s
I20260709 12:27:25.966812 26849 tablet_bootstrap.cc:654] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.973765 26731 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:25.974627 26913 raft_consensus.cc:493] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.975274 26913 raft_consensus.cc:515] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:25.979033 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:25.979849 26805 raft_consensus.cc:2468] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:25.980962 26850 tablet_bootstrap.cc:654] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:25.981695 26546 leader_election.cc:304] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:25.983002 26912 raft_consensus.cc:2804] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:25.983469 26912 raft_consensus.cc:493] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.983826 26912 raft_consensus.cc:3060] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:25.989430 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:25.990828 26730 raft_consensus.cc:2468] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:25.997718 26913 leader_election.cc:290] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.001704 26856 raft_consensus.cc:359] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.003033 26856 raft_consensus.cc:385] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.003453 26856 raft_consensus.cc:740] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.004652 26856 consensus_queue.cc:260] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.006443 26853 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.008256 26912 raft_consensus.cc:515] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.014613 26856 ts_tablet_manager.cc:1434] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.066s	user 0.010s	sys 0.000s
I20260709 12:27:26.016280 26856 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.020874 26850 tablet_bootstrap.cc:492] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.021579 26850 ts_tablet_manager.cc:1403] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.018s
I20260709 12:27:26.026121 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:26.026930 26805 raft_consensus.cc:3060] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.027944 26849 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:26.028152 26850 raft_consensus.cc:359] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.031513 26850 raft_consensus.cc:385] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.031028 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:26.031781 26849 ts_tablet_manager.cc:1403] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.119s	user 0.032s	sys 0.024s
I20260709 12:27:26.031939 26850 raft_consensus.cc:740] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.032410 26730 raft_consensus.cc:3060] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.032194 26853 tablet_bootstrap.cc:654] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.039417 26850 consensus_queue.cc:260] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.042080 26849 raft_consensus.cc:359] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.043074 26849 raft_consensus.cc:385] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.043448 26849 raft_consensus.cc:740] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.044471 26849 consensus_queue.cc:260] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.051838 26850 ts_tablet_manager.cc:1434] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.030s	user 0.001s	sys 0.013s
I20260709 12:27:26.031173 26912 leader_election.cc:290] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.057554 26850 tablet_bootstrap.cc:492] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:26.058923 26730 raft_consensus.cc:2468] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:26.070454 26856 tablet_bootstrap.cc:654] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.071578 26545 leader_election.cc:304] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:26.073103 26913 raft_consensus.cc:2804] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.073719 26913 raft_consensus.cc:697] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:26.074834 26913 consensus_queue.cc:237] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.061219 26805 raft_consensus.cc:2468] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:26.062693 26849 ts_tablet_manager.cc:1434] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 12:27:26.066037 26853 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.095304 26853 ts_tablet_manager.cc:1403] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.131s	user 0.020s	sys 0.003s
I20260709 12:27:26.095947 26849 tablet_bootstrap.cc:492] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:26.097806 26478 catalog_manager.cc:5697] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.067698 26850 tablet_bootstrap.cc:654] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.099073 26853 raft_consensus.cc:359] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.103464 26853 raft_consensus.cc:385] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.103869 26853 raft_consensus.cc:740] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.104900 26853 consensus_queue.cc:260] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.107832 26853 ts_tablet_manager.cc:1434] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:27:26.108224 26856 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:26.108914 26856 ts_tablet_manager.cc:1403] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.093s	user 0.021s	sys 0.012s
I20260709 12:27:26.109295 26853 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.112519 26856 raft_consensus.cc:359] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.113328 26856 raft_consensus.cc:385] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.113620 26856 raft_consensus.cc:740] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.113945 26849 tablet_bootstrap.cc:654] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.114470 26856 consensus_queue.cc:260] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.124956 26856 ts_tablet_manager.cc:1434] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.012s
I20260709 12:27:26.125432 26849 tablet_bootstrap.cc:492] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:26.126135 26849 ts_tablet_manager.cc:1403] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.006s
I20260709 12:27:26.126528 26856 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.129835 26849 raft_consensus.cc:359] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.130726 26849 raft_consensus.cc:385] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.131026 26849 raft_consensus.cc:740] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.132210 26930 consensus_queue.cc:1048] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:26.135769 26856 tablet_bootstrap.cc:654] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.139827 26853 tablet_bootstrap.cc:654] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.141788 26849 consensus_queue.cc:260] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.148334 26582 raft_consensus.cc:3060] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.155417 26850 tablet_bootstrap.cc:492] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.156204 26850 ts_tablet_manager.cc:1403] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.099s	user 0.013s	sys 0.015s
I20260709 12:27:26.160424 26850 raft_consensus.cc:359] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.161230 26850 raft_consensus.cc:385] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.161547 26850 raft_consensus.cc:740] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.162041 26853 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.162688 26853 ts_tablet_manager.cc:1403] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.002s
I20260709 12:27:26.162477 26850 consensus_queue.cc:260] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.165343 26850 ts_tablet_manager.cc:1434] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:27:26.166664 26850 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:26.168491 26853 raft_consensus.cc:359] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.169337 26853 raft_consensus.cc:385] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.169651 26853 raft_consensus.cc:740] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.174974 26910 raft_consensus.cc:493] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.175619 26910 raft_consensus.cc:515] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.176452 26900 raft_consensus.cc:493] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.177145 26900 raft_consensus.cc:515] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.170667 26853 consensus_queue.cc:260] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.179569 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.180464 26581 raft_consensus.cc:2468] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:26.181990 26900 leader_election.cc:290] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:26.182227 26621 leader_election.cc:304] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:26.183761 26916 raft_consensus.cc:2804] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.184233 26916 raft_consensus.cc:493] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.184633 26916 raft_consensus.cc:3060] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.185130 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.185873 26656 raft_consensus.cc:2468] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:26.186448 26853 ts_tablet_manager.cc:1434] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.004s
I20260709 12:27:26.187453 26768 leader_election.cc:304] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:26.187111 26849 ts_tablet_manager.cc:1434] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 12:27:26.188481 26900 raft_consensus.cc:2804] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.188938 26900 raft_consensus.cc:493] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.189419 26900 raft_consensus.cc:3060] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.194211 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.198401 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:26.198886 26769 leader_election.cc:343] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.199106 26730 raft_consensus.cc:2468] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:26.194967 26910 leader_election.cc:290] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.195206 26853 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.203589 26849 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:26.206975 26913 raft_consensus.cc:493] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.203116 26900 raft_consensus.cc:515] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.207593 26913 raft_consensus.cc:515] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.208684 26850 tablet_bootstrap.cc:654] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.211064 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:26.211498 26900 leader_election.cc:290] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:26.212489 26769 leader_election.cc:343] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.213735 26913 leader_election.cc:290] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.213845 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.215857 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:26.216269 26853 tablet_bootstrap.cc:654] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.216557 26655 raft_consensus.cc:3060] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.225060 26850 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.225878 26850 ts_tablet_manager.cc:1403] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.059s	user 0.008s	sys 0.004s
I20260709 12:27:26.227653 26930 consensus_queue.cc:1048] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:26.237881 26850 raft_consensus.cc:359] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.237881 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4761da8e934968ae1169a27136254a" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:26.238776 26850 raft_consensus.cc:385] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.239152 26850 raft_consensus.cc:740] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.240782 26850 consensus_queue.cc:260] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.240882 26916 raft_consensus.cc:515] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.242654 26850 ts_tablet_manager.cc:1434] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:27:26.245723 26849 tablet_bootstrap.cc:654] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.247335 26545 leader_election.cc:304] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.249837 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:26.250582 26582 raft_consensus.cc:3060] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.260489 26916 leader_election.cc:290] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.261845 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:26.260685 26655 raft_consensus.cc:2468] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:26.262600 26730 raft_consensus.cc:3060] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.262733 26853 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.263387 26853 ts_tablet_manager.cc:1403] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.010s
I20260709 12:27:26.264353 26768 leader_election.cc:304] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:26.265683 26900 raft_consensus.cc:2804] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.266888 26853 raft_consensus.cc:359] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.268020 26853 raft_consensus.cc:385] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.268359 26853 raft_consensus.cc:740] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.272439 26916 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.273129 26916 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.276153 26582 raft_consensus.cc:2468] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:26.276643 26916 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.277001 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.277464 26621 leader_election.cc:304] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:26.277904 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:26.278602 26621 leader_election.cc:343] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.278839 26856 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
W20260709 12:27:26.279430 26620 leader_election.cc:343] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.280150 26620 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.280804 26856 ts_tablet_manager.cc:1403] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.155s	user 0.038s	sys 0.044s
I20260709 12:27:26.281199 26910 raft_consensus.cc:2749] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.279301 26916 raft_consensus.cc:2804] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.284173 26916 raft_consensus.cc:697] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:26.286415 26912 raft_consensus.cc:2749] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:26.318249 26900 raft_consensus.cc:697] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:26.331867 26940 raft_consensus.cc:493] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.332576 26940 raft_consensus.cc:515] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.334095 26849 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:26.334733 26849 ts_tablet_manager.cc:1403] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.131s	user 0.016s	sys 0.008s
I20260709 12:27:26.336537 26850 tablet_bootstrap.cc:492] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:26.337097 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:26.338919 26769 leader_election.cc:343] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.338572 26916 consensus_queue.cc:237] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.343040 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:26.343540 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:26.344282 26853 consensus_queue.cc:260] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.345161 26913 raft_consensus.cc:493] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.345903 26913 raft_consensus.cc:515] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.351873 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:26.352456 26768 leader_election.cc:343] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.353065 26768 leader_election.cc:304] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.354079 26941 raft_consensus.cc:2749] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.355288 26913 leader_election.cc:290] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:26.356595 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.357599 26657 raft_consensus.cc:2393] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:26.359055 26545 leader_election.cc:304] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:26.360949 26867 raft_consensus.cc:3060] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.372820 26849 raft_consensus.cc:359] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.374001 26849 raft_consensus.cc:385] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.374666 26849 raft_consensus.cc:740] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.334048 26900 consensus_queue.cc:237] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.379222 26856 raft_consensus.cc:359] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.380928 26856 raft_consensus.cc:385] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.381673 26856 raft_consensus.cc:740] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.375881 26849 consensus_queue.cc:260] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.384392 26940 leader_election.cc:290] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:26.388046 26849 ts_tablet_manager.cc:1434] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.001s
I20260709 12:27:26.389499 26849 tablet_bootstrap.cc:492] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:26.390789 26867 raft_consensus.cc:2749] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.373986 26853 ts_tablet_manager.cc:1434] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.110s	user 0.019s	sys 0.012s
I20260709 12:27:26.393234 26853 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.384441 26856 consensus_queue.cc:260] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.345165 26730 raft_consensus.cc:2468] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:26.411187 26856 ts_tablet_manager.cc:1434] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.129s	user 0.008s	sys 0.002s
I20260709 12:27:26.418175 26478 catalog_manager.cc:5697] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:26.424124 26853 tablet_bootstrap.cc:654] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.426638 26850 tablet_bootstrap.cc:654] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.419499 26856 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.442519 26849 tablet_bootstrap.cc:654] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.444514 26856 tablet_bootstrap.cc:654] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.464607 26856 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:26.465143 26853 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.465305 26856 ts_tablet_manager.cc:1403] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.002s
I20260709 12:27:26.465811 26853 ts_tablet_manager.cc:1403] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.073s	user 0.017s	sys 0.006s
I20260709 12:27:26.468204 26479 catalog_manager.cc:5697] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:26.471150 26850 tablet_bootstrap.cc:492] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.471977 26850 ts_tablet_manager.cc:1403] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.136s	user 0.016s	sys 0.004s
I20260709 12:27:26.471752 26912 consensus_queue.cc:1048] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:26.475683 26850 raft_consensus.cc:359] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.476584 26850 raft_consensus.cc:385] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.476884 26850 raft_consensus.cc:740] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.477712 26850 consensus_queue.cc:260] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.479763 26910 raft_consensus.cc:493] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.480825 26910 raft_consensus.cc:515] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.482321 26850 ts_tablet_manager.cc:1434] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:27:26.483680 26850 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:26.485183 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60b354295114347b828b0acc3386536" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:26.486016 26730 raft_consensus.cc:2468] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:26.487730 26620 leader_election.cc:304] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:26.488288 26930 raft_consensus.cc:493] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.488096 26853 raft_consensus.cc:359] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.489712 26853 raft_consensus.cc:385] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.488876 26916 raft_consensus.cc:493] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.489928 26849 tablet_bootstrap.cc:492] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:26.490442 26853 raft_consensus.cc:740] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.490689 26916 raft_consensus.cc:515] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.491272 26849 ts_tablet_manager.cc:1403] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.007s
I20260709 12:27:26.491430 26853 consensus_queue.cc:260] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.492130 26850 tablet_bootstrap.cc:654] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.489670 26930 raft_consensus.cc:515] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.496562 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60b354295114347b828b0acc3386536" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:26.497309 26910 leader_election.cc:290] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.497822 26930 leader_election.cc:290] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:26.500032 26803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:26.500886 26803 raft_consensus.cc:2468] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:26.502372 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.502818 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:26.504372 26695 leader_election.cc:343] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.506955 26853 ts_tablet_manager.cc:1434] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.041s	user 0.009s	sys 0.000s
I20260709 12:27:26.507591 26621 leader_election.cc:304] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:26.508275 26853 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.511449 26916 leader_election.cc:290] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:26.511672 26695 leader_election.cc:343] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.512017 26916 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.512413 26695 leader_election.cc:304] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:26.512527 26916 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.515012 26916 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.497875 26910 raft_consensus.cc:2804] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.515607 26916 raft_consensus.cc:2804] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.515861 26910 raft_consensus.cc:493] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.516607 26910 raft_consensus.cc:3060] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.517832 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:26.519621 26620 leader_election.cc:343] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.519771 26849 raft_consensus.cc:359] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.515154 26930 raft_consensus.cc:2749] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.520510 26849 raft_consensus.cc:385] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.520853 26849 raft_consensus.cc:740] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.516060 26916 raft_consensus.cc:493] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.521566 26853 tablet_bootstrap.cc:654] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.523918 26916 raft_consensus.cc:3060] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.526950 26849 consensus_queue.cc:260] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.528014 26856 raft_consensus.cc:359] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.528683 26856 raft_consensus.cc:385] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.524436 26940 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.528925 26856 raft_consensus.cc:740] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.530499 26940 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.532698 26940 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.536422 26930 raft_consensus.cc:493] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.538329 26849 ts_tablet_manager.cc:1434] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.002s
I20260709 12:27:26.539324 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:26.542403 26621 leader_election.cc:343] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:26.542582 26768 leader_election.cc:343] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.543407 26849 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:26.545320 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:26.545555 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.546106 26804 raft_consensus.cc:2468] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:26.545781 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.547024 26657 raft_consensus.cc:2468] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:26.547951 26912 consensus_queue.cc:1048] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:26.548537 26768 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.549902 26940 raft_consensus.cc:2804] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.550375 26940 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.550801 26940 raft_consensus.cc:3060] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.553848 26849 tablet_bootstrap.cc:654] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.556830 26856 consensus_queue.cc:260] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.562101 26912 raft_consensus.cc:493] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.564757 26912 raft_consensus.cc:515] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.567215 26900 raft_consensus.cc:493] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.559696 26930 raft_consensus.cc:515] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.567932 26912 leader_election.cc:290] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.567848 26900 raft_consensus.cc:515] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.568796 26621 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.570024 26951 raft_consensus.cc:2804] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:27:26.570564 26621 leader_election.cc:343] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.570719 26951 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.571334 26951 raft_consensus.cc:3060] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.573110 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:26.574868 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:26.576877 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.576982 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.577934 26580 raft_consensus.cc:2468] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:26.579625 26695 leader_election.cc:304] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:26.579653 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:26.580312 26768 leader_election.cc:343] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.580787 26802 raft_consensus.cc:2468] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:26.581362 26875 raft_consensus.cc:2804] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.581779 26875 raft_consensus.cc:493] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.582136 26875 raft_consensus.cc:3060] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.582645 26546 leader_election.cc:304] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:26.584491 26900 leader_election.cc:290] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:26.586426 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.588192 26856 ts_tablet_manager.cc:1434] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.122s	user 0.003s	sys 0.004s
W20260709 12:27:26.588514 26769 leader_election.cc:343] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.589054 26769 leader_election.cc:304] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.589545 26856 tablet_bootstrap.cc:492] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.590328 26900 raft_consensus.cc:2749] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.591255 26912 raft_consensus.cc:2749] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.566468 26910 raft_consensus.cc:515] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.575631 26930 leader_election.cc:290] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:26.598606 26951 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.597609 26849 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:26.602062 26849 ts_tablet_manager.cc:1403] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.059s	user 0.022s	sys 0.005s
I20260709 12:27:26.601245 26875 raft_consensus.cc:515] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.602907 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:26.605276 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:26.606173 26804 raft_consensus.cc:3060] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.601244 26940 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.606231 26849 raft_consensus.cc:359] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.627128 26849 raft_consensus.cc:385] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.627429 26849 raft_consensus.cc:740] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.628302 26849 consensus_queue.cc:260] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.631220 26849 ts_tablet_manager.cc:1434] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:27:26.608260 26916 raft_consensus.cc:515] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.635200 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:26.636083 26657 raft_consensus.cc:2393] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:26.636667 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
W20260709 12:27:26.611632 26620 leader_election.cc:343] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.642671 26805 raft_consensus.cc:2393] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:26.612576 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60b354295114347b828b0acc3386536" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:26.645241 26802 raft_consensus.cc:3060] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.626029 26951 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.615398 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:26.612937 26875 leader_election.cc:290] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:26.652554 26581 raft_consensus.cc:3060] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.612694 26940 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.617478 26850 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.638892 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:26.608435 26910 leader_election.cc:290] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.614745 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a60b354295114347b828b0acc3386536" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:26.662294 26731 raft_consensus.cc:3060] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:26.659845 26768 leader_election.cc:343] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.662962 26768 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.663898 26900 raft_consensus.cc:2749] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.639779 26856 tablet_bootstrap.cc:654] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.635221 26950 raft_consensus.cc:493] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.670289 26950 raft_consensus.cc:515] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.671288 26804 raft_consensus.cc:2468] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:26.672325 26621 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.672952 26695 leader_election.cc:304] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:26.673880 26875 raft_consensus.cc:2804] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.674373 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:26.674595 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:26.675395 26804 raft_consensus.cc:2468] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
W20260709 12:27:26.676874 26621 leader_election.cc:343] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.644387 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:26.674397 26875 raft_consensus.cc:697] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:26.677732 26801 raft_consensus.cc:3060] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.678056 26951 raft_consensus.cc:2749] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.678668 26875 consensus_queue.cc:237] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.640806 26849 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:26.637151 26916 leader_election.cc:290] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:26.661182 26850 ts_tablet_manager.cc:1403] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.004s
I20260709 12:27:26.615851 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6147d09bb6fe4ec5be65fa9239d10536" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
W20260709 12:27:26.676910 26620 leader_election.cc:343] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.685943 26850 raft_consensus.cc:359] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.672964 26950 leader_election.cc:290] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.687533 26802 raft_consensus.cc:2468] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:26.686779 26580 raft_consensus.cc:3060] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.688336 26801 raft_consensus.cc:2468] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:26.687211 26850 raft_consensus.cc:385] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.699509 26850 raft_consensus.cc:740] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.698002 26479 catalog_manager.cc:5697] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:26.700492 26850 consensus_queue.cc:260] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.697624 26580 raft_consensus.cc:2468] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:26.702936 26621 leader_election.cc:304] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.714874 26950 raft_consensus.cc:2804] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.715293 26950 raft_consensus.cc:493] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.715530 26853 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.715619 26950 raft_consensus.cc:3060] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.721550 26731 raft_consensus.cc:2468] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:26.722435 26853 ts_tablet_manager.cc:1403] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.214s	user 0.009s	sys 0.008s
W20260709 12:27:26.722684 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 44896.1us
I20260709 12:27:26.724565 26621 leader_election.cc:304] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:26.714170 26856 tablet_bootstrap.cc:492] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:26.714038 26875 raft_consensus.cc:493] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.729992 26875 raft_consensus.cc:515] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.730396 26853 raft_consensus.cc:359] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.730484 26950 raft_consensus.cc:515] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.731205 26853 raft_consensus.cc:385] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.731514 26853 raft_consensus.cc:740] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.732311 26853 consensus_queue.cc:260] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.733717 26950 leader_election.cc:290] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.734306 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:26.734743 26853 ts_tablet_manager.cc:1434] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.007s
I20260709 12:27:26.735857 26853 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:26.736009 26620 leader_election.cc:343] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.733450 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.736769 26580 raft_consensus.cc:2468] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:26.737313 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.737526 26875 leader_election.cc:290] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:26.738806 26695 leader_election.cc:304] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
W20260709 12:27:26.739609 26694 leader_election.cc:343] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.739775 26875 raft_consensus.cc:2804] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:26.740531 26875 raft_consensus.cc:493] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.740947 26875 raft_consensus.cc:3060] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.742769 26856 ts_tablet_manager.cc:1403] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.154s	user 0.007s	sys 0.009s
I20260709 12:27:26.742873 26853 tablet_bootstrap.cc:654] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.745978 26856 raft_consensus.cc:359] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.746755 26856 raft_consensus.cc:385] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.747041 26856 raft_consensus.cc:740] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.747828 26856 consensus_queue.cc:260] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.750026 26856 ts_tablet_manager.cc:1434] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 12:27:26.726830 26849 tablet_bootstrap.cc:654] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.726761 26916 raft_consensus.cc:2804] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.750780 26916 raft_consensus.cc:697] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:26.752013 26916 consensus_queue.cc:237] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.727320 26912 raft_consensus.cc:493] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.754541 26856 tablet_bootstrap.cc:492] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.754917 26912 raft_consensus.cc:515] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.727589 26621 leader_election.cc:304] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:26.713487 26850 ts_tablet_manager.cc:1434] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:27:26.757290 26912 leader_election.cc:290] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.758496 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.759217 26853 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.759807 26853 ts_tablet_manager.cc:1403] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
W20260709 12:27:26.760689 26545 leader_election.cc:343] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.767136 26850 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:26.772982 26875 raft_consensus.cc:515] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.773307 26853 raft_consensus.cc:359] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.774046 26853 raft_consensus.cc:385] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.775014 26951 raft_consensus.cc:2804] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.776233 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:26.777891 26853 raft_consensus.cc:740] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.778146 26951 raft_consensus.cc:697] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:26.778328 26581 raft_consensus.cc:3060] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.778925 26853 consensus_queue.cc:260] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.779265 26951 consensus_queue.cc:237] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.780180 26849 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:26.776979 26875 leader_election.cc:290] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:26.777233 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:26.781400 26853 ts_tablet_manager.cc:1434] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
W20260709 12:27:26.782436 26694 leader_election.cc:343] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:26.777559 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 50084.6us
I20260709 12:27:26.780750 26849 ts_tablet_manager.cc:1403] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.140s	user 0.014s	sys 0.004s
I20260709 12:27:26.785478 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:26.786192 26801 raft_consensus.cc:3060] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.786648 26849 raft_consensus.cc:359] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.787391 26849 raft_consensus.cc:385] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.787672 26849 raft_consensus.cc:740] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.788648 26849 consensus_queue.cc:260] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:26.789980 26850 tablet_bootstrap.cc:654] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.767194 26856 tablet_bootstrap.cc:654] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.794191 26930 raft_consensus.cc:493] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.795595 26930 raft_consensus.cc:515] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.798244 26849 ts_tablet_manager.cc:1434] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:27:26.799458 26849 tablet_bootstrap.cc:492] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:26.801067 26853 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.802515 26930 leader_election.cc:290] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:26.803697 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.804629 26657 raft_consensus.cc:2393] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:26.785487 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:26.808143 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:26.808430 26545 leader_election.cc:343] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.808930 26545 leader_election.cc:304] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.810006 26695 leader_election.cc:304] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:26.812125 26849 tablet_bootstrap.cc:654] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.812537 26950 consensus_queue.cc:1048] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:26.814944 26930 raft_consensus.cc:3060] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.818482 26912 raft_consensus.cc:2749] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.819190 26801 raft_consensus.cc:2468] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:26.822729 26621 leader_election.cc:304] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.823984 26950 raft_consensus.cc:2804] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.824548 26950 raft_consensus.cc:697] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:26.825418 26950 consensus_queue.cc:237] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.832993 26940 consensus_queue.cc:1048] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:26.830336 26581 raft_consensus.cc:2468] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:26.844727 26695 leader_election.cc:304] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:26.849771 26875 raft_consensus.cc:2804] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:26.855458 26856 tablet_bootstrap.cc:492] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:26.856112 26856 ts_tablet_manager.cc:1403] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.102s	user 0.012s	sys 0.022s
I20260709 12:27:26.862831 26856 raft_consensus.cc:359] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.863718 26856 raft_consensus.cc:385] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.864075 26856 raft_consensus.cc:740] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.867564 26856 consensus_queue.cc:260] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.871045 26930 raft_consensus.cc:2749] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.879495 26853 tablet_bootstrap.cc:654] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.873759 26875 raft_consensus.cc:697] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:26.887354 26856 ts_tablet_manager.cc:1434] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.004s
I20260709 12:27:26.887777 26850 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.874300 26480 catalog_manager.cc:5697] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:26.881446 26900 consensus_queue.cc:1048] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:26.890861 26850 ts_tablet_manager.cc:1403] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.124s	user 0.021s	sys 0.009s
I20260709 12:27:26.893545 26875 consensus_queue.cc:237] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.893922 26480 catalog_manager.cc:5697] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.905124 26900 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.907645 26900 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.914244 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.911855 26900 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:26.915802 26769 leader_election.cc:343] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.920060 26856 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.921818 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.928210 26850 raft_consensus.cc:359] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.929061 26850 raft_consensus.cc:385] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.929374 26850 raft_consensus.cc:740] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.932355 26856 tablet_bootstrap.cc:654] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.932963 26853 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.935418 26853 ts_tablet_manager.cc:1403] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.135s	user 0.028s	sys 0.012s
I20260709 12:27:26.930104 26850 consensus_queue.cc:260] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:26.937903 26768 leader_election.cc:343] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.938437 26768 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:26.939529 26916 consensus_queue.cc:1048] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:26.939631 26850 ts_tablet_manager.cc:1434] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 12:27:26.939661 26900 raft_consensus.cc:2749] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.941529 26850 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:26.943892 26853 raft_consensus.cc:359] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.944746 26853 raft_consensus.cc:385] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.945148 26853 raft_consensus.cc:740] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.946026 26853 consensus_queue.cc:260] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.949084 26853 ts_tablet_manager.cc:1434] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 12:27:26.950569 26853 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:26.954229 26856 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:26.955065 26856 ts_tablet_manager.cc:1403] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.011s
I20260709 12:27:26.959302 26853 tablet_bootstrap.cc:654] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.959479 26856 raft_consensus.cc:359] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.960541 26856 raft_consensus.cc:385] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:26.960892 26856 raft_consensus.cc:740] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.962208 26912 raft_consensus.cc:493] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.961920 26856 consensus_queue.cc:260] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.962779 26912 raft_consensus.cc:515] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:26.962677 26850 tablet_bootstrap.cc:654] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.965184 26856 ts_tablet_manager.cc:1434] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:27:26.966475 26856 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:26.967784 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:26.969008 26801 raft_consensus.cc:2393] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:26.972424 26853 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:26.973219 26853 ts_tablet_manager.cc:1403] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 12:27:26.973328 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0fc2016a9b4f2390b55edc0d8289f0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:26.972448 26912 leader_election.cc:290] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:26.975196 26545 leader_election.cc:304] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:26.976504 26912 raft_consensus.cc:3060] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:26.977511 26850 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:26.978080 26850 ts_tablet_manager.cc:1403] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.000s
I20260709 12:27:26.981691 26950 raft_consensus.cc:493] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.976125 26856 tablet_bootstrap.cc:654] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:26.982203 26950 raft_consensus.cc:515] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:26.983137 26478 catalog_manager.cc:5697] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:26.984691 26950 leader_election.cc:290] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:26.985543 26912 raft_consensus.cc:2749] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:26.987075 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:26.988243 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:26.989019 26621 leader_election.cc:343] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.990249 26849 tablet_bootstrap.cc:492] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
W20260709 12:27:26.990411 26620 leader_election.cc:343] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:26.991115 26849 ts_tablet_manager.cc:1403] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.192s	user 0.010s	sys 0.020s
I20260709 12:27:26.991259 26620 leader_election.cc:304] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:26.992698 26950 raft_consensus.cc:2749] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:27.007542 26912 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.000159 26849 raft_consensus.cc:359] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:26.995958 26853 raft_consensus.cc:359] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.007799 26479 catalog_manager.cc:5697] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:26.994230 26856 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:27.009861 26849 raft_consensus.cc:385] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.009850 26912 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.010722 26856 ts_tablet_manager.cc:1403] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.006s
I20260709 12:27:27.010726 26849 raft_consensus.cc:740] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.013550 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.013671 26912 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.010037 26853 raft_consensus.cc:385] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.014297 26657 raft_consensus.cc:2468] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:27.012001 26849 consensus_queue.cc:260] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.014083 26856 raft_consensus.cc:359] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.015828 26856 raft_consensus.cc:385] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.016117 26856 raft_consensus.cc:740] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.017000 26856 consensus_queue.cc:260] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.017127 26849 ts_tablet_manager.cc:1434] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.012s
I20260709 12:27:27.019344 26856 ts_tablet_manager.cc:1434] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:27.019742 26545 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.020592 26856 tablet_bootstrap.cc:492] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:27.021015 26912 raft_consensus.cc:2804] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.021512 26912 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.021941 26912 raft_consensus.cc:3060] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.022251 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.022936 26731 raft_consensus.cc:2468] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:27.014425 26853 raft_consensus.cc:740] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:26.997233 26850 raft_consensus.cc:359] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.025152 26850 raft_consensus.cc:385] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.025470 26850 raft_consensus.cc:740] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.026314 26850 consensus_queue.cc:260] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.027987 26850 ts_tablet_manager.cc:1434] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 12:27:27.028296 26856 tablet_bootstrap.cc:654] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.029125 26850 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:27.019366 26849 tablet_bootstrap.cc:492] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.030987 26853 consensus_queue.cc:260] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.034929 26853 ts_tablet_manager.cc:1434] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.007s
I20260709 12:27:27.036334 26853 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:27.038005 26849 tablet_bootstrap.cc:654] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.043988 26850 tablet_bootstrap.cc:654] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.056097 26853 tablet_bootstrap.cc:654] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.059926 26849 tablet_bootstrap.cc:492] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.060552 26849 ts_tablet_manager.cc:1403] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.009s
I20260709 12:27:27.060442 26912 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.063060 26912 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.063760 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:27.064018 26849 raft_consensus.cc:359] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.064433 26657 raft_consensus.cc:3060] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.065279 26849 raft_consensus.cc:385] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.065613 26849 raft_consensus.cc:740] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.066547 26849 consensus_queue.cc:260] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.067701 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:27.068368 26731 raft_consensus.cc:3060] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.074898 26856 tablet_bootstrap.cc:492] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:27.075570 26856 ts_tablet_manager.cc:1403] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.000s
I20260709 12:27:27.078940 26856 raft_consensus.cc:359] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.079651 26856 raft_consensus.cc:385] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.079955 26856 raft_consensus.cc:740] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.080857 26856 consensus_queue.cc:260] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.084300 26856 ts_tablet_manager.cc:1434] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:27:27.085021 26849 ts_tablet_manager.cc:1434] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.003s
I20260709 12:27:27.086179 26849 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.088724 26657 raft_consensus.cc:2468] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:27.090448 26545 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.091497 26912 raft_consensus.cc:2804] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.092007 26912 raft_consensus.cc:697] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:27.093412 26849 tablet_bootstrap.cc:654] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.093453 26912 consensus_queue.cc:237] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.099551 26941 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.099736 26731 raft_consensus.cc:2468] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:27.100216 26941 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.105389 26875 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.105993 26875 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.107421 26853 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:27.108318 26853 ts_tablet_manager.cc:1403] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.004s
I20260709 12:27:27.110687 26941 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.111192 26875 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.110517 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.112706 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.113871 26850 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:27.113803 26731 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
W20260709 12:27:27.113919 26694 leader_election.cc:343] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.112804 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:27.116355 26801 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:27.116822 26856 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:27.114841 26850 ts_tablet_manager.cc:1403] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.032s
I20260709 12:27:27.112663 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.120968 26768 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.120849 26695 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0
W20260709 12:27:27.124970 26768 leader_election.cc:343] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.125363 26941 raft_consensus.cc:2804] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.127362 26941 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.127091 26850 raft_consensus.cc:359] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.127998 26941 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.128089 26850 raft_consensus.cc:385] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.128705 26850 raft_consensus.cc:740] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.125257 26875 raft_consensus.cc:2804] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.129484 26875 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.129884 26875 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.129632 26850 consensus_queue.cc:260] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.131632 26856 tablet_bootstrap.cc:654] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.132437 26850 ts_tablet_manager.cc:1434] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:27:27.134243 26850 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:27.139513 26849 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.140151 26849 ts_tablet_manager.cc:1403] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.008s
I20260709 12:27:27.140076 26875 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.143277 26853 raft_consensus.cc:359] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.144299 26853 raft_consensus.cc:385] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.144639 26853 raft_consensus.cc:740] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.147611 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:27.149363 26694 leader_election.cc:343] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.155885 26856 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:27.156553 26856 ts_tablet_manager.cc:1403] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.019s
I20260709 12:27:27.156932 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:27.157843 26853 consensus_queue.cc:260] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.158597 26875 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.160260 26856 raft_consensus.cc:359] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.161602 26856 raft_consensus.cc:385] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.162012 26856 raft_consensus.cc:740] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.162811 26856 consensus_queue.cc:260] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.165082 26856 ts_tablet_manager.cc:1434] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:27:27.166734 26853 ts_tablet_manager.cc:1434] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.058s	user 0.013s	sys 0.000s
I20260709 12:27:27.169543 26853 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:27.169996 26849 raft_consensus.cc:359] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.170842 26849 raft_consensus.cc:385] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.171141 26849 raft_consensus.cc:740] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.171900 26849 consensus_queue.cc:260] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.173095 26476 catalog_manager.cc:5697] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:27.177294 26849 ts_tablet_manager.cc:1434] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:27:27.177953 26940 raft_consensus.cc:493] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.178681 26849 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.178534 26940 raft_consensus.cc:515] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.178620 26941 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.182730 26801 raft_consensus.cc:2393] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:27.183223 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.183985 26580 raft_consensus.cc:2468] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:27.185456 26769 leader_election.cc:304] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.186492 26900 raft_consensus.cc:2804] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.186899 26900 raft_consensus.cc:493] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.187373 26900 raft_consensus.cc:3060] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.188439 26850 tablet_bootstrap.cc:654] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.188728 26940 leader_election.cc:290] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:27.189968 26695 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:27.191195 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.191563 26970 raft_consensus.cc:2749] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:27.191952 26731 raft_consensus.cc:2468] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:27.193852 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:27.195619 26768 leader_election.cc:343] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.196995 26849 tablet_bootstrap.cc:654] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.200590 26950 consensus_queue.cc:1048] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:27.204734 26875 raft_consensus.cc:493] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.205332 26875 raft_consensus.cc:515] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.207566 26875 leader_election.cc:290] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.209441 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413d45f35e654998908cd4c1c6b8a0f2" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.209830 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413d45f35e654998908cd4c1c6b8a0f2" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:27.210222 26655 raft_consensus.cc:2468] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:27.210589 26802 raft_consensus.cc:2468] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:27.211838 26694 leader_election.cc:304] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.191565 26856 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:27.213528 26970 consensus_queue.cc:1048] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.215243 26970 raft_consensus.cc:2804] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.215615 26970 raft_consensus.cc:493] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.215922 26970 raft_consensus.cc:3060] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.216857 26900 raft_consensus.cc:515] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.218941 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:27.219866 26731 raft_consensus.cc:2393] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in current term 1: Already voted for candidate f57551729f2f4037a4c769ee1df81853 in this term.
I20260709 12:27:27.221488 26768 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:27.222446 26940 raft_consensus.cc:2749] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:27.223594 26941 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.225425 26853 tablet_bootstrap.cc:654] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.230674 26856 tablet_bootstrap.cc:654] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.230772 26941 raft_consensus.cc:493] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.231403 26941 raft_consensus.cc:515] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.232131 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:27.232926 26731 raft_consensus.cc:3060] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.231930 26970 raft_consensus.cc:515] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.236505 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.237388 26580 raft_consensus.cc:2468] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:27.242758 26769 leader_election.cc:304] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.242861 26900 leader_election.cc:290] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:27.243793 26900 raft_consensus.cc:2804] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.244153 26912 raft_consensus.cc:493] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.244284 26900 raft_consensus.cc:493] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.244933 26900 raft_consensus.cc:3060] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.252909 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.253902 26581 raft_consensus.cc:3060] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.260573 26875 consensus_queue.cc:1048] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.244732 26912 raft_consensus.cc:515] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.257447 26951 consensus_queue.cc:1048] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.264204 26912 leader_election.cc:290] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.267151 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.267992 26730 raft_consensus.cc:2468] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:27.270612 26545 leader_election.cc:304] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.271245 26951 consensus_queue.cc:1048] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:27.273613 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.283562 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:27.284636 26804 raft_consensus.cc:2468] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:27.269730 26941 leader_election.cc:290] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.280969 26912 raft_consensus.cc:2804] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.289896 26655 raft_consensus.cc:2468] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:27.309603 26850 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:27.309890 26970 leader_election.cc:290] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.299719 26950 raft_consensus.cc:493] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.310227 26850 ts_tablet_manager.cc:1403] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.176s	user 0.008s	sys 0.008s
I20260709 12:27:27.327968 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413d45f35e654998908cd4c1c6b8a0f2" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:27.329253 26655 raft_consensus.cc:3060] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.310678 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413d45f35e654998908cd4c1c6b8a0f2" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:27.330106 26804 raft_consensus.cc:3060] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.340688 26900 raft_consensus.cc:515] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.302757 26912 raft_consensus.cc:493] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.356611 26850 raft_consensus.cc:359] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.344802 26912 raft_consensus.cc:3060] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.311693 26950 raft_consensus.cc:515] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.322572 26916 consensus_queue.cc:1048] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:27:27.362031 26849 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.375269 26849 ts_tablet_manager.cc:1403] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.197s	user 0.018s	sys 0.015s
I20260709 12:27:27.365021 26731 raft_consensus.cc:2468] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:27.365725 26856 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:27.376756 26856 ts_tablet_manager.cc:1403] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.211s	user 0.022s	sys 0.029s
I20260709 12:27:27.379246 26950 leader_election.cc:290] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.380270 26856 raft_consensus.cc:359] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.381022 26856 raft_consensus.cc:385] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.381088 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.381384 26856 raft_consensus.cc:740] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.366886 26804 raft_consensus.cc:2468] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:27.381879 26580 raft_consensus.cc:2468] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:27.383625 26695 leader_election.cc:304] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.384384 26621 leader_election.cc:304] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.384747 26968 raft_consensus.cc:2804] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.385320 26968 raft_consensus.cc:697] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:27.386380 26951 consensus_queue.cc:1048] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:27:27.372082 26850 raft_consensus.cc:385] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.382877 26856 consensus_queue.cc:260] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.387640 26850 raft_consensus.cc:740] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
W20260709 12:27:27.376482 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:27.390709 26997 raft_consensus.cc:2804] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.391259 26997 raft_consensus.cc:493] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.391618 26997 raft_consensus.cc:3060] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.390838 26856 ts_tablet_manager.cc:1434] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.002s
I20260709 12:27:27.386194 26968 consensus_queue.cc:237] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.399544 26900 leader_election.cc:290] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.400684 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.401361 26580 raft_consensus.cc:3060] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.403139 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:27.403848 26657 raft_consensus.cc:3060] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.405038 26856 tablet_bootstrap.cc:492] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:27.409837 26850 consensus_queue.cc:260] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.405877 26849 raft_consensus.cc:359] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.410750 26849 raft_consensus.cc:385] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.411057 26849 raft_consensus.cc:740] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.412107 26849 consensus_queue.cc:260] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.420094 26850 ts_tablet_manager.cc:1434] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.000s
I20260709 12:27:27.423537 26655 raft_consensus.cc:2468] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:27.421512 26930 raft_consensus.cc:493] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.426898 26849 ts_tablet_manager.cc:1434] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.003s
I20260709 12:27:27.390017 26950 consensus_queue.cc:1048] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:27:27.415014 26581 raft_consensus.cc:2468] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:27.429888 26930 raft_consensus.cc:515] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.433465 26849 tablet_bootstrap.cc:492] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.425760 26768 leader_election.cc:304] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.435986 26930 leader_election.cc:290] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.440322 26578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.441135 26578 raft_consensus.cc:2468] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:27.442483 26695 leader_election.cc:304] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.446496 26875 consensus_queue.cc:1048] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:27.446905 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.448024 26731 raft_consensus.cc:2468] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:27.403167 26997 raft_consensus.cc:515] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.451953 26875 raft_consensus.cc:2804] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.452263 26940 raft_consensus.cc:2804] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.452836 26940 raft_consensus.cc:697] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:27.453991 26997 leader_election.cc:290] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.456753 26875 raft_consensus.cc:493] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.459067 26875 raft_consensus.cc:3060] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.453758 26940 consensus_queue.cc:237] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.483317 26850 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:27.479552 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.471661 26912 raft_consensus.cc:515] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.474612 26580 raft_consensus.cc:2468] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:27.489511 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.496511 26656 raft_consensus.cc:2393] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:27.498637 26853 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:27.499255 26853 ts_tablet_manager.cc:1403] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.330s	user 0.011s	sys 0.011s
I20260709 12:27:27.499832 26769 leader_election.cc:304] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.501262 26900 raft_consensus.cc:2804] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.502204 26900 raft_consensus.cc:697] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:27.491873 26581 raft_consensus.cc:3060] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.503479 26900 consensus_queue.cc:237] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.512105 26853 raft_consensus.cc:359] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.513023 26853 raft_consensus.cc:385] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.513355 26853 raft_consensus.cc:740] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.514729 27001 consensus_queue.cc:1048] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.518193 26853 consensus_queue.cc:260] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.524211 26849 tablet_bootstrap.cc:654] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.526847 26656 raft_consensus.cc:3060] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.527581 26875 raft_consensus.cc:515] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.532385 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.533763 26581 raft_consensus.cc:2468] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:27.535317 26621 leader_election.cc:304] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.535987 26853 ts_tablet_manager.cc:1434] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:27:27.536697 26951 raft_consensus.cc:2804] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.537367 26853 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:27.537513 26951 raft_consensus.cc:697] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:27.544155 26875 leader_election.cc:290] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.552070 26849 tablet_bootstrap.cc:492] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.552742 26849 ts_tablet_manager.cc:1403] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.121s	user 0.015s	sys 0.004s
I20260709 12:27:27.553263 26951 consensus_queue.cc:237] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.564271 26849 raft_consensus.cc:359] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.564880 26853 tablet_bootstrap.cc:654] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.565155 26849 raft_consensus.cc:385] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.558470 26479 catalog_manager.cc:5697] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:27.565685 26849 raft_consensus.cc:740] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.565068 26657 raft_consensus.cc:2468] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:27.579519 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:27.580935 26730 raft_consensus.cc:2393] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate f57551729f2f4037a4c769ee1df81853 in this term.
I20260709 12:27:27.578557 26849 consensus_queue.cc:260] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.585520 26849 ts_tablet_manager.cc:1434] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 12:27:27.593639 26850 tablet_bootstrap.cc:654] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.600339 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:27.600729 26998 raft_consensus.cc:493] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.601403 26730 raft_consensus.cc:3060] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.601425 26998 raft_consensus.cc:515] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.604019 26997 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.606612 26997 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.607360 26998 leader_election.cc:290] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.608721 26997 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:27.609742 26849 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.611143 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:27.611863 26804 raft_consensus.cc:2468] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:27.612272 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.618472 26478 catalog_manager.cc:5697] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:27.619846 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.611892 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.621665 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdc9b665b7de4e61ae5821f49d7459ec" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:27.623430 26875 consensus_queue.cc:1048] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.624557 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:27.625344 26804 raft_consensus.cc:3060] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.624452 26694 leader_election.cc:304] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:27.627914 26856 tablet_bootstrap.cc:654] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.628191 26968 raft_consensus.cc:2749] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:27.628729 26912 leader_election.cc:290] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.629400 27001 consensus_queue.cc:1048] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.635190 26970 raft_consensus.cc:493] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.636140 26970 raft_consensus.cc:515] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.636466 26621 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:27.638329 26950 raft_consensus.cc:2804] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.638787 26950 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.638960 26970 leader_election.cc:290] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.639175 26950 raft_consensus.cc:3060] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.642993 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.643842 26655 raft_consensus.cc:2468] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:27.645354 26900 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.645920 26900 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.643806 26621 leader_election.cc:304] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:27.623528 26478 catalog_manager.cc:5697] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:27.668721 26997 raft_consensus.cc:2749] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:27:27.669662 26621 leader_election.cc:343] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.659809 26694 leader_election.cc:304] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.683683 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.684665 26900 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:27.685593 26768 leader_election.cc:343] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.686085 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:27.691128 26769 leader_election.cc:343] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.691689 26769 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:27.700690 26853 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:27.701126 26804 raft_consensus.cc:2468] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
W20260709 12:27:27.702929 26694 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 48465.6us
I20260709 12:27:27.702812 26477 catalog_manager.cc:5697] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:27.702746 26875 raft_consensus.cc:2804] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.705217 26875 raft_consensus.cc:493] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.705579 26875 raft_consensus.cc:3060] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.705806 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.706837 26579 raft_consensus.cc:2468] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:27.713726 26940 raft_consensus.cc:2749] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:27.701370 26853 ts_tablet_manager.cc:1403] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.010s
I20260709 12:27:27.722126 26546 leader_election.cc:304] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:27.723388 26867 raft_consensus.cc:2804] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.723641 26940 raft_consensus.cc:493] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.723975 26867 raft_consensus.cc:697] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:27.724149 26940 raft_consensus.cc:515] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.724921 26867 consensus_queue.cc:237] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.727555 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.728654 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:27.723306 26853 raft_consensus.cc:359] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.729213 26940 leader_election.cc:290] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:27.730995 26950 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.731817 26853 raft_consensus.cc:385] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:27.732997 26768 leader_election.cc:343] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.733307 26853 raft_consensus.cc:740] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.734979 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:27.735826 26804 raft_consensus.cc:3060] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.735118 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.735035 26853 consensus_queue.cc:260] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:27.737293 26621 leader_election.cc:343] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:27.734877 26768 leader_election.cc:343] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.740204 26853 ts_tablet_manager.cc:1434] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:27:27.726078 26730 raft_consensus.cc:2468] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:27.742143 26853 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:27.742805 26849 tablet_bootstrap.cc:654] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.740501 26768 leader_election.cc:304] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:27.745838 26940 raft_consensus.cc:2749] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:27.740276 26850 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:27.747059 26850 ts_tablet_manager.cc:1403] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.266s	user 0.024s	sys 0.011s
I20260709 12:27:27.733995 26950 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:27.747918 26804 raft_consensus.cc:2468] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:27.754007 26849 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.755045 26849 ts_tablet_manager.cc:1403] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.146s	user 0.015s	sys 0.002s
I20260709 12:27:27.754799 26850 raft_consensus.cc:359] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.755707 26850 raft_consensus.cc:385] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.757714 26850 raft_consensus.cc:740] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.759835 26849 raft_consensus.cc:359] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.761404 26849 raft_consensus.cc:385] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.761893 26849 raft_consensus.cc:740] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.763029 26849 consensus_queue.cc:260] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.766176 26849 ts_tablet_manager.cc:1434] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:27:27.767835 26849 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.767913 26853 tablet_bootstrap.cc:654] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.759835 26850 consensus_queue.cc:260] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.773586 26850 ts_tablet_manager.cc:1434] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:27:27.775285 26850 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:27.781347 26849 tablet_bootstrap.cc:654] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.782081 26850 tablet_bootstrap.cc:654] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.790460 26621 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:27.792970 26950 raft_consensus.cc:2804] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.793571 26950 raft_consensus.cc:697] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:27.794528 26950 consensus_queue.cc:237] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.800879 26849 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.801892 26849 ts_tablet_manager.cc:1403] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.034s	user 0.024s	sys 0.004s
I20260709 12:27:27.806402 26849 raft_consensus.cc:359] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.807647 26849 raft_consensus.cc:385] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.808115 26849 raft_consensus.cc:740] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.809363 26849 consensus_queue.cc:260] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.813019 26849 ts_tablet_manager.cc:1434] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:27:27.821379 26849 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.829197 26853 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:27.830060 26968 consensus_queue.cc:1048] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.840287 26853 ts_tablet_manager.cc:1403] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.000s
I20260709 12:27:27.849252 26853 raft_consensus.cc:359] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.818042 26875 raft_consensus.cc:515] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.854233 26849 tablet_bootstrap.cc:654] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.864046 26856 tablet_bootstrap.cc:492] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:27.864717 26856 ts_tablet_manager.cc:1403] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.460s	user 0.048s	sys 0.055s
I20260709 12:27:27.868230 26856 raft_consensus.cc:359] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.869059 26856 raft_consensus.cc:385] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.869387 26856 raft_consensus.cc:740] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.869521 26900 raft_consensus.cc:493] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.870091 26900 raft_consensus.cc:515] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.870332 26856 consensus_queue.cc:260] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:27.872390 26900 leader_election.cc:290] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.873770 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:27.875821 26657 raft_consensus.cc:2468] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:27.874927 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:27.877151 26768 leader_election.cc:304] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
W20260709 12:27:27.878230 26769 leader_election.cc:343] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.879719 26940 raft_consensus.cc:2804] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.880136 26940 raft_consensus.cc:493] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.880483 26940 raft_consensus.cc:3060] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.880414 26930 consensus_queue.cc:1048] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.881363 27001 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.887360 27001 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.850076 26853 raft_consensus.cc:385] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.895212 26850 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:27.895709 27001 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.897014 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:27.898674 26545 leader_election.cc:343] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.898487 26941 consensus_queue.cc:1048] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:27.900028 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:27.900622 26875 leader_election.cc:290] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.900775 26804 raft_consensus.cc:2468] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:27.890697 26853 raft_consensus.cc:740] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.901774 26853 consensus_queue.cc:260] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.895992 26850 ts_tablet_manager.cc:1403] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.121s	user 0.017s	sys 0.005s
I20260709 12:27:27.903290 26856 ts_tablet_manager.cc:1434] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:27:27.905315 26940 raft_consensus.cc:515] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.905803 26546 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:27.906741 26912 raft_consensus.cc:2804] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:27.907172 26912 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.907518 26912 raft_consensus.cc:3060] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.907719 26940 leader_election.cc:290] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:27.905992 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:27.913370 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.914654 26853 ts_tablet_manager.cc:1434] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.069s	user 0.004s	sys 0.004s
I20260709 12:27:27.916050 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:27.916862 26849 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:27.917248 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:27.920066 26849 ts_tablet_manager.cc:1403] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.105s	user 0.017s	sys 0.000s
I20260709 12:27:27.919185 26579 raft_consensus.cc:3060] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.919152 26656 raft_consensus.cc:3060] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.925763 26655 raft_consensus.cc:3060] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.919078 26912 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:27.932889 26853 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:27.933698 26769 leader_election.cc:343] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.944501 26912 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:27.944350 26478 catalog_manager.cc:5697] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:27.950743 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:27.951830 26849 raft_consensus.cc:359] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:27.952560 26849 raft_consensus.cc:385] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.952833 26849 raft_consensus.cc:740] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.953552 26849 consensus_queue.cc:260] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:27.955427 26545 leader_election.cc:343] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:27.956029 26849 ts_tablet_manager.cc:1434] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:27:27.957191 26849 tablet_bootstrap.cc:492] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:27.961553 26900 consensus_queue.cc:1048] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.963692 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:27.964406 26804 raft_consensus.cc:3060] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:27.964824 26850 raft_consensus.cc:359] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.965553 26850 raft_consensus.cc:385] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:27.965816 26850 raft_consensus.cc:740] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:27.966679 26856 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:27.980007 26579 raft_consensus.cc:2468] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:27.980926 26950 consensus_queue.cc:1048] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:27.983184 26695 leader_election.cc:304] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:27.984372 26930 raft_consensus.cc:2804] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:27.984928 26930 raft_consensus.cc:697] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:27.985178 26941 consensus_queue.cc:1048] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:27.985805 26930 consensus_queue.cc:237] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.989692 26997 consensus_queue.cc:1048] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:27.995023 26480 catalog_manager.cc:5697] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:28.004724 26804 raft_consensus.cc:2468] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:28.015079 26656 raft_consensus.cc:2468] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:28.016300 26546 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.019816 26912 raft_consensus.cc:2804] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:28.020412 26912 raft_consensus.cc:697] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:28.021360 26912 consensus_queue.cc:237] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.021809 26940 consensus_queue.cc:1048] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:28.022608 26856 tablet_bootstrap.cc:654] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:27.989945 26850 consensus_queue.cc:260] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:27.942139 26655 raft_consensus.cc:2468] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:28.026482 26849 tablet_bootstrap.cc:654] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.026602 26768 leader_election.cc:304] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:28.027918 26941 raft_consensus.cc:2804] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:28.028402 26850 ts_tablet_manager.cc:1434] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.000s
I20260709 12:27:28.028402 26941 raft_consensus.cc:697] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:28.029940 26941 consensus_queue.cc:237] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.032210 26853 tablet_bootstrap.cc:654] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.036036 26850 tablet_bootstrap.cc:492] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:28.049980 26853 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:28.050647 26853 ts_tablet_manager.cc:1403] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.005s
I20260709 12:27:28.065918 26856 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:28.066674 26856 ts_tablet_manager.cc:1403] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.100s	user 0.012s	sys 0.009s
I20260709 12:27:28.080420 26477 catalog_manager.cc:5697] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:28.083738 26480 catalog_manager.cc:5697] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:28.053733 26853 raft_consensus.cc:359] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.090956 26849 tablet_bootstrap.cc:492] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:28.091647 26849 ts_tablet_manager.cc:1403] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.135s	user 0.014s	sys 0.004s
I20260709 12:27:28.092200 26479 catalog_manager.cc:5697] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:28.089116 26853 raft_consensus.cc:385] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.094275 26853 raft_consensus.cc:740] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.095084 26853 consensus_queue.cc:260] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.097520 26853 ts_tablet_manager.cc:1434] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.002s
I20260709 12:27:28.099071 26853 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:28.102094 26850 tablet_bootstrap.cc:654] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.106813 26853 tablet_bootstrap.cc:654] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.111769 26849 raft_consensus.cc:359] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.112841 26849 raft_consensus.cc:385] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.113217 26849 raft_consensus.cc:740] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.115414 26856 raft_consensus.cc:359] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.116467 26856 raft_consensus.cc:385] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.116822 26856 raft_consensus.cc:740] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.118166 26849 consensus_queue.cc:260] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.119910 26856 consensus_queue.cc:260] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.126691 26853 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:28.127401 26853 ts_tablet_manager.cc:1403] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.003s
I20260709 12:27:28.127421 26850 tablet_bootstrap.cc:492] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:28.128093 26850 ts_tablet_manager.cc:1403] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.092s	user 0.019s	sys 0.009s
I20260709 12:27:28.130774 26849 ts_tablet_manager.cc:1434] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.003s
I20260709 12:27:28.130820 26853 raft_consensus.cc:359] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.131317 26850 raft_consensus.cc:359] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.131987 26853 raft_consensus.cc:385] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.132133 26849 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:28.132057 26850 raft_consensus.cc:385] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.132526 26853 raft_consensus.cc:740] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.132695 26850 raft_consensus.cc:740] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.133602 26850 consensus_queue.cc:260] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.133687 26853 consensus_queue.cc:260] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.135809 26941 raft_consensus.cc:493] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.136330 26850 ts_tablet_manager.cc:1434] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:27:28.136780 26853 ts_tablet_manager.cc:1434] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:27:28.136344 26941 raft_consensus.cc:515] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.137717 26850 tablet_bootstrap.cc:492] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:28.137879 26853 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:28.139483 26856 ts_tablet_manager.cc:1434] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.003s
I20260709 12:27:28.140751 26867 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.141309 26867 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.143021 26849 tablet_bootstrap.cc:654] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.143608 26867 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.140908 26856 tablet_bootstrap.cc:492] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:28.144966 26850 tablet_bootstrap.cc:654] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.147128 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.151118 26916 raft_consensus.cc:493] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.151664 26916 raft_consensus.cc:515] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.153833 26916 leader_election.cc:290] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.155529 26850 tablet_bootstrap.cc:492] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:28.155889 26856 tablet_bootstrap.cc:654] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.156037 26850 ts_tablet_manager.cc:1403] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 12:27:28.156216 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.156929 26802 raft_consensus.cc:2468] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:28.157502 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.157905 26849 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:28.158370 26804 raft_consensus.cc:2393] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:28.158530 26849 ts_tablet_manager.cc:1403] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.008s
I20260709 12:27:28.159332 26850 raft_consensus.cc:359] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.160064 26850 raft_consensus.cc:385] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.160351 26850 raft_consensus.cc:740] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.161078 26850 consensus_queue.cc:260] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.161958 26545 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:28.162879 26912 raft_consensus.cc:3060] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.163589 26850 ts_tablet_manager.cc:1434] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:28.164278 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.164812 26850 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:28.166564 26621 leader_election.cc:304] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:28.167706 26950 raft_consensus.cc:2804] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:28.168104 26950 raft_consensus.cc:493] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.168430 26950 raft_consensus.cc:3060] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:28.168671 26620 leader_election.cc:343] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.170204 26941 leader_election.cc:290] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.171335 26912 raft_consensus.cc:2749] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.171991 26850 tablet_bootstrap.cc:654] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.162173 26849 raft_consensus.cc:359] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.172737 26849 raft_consensus.cc:385] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.172648 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5150550259ec4e3fac2af6e10b6ca9b5" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:28.174780 26768 leader_election.cc:343] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.175509 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5150550259ec4e3fac2af6e10b6ca9b5" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.175617 26849 raft_consensus.cc:740] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.176270 26580 raft_consensus.cc:2468] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:28.177161 26849 consensus_queue.cc:260] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.178102 26769 leader_election.cc:304] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:28.179594 26940 raft_consensus.cc:2804] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:28.179827 26849 ts_tablet_manager.cc:1434] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:27:28.179967 26940 raft_consensus.cc:493] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.180648 26940 raft_consensus.cc:3060] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.189785 26912 consensus_queue.cc:1048] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.192171 26849 tablet_bootstrap.cc:492] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:28.208703 26950 raft_consensus.cc:515] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.209156 26951 consensus_queue.cc:1048] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:28.203265 26940 raft_consensus.cc:515] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.217499 26853 tablet_bootstrap.cc:654] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.218986 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5150550259ec4e3fac2af6e10b6ca9b5" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:28.222563 26579 raft_consensus.cc:3060] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.224997 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5150550259ec4e3fac2af6e10b6ca9b5" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:28.227946 26768 leader_election.cc:343] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.236131 26916 raft_consensus.cc:493] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.228907 26940 leader_election.cc:290] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.237970 26849 tablet_bootstrap.cc:654] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.238083 26875 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.240022 26875 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.251210 26875 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.239104 26916 raft_consensus.cc:515] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.258668 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.259151 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.259653 26801 raft_consensus.cc:2393] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:28.260243 26657 raft_consensus.cc:2393] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:28.263965 26941 raft_consensus.cc:493] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.264552 26941 raft_consensus.cc:515] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.264936 26916 leader_election.cc:290] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:28.268268 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.268460 26694 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
W20260709 12:27:28.270689 26620 leader_election.cc:343] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.271713 26930 raft_consensus.cc:3060] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.275408 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:28.277168 26768 leader_election.cc:343] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.277575 26579 raft_consensus.cc:2468] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:28.271592 26941 leader_election.cc:290] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:28.278445 27041 raft_consensus.cc:493] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.279007 27041 raft_consensus.cc:515] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.298373 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.282234 27041 leader_election.cc:290] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:28.302889 26853 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
W20260709 12:27:28.302862 26768 leader_election.cc:343] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.303623 26853 ts_tablet_manager.cc:1403] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.166s	user 0.019s	sys 0.003s
I20260709 12:27:28.288699 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
W20260709 12:27:28.305274 26620 leader_election.cc:343] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.305351 26769 leader_election.cc:304] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:28.306703 27040 raft_consensus.cc:2804] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:28.306895 26853 raft_consensus.cc:359] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.307766 26853 raft_consensus.cc:385] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.307781 27040 raft_consensus.cc:697] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:28.308144 26853 raft_consensus.cc:740] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.308612 26912 consensus_queue.cc:1048] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.308815 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.310024 26853 consensus_queue.cc:260] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.312638 26856 tablet_bootstrap.cc:492] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:28.313174 26856 ts_tablet_manager.cc:1403] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.173s	user 0.015s	sys 0.018s
I20260709 12:27:28.316606 26856 raft_consensus.cc:359] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.317318 26856 raft_consensus.cc:385] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.317602 26856 raft_consensus.cc:740] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.318288 26856 consensus_queue.cc:260] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.321506 26856 ts_tablet_manager.cc:1434] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:27:28.321931 26849 tablet_bootstrap.cc:492] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:28.286958 26950 leader_election.cc:290] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.323571 26849 ts_tablet_manager.cc:1403] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.132s	user 0.016s	sys 0.001s
I20260709 12:27:28.328691 26856 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:28.329523 26853 ts_tablet_manager.cc:1434] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:27:28.330562 26853 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:28.330775 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.332545 26849 raft_consensus.cc:359] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.333346 26849 raft_consensus.cc:385] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.333657 26849 raft_consensus.cc:740] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.334652 26849 consensus_queue.cc:260] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.337565 26853 tablet_bootstrap.cc:654] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.337885 26849 ts_tablet_manager.cc:1434] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 12:27:28.339087 26849 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
W20260709 12:27:28.341121 26769 leader_election.cc:343] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.341711 26769 leader_election.cc:304] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.342758 27041 raft_consensus.cc:2749] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.345245 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:28.345383 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:28.345670 26621 leader_election.cc:343] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.345911 26804 raft_consensus.cc:3060] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.346175 26621 leader_election.cc:304] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.353400 26951 raft_consensus.cc:2749] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.346560 27040 consensus_queue.cc:237] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:28.355684 26768 leader_election.cc:343] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.356640 26768 leader_election.cc:304] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.348699 26850 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:28.358806 26850 ts_tablet_manager.cc:1403] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.194s	user 0.055s	sys 0.050s
I20260709 12:27:28.373489 26930 raft_consensus.cc:2749] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.359826 26940 raft_consensus.cc:2749] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.383767 26850 raft_consensus.cc:359] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.384558 26850 raft_consensus.cc:385] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.384804 26850 raft_consensus.cc:740] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.385690 26850 consensus_queue.cc:260] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.391744 26856 tablet_bootstrap.cc:654] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.395195 26850 ts_tablet_manager.cc:1434] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.002s
I20260709 12:27:28.411454 26850 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:28.422029 26951 consensus_queue.cc:1048] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.425347 26853 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:28.425974 26853 ts_tablet_manager.cc:1403] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.039s
I20260709 12:27:28.429080 26853 raft_consensus.cc:359] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.429806 26853 raft_consensus.cc:385] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.430085 26853 raft_consensus.cc:740] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.461109 26849 tablet_bootstrap.cc:654] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.445072 26853 consensus_queue.cc:260] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.457067 26970 raft_consensus.cc:493] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.454980 26912 consensus_queue.cc:1048] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:27:28.464843 26970 raft_consensus.cc:515] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.470615 26875 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.471333 26875 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.472510 26853 ts_tablet_manager.cc:1434] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:27:28.473587 26875 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:28.454245 26479 catalog_manager.cc:5697] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:28.473754 26853 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:28.486178 26970 leader_election.cc:290] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.451215 26930 consensus_queue.cc:1048] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 12:27:28.489292 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.490235 26657 raft_consensus.cc:2393] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:28.490579 26853 tablet_bootstrap.cc:654] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.478708 26730 raft_consensus.cc:3060] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.479604 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:28.441780 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:28.500612 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.430074 26804 raft_consensus.cc:2468] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:28.508162 26950 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.508800 26950 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.513103 26930 consensus_queue.cc:1048] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.499871 26912 raft_consensus.cc:493] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.517850 26695 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:28.517217 26912 raft_consensus.cc:515] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.536978 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1d43a3098bd4f5d80095bc30638c181" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.537927 26805 raft_consensus.cc:2393] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:28.537751 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.538734 26951 raft_consensus.cc:493] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.539244 26951 raft_consensus.cc:515] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.541496 26580 raft_consensus.cc:3060] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.542586 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.543295 26581 raft_consensus.cc:2468] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:28.544728 26621 leader_election.cc:304] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:28.545897 26916 raft_consensus.cc:2804] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:28.546469 26916 raft_consensus.cc:493] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.546912 26916 raft_consensus.cc:3060] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.520853 26940 consensus_queue.cc:1048] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:27:28.550788 27041 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.556985 27041 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.557612 26621 leader_election.cc:304] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.558844 26997 raft_consensus.cc:2804] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:28.564900 26912 leader_election.cc:290] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.565963 26950 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.559373 27041 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.566771 26951 leader_election.cc:290] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:28.566344 26695 leader_election.cc:304] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:28.582101 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.584714 26970 raft_consensus.cc:3060] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.587452 26731 raft_consensus.cc:2393] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate f57551729f2f4037a4c769ee1df81853 in this term.
I20260709 12:27:28.589692 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.589931 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:28.591737 26769 leader_election.cc:343] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:28.592271 26620 leader_election.cc:343] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.590449 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.597124 26655 raft_consensus.cc:2393] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:28.599835 26545 leader_election.cc:304] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:28.608408 26850 tablet_bootstrap.cc:654] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.610235 27057 raft_consensus.cc:2749] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.612601 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.613585 26801 raft_consensus.cc:2393] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:28.615898 26930 raft_consensus.cc:2749] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.617491 26853 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:28.618083 26853 ts_tablet_manager.cc:1403] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.145s	user 0.028s	sys 0.031s
I20260709 12:27:28.621387 26853 raft_consensus.cc:359] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.622043 26853 raft_consensus.cc:385] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.622506 26853 raft_consensus.cc:740] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.623404 26853 consensus_queue.cc:260] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.617671 26940 consensus_queue.cc:1048] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.623755 26916 raft_consensus.cc:515] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.631615 26970 raft_consensus.cc:2749] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.640563 26856 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:28.643255 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
W20260709 12:27:28.645201 26620 leader_election.cc:343] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.645181 26856 ts_tablet_manager.cc:1403] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.317s	user 0.012s	sys 0.004s
I20260709 12:27:28.634164 26621 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.636880 27055 consensus_queue.cc:1048] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.652633 26853 ts_tablet_manager.cc:1434] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:27:28.658844 26997 raft_consensus.cc:697] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:28.659947 26997 consensus_queue.cc:237] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.661772 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.666839 26950 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.671093 26853 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:28.678217 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:28.678691 26916 leader_election.cc:290] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:28.679031 26581 raft_consensus.cc:3060] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:28.694737 26768 leader_election.cc:343] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.695619 26768 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:28.696784 26940 raft_consensus.cc:2749] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.699342 26856 raft_consensus.cc:359] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.699478 26853 tablet_bootstrap.cc:654] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.700361 26856 raft_consensus.cc:385] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.703380 26856 raft_consensus.cc:740] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.704625 26856 consensus_queue.cc:260] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.708911 26856 ts_tablet_manager.cc:1434] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 12:27:28.710551 26856 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:28.713289 26940 raft_consensus.cc:493] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.713905 26940 raft_consensus.cc:515] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.716495 26940 leader_election.cc:290] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:28.719286 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb970d065d54c94be1a04a340a5821c" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:28.721582 26768 leader_election.cc:343] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:28.723584 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 47155.8us
I20260709 12:27:28.725972 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb970d065d54c94be1a04a340a5821c" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.726583 26916 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.726666 26950 raft_consensus.cc:2749] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.727265 26916 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.728161 26581 raft_consensus.cc:2468] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:28.731892 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.732635 26731 raft_consensus.cc:2468] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:28.734087 26620 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:28.735241 27061 raft_consensus.cc:2804] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:28.735622 27061 raft_consensus.cc:493] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.735953 27061 raft_consensus.cc:3060] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:28.738854 26621 leader_election.cc:304] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.741456 26849 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:28.741791 26951 raft_consensus.cc:2804] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:28.742022 26849 ts_tablet_manager.cc:1403] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.403s	user 0.032s	sys 0.022s
I20260709 12:27:28.742405 26951 raft_consensus.cc:697] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
W20260709 12:27:28.743274 26768 leader_election.cc:343] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.743381 26951 consensus_queue.cc:237] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.743886 26768 leader_election.cc:304] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:28.742411 26916 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.744735 26941 raft_consensus.cc:2749] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:28.744175 26479 catalog_manager.cc:5697] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:28.745213 26849 raft_consensus.cc:359] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.745949 26849 raft_consensus.cc:385] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.746235 26849 raft_consensus.cc:740] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.747867 26849 consensus_queue.cc:260] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.750010 26849 ts_tablet_manager.cc:1434] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.003s
I20260709 12:27:28.751124 26849 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:28.753218 27061 raft_consensus.cc:515] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.756968 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.757753 26802 raft_consensus.cc:2468] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
W20260709 12:27:28.760877 26768 consensus_peers.cc:597] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:28.762092 26850 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:28.762710 26850 ts_tablet_manager.cc:1403] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.355s	user 0.030s	sys 0.010s
I20260709 12:27:28.765486 26849 tablet_bootstrap.cc:654] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.766845 27057 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.767521 27057 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.771469 26853 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:28.772168 26853 ts_tablet_manager.cc:1403] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.101s	user 0.014s	sys 0.001s
I20260709 12:27:28.726524 26856 tablet_bootstrap.cc:654] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.782161 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.788745 26802 raft_consensus.cc:2468] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:28.789752 26856 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:28.782673 27057 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.790313 26856 ts_tablet_manager.cc:1403] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.080s	user 0.014s	sys 0.000s
I20260709 12:27:28.783237 27061 leader_election.cc:290] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.782545 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:28.791911 26731 raft_consensus.cc:3060] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:28.794145 26856 raft_consensus.cc:359] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.794992 26856 raft_consensus.cc:385] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.795272 26856 raft_consensus.cc:740] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.795203 26546 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:28.796321 27055 raft_consensus.cc:2804] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:28.796252 26856 consensus_queue.cc:260] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.796763 27055 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.797160 27055 raft_consensus.cc:3060] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.766184 26850 raft_consensus.cc:359] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.797849 26850 raft_consensus.cc:385] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.798146 26850 raft_consensus.cc:740] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.798933 26850 consensus_queue.cc:260] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.800884 26850 ts_tablet_manager.cc:1434] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:27:28.802062 26850 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:28.808947 26856 ts_tablet_manager.cc:1434] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:27:28.810492 26856 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:28.817873 26850 tablet_bootstrap.cc:654] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.786021 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.821071 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bba92ba095b54d48914151dcac5033d1" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
W20260709 12:27:28.821489 26545 leader_election.cc:343] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.785027 26853 raft_consensus.cc:359] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.822592 26853 raft_consensus.cc:385] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.822902 26853 raft_consensus.cc:740] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.823630 26853 consensus_queue.cc:260] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.824432 26856 tablet_bootstrap.cc:654] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.839120 26941 raft_consensus.cc:493] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.840360 26941 raft_consensus.cc:515] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.821908 26802 raft_consensus.cc:3060] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:28.852639 27055 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.857967 26731 raft_consensus.cc:2468] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 2.
I20260709 12:27:28.865952 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:28.867830 27055 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:28.868274 26620 leader_election.cc:304] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
W20260709 12:27:28.869784 26545 leader_election.cc:343] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.870030 27041 consensus_queue.cc:1048] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:28.872843 27071 raft_consensus.cc:2804] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:28.874698 27071 raft_consensus.cc:697] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:28.875419 26850 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:28.876006 26850 ts_tablet_manager.cc:1403] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.017s
I20260709 12:27:28.879333 26850 raft_consensus.cc:359] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.880810 26850 raft_consensus.cc:385] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:28.881080 26850 raft_consensus.cc:740] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:28.881821 26850 consensus_queue.cc:260] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:28.884547 26850 ts_tablet_manager.cc:1434] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:28.885649 26850 tablet_bootstrap.cc:492] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:28.875617 27071 consensus_queue.cc:237] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.894488 26853 ts_tablet_manager.cc:1434] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.122s	user 0.009s	sys 0.000s
I20260709 12:27:28.913862 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:28.914893 26801 raft_consensus.cc:3060] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:28.918721 26970 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.919302 26970 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:28.921451 26970 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.928254 26950 raft_consensus.cc:493] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.933017 26950 raft_consensus.cc:515] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:28.935407 26950 leader_election.cc:290] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:28.938428 26941 leader_election.cc:290] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:28.939147 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.939911 26805 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:28.940032 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.939158 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:28.945945 26853 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:28.949968 26850 tablet_bootstrap.cc:654] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:28.951424 26867 raft_consensus.cc:493] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.957624 26867 raft_consensus.cc:515] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.960197 26867 leader_election.cc:290] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:28.960951 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:28.961808 26730 raft_consensus.cc:2468] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:28.963287 26545 leader_election.cc:304] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:28.964586 26695 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:28.965658 26970 raft_consensus.cc:2804] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:28.965315 26867 raft_consensus.cc:2804] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:28.966135 26970 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.966281 26867 raft_consensus.cc:493] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.941753 26656 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:28.969201 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.970171 26580 raft_consensus.cc:2468] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:28.967495 26970 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:28.967419 26867 raft_consensus.cc:3060] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:28.966918 26768 leader_election.cc:343] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:28.982357 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5150550259ec4e3fac2af6e10b6ca9b5" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:28.987321 26480 catalog_manager.cc:5697] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:28.988346 26802 raft_consensus.cc:2468] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 2.
I20260709 12:27:28.988806 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5150550259ec4e3fac2af6e10b6ca9b5" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.990850 26480 catalog_manager.cc:5697] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 2, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:28.993476 26867 raft_consensus.cc:515] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:28.995692 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:28.998571 26805 raft_consensus.cc:2468] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:28.995812 26769 leader_election.cc:304] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:29.000489 26867 leader_election.cc:290] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.000880 27039 raft_consensus.cc:2804] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.001385 27039 raft_consensus.cc:493] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.001765 27039 raft_consensus.cc:3060] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.002162 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:29.002871 26730 raft_consensus.cc:3060] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.004868 26801 raft_consensus.cc:2468] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:29.006412 26849 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:29.006970 26849 ts_tablet_manager.cc:1403] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.256s	user 0.071s	sys 0.116s
I20260709 12:27:29.002830 26621 leader_election.cc:304] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:29.009269 27071 raft_consensus.cc:2749] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:29.022203 26856 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:29.022933 26856 ts_tablet_manager.cc:1403] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.213s	user 0.026s	sys 0.035s
I20260709 12:27:29.016419 26546 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:29.023890 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:29.024626 26801 raft_consensus.cc:3060] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.026305 27055 raft_consensus.cc:2804] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.026577 26856 raft_consensus.cc:359] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.027262 27055 raft_consensus.cc:697] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:29.027349 26856 raft_consensus.cc:385] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.027849 26856 raft_consensus.cc:740] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.028287 27055 consensus_queue.cc:237] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.029479 26856 consensus_queue.cc:260] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.038183 26853 tablet_bootstrap.cc:654] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.043272 26856 ts_tablet_manager.cc:1434] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:27:29.044551 26856 tablet_bootstrap.cc:492] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:29.045360 26970 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.049043 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:29.049810 26656 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:29.058620 26730 raft_consensus.cc:2468] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:29.061996 26856 tablet_bootstrap.cc:654] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.061285 27039 raft_consensus.cc:515] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.069221 26545 leader_election.cc:304] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.070219 26849 raft_consensus.cc:359] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.074714 26849 raft_consensus.cc:385] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.077056 26849 raft_consensus.cc:740] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.070550 26867 raft_consensus.cc:2804] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.078397 26850 tablet_bootstrap.cc:492] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:29.079695 26850 ts_tablet_manager.cc:1403] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.194s	user 0.020s	sys 0.005s
I20260709 12:27:29.080859 26867 raft_consensus.cc:697] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:29.081036 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:29.081908 26867 consensus_queue.cc:237] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:29.082943 26768 leader_election.cc:343] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.083319 26850 raft_consensus.cc:359] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.084076 26850 raft_consensus.cc:385] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.084359 26850 raft_consensus.cc:740] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.071187 27039 leader_election.cc:290] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:29.085187 26850 consensus_queue.cc:260] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.082998 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:29.090583 26849 consensus_queue.cc:260] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.096634 26850 ts_tablet_manager.cc:1434] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:27:29.096966 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:29.097616 26805 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:29.098078 26850 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:29.087513 26580 raft_consensus.cc:3060] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.108944 26849 ts_tablet_manager.cc:1434] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.102s	user 0.010s	sys 0.000s
I20260709 12:27:29.112335 26849 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:29.110822 26801 raft_consensus.cc:2468] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:29.120151 26479 catalog_manager.cc:5697] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:29.089972 26970 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.122633 26853 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:29.124728 26656 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
I20260709 12:27:29.125569 26853 ts_tablet_manager.cc:1403] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.202s	user 0.020s	sys 0.004s
I20260709 12:27:29.129776 26694 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.130311 26853 raft_consensus.cc:359] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.131114 26853 raft_consensus.cc:385] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.131388 26853 raft_consensus.cc:740] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.132184 26849 tablet_bootstrap.cc:654] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.132745 26970 raft_consensus.cc:2804] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:29.134059 26970 raft_consensus.cc:697] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:29.135119 26970 consensus_queue.cc:237] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.132115 26853 consensus_queue.cc:260] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.139853 26580 raft_consensus.cc:2468] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:29.153003 26769 leader_election.cc:304] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:29.153779 26805 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
I20260709 12:27:29.154459 27039 raft_consensus.cc:2804] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.157922 27039 raft_consensus.cc:697] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:29.159053 26853 ts_tablet_manager.cc:1434] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 12:27:29.159335 27039 consensus_queue.cc:237] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.161989 26850 tablet_bootstrap.cc:654] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.169245 26849 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:29.171797 26849 ts_tablet_manager.cc:1403] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.019s
I20260709 12:27:29.169507 26856 tablet_bootstrap.cc:492] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:29.173620 26856 ts_tablet_manager.cc:1403] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.129s	user 0.012s	sys 0.005s
I20260709 12:27:29.179488 26730 raft_consensus.cc:3060] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.182720 26856 raft_consensus.cc:359] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.183828 26856 raft_consensus.cc:385] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.184180 26856 raft_consensus.cc:740] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.185624 26853 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:29.185144 26856 consensus_queue.cc:260] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.187211 26476 catalog_manager.cc:5697] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 2, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:29.182729 26479 catalog_manager.cc:5697] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:29.194172 26951 consensus_queue.cc:1048] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:29.202040 26853 tablet_bootstrap.cc:654] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.215476 26856 ts_tablet_manager.cc:1434] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:27:29.216945 27071 raft_consensus.cc:493] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.217869 27071 raft_consensus.cc:515] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.219292 26849 raft_consensus.cc:359] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.220073 26849 raft_consensus.cc:385] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.220232 27071 leader_election.cc:290] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.220422 26849 raft_consensus.cc:740] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.216822 26856 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:29.222074 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:29.223824 26620 leader_election.cc:343] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.226694 26850 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:29.227321 26850 ts_tablet_manager.cc:1403] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.130s	user 0.020s	sys 0.018s
I20260709 12:27:29.229616 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:29.230494 26856 tablet_bootstrap.cc:654] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.230443 26801 raft_consensus.cc:2468] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:29.231062 26850 raft_consensus.cc:359] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.231832 26850 raft_consensus.cc:385] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.232117 26850 raft_consensus.cc:740] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.232550 26621 leader_election.cc:304] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:29.235504 26951 raft_consensus.cc:2804] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.236150 26951 raft_consensus.cc:493] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.236670 26951 raft_consensus.cc:3060] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.239459 27071 consensus_queue.cc:1048] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:29.243067 26853 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:29.242409 26849 consensus_queue.cc:260] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.243654 26853 ts_tablet_manager.cc:1403] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.006s
I20260709 12:27:29.245560 26849 ts_tablet_manager.cc:1434] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.073s	user 0.006s	sys 0.000s
I20260709 12:27:29.246824 26849 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:29.246973 26853 raft_consensus.cc:359] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.247728 26853 raft_consensus.cc:385] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.248004 26853 raft_consensus.cc:740] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.248741 26853 consensus_queue.cc:260] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.232834 26850 consensus_queue.cc:260] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.251247 27071 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.252818 26853 ts_tablet_manager.cc:1434] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:27:29.253055 26850 ts_tablet_manager.cc:1434] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 12:27:29.254454 26853 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:29.255678 26850 tablet_bootstrap.cc:492] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:29.251893 27071 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.261000 26849 tablet_bootstrap.cc:654] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.264197 26853 tablet_bootstrap.cc:654] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.270900 26850 tablet_bootstrap.cc:654] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.281155 26997 consensus_queue.cc:1048] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:29.290942 26850 tablet_bootstrap.cc:492] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:29.291798 26850 ts_tablet_manager.cc:1403] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.001s
I20260709 12:27:29.292598 26801 raft_consensus.cc:1275] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:27:29.295094 26850 raft_consensus.cc:359] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.296077 26850 raft_consensus.cc:385] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.296422 26850 raft_consensus.cc:740] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.297087 26656 raft_consensus.cc:3060] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.297304 26850 consensus_queue.cc:260] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.307987 27039 raft_consensus.cc:493] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.308774 27039 raft_consensus.cc:515] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.303112 27071 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:29.311723 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.312041 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d89d07b1f88649de8fe8c262f31a0ac6" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.312767 26581 raft_consensus.cc:2468] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:29.312846 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:29.313699 26802 raft_consensus.cc:2393] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate cb56388a4ff14b5a9f8f1196dc94461f in this term.
I20260709 12:27:29.314164 26769 leader_election.cc:304] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:29.315166 26940 raft_consensus.cc:2804] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.315637 26940 raft_consensus.cc:493] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.236688 26479 catalog_manager.cc:5697] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:29.316063 26940 raft_consensus.cc:3060] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.316812 27039 leader_election.cc:290] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:29.345934 26856 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:29.345821 26940 raft_consensus.cc:515] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.346194 26951 raft_consensus.cc:515] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.346939 27100 consensus_queue.cc:1048] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:27:29.346704 26856 ts_tablet_manager.cc:1403] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.130s	user 0.027s	sys 0.028s
I20260709 12:27:29.348428 26951 leader_election.cc:290] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.349911 26850 ts_tablet_manager.cc:1434] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 12:27:29.351023 26850 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:29.352200 26856 raft_consensus.cc:359] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.358464 26856 raft_consensus.cc:385] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.358860 26856 raft_consensus.cc:740] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.352546 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:29.359988 26856 consensus_queue.cc:260] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:29.361011 26620 leader_election.cc:343] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.362545 26856 ts_tablet_manager.cc:1434] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:27:29.363667 26856 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:29.354584 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d89d07b1f88649de8fe8c262f31a0ac6" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:29.355615 26849 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:29.356089 26731 raft_consensus.cc:1275] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:27:29.371248 26849 ts_tablet_manager.cc:1403] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.125s	user 0.022s	sys 0.005s
I20260709 12:27:29.371896 26856 tablet_bootstrap.cc:654] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.373000 27100 consensus_queue.cc:1048] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:29.375496 26940 leader_election.cc:290] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:29.377213 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d89d07b1f88649de8fe8c262f31a0ac6" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:29.378165 26581 raft_consensus.cc:3060] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.381642 26849 raft_consensus.cc:359] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.382741 26849 raft_consensus.cc:385] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.383122 26849 raft_consensus.cc:740] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.384003 26849 consensus_queue.cc:260] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.385140 27039 raft_consensus.cc:493] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.385622 27039 raft_consensus.cc:515] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.386596 26856 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:29.387224 26856 ts_tablet_manager.cc:1403] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:27:29.387931 27039 leader_election.cc:290] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:29.390136 26849 ts_tablet_manager.cc:1434] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:27:29.390815 26856 raft_consensus.cc:359] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.391501 26849 tablet_bootstrap.cc:492] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:29.391635 26856 raft_consensus.cc:385] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.392067 26856 raft_consensus.cc:740] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.392899 26856 consensus_queue.cc:260] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.393118 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cde6c3890534a20956dd0622e8cb1dd" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:29.395891 26856 ts_tablet_manager.cc:1434] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:27:29.394084 26729 raft_consensus.cc:2468] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:29.397394 26856 tablet_bootstrap.cc:492] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:29.398669 26849 tablet_bootstrap.cc:654] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.395839 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cde6c3890534a20956dd0622e8cb1dd" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.398782 26768 leader_election.cc:304] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.402632 27039 raft_consensus.cc:2804] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.403179 27039 raft_consensus.cc:493] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.403620 27039 raft_consensus.cc:3060] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.403863 26850 tablet_bootstrap.cc:654] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.405110 26856 tablet_bootstrap.cc:654] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:29.408941 26769 leader_election.cc:343] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.356436 26997 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.358086 26853 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:29.411867 26997 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.418411 27066 raft_consensus.cc:493] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.419142 27066 raft_consensus.cc:515] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:29.417739 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 59833.6us
I20260709 12:27:29.422511 26621 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:29.426209 26853 ts_tablet_manager.cc:1403] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.172s	user 0.042s	sys 0.010s
I20260709 12:27:29.428128 27100 raft_consensus.cc:3060] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.428030 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:29.431075 26853 raft_consensus.cc:359] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:29.431440 26620 leader_election.cc:343] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.431859 26853 raft_consensus.cc:385] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.432183 26853 raft_consensus.cc:740] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.433023 26853 consensus_queue.cc:260] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.439517 26970 raft_consensus.cc:493] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.440167 26970 raft_consensus.cc:515] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.443826 26970 leader_election.cc:290] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.449446 26850 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:29.450054 26850 ts_tablet_manager.cc:1403] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.099s	user 0.011s	sys 0.004s
I20260709 12:27:29.453368 26856 tablet_bootstrap.cc:492] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:29.453128 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb970d065d54c94be1a04a340a5821c" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:29.453840 27100 raft_consensus.cc:2749] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:29.453953 26856 ts_tablet_manager.cc:1403] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.002s
I20260709 12:27:29.456435 26853 ts_tablet_manager.cc:1434] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.001s
I20260709 12:27:29.457410 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.458118 26580 raft_consensus.cc:2468] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:29.458673 26849 tablet_bootstrap.cc:492] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:29.459220 26849 ts_tablet_manager.cc:1403] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.003s
I20260709 12:27:29.459642 26695 leader_election.cc:304] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.454013 26656 raft_consensus.cc:2468] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:29.462203 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb970d065d54c94be1a04a340a5821c" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:29.464135 26849 raft_consensus.cc:359] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.465076 26849 raft_consensus.cc:385] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.465363 26849 raft_consensus.cc:740] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.466463 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:29.467207 26656 raft_consensus.cc:2468] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:29.467123 26850 raft_consensus.cc:359] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.467942 26850 raft_consensus.cc:385] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.468230 26850 raft_consensus.cc:740] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.468524 26694 leader_election.cc:304] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.462940 26805 raft_consensus.cc:2468] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:29.469410 26850 consensus_queue.cc:260] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.444504 26997 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:29.476826 26849 consensus_queue.cc:260] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.456970 27066 leader_election.cc:290] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:29.476099 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.486092 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d89d07b1f88649de8fe8c262f31a0ac6" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:29.474006 26970 raft_consensus.cc:2804] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.494302 26970 raft_consensus.cc:493] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.494747 26970 raft_consensus.cc:3060] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.495982 26850 ts_tablet_manager.cc:1434] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:27:29.499343 27066 raft_consensus.cc:2804] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.492511 26853 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:29.502982 26768 leader_election.cc:343] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.509481 27041 raft_consensus.cc:493] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.510035 27041 raft_consensus.cc:515] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.510352 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:29.510994 26802 raft_consensus.cc:3060] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.512326 27041 leader_election.cc:290] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:29.513381 26856 raft_consensus.cc:359] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.514479 26856 raft_consensus.cc:385] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.514833 26856 raft_consensus.cc:740] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.502381 27066 raft_consensus.cc:493] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.518967 27066 raft_consensus.cc:3060] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.519798 27055 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.520357 27055 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:29.521561 26621 leader_election.cc:343] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.522110 26621 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:29.525758 26950 raft_consensus.cc:2749] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:29.526678 27055 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:29.527757 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:29.528491 26801 raft_consensus.cc:2468] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:29.530607 27102 consensus_queue.cc:1048] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:29.529928 26768 leader_election.cc:343] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.536558 26856 consensus_queue.cc:260] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.500363 26581 raft_consensus.cc:2468] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:29.540215 26849 ts_tablet_manager.cc:1434] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.001s
I20260709 12:27:29.538157 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:29.508718 27039 raft_consensus.cc:515] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.551146 26850 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:29.548415 26546 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:29.558120 27057 raft_consensus.cc:2804] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:29.558640 27057 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.559070 27057 raft_consensus.cc:3060] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:29.545858 26694 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 77443.1us
I20260709 12:27:29.572873 26856 ts_tablet_manager.cc:1434] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.117s	user 0.007s	sys 0.001s
I20260709 12:27:29.578411 27110 raft_consensus.cc:493] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.581135 26769 leader_election.cc:304] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:29.581657 27110 raft_consensus.cc:515] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.583786 26849 tablet_bootstrap.cc:492] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:29.553109 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:29.596781 27110 leader_election.cc:290] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:29.597681 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.598661 26580 raft_consensus.cc:2393] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:29.586110 27041 raft_consensus.cc:2804] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.601960 27041 raft_consensus.cc:697] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:29.605008 27041 consensus_queue.cc:237] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:29.607826 26545 leader_election.cc:343] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.609043 26853 tablet_bootstrap.cc:654] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.611500 26856 tablet_bootstrap.cc:492] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:29.611339 27055 consensus_queue.cc:1048] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:27:29.633404 26802 raft_consensus.cc:2468] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
W20260709 12:27:29.634377 26768 leader_election.cc:343] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.641402 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:29.645512 27039 leader_election.cc:290] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:29.609220 26970 raft_consensus.cc:515] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.646425 26804 raft_consensus.cc:1275] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:27:29.647797 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:29.648010 26970 leader_election.cc:290] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:29.654096 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:29.655278 26655 raft_consensus.cc:3060] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.662113 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:29.662935 26580 raft_consensus.cc:3060] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.663733 26621 leader_election.cc:304] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:29.664160 26656 raft_consensus.cc:1275] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:27:29.664819 27102 raft_consensus.cc:2804] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.665333 27102 raft_consensus.cc:697] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:29.666302 27102 consensus_queue.cc:237] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.662842 27110 consensus_queue.cc:1048] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:29.662907 26849 tablet_bootstrap.cc:654] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.675222 27108 consensus_queue.cc:1048] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 12:27:29.678643 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:29.680141 26856 tablet_bootstrap.cc:654] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.680384 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cde6c3890534a20956dd0622e8cb1dd" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:29.681422 26730 raft_consensus.cc:3060] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.683260 27066 raft_consensus.cc:515] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
W20260709 12:27:29.684984 27104 log.cc:927] Time spent T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:27:29.685539 27066 leader_election.cc:290] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.687162 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb970d065d54c94be1a04a340a5821c" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:29.687842 26657 raft_consensus.cc:3060] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.691224 26850 tablet_bootstrap.cc:654] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:29.716866 26769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 86225us
I20260709 12:27:29.717592 27057 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:29.755864 26655 raft_consensus.cc:2468] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:29.774871 26476 catalog_manager.cc:5697] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:29.761801 26657 raft_consensus.cc:2468] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:29.761940 27057 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:29.803148 26578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cde6c3890534a20956dd0622e8cb1dd" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:29.815335 27110 raft_consensus.cc:493] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.816113 27110 raft_consensus.cc:515] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.822877 26853 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:29.823571 26853 ts_tablet_manager.cc:1403] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.349s	user 0.016s	sys 0.023s
I20260709 12:27:29.825474 26695 leader_election.cc:304] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:29.844219 27108 raft_consensus.cc:3060] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.846541 26856 tablet_bootstrap.cc:492] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:29.851703 26856 ts_tablet_manager.cc:1403] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.263s	user 0.017s	sys 0.022s
I20260709 12:27:29.835410 26694 leader_election.cc:304] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.863612 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:29.865744 27066 raft_consensus.cc:2804] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.867650 27066 raft_consensus.cc:697] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:29.868700 27066 consensus_queue.cc:237] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.877758 26730 raft_consensus.cc:2468] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:29.879372 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:29.892983 27110 leader_election.cc:290] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:29.882025 26853 raft_consensus.cc:359] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.895673 26853 raft_consensus.cc:385] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.895985 26853 raft_consensus.cc:740] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.896818 26853 consensus_queue.cc:260] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.877914 27108 raft_consensus.cc:2749] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:29.885134 27057 consensus_queue.cc:1048] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:27:29.901423 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cb970d065d54c94be1a04a340a5821c" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:29.902071 26801 raft_consensus.cc:3060] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:29.887943 26480 catalog_manager.cc:5697] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:29.887954 26545 leader_election.cc:343] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.905066 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb4745cc61545b0a28693865420f43f" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:29.905943 27057 consensus_queue.cc:1048] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:29.906523 26768 leader_election.cc:343] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.907047 26768 leader_election.cc:304] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:29.907864 27126 raft_consensus.cc:2749] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:29.879135 26694 leader_election.cc:304] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:29.892045 26849 tablet_bootstrap.cc:492] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:29.899763 26853 ts_tablet_manager.cc:1434] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.004s
I20260709 12:27:29.883658 26856 raft_consensus.cc:359] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.911320 26849 ts_tablet_manager.cc:1403] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.343s	user 0.025s	sys 0.007s
I20260709 12:27:29.911410 26856 raft_consensus.cc:385] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.880504 26850 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:29.915203 26950 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.905956 26802 raft_consensus.cc:2393] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:29.915812 26950 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:29.912192 26856 raft_consensus.cc:740] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.917258 26850 ts_tablet_manager.cc:1403] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.368s	user 0.016s	sys 0.020s
I20260709 12:27:29.919163 27110 raft_consensus.cc:2804] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:29.921068 26950 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:29.924520 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:29.925303 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:29.926378 26621 leader_election.cc:343] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.928730 27110 raft_consensus.cc:697] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:29.931483 26853 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:29.938416 27110 consensus_queue.cc:237] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.888005 26580 raft_consensus.cc:2468] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
W20260709 12:27:29.899786 26769 leader_election.cc:343] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.947446 26856 consensus_queue.cc:260] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
W20260709 12:27:29.953490 26769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 60944.7us
I20260709 12:27:29.968405 26856 ts_tablet_manager.cc:1434] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.116s	user 0.006s	sys 0.004s
W20260709 12:27:29.963510 26620 leader_election.cc:343] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:29.977298 27126 consensus_queue.cc:1048] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:29.980195 26768 leader_election.cc:304] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:29.981236 27116 raft_consensus.cc:2804] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:27:29.981379 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 73176.6us
I20260709 12:27:29.981805 27116 raft_consensus.cc:697] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:29.982805 27116 consensus_queue.cc:237] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:29.982075 26853 tablet_bootstrap.cc:654] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:29.995599 26849 raft_consensus.cc:359] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:29.996690 26849 raft_consensus.cc:385] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:29.997063 26849 raft_consensus.cc:740] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:29.998036 26849 consensus_queue.cc:260] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.012666 26849 ts_tablet_manager.cc:1434] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.101s	user 0.011s	sys 0.000s
I20260709 12:27:30.016834 26849 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:29.994561 26620 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:30.021118 27102 raft_consensus.cc:2749] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:30.022357 26620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 60256.8us
I20260709 12:27:30.033457 26695 leader_election.cc:304] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:30.034989 27109 raft_consensus.cc:3060] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.037878 26856 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:30.050415 27109 raft_consensus.cc:2749] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:30.052701 26849 tablet_bootstrap.cc:654] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.052708 26850 raft_consensus.cc:359] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:30.068380 26850 raft_consensus.cc:385] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.070868 26850 raft_consensus.cc:740] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.067848 26801 raft_consensus.cc:2468] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:30.079257 26853 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:30.079859 26853 ts_tablet_manager.cc:1403] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.149s	user 0.025s	sys 0.011s
I20260709 12:27:30.083269 26853 raft_consensus.cc:359] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.084671 26853 raft_consensus.cc:385] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.085017 26853 raft_consensus.cc:740] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.086071 26853 consensus_queue.cc:260] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:30.085402 26768 consensus_peers.cc:597] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:30.089545 26850 consensus_queue.cc:260] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:30.095779 26853 ts_tablet_manager.cc:1434] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 12:27:30.098335 26853 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:30.101069 27109 raft_consensus.cc:493] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.110404 27131 log.cc:927] Time spent T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:27:30.113106 26853 tablet_bootstrap.cc:654] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.113515 26480 catalog_manager.cc:5697] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:30.101686 27109 raft_consensus.cc:515] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.140301 27109 leader_election.cc:290] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:30.113956 27126 consensus_queue.cc:1048] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:27:30.144927 26850 ts_tablet_manager.cc:1434] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.227s	user 0.005s	sys 0.004s
I20260709 12:27:30.156262 26850 tablet_bootstrap.cc:492] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:30.163452 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:30.164565 26800 raft_consensus.cc:2468] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:30.171010 26849 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:30.171643 26849 ts_tablet_manager.cc:1403] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.155s	user 0.030s	sys 0.041s
I20260709 12:27:30.173115 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:30.173846 26804 raft_consensus.cc:3060] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.174794 26849 raft_consensus.cc:359] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.175565 26849 raft_consensus.cc:385] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.175840 26849 raft_consensus.cc:740] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.176577 26849 consensus_queue.cc:260] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.179275 26849 ts_tablet_manager.cc:1434] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:30.177122 27057 consensus_queue.cc:1048] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:27:30.160507 26856 tablet_bootstrap.cc:654] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.195080 26695 leader_election.cc:304] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:30.196494 26849 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:30.198995 27108 raft_consensus.cc:2804] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:30.199532 27108 raft_consensus.cc:493] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.199901 27108 raft_consensus.cc:3060] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.204584 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:30.216363 26849 tablet_bootstrap.cc:654] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.236912 26950 raft_consensus.cc:493] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.237558 26950 raft_consensus.cc:515] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:30.240463 26950 leader_election.cc:290] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:30.242550 26694 leader_election.cc:343] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.254838 27102 consensus_queue.cc:1048] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:30.266602 26853 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:30.267382 26853 ts_tablet_manager.cc:1403] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.169s	user 0.020s	sys 0.022s
I20260709 12:27:30.269666 26479 catalog_manager.cc:5697] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:30.276787 26804 raft_consensus.cc:2468] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:30.283401 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d89d07b1f88649de8fe8c262f31a0ac6" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:30.284354 26581 raft_consensus.cc:2393] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:30.289316 26546 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:30.289546 27113 raft_consensus.cc:493] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.290573 27115 raft_consensus.cc:2804] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:30.290380 27113 raft_consensus.cc:515] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.291184 27115 raft_consensus.cc:697] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:30.292281 27115 consensus_queue.cc:237] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:30.293170 27113 leader_election.cc:290] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:30.298327 26476 catalog_manager.cc:5697] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:30.343050 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf5c3113a22464e81d4cb8462e8ac67" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.346241 27118 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.346781 26768 leader_election.cc:343] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.347177 27118 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.346517 27110 consensus_queue.cc:1048] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:30.350138 27118 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:30.352272 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:30.353065 26656 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:30.353195 26950 consensus_queue.cc:1048] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:30.354091 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf5c3113a22464e81d4cb8462e8ac67" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:30.356139 26545 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:30.357172 27118 raft_consensus.cc:2804] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:30.357554 27118 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.357901 27118 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.358587 27114 raft_consensus.cc:493] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.359123 27114 raft_consensus.cc:515] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.361356 27114 leader_election.cc:290] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:30.362401 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.362881 27113 consensus_queue.cc:1048] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:30.368362 26545 leader_election.cc:343] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.370131 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cde6c3890534a20956dd0622e8cb1dd" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.371060 26731 raft_consensus.cc:2393] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:30.370774 26853 raft_consensus.cc:359] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.371512 26853 raft_consensus.cc:385] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.371806 26853 raft_consensus.cc:740] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.372716 26853 consensus_queue.cc:260] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.374852 26853 ts_tablet_manager.cc:1434] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.001s
W20260709 12:27:30.377125 26769 leader_election.cc:343] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.377862 26769 leader_election.cc:304] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:30.379586 27113 raft_consensus.cc:2749] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:30.382220 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cde6c3890534a20956dd0622e8cb1dd" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:30.389961 26853 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:30.392894 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d89d07b1f88649de8fe8c262f31a0ac6" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:30.407349 26621 leader_election.cc:304] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:30.411783 27102 raft_consensus.cc:3060] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.418865 27108 raft_consensus.cc:515] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.434167 26546 leader_election.cc:304] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:30.436365 26850 tablet_bootstrap.cc:654] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.436847 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:30.446894 26694 leader_election.cc:343] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.437182 27115 raft_consensus.cc:3060] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.465772 26853 tablet_bootstrap.cc:654] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.465435 27110 consensus_queue.cc:1048] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:30.481523 26856 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:30.482437 26856 ts_tablet_manager.cc:1403] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.505s	user 0.021s	sys 0.007s
I20260709 12:27:30.491153 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:30.491984 26800 raft_consensus.cc:3060] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.493459 27109 consensus_queue.cc:1048] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:30.494585 27039 raft_consensus.cc:493] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.495157 27039 raft_consensus.cc:515] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.497614 27039 leader_election.cc:290] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:30.497336 26480 catalog_manager.cc:5697] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:30.499872 27108 leader_election.cc:290] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:30.500923 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc276f88554143c3bd0a5b4f74143b2f" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.499775 26856 raft_consensus.cc:359] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.507773 26856 raft_consensus.cc:385] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:27:30.507975 26768 leader_election.cc:343] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.508227 26856 raft_consensus.cc:740] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.509115 26856 consensus_queue.cc:260] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.511701 26856 ts_tablet_manager.cc:1434] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:27:30.516412 26856 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:30.523519 26849 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:30.524319 26849 ts_tablet_manager.cc:1403] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.333s	user 0.024s	sys 0.010s
I20260709 12:27:30.525717 26856 tablet_bootstrap.cc:654] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.529807 26850 tablet_bootstrap.cc:492] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:30.530602 26850 ts_tablet_manager.cc:1403] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.378s	user 0.019s	sys 0.007s
I20260709 12:27:30.536616 27115 raft_consensus.cc:2749] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:30.537279 27146 log.cc:927] Time spent T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:27:30.538583 26850 raft_consensus.cc:359] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.546924 26850 raft_consensus.cc:385] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.547353 26850 raft_consensus.cc:740] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.549284 26850 consensus_queue.cc:260] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.526657 27102 raft_consensus.cc:2749] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:30.559229 26850 ts_tablet_manager.cc:1434] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:27:30.560529 26850 tablet_bootstrap.cc:492] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:30.565815 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc276f88554143c3bd0a5b4f74143b2f" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:30.567950 26850 tablet_bootstrap.cc:654] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.575479 27039 consensus_queue.cc:1048] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:30.586619 26768 leader_election.cc:343] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.598683 26768 leader_election.cc:304] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:30.600364 26849 raft_consensus.cc:359] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.610901 26849 raft_consensus.cc:385] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.611644 26849 raft_consensus.cc:740] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.601732 27039 raft_consensus.cc:2749] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:30.613441 26849 consensus_queue.cc:260] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.622061 27114 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.616310 27118 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.622920 27114 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.591082 27108 consensus_queue.cc:1048] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:27:30.643949 26654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:30.644703 26654 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.645591 26849 ts_tablet_manager.cc:1434] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.121s	user 0.008s	sys 0.003s
I20260709 12:27:30.655169 27039 raft_consensus.cc:493] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.654246 27118 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:30.656466 26800 raft_consensus.cc:2468] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:30.664147 27114 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:30.682518 26849 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:30.687140 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:30.669179 27039 raft_consensus.cc:515] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.721890 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:30.722776 26656 raft_consensus.cc:2468] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:30.713900 26695 leader_election.cc:304] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0
I20260709 12:27:30.716866 26730 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.753494 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.755872 27039 leader_election.cc:290] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:30.757884 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 62215.3us
I20260709 12:27:30.758129 26856 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:30.758780 26856 ts_tablet_manager.cc:1403] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.243s	user 0.022s	sys 0.019s
I20260709 12:27:30.758930 27066 raft_consensus.cc:2804] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:30.759502 27066 raft_consensus.cc:697] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:30.727376 27126 consensus_queue.cc:1048] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 12:27:30.760483 27066 consensus_queue.cc:237] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
W20260709 12:27:30.762518 26545 leader_election.cc:343] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.762202 26856 raft_consensus.cc:359] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.765843 26856 raft_consensus.cc:385] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.766130 26856 raft_consensus.cc:740] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.766875 26856 consensus_queue.cc:260] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.771631 26850 tablet_bootstrap.cc:492] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:30.772275 26850 ts_tablet_manager.cc:1403] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.212s	user 0.011s	sys 0.005s
I20260709 12:27:30.782593 26545 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:30.785838 27115 raft_consensus.cc:2804] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:30.786379 27115 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.786810 27115 raft_consensus.cc:3060] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.796432 26850 raft_consensus.cc:359] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.797341 26850 raft_consensus.cc:385] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.797688 26850 raft_consensus.cc:740] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.812860 26730 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:30.814599 26850 consensus_queue.cc:260] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
W20260709 12:27:30.844463 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:27:30.852211 26694 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 45512us
I20260709 12:27:30.862807 26997 raft_consensus.cc:493] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.863701 26997 raft_consensus.cc:515] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:30.875717 26856 ts_tablet_manager.cc:1434] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.117s	user 0.005s	sys 0.003s
I20260709 12:27:30.875818 26850 ts_tablet_manager.cc:1434] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.103s	user 0.009s	sys 0.002s
W20260709 12:27:30.878656 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 158649us
I20260709 12:27:30.870608 27110 raft_consensus.cc:493] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.880952 26997 leader_election.cc:290] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:30.883749 26853 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:30.883458 27110 raft_consensus.cc:515] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.884577 26853 ts_tablet_manager.cc:1403] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.495s	user 0.013s	sys 0.020s
I20260709 12:27:30.885211 26654 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:30.885764 27110 leader_election.cc:290] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:30.886869 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.887610 26850 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:30.889585 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf5c3113a22464e81d4cb8462e8ac67" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:30.892498 26695 leader_election.cc:343] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.894332 26849 tablet_bootstrap.cc:654] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.894472 26545 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:30.895761 27057 raft_consensus.cc:2804] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:30.896277 27057 raft_consensus.cc:697] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:30.897292 27057 consensus_queue.cc:237] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.901175 26853 raft_consensus.cc:359] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.905068 26478 catalog_manager.cc:5697] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:30.906899 26853 raft_consensus.cc:385] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:30.908174 26853 raft_consensus.cc:740] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:30.909678 26856 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:30.909225 26853 consensus_queue.cc:260] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:30.912038 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e6beba6a6d747ebbcd858a8b61ad68b" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:30.942838 26800 raft_consensus.cc:2393] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 643057f6f77241529979f572a42ed9b0 in current term 1: Already voted for candidate f57551729f2f4037a4c769ee1df81853 in this term.
I20260709 12:27:30.912573 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293707805bab498a9449cd938d3b1b6b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:30.920528 27115 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:30.921396 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293707805bab498a9449cd938d3b1b6b" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:30.945439 26856 tablet_bootstrap.cc:654] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.935549 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf5c3113a22464e81d4cb8462e8ac67" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:30.940203 27055 consensus_queue.cc:1048] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:30.922552 26545 consensus_peers.cc:597] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:30.947678 27115 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:30.949589 26804 raft_consensus.cc:2468] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:30.950423 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:30.951841 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:30.952292 26545 leader_election.cc:343] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.952513 26656 raft_consensus.cc:3060] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.953744 26850 tablet_bootstrap.cc:654] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.952301 26853 ts_tablet_manager.cc:1434] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.005s
W20260709 12:27:30.960065 26768 leader_election.cc:343] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:30.961091 26621 leader_election.cc:304] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:30.962085 26997 raft_consensus.cc:3060] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.960109 26853 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:30.975817 26997 raft_consensus.cc:2749] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:30.977083 26695 leader_election.cc:304] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:30.984031 27066 raft_consensus.cc:2804] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:30.984648 27066 raft_consensus.cc:493] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.985088 27066 raft_consensus.cc:3060] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:30.994220 26850 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:30.995042 26850 ts_tablet_manager.cc:1403] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.108s	user 0.016s	sys 0.001s
I20260709 12:27:30.996436 26853 tablet_bootstrap.cc:654] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:30.998960 26850 raft_consensus.cc:359] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:30.999886 26850 raft_consensus.cc:385] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.000324 26850 raft_consensus.cc:740] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.001086 26850 consensus_queue.cc:260] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.003870 26850 ts_tablet_manager.cc:1434] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:27:31.005648 26850 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:31.014595 26850 tablet_bootstrap.cc:654] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.027611 26853 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:31.028220 26853 ts_tablet_manager.cc:1403] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.004s
I20260709 12:27:31.029277 26850 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:31.029848 26850 ts_tablet_manager.cc:1403] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.015s
I20260709 12:27:31.031359 26656 raft_consensus.cc:2468] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:31.031381 26853 raft_consensus.cc:359] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.032856 26545 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:31.032358 26853 raft_consensus.cc:385] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.034670 26853 raft_consensus.cc:740] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.035068 26850 raft_consensus.cc:359] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.035802 26850 raft_consensus.cc:385] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.035455 26853 consensus_queue.cc:260] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.036059 26850 raft_consensus.cc:740] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.036841 27114 raft_consensus.cc:2804] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:31.037415 27114 raft_consensus.cc:697] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:31.038457 27114 consensus_queue.cc:237] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.044612 26853 ts_tablet_manager.cc:1434] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:27:31.047686 26853 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:31.036860 26850 consensus_queue.cc:260] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.045691 27066 raft_consensus.cc:515] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.059770 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf5c3113a22464e81d4cb8462e8ac67" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:31.060750 26804 raft_consensus.cc:3060] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.060938 26478 catalog_manager.cc:5697] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:31.062903 26850 ts_tablet_manager.cc:1434] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.002s
I20260709 12:27:31.079865 26577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bf5c3113a22464e81d4cb8462e8ac67" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:31.080957 26577 raft_consensus.cc:3060] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.091826 26849 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:31.092437 26850 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:31.100754 26849 ts_tablet_manager.cc:1403] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.432s	user 0.013s	sys 0.007s
W20260709 12:27:31.093190 26546 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 153443us
I20260709 12:27:31.106602 27066 leader_election.cc:290] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:31.104538 27118 raft_consensus.cc:493] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.117012 27118 raft_consensus.cc:515] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.120016 26804 raft_consensus.cc:2468] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:31.121389 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f355a7f12b14432844a7a4508117120" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:31.123427 26769 leader_election.cc:343] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:31.123605 26545 leader_election.cc:343] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:31.123962 26769 leader_election.cc:304] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:31.125432 27116 raft_consensus.cc:2749] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:31.125710 27118 leader_election.cc:290] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:31.126461 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f355a7f12b14432844a7a4508117120" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:31.127947 26545 leader_election.cc:343] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:31.128407 26545 leader_election.cc:304] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:31.130883 27169 raft_consensus.cc:2749] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:31.109328 26850 tablet_bootstrap.cc:654] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.128286 26695 leader_election.cc:304] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:31.137846 26849 raft_consensus.cc:359] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.138579 26970 raft_consensus.cc:2804] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:31.139432 26970 raft_consensus.cc:697] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:31.140659 26970 consensus_queue.cc:237] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.151677 26856 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:31.152351 26856 ts_tablet_manager.cc:1403] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.247s	user 0.052s	sys 0.121s
I20260709 12:27:31.152684 26849 raft_consensus.cc:385] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.153028 26849 raft_consensus.cc:740] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.110638 26853 tablet_bootstrap.cc:654] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.154521 26849 consensus_queue.cc:260] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.158322 26856 raft_consensus.cc:359] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.159040 26856 raft_consensus.cc:385] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.159394 26856 raft_consensus.cc:740] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.161571 26856 consensus_queue.cc:260] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.162742 27102 raft_consensus.cc:493] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.163292 27102 raft_consensus.cc:515] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.165400 27102 leader_election.cc:290] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:31.166366 26850 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:31.168053 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc276f88554143c3bd0a5b4f74143b2f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:31.169287 26856 ts_tablet_manager.cc:1434] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
W20260709 12:27:31.169690 26620 leader_election.cc:343] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:31.170575 26856 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:31.178053 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc276f88554143c3bd0a5b4f74143b2f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:31.158356 26849 ts_tablet_manager.cc:1434] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:27:31.180464 26804 raft_consensus.cc:2468] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:31.181746 26849 tablet_bootstrap.cc:492] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:31.178429 26577 raft_consensus.cc:2468] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:31.178635 26856 tablet_bootstrap.cc:654] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.166950 26850 ts_tablet_manager.cc:1403] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.004s
I20260709 12:27:31.181805 26621 leader_election.cc:304] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:31.189589 27102 raft_consensus.cc:2804] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:31.190063 27102 raft_consensus.cc:493] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.191352 27102 raft_consensus.cc:3060] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.177881 26478 catalog_manager.cc:5697] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:31.193244 26853 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:31.202517 26853 ts_tablet_manager.cc:1403] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.155s	user 0.008s	sys 0.009s
I20260709 12:27:31.189378 26850 raft_consensus.cc:359] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.206560 26850 raft_consensus.cc:385] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.209874 26850 raft_consensus.cc:740] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.217087 26853 raft_consensus.cc:359] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.217940 26853 raft_consensus.cc:385] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.211727 26850 consensus_queue.cc:260] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.218187 26853 raft_consensus.cc:740] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.219331 26856 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:31.194886 26849 tablet_bootstrap.cc:654] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.228688 26850 ts_tablet_manager.cc:1434] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 12:27:31.230181 26850 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:31.219048 26853 consensus_queue.cc:260] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.219888 26856 ts_tablet_manager.cc:1403] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.013s
I20260709 12:27:31.237632 26849 tablet_bootstrap.cc:492] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:31.250051 26856 raft_consensus.cc:359] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.242978 27102 raft_consensus.cc:515] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.263192 26856 raft_consensus.cc:385] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.262259 26849 ts_tablet_manager.cc:1403] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.081s	user 0.017s	sys 0.003s
I20260709 12:27:31.263566 26856 raft_consensus.cc:740] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.264483 26856 consensus_queue.cc:260] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.256771 26476 catalog_manager.cc:5697] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:31.267144 26856 ts_tablet_manager.cc:1434] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:27:31.268311 26856 tablet_bootstrap.cc:492] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:31.271138 27102 leader_election.cc:290] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:31.272416 26804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc276f88554143c3bd0a5b4f74143b2f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:31.273111 26804 raft_consensus.cc:3060] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.237203 27066 consensus_queue.cc:1048] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.038s
I20260709 12:27:31.251948 26853 ts_tablet_manager.cc:1434] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.049s	user 0.002s	sys 0.006s
I20260709 12:27:31.278846 26853 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:31.280330 26856 tablet_bootstrap.cc:654] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.280807 26849 raft_consensus.cc:359] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.281607 26849 raft_consensus.cc:385] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.284235 26804 raft_consensus.cc:2468] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:31.284678 26970 consensus_queue.cc:1048] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:31.287118 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc276f88554143c3bd0a5b4f74143b2f" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:31.287480 26621 leader_election.cc:304] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:31.288777 26853 tablet_bootstrap.cc:654] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.288856 26997 raft_consensus.cc:2804] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:31.262239 26850 tablet_bootstrap.cc:654] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:31.290876 26620 leader_election.cc:343] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:31.293269 26997 raft_consensus.cc:697] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:31.294457 26997 consensus_queue.cc:237] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.281934 26849 raft_consensus.cc:740] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.290021 27102 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.306084 26856 tablet_bootstrap.cc:492] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:31.306952 26856 ts_tablet_manager.cc:1403] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.007s
I20260709 12:27:31.306778 27102 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.307101 26849 consensus_queue.cc:260] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.310034 26849 ts_tablet_manager.cc:1434] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 12:27:31.311185 26849 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:31.311430 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:31.312636 26856 raft_consensus.cc:359] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.313470 26856 raft_consensus.cc:385] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.313771 26856 raft_consensus.cc:740] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.314729 26856 consensus_queue.cc:260] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.317214 26856 ts_tablet_manager.cc:1434] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:27:31.325103 26856 tablet_bootstrap.cc:492] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:31.325528 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:31.327185 26950 raft_consensus.cc:493] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.327584 26850 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:31.328110 26850 ts_tablet_manager.cc:1403] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.004s
I20260709 12:27:31.327757 26950 raft_consensus.cc:515] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.328969 26621 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:31.330436 27100 raft_consensus.cc:2749] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:31.330482 27102 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:31.338018 27114 consensus_queue.cc:1048] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:27:31.343320 26853 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:31.332787 26850 raft_consensus.cc:359] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.343773 26850 raft_consensus.cc:385] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.343936 26853 ts_tablet_manager.cc:1403] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.007s
I20260709 12:27:31.344102 26850 raft_consensus.cc:740] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.345105 26850 consensus_queue.cc:260] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.348146 26850 ts_tablet_manager.cc:1434] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.000s
I20260709 12:27:31.349797 26850 tablet_bootstrap.cc:492] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:31.352420 26950 leader_election.cc:290] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:31.354144 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47d7e2ee9fa8426f9ce60485d6e4bb7a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:31.354748 26654 raft_consensus.cc:3060] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.355196 26582 raft_consensus.cc:2468] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:31.347632 26853 raft_consensus.cc:359] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.340569 26856 tablet_bootstrap.cc:654] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.356218 26853 raft_consensus.cc:385] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.340209 26849 tablet_bootstrap.cc:654] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.356591 26853 raft_consensus.cc:740] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.357540 26621 leader_election.cc:304] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:31.354295 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47d7e2ee9fa8426f9ce60485d6e4bb7a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:31.359838 26970 raft_consensus.cc:493] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.360086 26729 raft_consensus.cc:2468] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:31.360460 26970 raft_consensus.cc:515] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.358664 26950 raft_consensus.cc:2804] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:31.365279 26950 raft_consensus.cc:493] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.365795 26950 raft_consensus.cc:3060] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.357564 26853 consensus_queue.cc:260] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.371037 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:31.371829 26802 raft_consensus.cc:2468] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:31.373278 26695 leader_election.cc:304] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:31.375277 27110 raft_consensus.cc:2804] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:31.376248 27110 raft_consensus.cc:493] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.376659 27110 raft_consensus.cc:3060] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.390239 26970 leader_election.cc:290] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:31.374114 26850 tablet_bootstrap.cc:654] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.417538 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:31.433485 26853 ts_tablet_manager.cc:1434] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.089s	user 0.014s	sys 0.001s
I20260709 12:27:31.435446 26853 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:31.448127 27193 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.448859 27193 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:31.451005 26694 leader_election.cc:343] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:31.455397 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 67876.1us
I20260709 12:27:31.461826 27193 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:31.464056 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:31.465323 26656 raft_consensus.cc:2393] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate cb56388a4ff14b5a9f8f1196dc94461f in this term.
I20260709 12:27:31.466886 27118 consensus_queue.cc:1048] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:31.468132 26853 tablet_bootstrap.cc:654] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.457203 26850 tablet_bootstrap.cc:492] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:31.470781 26850 ts_tablet_manager.cc:1403] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.121s	user 0.012s	sys 0.006s
I20260709 12:27:31.474592 26850 raft_consensus.cc:359] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.474840 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:31.476854 26695 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:31.483608 27126 raft_consensus.cc:493] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.484340 27126 raft_consensus.cc:515] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.486814 27126 leader_election.cc:290] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:31.487370 27191 raft_consensus.cc:3060] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.492151 26577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:31.475337 26850 raft_consensus.cc:385] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.493296 26577 raft_consensus.cc:2468] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:31.493665 26850 raft_consensus.cc:740] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.494859 26769 leader_election.cc:304] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:31.494580 26850 consensus_queue.cc:260] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.495744 27126 raft_consensus.cc:2804] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:31.496148 27126 raft_consensus.cc:493] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.496506 27126 raft_consensus.cc:3060] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.503177 26850 ts_tablet_manager.cc:1434] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 12:27:31.505164 26850 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:31.512817 27126 raft_consensus.cc:515] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.519958 26853 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:31.520741 26853 ts_tablet_manager.cc:1403] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.086s	user 0.014s	sys 0.002s
I20260709 12:27:31.542356 26476 catalog_manager.cc:5697] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:31.558055 26950 raft_consensus.cc:515] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.564020 26950 leader_election.cc:290] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:31.577766 27126 leader_election.cc:290] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:31.591822 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47d7e2ee9fa8426f9ce60485d6e4bb7a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:31.592650 26729 raft_consensus.cc:3060] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.555526 27191 raft_consensus.cc:2749] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:31.593844 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47d7e2ee9fa8426f9ce60485d6e4bb7a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:31.594519 26582 raft_consensus.cc:3060] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.596379 26654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:31.597175 26654 raft_consensus.cc:2468] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:31.602860 26850 tablet_bootstrap.cc:654] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.607405 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:31.608160 26657 raft_consensus.cc:3060] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.589928 26577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:31.609766 26577 raft_consensus.cc:3060] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.615592 27110 raft_consensus.cc:515] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.616626 26853 raft_consensus.cc:359] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.617414 26853 raft_consensus.cc:385] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.617679 26853 raft_consensus.cc:740] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.619269 26853 consensus_queue.cc:260] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.621726 26853 ts_tablet_manager.cc:1434] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.101s	user 0.010s	sys 0.000s
I20260709 12:27:31.630231 26856 tablet_bootstrap.cc:492] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:31.630815 26853 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:31.630934 26856 ts_tablet_manager.cc:1403] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.313s	user 0.067s	sys 0.066s
I20260709 12:27:31.631290 26970 consensus_queue.cc:1048] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:31.641067 27110 leader_election.cc:290] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
W20260709 12:27:31.694957 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 84706.2us
I20260709 12:27:31.714159 26856 raft_consensus.cc:359] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.715057 26856 raft_consensus.cc:385] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.715342 26856 raft_consensus.cc:740] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.716004 26856 consensus_queue.cc:260] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:31.710498 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:31.721837 26801 raft_consensus.cc:3060] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:31.690634 26545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 56124.5us
I20260709 12:27:31.729359 26657 raft_consensus.cc:2468] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:31.734378 26856 ts_tablet_manager.cc:1434] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.103s	user 0.005s	sys 0.003s
I20260709 12:27:31.744637 26856 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:31.744801 26577 raft_consensus.cc:2468] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:31.745707 26582 raft_consensus.cc:2468] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:31.747121 26849 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:31.747458 26853 tablet_bootstrap.cc:654] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.747687 26849 ts_tablet_manager.cc:1403] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.437s	user 0.029s	sys 0.029s
I20260709 12:27:31.750779 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:31.751552 26769 leader_election.cc:304] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:31.732919 26729 raft_consensus.cc:2468] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:31.752839 27163 raft_consensus.cc:2804] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:31.753784 27163 raft_consensus.cc:697] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:31.754951 27163 consensus_queue.cc:237] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:31.756207 26620 leader_election.cc:304] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:31.751591 26655 raft_consensus.cc:3060] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.805841 26950 raft_consensus.cc:2804] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:31.821230 26950 raft_consensus.cc:697] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:31.822293 26950 consensus_queue.cc:237] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.820372 26801 raft_consensus.cc:2468] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:31.832722 27118 consensus_queue.cc:1048] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:31.773851 27204 raft_consensus.cc:493] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.852277 26849 raft_consensus.cc:359] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.871814 26849 raft_consensus.cc:385] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.872406 26849 raft_consensus.cc:740] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.865052 26850 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:31.876013 26850 ts_tablet_manager.cc:1403] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.371s	user 0.029s	sys 0.038s
I20260709 12:27:31.879433 26850 raft_consensus.cc:359] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.861718 27204 raft_consensus.cc:515] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.882335 27204 leader_election.cc:290] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:31.861243 26856 tablet_bootstrap.cc:654] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.873340 26849 consensus_queue.cc:260] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.901749 27203 raft_consensus.cc:493] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.902460 27203 raft_consensus.cc:515] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.907472 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:31.906211 27203 leader_election.cc:290] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:31.908792 26849 ts_tablet_manager.cc:1434] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.159s	user 0.006s	sys 0.001s
I20260709 12:27:31.916951 26849 tablet_bootstrap.cc:492] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
W20260709 12:27:31.917101 26620 leader_election.cc:343] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:31.910346 26850 raft_consensus.cc:385] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.920542 26850 raft_consensus.cc:740] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
W20260709 12:27:31.926461 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 166069us
I20260709 12:27:31.928745 26850 consensus_queue.cc:260] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:31.938931 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:31.939946 26580 raft_consensus.cc:2468] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:31.942766 26621 leader_election.cc:304] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:31.946038 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:31.952701 26853 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:31.953387 26853 ts_tablet_manager.cc:1403] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.323s	user 0.023s	sys 0.008s
W20260709 12:27:31.955763 26769 leader_election.cc:343] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:31.958115 26853 raft_consensus.cc:359] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.959719 26853 raft_consensus.cc:385] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.960054 26853 raft_consensus.cc:740] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:31.960763 26853 consensus_queue.cc:260] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.962505 27102 raft_consensus.cc:2804] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:31.962981 27102 raft_consensus.cc:493] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.963373 27102 raft_consensus.cc:3060] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.966226 26729 raft_consensus.cc:3060] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:31.948872 26655 raft_consensus.cc:2468] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:31.973735 26849 tablet_bootstrap.cc:654] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:31.982614 26856 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:31.983309 26856 ts_tablet_manager.cc:1403] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.244s	user 0.016s	sys 0.004s
I20260709 12:27:31.986749 26856 raft_consensus.cc:359] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:31.987557 26856 raft_consensus.cc:385] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:31.987852 26856 raft_consensus.cc:740] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
W20260709 12:27:31.988988 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 46325us
I20260709 12:27:31.991961 26950 consensus_queue.cc:1048] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:31.996044 26695 leader_election.cc:304] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:31.998478 27190 raft_consensus.cc:2804] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:31.999042 27190 raft_consensus.cc:697] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:31.998688 26476 catalog_manager.cc:5697] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:31.999212 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:31.999953 27190 consensus_queue.cc:237] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.000825 26850 ts_tablet_manager.cc:1434] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.124s	user 0.008s	sys 0.000s
I20260709 12:27:32.003677 26850 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:32.010085 27118 consensus_queue.cc:1048] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:32.012019 26546 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 55019.3us
I20260709 12:27:32.023742 26478 catalog_manager.cc:5697] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:32.045075 27102 raft_consensus.cc:515] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.048731 26853 ts_tablet_manager.cc:1434] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.095s	user 0.008s	sys 0.000s
I20260709 12:27:32.061180 26856 consensus_queue.cc:260] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.066617 26853 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:32.066426 26950 consensus_queue.cc:1048] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:32.073561 26856 ts_tablet_manager.cc:1434] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.000s
I20260709 12:27:32.083688 26850 tablet_bootstrap.cc:654] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.085530 27102 leader_election.cc:290] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:32.089425 26479 catalog_manager.cc:5697] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:32.093554 26856 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:32.096160 26970 consensus_queue.cc:1048] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:32.102392 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:32.112941 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
W20260709 12:27:32.114475 26768 leader_election.cc:343] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.115262 26768 leader_election.cc:304] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:32.117972 26581 raft_consensus.cc:3060] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.156759 26849 tablet_bootstrap.cc:492] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:32.157495 26849 ts_tablet_manager.cc:1403] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.241s	user 0.032s	sys 0.058s
I20260709 12:27:32.124929 27126 raft_consensus.cc:2749] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:32.157732 26620 leader_election.cc:343] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:32.174171 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:32.210225 26856 tablet_bootstrap.cc:654] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.221102 26581 raft_consensus.cc:2468] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:32.226704 26853 tablet_bootstrap.cc:654] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.229954 27206 consensus_queue.cc:1048] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:32.233156 26849 raft_consensus.cc:359] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.234232 26849 raft_consensus.cc:385] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.234633 26849 raft_consensus.cc:740] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.236212 26849 consensus_queue.cc:260] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.241844 27066 raft_consensus.cc:493] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.243018 26849 ts_tablet_manager.cc:1434] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.004s
I20260709 12:27:32.242488 27066 raft_consensus.cc:515] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.256335 26849 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:32.252144 26621 leader_election.cc:304] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:32.259282 27204 raft_consensus.cc:2804] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:32.259856 27204 raft_consensus.cc:697] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:32.260808 27204 consensus_queue.cc:237] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.264115 26850 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:32.261214 27066 leader_election.cc:290] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:32.264688 26850 ts_tablet_manager.cc:1403] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.262s	user 0.013s	sys 0.053s
I20260709 12:27:32.299922 27205 raft_consensus.cc:493] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.300556 27205 raft_consensus.cc:515] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.303879 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293707805bab498a9449cd938d3b1b6b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:32.305302 26856 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:32.305840 26856 ts_tablet_manager.cc:1403] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.213s	user 0.016s	sys 0.004s
I20260709 12:27:32.326511 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293707805bab498a9449cd938d3b1b6b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:32.343014 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f355a7f12b14432844a7a4508117120" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:32.344180 26582 raft_consensus.cc:2468] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:32.346310 26850 raft_consensus.cc:359] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.345503 26856 raft_consensus.cc:359] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.347390 26850 raft_consensus.cc:385] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.347594 26856 raft_consensus.cc:385] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.348112 26856 raft_consensus.cc:740] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.351877 26621 leader_election.cc:304] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:32.347880 26850 raft_consensus.cc:740] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.352856 26950 raft_consensus.cc:2804] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:32.353228 26950 raft_consensus.cc:493] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.353677 26950 raft_consensus.cc:3060] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.353142 26850 consensus_queue.cc:260] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.362407 27205 leader_election.cc:290] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:32.340229 26800 raft_consensus.cc:2468] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:32.365439 26849 tablet_bootstrap.cc:654] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:32.367074 26695 leader_election.cc:343] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.368554 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f355a7f12b14432844a7a4508117120" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:32.368633 26695 leader_election.cc:304] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:32.370632 27191 raft_consensus.cc:2804] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:32.371192 27191 raft_consensus.cc:493] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.371646 27191 raft_consensus.cc:3060] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.403923 26856 consensus_queue.cc:260] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.414781 26850 ts_tablet_manager.cc:1434] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.150s	user 0.008s	sys 0.004s
I20260709 12:27:32.415014 26950 raft_consensus.cc:515] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.426641 26850 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:32.427176 26856 ts_tablet_manager.cc:1434] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.121s	user 0.015s	sys 0.000s
I20260709 12:27:32.431344 26950 leader_election.cc:290] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:32.420228 26731 raft_consensus.cc:2468] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:32.436995 26856 tablet_bootstrap.cc:492] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:32.450450 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f355a7f12b14432844a7a4508117120" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:32.430887 27102 consensus_queue.cc:1048] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:27:32.431461 27118 raft_consensus.cc:493] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.458354 27118 raft_consensus.cc:515] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.464042 27118 leader_election.cc:290] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:32.470288 27203 consensus_queue.cc:1048] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:32.476830 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:32.486626 26802 raft_consensus.cc:2468] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:32.492877 26849 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:32.494680 26849 ts_tablet_manager.cc:1403] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.239s	user 0.021s	sys 0.017s
I20260709 12:27:32.500026 26478 catalog_manager.cc:5697] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:32.507021 26853 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:32.507795 26853 ts_tablet_manager.cc:1403] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.458s	user 0.021s	sys 0.010s
I20260709 12:27:32.522851 26850 tablet_bootstrap.cc:654] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.526109 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:32.528666 26545 leader_election.cc:343] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.533648 26856 tablet_bootstrap.cc:654] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.544899 26546 leader_election.cc:304] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:32.547870 27114 raft_consensus.cc:2804] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:32.548341 27114 raft_consensus.cc:493] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.548691 27114 raft_consensus.cc:3060] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.552196 26853 raft_consensus.cc:359] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.553020 26853 raft_consensus.cc:385] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.553303 26853 raft_consensus.cc:740] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.553980 26853 consensus_queue.cc:260] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.554296 26849 raft_consensus.cc:359] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.558121 26849 raft_consensus.cc:385] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.558701 26849 raft_consensus.cc:740] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.559432 26849 consensus_queue.cc:260] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.536599 27191 raft_consensus.cc:515] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.562074 26849 ts_tablet_manager.cc:1434] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.000s
I20260709 12:27:32.563341 26849 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:32.564383 27191 leader_election.cc:290] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:32.575898 27206 raft_consensus.cc:493] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.580713 26853 ts_tablet_manager.cc:1434] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.073s	user 0.010s	sys 0.003s
I20260709 12:27:32.596148 26853 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:32.576550 27206 raft_consensus.cc:515] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.590772 27066 consensus_queue.cc:1048] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:27:32.638031 26850 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:32.638948 26850 ts_tablet_manager.cc:1403] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.222s	user 0.012s	sys 0.011s
I20260709 12:27:32.647219 27205 raft_consensus.cc:493] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.647908 27205 raft_consensus.cc:515] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.654389 27206 leader_election.cc:290] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:32.657414 26853 tablet_bootstrap.cc:654] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.658577 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8437e96f55e2448e883f6f8bcd057be2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:32.659456 26802 raft_consensus.cc:2468] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:32.666674 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ceb8aabf0a94f9385121858268a3169" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:32.666823 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f355a7f12b14432844a7a4508117120" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:32.671391 26581 raft_consensus.cc:3060] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.672761 26856 tablet_bootstrap.cc:492] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:32.666064 26850 raft_consensus.cc:359] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.673277 26850 raft_consensus.cc:385] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.673362 26856 ts_tablet_manager.cc:1403] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.237s	user 0.031s	sys 0.011s
I20260709 12:27:32.662168 26849 tablet_bootstrap.cc:654] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.673599 26850 raft_consensus.cc:740] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.684659 26856 raft_consensus.cc:359] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.685549 26856 raft_consensus.cc:385] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.685850 26856 raft_consensus.cc:740] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
W20260709 12:27:32.686743 26768 leader_election.cc:343] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.686658 26856 consensus_queue.cc:260] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.688794 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293707805bab498a9449cd938d3b1b6b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:32.689412 26805 raft_consensus.cc:3060] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.682061 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ceb8aabf0a94f9385121858268a3169" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:32.696357 27205 leader_election.cc:290] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:32.699967 26805 raft_consensus.cc:2468] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:32.703162 26695 leader_election.cc:304] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:32.704067 27066 raft_consensus.cc:2804] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:32.704574 27066 raft_consensus.cc:697] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:32.705495 27066 consensus_queue.cc:237] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.706200 26621 leader_election.cc:304] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:32.707607 27204 raft_consensus.cc:2804] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:32.708106 27204 raft_consensus.cc:493] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.708536 27204 raft_consensus.cc:3060] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.711001 27193 consensus_queue.cc:1048] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:27:32.711817 26768 leader_election.cc:343] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.703361 26856 ts_tablet_manager.cc:1434] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.005s
I20260709 12:27:32.712311 26768 leader_election.cc:304] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:32.713403 26856 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
W20260709 12:27:32.711966 26620 consensus_peers.cc:597] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:32.713974 27116 raft_consensus.cc:2749] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:32.699644 27114 raft_consensus.cc:515] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.733791 26856 tablet_bootstrap.cc:654] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.726632 26581 raft_consensus.cc:2468] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:32.726163 26850 consensus_queue.cc:260] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.755096 26850 ts_tablet_manager.cc:1434] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.000s
I20260709 12:27:32.756381 26850 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:32.756213 27114 leader_election.cc:290] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:32.780687 27204 raft_consensus.cc:515] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.783185 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
W20260709 12:27:32.784853 26545 leader_election.cc:343] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.786727 26621 leader_election.cc:304] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:32.788909 26950 consensus_queue.cc:1048] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:32.790688 26950 raft_consensus.cc:2804] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:32.791940 26577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8437e96f55e2448e883f6f8bcd057be2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:32.794533 26850 tablet_bootstrap.cc:654] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.795214 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "293707805bab498a9449cd938d3b1b6b" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:32.796082 26582 raft_consensus.cc:3060] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:32.802212 26621 leader_election.cc:343] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.805063 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:32.803341 26950 raft_consensus.cc:697] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:32.809087 26950 consensus_queue.cc:237] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:32.811473 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8437e96f55e2448e883f6f8bcd057be2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:32.811846 26800 raft_consensus.cc:3060] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.812188 26805 raft_consensus.cc:3060] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:32.804893 27204 leader_election.cc:290] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:32.826143 26805 raft_consensus.cc:2468] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:32.834306 26849 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:32.842573 26849 ts_tablet_manager.cc:1403] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.279s	user 0.022s	sys 0.001s
I20260709 12:27:32.836215 26621 leader_election.cc:304] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:32.863727 26578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8437e96f55e2448e883f6f8bcd057be2" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:32.852938 27204 raft_consensus.cc:2804] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:32.867882 27204 raft_consensus.cc:697] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:32.869059 27204 consensus_queue.cc:237] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.871106 26800 raft_consensus.cc:2468] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:32.879070 26582 raft_consensus.cc:2468] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:32.887825 26546 leader_election.cc:304] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:32.889523 27169 raft_consensus.cc:2804] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:32.890076 27169 raft_consensus.cc:697] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:32.891048 27169 consensus_queue.cc:237] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:32.900365 26621 leader_election.cc:343] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:32.912966 27102 consensus_queue.cc:1048] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:32.863826 26849 raft_consensus.cc:359] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:32.924153 26849 raft_consensus.cc:385] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.924577 26849 raft_consensus.cc:740] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.942764 26476 catalog_manager.cc:5697] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:32.946152 26850 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:32.946843 26850 ts_tablet_manager.cc:1403] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.191s	user 0.009s	sys 0.039s
I20260709 12:27:32.946138 26853 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:32.949944 26853 ts_tablet_manager.cc:1403] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.354s	user 0.035s	sys 0.047s
I20260709 12:27:32.952615 26850 raft_consensus.cc:359] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.953402 26850 raft_consensus.cc:385] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.953882 26850 raft_consensus.cc:740] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.954716 26850 consensus_queue.cc:260] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.957821 26850 ts_tablet_manager.cc:1434] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:27:32.958933 26850 tablet_bootstrap.cc:492] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:32.965564 26850 tablet_bootstrap.cc:654] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.952935 26853 raft_consensus.cc:359] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.970592 26853 raft_consensus.cc:385] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.971019 26853 raft_consensus.cc:740] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.973392 26853 consensus_queue.cc:260] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:32.981257 26853 ts_tablet_manager.cc:1434] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:27:32.982508 26853 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:32.992965 26850 tablet_bootstrap.cc:492] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:32.993604 26850 ts_tablet_manager.cc:1403] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.004s
I20260709 12:27:32.997045 26850 raft_consensus.cc:359] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:32.997766 26850 raft_consensus.cc:385] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:32.998075 26850 raft_consensus.cc:740] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:32.998319 26853 tablet_bootstrap.cc:654] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:32.999027 26850 consensus_queue.cc:260] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.000171 26849 consensus_queue.cc:260] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.001770 26850 ts_tablet_manager.cc:1434] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:27:33.002197 26849 ts_tablet_manager.cc:1434] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.159s	user 0.006s	sys 0.001s
I20260709 12:27:33.003172 26850 tablet_bootstrap.cc:492] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:33.003194 26849 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:33.025725 26476 catalog_manager.cc:5697] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.027287 26856 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:33.027817 26856 ts_tablet_manager.cc:1403] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.315s	user 0.009s	sys 0.007s
I20260709 12:27:33.031041 26856 raft_consensus.cc:359] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.031706 26856 raft_consensus.cc:385] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.031981 26856 raft_consensus.cc:740] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.032611 26856 consensus_queue.cc:260] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.034839 26856 ts_tablet_manager.cc:1434] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:33.035836 26856 tablet_bootstrap.cc:492] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:33.038394 27206 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.039002 27206 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.051308 26849 tablet_bootstrap.cc:654] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.054972 26856 tablet_bootstrap.cc:654] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.055445 26850 tablet_bootstrap.cc:654] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.074390 26477 catalog_manager.cc:5697] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:33.083009 27206 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:33.084281 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:33.085482 26477 catalog_manager.cc:5697] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:33.094336 26768 leader_election.cc:343] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.102660 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:33.108136 26769 leader_election.cc:343] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.108676 26769 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:33.110543 27206 raft_consensus.cc:2749] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:33.116701 26856 tablet_bootstrap.cc:492] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:33.117362 26856 ts_tablet_manager.cc:1403] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.003s
I20260709 12:27:33.123461 27239 raft_consensus.cc:493] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.124969 27239 raft_consensus.cc:515] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.120416 26856 raft_consensus.cc:359] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.127871 26856 raft_consensus.cc:385] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.128237 26856 raft_consensus.cc:740] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.129027 26856 consensus_queue.cc:260] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.150669 26849 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:33.151297 26849 ts_tablet_manager.cc:1403] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.148s	user 0.019s	sys 0.005s
I20260709 12:27:33.156730 27239 leader_election.cc:290] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:33.157541 26849 raft_consensus.cc:359] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.158317 26849 raft_consensus.cc:385] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.158587 26849 raft_consensus.cc:740] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.159487 26849 consensus_queue.cc:260] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.168619 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.167753 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:33.171193 26950 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.172017 26950 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.177229 26849 ts_tablet_manager.cc:1434] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:27:33.179199 26695 leader_election.cc:304] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:33.182963 26950 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:33.183300 26856 ts_tablet_manager.cc:1434] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.066s	user 0.010s	sys 0.000s
I20260709 12:27:33.184499 26849 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:33.186304 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:33.182102 27190 raft_consensus.cc:2749] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:33.187705 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:33.187741 26621 leader_election.cc:343] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:33.190483 26621 leader_election.cc:343] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.190855 26621 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:33.191672 26950 raft_consensus.cc:2749] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:33.192760 26856 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:33.211203 26856 tablet_bootstrap.cc:654] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.192286 27239 raft_consensus.cc:493] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.213696 27239 raft_consensus.cc:515] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.223330 26853 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:33.224066 26853 ts_tablet_manager.cc:1403] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.242s	user 0.056s	sys 0.072s
I20260709 12:27:33.224557 27239 leader_election.cc:290] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:33.227428 26853 raft_consensus.cc:359] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.228262 26853 raft_consensus.cc:385] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.228534 26853 raft_consensus.cc:740] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.229238 26853 consensus_queue.cc:260] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.234488 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ceb8aabf0a94f9385121858268a3169" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:33.235998 26853 ts_tablet_manager.cc:1434] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:27:33.237272 26853 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:33.244277 26853 tablet_bootstrap.cc:654] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.235626 26802 raft_consensus.cc:2468] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:33.254340 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ceb8aabf0a94f9385121858268a3169" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.255509 26652 raft_consensus.cc:2468] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
W20260709 12:27:33.255893 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:33.234534 26849 tablet_bootstrap.cc:654] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.262617 26694 leader_election.cc:304] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:33.266517 27240 raft_consensus.cc:2804] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:33.267019 27240 raft_consensus.cc:493] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.267385 27240 raft_consensus.cc:3060] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.270744 26850 tablet_bootstrap.cc:492] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:33.271363 26850 ts_tablet_manager.cc:1403] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.268s	user 0.016s	sys 0.007s
I20260709 12:27:33.273460 26728 raft_consensus.cc:3060] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.285140 27240 raft_consensus.cc:515] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.287837 27240 leader_election.cc:290] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:33.299939 26850 raft_consensus.cc:359] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.301196 26850 raft_consensus.cc:385] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.301543 26850 raft_consensus.cc:740] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.302511 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ceb8aabf0a94f9385121858268a3169" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:33.302467 26850 consensus_queue.cc:260] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.303280 26656 raft_consensus.cc:3060] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.307925 27169 raft_consensus.cc:493] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.308879 26853 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:33.308501 27169 raft_consensus.cc:515] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.309448 26853 ts_tablet_manager.cc:1403] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.004s
I20260709 12:27:33.312602 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8437e96f55e2448e883f6f8bcd057be2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:33.312754 26853 raft_consensus.cc:359] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.313452 26853 raft_consensus.cc:385] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.313561 26805 raft_consensus.cc:2393] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 643057f6f77241529979f572a42ed9b0 in this term.
I20260709 12:27:33.313764 26853 raft_consensus.cc:740] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.314534 26853 consensus_queue.cc:260] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.316799 27169 leader_election.cc:290] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:33.336124 26850 ts_tablet_manager.cc:1434] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.000s
I20260709 12:27:33.331974 26656 raft_consensus.cc:2468] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:33.347995 26850 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:33.348261 26853 ts_tablet_manager.cc:1434] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.004s
I20260709 12:27:33.345057 27239 raft_consensus.cc:493] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.332347 27240 consensus_queue.cc:1048] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:27:33.358021 26856 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:33.339229 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ceb8aabf0a94f9385121858268a3169" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:33.364039 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8437e96f55e2448e883f6f8bcd057be2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.364295 26694 leader_election.cc:304] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:33.372707 26856 ts_tablet_manager.cc:1403] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.180s	user 0.026s	sys 0.034s
I20260709 12:27:33.373590 26802 raft_consensus.cc:3060] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.375895 26856 raft_consensus.cc:359] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.376641 26856 raft_consensus.cc:385] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.376842 27240 raft_consensus.cc:2804] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:33.377239 26856 raft_consensus.cc:740] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.377660 27240 raft_consensus.cc:697] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:33.378055 26856 consensus_queue.cc:260] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.381508 26950 consensus_queue.cc:1048] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:33.385385 26853 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:33.389297 26545 leader_election.cc:304] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:33.394712 26849 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:33.395465 26849 ts_tablet_manager.cc:1403] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.211s	user 0.026s	sys 0.033s
I20260709 12:27:33.395952 27169 raft_consensus.cc:3060] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.426414 27240 consensus_queue.cc:237] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.428810 26850 tablet_bootstrap.cc:654] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.380178 27066 consensus_queue.cc:1048] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:27:33.433899 27206 raft_consensus.cc:493] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.434605 27206 raft_consensus.cc:515] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.435145 26849 raft_consensus.cc:359] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.437330 26849 raft_consensus.cc:385] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.437693 26849 raft_consensus.cc:740] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.378978 27239 raft_consensus.cc:515] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.441457 26849 consensus_queue.cc:260] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.450513 26950 consensus_queue.cc:1048] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:33.453560 26694 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 70087.8us
W20260709 12:27:33.463064 26545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 62131.5us
I20260709 12:27:33.467088 26849 ts_tablet_manager.cc:1434] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.002s
I20260709 12:27:33.449635 27197 raft_consensus.cc:493] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.489014 27206 leader_election.cc:290] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:33.476805 27197 raft_consensus.cc:515] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.492868 26849 tablet_bootstrap.cc:492] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:33.493237 27197 leader_election.cc:290] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:33.495424 27239 leader_election.cc:290] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:33.498243 26803 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d187e1b239c246cea5ae376f42a72d6a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:33.505335 26849 tablet_bootstrap.cc:654] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.506865 26853 tablet_bootstrap.cc:654] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:33.508550 26695 leader_election.cc:343] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.510555 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc7a25d1e78a4401b1347ed2f07e858d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:33.513288 26768 leader_election.cc:343] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:33.517431 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 70881.3us
I20260709 12:27:33.519526 26731 raft_consensus.cc:3060] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:33.534907 26545 consensus_peers.cc:597] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:33.537873 26856 ts_tablet_manager.cc:1434] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.165s	user 0.027s	sys 0.013s
I20260709 12:27:33.539338 26856 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:33.524219 27206 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.543926 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d187e1b239c246cea5ae376f42a72d6a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.544889 26656 raft_consensus.cc:2468] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:33.549147 26578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc7a25d1e78a4401b1347ed2f07e858d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:33.549963 26578 raft_consensus.cc:2468] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:33.568492 26769 leader_election.cc:304] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:33.532886 26802 raft_consensus.cc:2468] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:33.574147 27116 raft_consensus.cc:2804] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:33.602375 26694 leader_election.cc:304] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:33.603436 27116 raft_consensus.cc:493] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.611052 27116 raft_consensus.cc:3060] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.614357 26581 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e1d3b6a6a41bc81db8ef63b2cc50a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:33.615666 27263 log.cc:927] Time spent T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:27:33.553486 27206 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:33.626991 26621 leader_election.cc:343] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.628265 27114 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.628101 27190 raft_consensus.cc:2804] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:33.657600 27114 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.649679 27273 consensus_queue.cc:1048] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:27:33.660043 27114 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:33.662791 27206 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:33.663583 26850 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:33.664191 26850 ts_tablet_manager.cc:1403] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.317s	user 0.017s	sys 0.014s
I20260709 12:27:33.685842 26856 tablet_bootstrap.cc:654] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.652158 27190 raft_consensus.cc:493] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.690167 27190 raft_consensus.cc:3060] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.690640 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.690968 26850 raft_consensus.cc:359] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.691423 26656 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:33.691771 26850 raft_consensus.cc:385] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.692124 26850 raft_consensus.cc:740] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.693159 26545 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:33.694120 27213 raft_consensus.cc:2804] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:33.694567 27213 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.694941 27213 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.702225 27116 raft_consensus.cc:515] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.680088 27169 raft_consensus.cc:2749] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:33.700069 26850 consensus_queue.cc:260] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:33.722877 26769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 94977.5us
W20260709 12:27:33.727351 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 95932us
I20260709 12:27:33.710096 27261 raft_consensus.cc:493] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.741891 26850 ts_tablet_manager.cc:1434] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.003s
I20260709 12:27:33.743498 26850 tablet_bootstrap.cc:492] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:33.758436 26849 tablet_bootstrap.cc:492] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:33.759217 26849 ts_tablet_manager.cc:1403] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.288s	user 0.023s	sys 0.022s
I20260709 12:27:33.735004 27169 consensus_queue.cc:1048] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:27:33.758162 26575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:33.751631 27261 raft_consensus.cc:515] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.798929 26657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.799875 26657 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:33.800182 26849 raft_consensus.cc:359] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.800958 26849 raft_consensus.cc:385] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.801268 26849 raft_consensus.cc:740] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.802006 26849 consensus_queue.cc:260] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:33.831243 26849 ts_tablet_manager.cc:1434] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
I20260709 12:27:33.837651 26578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc7a25d1e78a4401b1347ed2f07e858d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:33.838476 26578 raft_consensus.cc:3060] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.843145 26850 tablet_bootstrap.cc:654] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:33.829460 27116 leader_election.cc:290] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:33.846240 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc7a25d1e78a4401b1347ed2f07e858d" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:33.854007 26853 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:33.854938 26853 ts_tablet_manager.cc:1403] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.474s	user 0.027s	sys 0.007s
I20260709 12:27:33.854570 27190 raft_consensus.cc:515] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
W20260709 12:27:33.857457 26768 leader_election.cc:343] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.859674 27190 leader_election.cc:290] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:33.863972 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c629cb845a5247089d3810e037fe2035" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:33.868297 26654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d187e1b239c246cea5ae376f42a72d6a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:33.869208 26654 raft_consensus.cc:3060] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.877313 26849 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:33.865190 26798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d187e1b239c246cea5ae376f42a72d6a" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:33.867122 27213 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:33.889916 26574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c629cb845a5247089d3810e037fe2035" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:33.892460 27213 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:33.868300 26656 raft_consensus.cc:2468] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:33.864341 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e1d3b6a6a41bc81db8ef63b2cc50a" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:33.886005 27273 consensus_queue.cc:1048] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:27:33.862105 27261 leader_election.cc:290] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:33.894233 26575 raft_consensus.cc:2393] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in current term 1: Already voted for candidate cb56388a4ff14b5a9f8f1196dc94461f in this term.
I20260709 12:27:33.896483 26798 raft_consensus.cc:3060] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:33.897596 26621 leader_election.cc:343] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:33.898130 26621 leader_election.cc:304] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:33.907754 26694 leader_election.cc:304] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:33.914577 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:33.915565 26656 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.925423 27265 raft_consensus.cc:2804] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:33.925981 27265 raft_consensus.cc:493] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.926357 27265 raft_consensus.cc:3060] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:33.906862 27273 raft_consensus.cc:2749] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:33.920943 26768 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:33.949656 26856 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:33.950500 26856 ts_tablet_manager.cc:1403] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.411s	user 0.035s	sys 0.013s
I20260709 12:27:33.954624 26856 raft_consensus.cc:359] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.955992 26856 raft_consensus.cc:385] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:33.957050 26856 raft_consensus.cc:740] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:33.958195 26856 consensus_queue.cc:260] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:33.989163 26856 ts_tablet_manager.cc:1434] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:27:33.990375 26856 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:33.949949 27265 raft_consensus.cc:515] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:33.967141 27206 raft_consensus.cc:2804] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:34.015196 26853 raft_consensus.cc:359] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:34.016224 26853 raft_consensus.cc:385] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.023753 26656 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:34.016530 26853 raft_consensus.cc:740] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.033665 26853 consensus_queue.cc:260] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:34.033529 26545 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:34.026378 27206 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.051970 27206 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.036830 26578 raft_consensus.cc:2468] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:34.073784 26849 tablet_bootstrap.cc:654] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.073117 27114 raft_consensus.cc:2804] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:27:34.114663 26769 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 159538us
I20260709 12:27:34.122650 27114 raft_consensus.cc:697] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:34.123088 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
W20260709 12:27:34.124563 27291 log.cc:927] Time spent T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:27:34.124276 27114 consensus_queue.cc:237] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.130286 26654 raft_consensus.cc:2468] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:34.130319 26769 leader_election.cc:304] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:34.105434 26798 raft_consensus.cc:2468] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:34.148772 27265 leader_election.cc:290] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:34.161693 27286 raft_consensus.cc:2804] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:34.158165 26856 tablet_bootstrap.cc:654] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.164963 27261 consensus_queue.cc:1048] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:27:34.188403 26695 leader_election.cc:304] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:34.183316 27286 raft_consensus.cc:697] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:34.203275 26853 ts_tablet_manager.cc:1434] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.348s	user 0.012s	sys 0.003s
I20260709 12:27:34.199117 27190 raft_consensus.cc:2804] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:34.209921 27286 consensus_queue.cc:237] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.210878 27190 raft_consensus.cc:697] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:34.211911 27190 consensus_queue.cc:237] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:34.216890 26850 tablet_bootstrap.cc:492] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:34.217542 26850 ts_tablet_manager.cc:1403] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.474s	user 0.020s	sys 0.032s
I20260709 12:27:34.219650 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:34.213179 26476 catalog_manager.cc:5697] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:34.221643 26850 raft_consensus.cc:359] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:34.228365 27206 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.229722 26800 raft_consensus.cc:2393] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cb56388a4ff14b5a9f8f1196dc94461f in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:34.231797 26574 raft_consensus.cc:2468] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:34.222404 26850 raft_consensus.cc:385] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.246300 26853 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:34.243567 26850 raft_consensus.cc:740] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.262778 26850 consensus_queue.cc:260] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:34.265703 27261 raft_consensus.cc:493] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.266566 27261 raft_consensus.cc:515] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.269429 27261 leader_election.cc:290] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:34.270634 27284 consensus_queue.cc:1048] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:34.270879 26850 ts_tablet_manager.cc:1434] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.003s
I20260709 12:27:34.275174 26853 tablet_bootstrap.cc:654] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.280627 26850 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:34.312376 27206 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:34.308470 26798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc7a25d1e78a4401b1347ed2f07e858d" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:34.486119 26849 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:34.498879 26654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c629cb845a5247089d3810e037fe2035" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
W20260709 12:27:34.491745 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 157519us
I20260709 12:27:34.497853 26574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c629cb845a5247089d3810e037fe2035" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:34.506970 26574 raft_consensus.cc:3060] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.507205 26654 raft_consensus.cc:3060] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:34.486605 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 102630us
I20260709 12:27:34.497517 26856 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:34.524225 26856 ts_tablet_manager.cc:1403] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.534s	user 0.013s	sys 0.010s
I20260709 12:27:34.528889 26575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc7a25d1e78a4401b1347ed2f07e858d" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:34.529162 26545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 254778us
I20260709 12:27:34.529949 26575 raft_consensus.cc:2393] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in this term.
I20260709 12:27:34.537760 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:34.538968 26652 raft_consensus.cc:2393] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in current term 1: Already voted for candidate cb56388a4ff14b5a9f8f1196dc94461f in this term.
I20260709 12:27:34.506096 26849 ts_tablet_manager.cc:1403] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.629s	user 0.027s	sys 0.011s
I20260709 12:27:34.561874 26695 leader_election.cc:304] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:34.563154 27066 raft_consensus.cc:3060] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:34.576049 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:34.571262 26856 raft_consensus.cc:359] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.579687 26856 raft_consensus.cc:385] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.580125 26856 raft_consensus.cc:740] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.581029 26856 consensus_queue.cc:260] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.588313 26850 tablet_bootstrap.cc:700] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: Time spent opening tablet: real 0.103s	user 0.000s	sys 0.004s
I20260709 12:27:34.598551 26853 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:34.599253 26853 ts_tablet_manager.cc:1403] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.394s	user 0.013s	sys 0.013s
I20260709 12:27:34.601496 26654 raft_consensus.cc:2468] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:34.602471 26853 raft_consensus.cc:359] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.603235 26853 raft_consensus.cc:385] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.603505 26853 raft_consensus.cc:740] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.604995 26853 consensus_queue.cc:260] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.608561 26856 ts_tablet_manager.cc:1434] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 12:27:34.609530 26850 tablet_bootstrap.cc:654] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.609726 26856 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:34.612082 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:34.614918 27066 raft_consensus.cc:2749] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:34.616991 26849 raft_consensus.cc:359] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.617772 26849 raft_consensus.cc:385] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.618345 26725 raft_consensus.cc:3060] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.618047 26849 raft_consensus.cc:740] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.625242 26694 leader_election.cc:304] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:34.623023 26849 consensus_queue.cc:260] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.629597 26853 ts_tablet_manager.cc:1434] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 12:27:34.634745 27262 raft_consensus.cc:2804] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:34.635386 27262 raft_consensus.cc:697] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:34.635896 26849 ts_tablet_manager.cc:1434] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.077s	user 0.002s	sys 0.004s
I20260709 12:27:34.636303 27262 consensus_queue.cc:237] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:34.649986 26853 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:34.647240 26574 raft_consensus.cc:2468] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:34.672708 26769 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:34.671458 26849 tablet_bootstrap.cc:492] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:34.696322 27116 raft_consensus.cc:2749] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:34.682493 27279 consensus_queue.cc:1048] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:27:34.709658 26856 tablet_bootstrap.cc:654] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:34.730585 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 47510.2us
I20260709 12:27:34.731607 27313 consensus_queue.cc:1048] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:34.768569 26849 tablet_bootstrap.cc:654] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.770606 26853 tablet_bootstrap.cc:654] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.776639 26850 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:34.777210 26850 ts_tablet_manager.cc:1403] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.500s	user 0.009s	sys 0.022s
I20260709 12:27:34.777591 26856 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:34.778448 26856 ts_tablet_manager.cc:1403] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.169s	user 0.012s	sys 0.001s
W20260709 12:27:34.785595 26694 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 44930.9us
I20260709 12:27:34.788187 27261 consensus_queue.cc:1048] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:34.792080 26850 raft_consensus.cc:359] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:34.792891 26850 raft_consensus.cc:385] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.793186 26850 raft_consensus.cc:740] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.793926 26850 consensus_queue.cc:260] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:34.796386 26850 ts_tablet_manager.cc:1434] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 12:27:34.797407 26850 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0: Bootstrap starting.
I20260709 12:27:34.799993 27304 raft_consensus.cc:493] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.800729 27304 raft_consensus.cc:515] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:34.804436 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f229d58a8c304ca1b723a4954aed76f4" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:34.806180 26620 leader_election.cc:343] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:34.807511 26856 raft_consensus.cc:359] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.808501 26856 raft_consensus.cc:385] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.808743 27304 leader_election.cc:290] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:34.814632 26850 tablet_bootstrap.cc:654] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:34.815651 27296 raft_consensus.cc:493] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.816278 27296 raft_consensus.cc:515] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.818753 27296 leader_election.cc:290] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:34.822660 27315 consensus_queue.cc:1048] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:34.824653 27280 consensus_queue.cc:1048] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:34.827790 26850 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0: No bootstrap required, opened a new log
I20260709 12:27:34.837387 26850 ts_tablet_manager.cc:1403] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.003s
I20260709 12:27:34.840790 26850 raft_consensus.cc:359] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.841585 26850 raft_consensus.cc:385] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.841878 26850 raft_consensus.cc:740] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.852375 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e1d3b6a6a41bc81db8ef63b2cc50a" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:34.854449 26850 consensus_queue.cc:260] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.855545 27197 raft_consensus.cc:493] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.856034 27197 raft_consensus.cc:515] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.859999 27197 leader_election.cc:290] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:34.858029 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f229d58a8c304ca1b723a4954aed76f4" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:34.862468 26580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2e5dc8624444d368676b1a1b3fa878e" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
W20260709 12:27:34.864212 26621 leader_election.cc:343] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:34.866009 26850 ts_tablet_manager.cc:1434] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.002s
I20260709 12:27:34.878839 27313 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.879635 27313 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.883579 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:34.885668 27313 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:34.808869 26856 raft_consensus.cc:740] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.892736 26856 consensus_queue.cc:260] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.893959 26725 raft_consensus.cc:2468] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:34.894995 26856 ts_tablet_manager.cc:1434] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.116s	user 0.008s	sys 0.000s
I20260709 12:27:34.896519 26856 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:34.898227 26545 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:34.899077 26653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e1d3b6a6a41bc81db8ef63b2cc50a" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:34.899312 27313 raft_consensus.cc:2804] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:27:34.899478 26621 leader_election.cc:343] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:34.899837 26653 raft_consensus.cc:2468] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:34.899940 27313 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.900007 26621 leader_election.cc:304] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:34.900628 27313 raft_consensus.cc:3060] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.902153 27297 raft_consensus.cc:2749] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:27:34.909482 26769 leader_election.cc:343] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:34.915508 27313 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.919005 26731 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:34.919920 26731 raft_consensus.cc:3060] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.922088 26853 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:34.925247 26853 ts_tablet_manager.cc:1403] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.276s	user 0.049s	sys 0.049s
I20260709 12:27:34.926759 26768 leader_election.cc:304] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:34.927508 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:34.935669 26802 raft_consensus.cc:2468] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:34.927416 27190 consensus_queue.cc:1048] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:27:34.933051 27280 raft_consensus.cc:2804] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:34.939932 27280 raft_consensus.cc:493] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.940340 27280 raft_consensus.cc:3060] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.948014 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2e5dc8624444d368676b1a1b3fa878e" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:34.952646 26849 tablet_bootstrap.cc:492] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:34.954704 26849 ts_tablet_manager.cc:1403] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.284s	user 0.016s	sys 0.002s
I20260709 12:27:34.957207 26476 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43906 (request call id 75) took 1196 ms. Trace:
I20260709 12:27:34.962205 26853 raft_consensus.cc:359] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.963145 26853 raft_consensus.cc:385] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:34.963464 26853 raft_consensus.cc:740] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:34.964223 26853 consensus_queue.cc:260] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.965173 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:34.965978 26799 raft_consensus.cc:3060] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:34.969512 26853 ts_tablet_manager.cc:1434] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.001s
I20260709 12:27:34.971858 26731 raft_consensus.cc:2468] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:34.973647 26545 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:34.973634 27280 raft_consensus.cc:515] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:34.974596 27313 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:34.974982 27306 raft_consensus.cc:2804] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:34.975543 27306 raft_consensus.cc:697] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:34.976645 27306 consensus_queue.cc:237] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:34.957736 26476 rpcz_store.cc:276] 0709 12:27:33.760894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.761144 (+   250us) service_pool.cc:224] Handling call
0709 12:27:34.957098 (+1195954us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:34.492412 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:34.493184 (+   772us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:34.493241 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:27:34.493265 (+    24us) tablet.cc:662] Decoding operations
0709 12:27:34.493806 (+   541us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:34.493894 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:34.493938 (+    44us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:27:34.494196 (+   258us) tablet.cc:701] Row locks acquired
0709 12:27:34.494234 (+    38us) write_op.cc:260] PREPARE: finished
0709 12:27:34.586805 (+ 92571us) write_op.cc:270] Start()
0709 12:27:34.612824 (+ 26019us) write_op.cc:276] Timestamp: P: 1783600054584017 usec, L: 0
0709 12:27:34.612848 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:34.619662 (+  6814us) log.cc:844] Serialized 537 byte log entry
0709 12:27:34.801353 (+181691us) op_driver.cc:464] REPLICATION: finished
0709 12:27:34.921701 (+120348us) write_op.cc:301] APPLY: starting
0709 12:27:34.921814 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:34.921942 (+   128us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:34.922069 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:34.922354 (+   285us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:34.922539 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:27:34.922598 (+    59us) write_op.cc:312] APPLY: finished
0709 12:27:34.927520 (+  4922us) log.cc:844] Serialized 22 byte log entry
0709 12:27:34.933535 (+  6015us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:34.949899 (+ 16364us) write_op.cc:454] Released schema lock
0709 12:27:34.949955 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:34.951092 (+  1137us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:34.951127 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":120055,"apply.run_cpu_time_us":2130,"apply.run_wall_time_us":29411,"num_ops":1,"prepare.queue_time_us":256163,"prepare.run_cpu_time_us":47700,"prepare.run_wall_time_us":296072,"raft.queue_time_us":17356,"raft.run_cpu_time_us":64888,"raft.run_wall_time_us":120133,"replication_time_us":182350,"thread_start_us":403812,"threads_started":4,"wal-append.queue_time_us":162058}]]}
I20260709 12:27:34.972841 26478 catalog_manager.cc:5697] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:34.971517 26853 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:34.993455 26849 raft_consensus.cc:359] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
W20260709 12:27:34.998133 26621 leader_election.cc:343] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:35.015956 26849 raft_consensus.cc:385] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.016268 26621 leader_election.cc:304] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:35.017715 26950 raft_consensus.cc:2749] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:34.994179 27280 leader_election.cc:290] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:34.999967 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e1d3b6a6a41bc81db8ef63b2cc50a" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:35.007593 26853 tablet_bootstrap.cc:654] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.020502 26582 raft_consensus.cc:3060] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.027334 26653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d4e1d3b6a6a41bc81db8ef63b2cc50a" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:35.028157 26653 raft_consensus.cc:3060] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.034462 26582 raft_consensus.cc:2468] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:35.038661 26653 raft_consensus.cc:2468] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:35.039227 26769 leader_election.cc:304] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:35.016364 26849 raft_consensus.cc:740] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.041395 27286 raft_consensus.cc:2804] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:35.040309 26477 catalog_manager.cc:5697] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } }
I20260709 12:27:35.041903 27286 raft_consensus.cc:697] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:35.042444 26477 catalog_manager.cc:5697] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } }
I20260709 12:27:35.042882 27286 consensus_queue.cc:237] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.007543 26856 tablet_bootstrap.cc:654] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.062542 26799 raft_consensus.cc:2468] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:35.063299 26849 consensus_queue.cc:260] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:35.069763 26849 ts_tablet_manager.cc:1434] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.115s	user 0.006s	sys 0.001s
I20260709 12:27:35.086187 26849 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:35.105573 26478 catalog_manager.cc:5697] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 0 to 1, leader changed from <none> to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.136808 26477 catalog_manager.cc:5697] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.140844 26849 tablet_bootstrap.cc:654] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.179612 26853 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:35.180253 26853 ts_tablet_manager.cc:1403] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.210s	user 0.022s	sys 0.015s
I20260709 12:27:35.189591 26856 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:35.190191 26856 ts_tablet_manager.cc:1403] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.294s	user 0.025s	sys 0.013s
I20260709 12:27:35.193220 26856 raft_consensus.cc:359] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.193924 26856 raft_consensus.cc:385] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.194190 26856 raft_consensus.cc:740] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.195170 26856 consensus_queue.cc:260] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.199126 26853 raft_consensus.cc:359] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.199935 26853 raft_consensus.cc:385] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.200207 26853 raft_consensus.cc:740] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.202906 26856 ts_tablet_manager.cc:1434] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:27:35.203933 26856 tablet_bootstrap.cc:492] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:35.186360 26477 catalog_manager.cc:5697] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:35.211174 27262 consensus_queue.cc:1048] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:35.213634 26853 consensus_queue.cc:260] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.216583 26853 ts_tablet_manager.cc:1434] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:27:35.218042 26853 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:35.223286 27297 raft_consensus.cc:493] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.223899 27297 raft_consensus.cc:515] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.230386 26853 tablet_bootstrap.cc:654] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.231091 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673d53dde2dc42e1b7feb31f4e28bb3c" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:35.231969 26579 raft_consensus.cc:2468] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:35.234028 26726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673d53dde2dc42e1b7feb31f4e28bb3c" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:35.234660 26621 leader_election.cc:304] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:35.234895 26726 raft_consensus.cc:2468] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 0.
I20260709 12:27:35.235569 26950 raft_consensus.cc:2804] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:35.235950 26950 raft_consensus.cc:493] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.236260 26950 raft_consensus.cc:3060] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.238446 27297 leader_election.cc:290] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:35.240640 26856 tablet_bootstrap.cc:654] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.254382 26849 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:35.255075 26849 ts_tablet_manager.cc:1403] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.184s	user 0.015s	sys 0.000s
I20260709 12:27:35.272328 27261 consensus_queue.cc:1048] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:35.260821 26849 raft_consensus.cc:359] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.276682 26849 raft_consensus.cc:385] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.277128 26849 raft_consensus.cc:740] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.252419 26950 raft_consensus.cc:515] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.279671 26849 consensus_queue.cc:260] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.286664 26950 leader_election.cc:290] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:35.287493 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673d53dde2dc42e1b7feb31f4e28bb3c" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:35.288175 26579 raft_consensus.cc:3060] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.291941 26726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "673d53dde2dc42e1b7feb31f4e28bb3c" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:35.292635 26726 raft_consensus.cc:3060] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.307662 26856 tablet_bootstrap.cc:492] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:35.308347 26856 ts_tablet_manager.cc:1403] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.020s
I20260709 12:27:35.309093 26853 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:35.309638 26853 ts_tablet_manager.cc:1403] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.092s	user 0.013s	sys 0.001s
I20260709 12:27:35.311668 26856 raft_consensus.cc:359] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:35.312470 26856 raft_consensus.cc:385] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.312750 26856 raft_consensus.cc:740] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.321144 26853 raft_consensus.cc:359] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.322007 26853 raft_consensus.cc:385] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.322755 26849 ts_tablet_manager.cc:1434] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.000s
I20260709 12:27:35.322324 26853 raft_consensus.cc:740] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.327380 26853 consensus_queue.cc:260] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.323062 26856 consensus_queue.cc:260] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:35.341029 26856 ts_tablet_manager.cc:1434] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:27:35.341513 26726 raft_consensus.cc:2468] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:35.342080 26856 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: Bootstrap starting.
I20260709 12:27:35.350409 26849 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa: Bootstrap starting.
I20260709 12:27:35.353722 26853 ts_tablet_manager.cc:1434] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:27:35.344422 26579 raft_consensus.cc:2468] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:35.359752 26620 leader_election.cc:304] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:35.360867 27304 raft_consensus.cc:2804] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:35.361382 27304 raft_consensus.cc:697] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:35.363816 26856 tablet_bootstrap.cc:654] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.363529 27304 consensus_queue.cc:237] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:35.354971 26853 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:35.392844 27296 raft_consensus.cc:493] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.393404 27296 raft_consensus.cc:515] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:35.394058 26849 tablet_bootstrap.cc:654] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.398021 26726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f229d58a8c304ca1b723a4954aed76f4" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:35.407006 26726 raft_consensus.cc:2468] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:35.398826 26653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f229d58a8c304ca1b723a4954aed76f4" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:35.408500 26653 raft_consensus.cc:2468] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:35.396574 27296 leader_election.cc:290] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:35.411120 26768 leader_election.cc:304] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:35.390601 26853 tablet_bootstrap.cc:654] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:35.416452 27306 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.416992 27306 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.417610 26856 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: No bootstrap required, opened a new log
I20260709 12:27:35.418154 26856 ts_tablet_manager.cc:1403] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: Time spent bootstrapping tablet: real 0.076s	user 0.025s	sys 0.020s
I20260709 12:27:35.419299 27306 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:35.424057 26726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:35.427585 26545 leader_election.cc:343] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:35.429836 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:35.432300 26801 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 0.
I20260709 12:27:35.434059 26546 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: f57551729f2f4037a4c769ee1df81853
I20260709 12:27:35.434973 27313 raft_consensus.cc:2804] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:35.435380 27313 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.433059 26856 raft_consensus.cc:359] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.435755 27313 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.436020 26856 raft_consensus.cc:385] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.436601 26856 raft_consensus.cc:740] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.437788 26856 consensus_queue.cc:260] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.421631 27296 raft_consensus.cc:2804] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:35.441772 27296 raft_consensus.cc:493] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.442669 27296 raft_consensus.cc:3060] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.455379 26856 ts_tablet_manager.cc:1434] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:27:35.466444 26849 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa: No bootstrap required, opened a new log
I20260709 12:27:35.467197 26849 ts_tablet_manager.cc:1403] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent bootstrapping tablet: real 0.144s	user 0.010s	sys 0.022s
I20260709 12:27:35.470412 26849 raft_consensus.cc:359] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.471168 26849 raft_consensus.cc:385] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.471439 26849 raft_consensus.cc:740] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.475890 26477 catalog_manager.cc:5697] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:35.479722 27306 consensus_queue.cc:1048] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:35.486963 26849 consensus_queue.cc:260] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.491617 26853 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: No bootstrap required, opened a new log
I20260709 12:27:35.492251 26853 ts_tablet_manager.cc:1403] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.138s	user 0.009s	sys 0.011s
I20260709 12:27:35.492640 26849 ts_tablet_manager.cc:1434] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:27:35.495416 26853 raft_consensus.cc:359] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.496157 26853 raft_consensus.cc:385] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:35.496421 26853 raft_consensus.cc:740] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:35.489163 27306 consensus_queue.cc:1048] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:27:35.497170 27296 raft_consensus.cc:515] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:35.497179 26853 consensus_queue.cc:260] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.539804 26853 ts_tablet_manager.cc:1434] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:27:35.538911 27286 consensus_queue.cc:1048] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:27:35.521605 27313 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.557358 27296 leader_election.cc:290] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:35.560169 27313 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:35.571372 26726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f229d58a8c304ca1b723a4954aed76f4" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:35.572127 27265 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.572239 26726 raft_consensus.cc:3060] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.572849 27265 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.577147 26728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:35.581612 26728 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.582526 27265 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:35.583595 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:35.584466 26582 raft_consensus.cc:2393] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f57551729f2f4037a4c769ee1df81853 in current term 1: Already voted for candidate cb56388a4ff14b5a9f8f1196dc94461f in this term.
I20260709 12:27:35.586822 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:35.587498 26802 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 0.
I20260709 12:27:35.578601 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f229d58a8c304ca1b723a4954aed76f4" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:35.588752 26695 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:35.589721 26655 raft_consensus.cc:3060] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.603312 26726 raft_consensus.cc:2468] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:35.607496 26768 leader_election.cc:304] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:35.612875 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:35.613718 26801 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.630291 26801 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:35.631521 27286 raft_consensus.cc:2804] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:35.632419 27286 raft_consensus.cc:697] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:35.633368 27286 consensus_queue.cc:237] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:35.667624 27296 raft_consensus.cc:493] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.668321 27296 raft_consensus.cc:515] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.671706 27296 leader_election.cc:290] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:35.675617 26654 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2e5dc8624444d368676b1a1b3fa878e" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:35.676481 26654 raft_consensus.cc:2468] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:35.677731 26546 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:35.682755 27355 raft_consensus.cc:2804] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:35.683284 27355 raft_consensus.cc:697] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:35.684187 27355 consensus_queue.cc:237] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:35.680452 26768 leader_election.cc:304] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:35.686839 27280 consensus_queue.cc:1048] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:35.689314 27279 raft_consensus.cc:2804] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:35.689795 27279 raft_consensus.cc:493] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.690151 27279 raft_consensus.cc:3060] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.692332 26582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2e5dc8624444d368676b1a1b3fa878e" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" is_pre_election: true
I20260709 12:27:35.693104 26582 raft_consensus.cc:2468] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 0.
I20260709 12:27:35.722656 26655 raft_consensus.cc:2468] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:35.667193 26728 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 1.
I20260709 12:27:35.739015 27261 raft_consensus.cc:2764] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:27:35.750850 27279 raft_consensus.cc:515] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:35.754659 27279 leader_election.cc:290] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:35.841604 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2e5dc8624444d368676b1a1b3fa878e" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:35.841557 26579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2e5dc8624444d368676b1a1b3fa878e" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
I20260709 12:27:35.859266 26579 raft_consensus.cc:3060] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:35.923724 26579 raft_consensus.cc:2468] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:35.945842 26769 leader_election.cc:304] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:35.949437 27286 raft_consensus.cc:2804] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:35.950377 27286 raft_consensus.cc:697] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:35.989815 26476 catalog_manager.cc:5697] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:35.978050 26655 raft_consensus.cc:3060] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:36.028730 27368 log.cc:927] Time spent T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.002s
I20260709 12:27:36.105485 27286 consensus_queue.cc:237] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:36.126374 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 214896us
I20260709 12:27:36.207671 27296 consensus_queue.cc:1048] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:36.216449 27296 consensus_queue.cc:1048] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:36.244686 26655 raft_consensus.cc:2468] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:36.271588 27197 consensus_queue.cc:1048] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:27:36.278841 27372 consensus_queue.cc:1048] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:27:36.287818 26527 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:36.301184 27266 consensus_queue.cc:1048] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:27:36.387126 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 101820us
I20260709 12:27:36.464195 27313 consensus_queue.cc:1048] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:27:36.553638 26479 catalog_manager.cc:5697] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:36.636379 26476 catalog_manager.cc:5697] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 1 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:36.642760 27279 consensus_queue.cc:1048] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:36.657766 27279 consensus_queue.cc:1048] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:37.229575 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
W20260709 12:27:37.236713 26621 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 12:27:37.259178 26621 consensus_peers.cc:597] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.265383 26621 consensus_peers.cc:597] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.272825 26769 consensus_peers.cc:597] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.283030 26695 consensus_peers.cc:597] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.290953 26621 consensus_peers.cc:597] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.299196 26621 consensus_peers.cc:597] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.323426 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:37.330250 12469 tablet_replica.cc:333] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.331832 12469 raft_consensus.cc:2243] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.332597 12469 raft_consensus.cc:2272] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.339306 12469 tablet_replica.cc:333] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.343232 12469 raft_consensus.cc:2243] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.343921 12469 raft_consensus.cc:2272] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.348620 26695 consensus_peers.cc:597] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.349509 12469 tablet_replica.cc:333] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
W20260709 12:27:37.350638 26769 consensus_peers.cc:597] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.351200 12469 raft_consensus.cc:2243] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.352697 12469 raft_consensus.cc:2272] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.353972 26621 consensus_peers.cc:597] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.356462 12469 tablet_replica.cc:333] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.357335 12469 raft_consensus.cc:2243] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.357894 12469 raft_consensus.cc:2272] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.361215 12469 tablet_replica.cc:333] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.362455 12469 raft_consensus.cc:2243] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.363107 12469 raft_consensus.cc:2272] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.366612 12469 tablet_replica.cc:333] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.367765 12469 raft_consensus.cc:2243] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.369065 12469 raft_consensus.cc:2272] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.372510 12469 tablet_replica.cc:333] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.373406 12469 raft_consensus.cc:2243] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.374126 12469 raft_consensus.cc:2272] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.375344 26621 consensus_peers.cc:597] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.378051 12469 tablet_replica.cc:333] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.379190 12469 raft_consensus.cc:2243] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.379796 12469 raft_consensus.cc:2272] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.383558 12469 tablet_replica.cc:333] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
W20260709 12:27:37.384326 26695 consensus_peers.cc:597] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.384678 12469 raft_consensus.cc:2243] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.385388 12469 raft_consensus.cc:2272] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.387954 12469 tablet_replica.cc:333] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.388720 12469 raft_consensus.cc:2243] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.389219 12469 raft_consensus.cc:2272] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.392179 12469 tablet_replica.cc:333] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.393036 12469 raft_consensus.cc:2243] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.395161 12469 raft_consensus.cc:2272] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.398144 12469 tablet_replica.cc:333] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.398890 12469 raft_consensus.cc:2243] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.399943 12469 raft_consensus.cc:2272] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.402642 12469 tablet_replica.cc:333] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.403221 12469 raft_consensus.cc:2243] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.403648 12469 raft_consensus.cc:2272] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.406397 12469 tablet_replica.cc:333] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.407231 12469 raft_consensus.cc:2243] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.407933 12469 raft_consensus.cc:2272] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.411015 12469 tablet_replica.cc:333] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
W20260709 12:27:37.410893 26769 consensus_peers.cc:597] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.412020 12469 raft_consensus.cc:2243] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.412541 12469 raft_consensus.cc:2272] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.415925 12469 tablet_replica.cc:333] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.416766 12469 raft_consensus.cc:2243] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.417284 12469 raft_consensus.cc:2272] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.417635 26695 consensus_peers.cc:597] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.420620 12469 tablet_replica.cc:333] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.421584 12469 raft_consensus.cc:2243] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.422204 12469 raft_consensus.cc:2272] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.422564 26769 consensus_peers.cc:597] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.425840 12469 tablet_replica.cc:333] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.427042 12469 raft_consensus.cc:2243] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.427704 12469 raft_consensus.cc:2272] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.430565 26621 consensus_peers.cc:597] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.431577 12469 tablet_replica.cc:333] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.432561 12469 raft_consensus.cc:2243] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:27:37.433877 26769 consensus_peers.cc:597] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.434131 12469 raft_consensus.cc:2272] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.437626 12469 tablet_replica.cc:333] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.438419 12469 raft_consensus.cc:2243] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.439406 12469 raft_consensus.cc:2272] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.442619 12469 tablet_replica.cc:333] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.443291 12469 raft_consensus.cc:2243] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.443800 12469 raft_consensus.cc:2272] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.446548 12469 tablet_replica.cc:333] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.447326 12469 raft_consensus.cc:2243] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.447846 12469 raft_consensus.cc:2272] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.450865 12469 tablet_replica.cc:333] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.451771 12469 raft_consensus.cc:2243] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:27:37.452807 26695 consensus_peers.cc:597] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.452874 12469 raft_consensus.cc:2272] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.455770 12469 tablet_replica.cc:333] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.456756 12469 raft_consensus.cc:2243] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.457647 12469 raft_consensus.cc:2272] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.459900 26695 consensus_peers.cc:597] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.461392 12469 tablet_replica.cc:333] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.462090 12469 raft_consensus.cc:2243] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.464944 12469 raft_consensus.cc:2272] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.470903 12469 tablet_replica.cc:333] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.472214 12469 raft_consensus.cc:2243] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.473001 12469 raft_consensus.cc:2272] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.474375 26769 consensus_peers.cc:597] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.476540 12469 tablet_replica.cc:333] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.477496 12469 raft_consensus.cc:2243] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.478112 12469 raft_consensus.cc:2272] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.481786 12469 tablet_replica.cc:333] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.482659 12469 raft_consensus.cc:2243] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.483309 12469 raft_consensus.cc:2272] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.485793 12469 tablet_replica.cc:333] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.486428 12469 raft_consensus.cc:2243] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.487002 12469 raft_consensus.cc:2272] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.490595 12469 tablet_replica.cc:333] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.491451 12469 raft_consensus.cc:2243] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.492357 12469 raft_consensus.cc:2272] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.494993 12469 tablet_replica.cc:333] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.495906 12469 raft_consensus.cc:2243] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.496562 12469 raft_consensus.cc:2272] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.499555 12469 tablet_replica.cc:333] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.500245 12469 raft_consensus.cc:2243] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.500859 12469 raft_consensus.cc:2272] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.504339 12469 tablet_replica.cc:333] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.504974 12469 raft_consensus.cc:2243] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.505620 12469 raft_consensus.cc:2272] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.508883 12469 tablet_replica.cc:333] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.509735 12469 raft_consensus.cc:2243] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.510819 12469 raft_consensus.cc:2272] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.513314 12469 tablet_replica.cc:333] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.513935 12469 raft_consensus.cc:2243] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.514979 12469 raft_consensus.cc:2272] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.517235 12469 tablet_replica.cc:333] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.517831 12469 raft_consensus.cc:2243] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.518414 12469 raft_consensus.cc:2272] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.520915 12469 tablet_replica.cc:333] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.521589 12469 raft_consensus.cc:2243] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.522051 12469 raft_consensus.cc:2272] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.525002 26769 consensus_peers.cc:597] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.525121 12469 tablet_replica.cc:333] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.526876 12469 raft_consensus.cc:2243] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.527516 12469 raft_consensus.cc:2272] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.530211 12469 tablet_replica.cc:333] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.530884 12469 raft_consensus.cc:2243] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.531869 12469 raft_consensus.cc:2272] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.534607 12469 tablet_replica.cc:333] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.535184 12469 raft_consensus.cc:2243] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.535575 12469 raft_consensus.cc:2272] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.538349 12469 tablet_replica.cc:333] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.539103 12469 raft_consensus.cc:2243] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.539803 12469 raft_consensus.cc:2272] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.543190 12469 tablet_replica.cc:333] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.543793 12469 raft_consensus.cc:2243] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.544272 12469 raft_consensus.cc:2272] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.546653 12469 tablet_replica.cc:333] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.547268 12469 raft_consensus.cc:2243] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.547843 12469 raft_consensus.cc:2272] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.551178 12469 tablet_replica.cc:333] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.551821 12469 raft_consensus.cc:2243] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:37.552418 12469 raft_consensus.cc:2272] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:37.555491 12469 tablet_replica.cc:333] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: stopping tablet replica
I20260709 12:27:37.556102 12469 raft_consensus.cc:2243] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:37.557273 12469 raft_consensus.cc:2272] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:37.563048 26695 consensus_peers.cc:597] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.571318 26769 consensus_peers.cc:597] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.574601 26621 consensus_peers.cc:597] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.576189 26621 consensus_peers.cc:597] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.608120 26621 consensus_peers.cc:597] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.637614 26769 consensus_peers.cc:597] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.650235 26769 consensus_peers.cc:597] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.665721 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
W20260709 12:27:37.680747 26769 consensus_peers.cc:597] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.712813 26695 consensus_peers.cc:597] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:37.732600 26769 consensus_peers.cc:597] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.803431 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:37.813844 27409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:37.815322 27410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:37.831813 26769 consensus_peers.cc:597] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:37.838423 12469 server_base.cc:1061] running on GCE node
W20260709 12:27:37.853042 27413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:37.856745 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:37.857252 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:37.857553 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600057857529 us; error 0 us; skew 500 ppm
I20260709 12:27:37.858434 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:37.886755 12469 webserver.cc:533] Webserver started at http://127.12.45.65:40879/ using document root <none> and password file <none>
I20260709 12:27:37.887693 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:37.888031 12469 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:27:37.894704 12469 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:27:37.908010 27417 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:37.909461 12469 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:27:37.909937 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
format_stamp: "Formatted at 2026-07-09 12:27:22 on dist-test-slave-90bm"
I20260709 12:27:37.910439 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:37.966626 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:37.968143 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:37.970000 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:37.975626 27424 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:27:38.331063 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:27:38.331528 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.357s	user 0.011s	sys 0.000s
I20260709 12:27:38.331943 12469 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:27:38.338618 27424 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.353936 27424 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.355046 27424 tablet_bootstrap.cc:492] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.355783 27424 ts_tablet_manager.cc:1403] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:27:38.358669 27424 raft_consensus.cc:359] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.360409 27424 raft_consensus.cc:740] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.361286 27424 consensus_queue.cc:260] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.364118 27424 ts_tablet_manager.cc:1434] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:27:38.364995 27424 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.372578 27372 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.373152 27372 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.376502 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.377043 27372 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:38.384204 26621 leader_election.cc:336] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.384806 26621 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.386163 27372 raft_consensus.cc:2749] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.388285 27424 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.390566 27424 tablet_bootstrap.cc:492] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.391462 27424 ts_tablet_manager.cc:1403] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.006s
I20260709 12:27:38.394855 27424 raft_consensus.cc:359] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.395605 27424 raft_consensus.cc:740] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.396319 27424 consensus_queue.cc:260] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.398391 27424 ts_tablet_manager.cc:1434] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:27:38.399369 27424 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.448421 27424 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.449581 27424 tablet_bootstrap.cc:492] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.450383 27424 ts_tablet_manager.cc:1403] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.051s	user 0.022s	sys 0.028s
I20260709 12:27:38.453392 27424 raft_consensus.cc:359] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.454162 27424 raft_consensus.cc:740] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.454972 27424 consensus_queue.cc:260] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.456987 27424 ts_tablet_manager.cc:1434] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:38.457824 27424 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.473232 27424 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.474339 27424 tablet_bootstrap.cc:492] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.475101 27424 ts_tablet_manager.cc:1403] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:27:38.477885 27424 raft_consensus.cc:359] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.478652 27424 raft_consensus.cc:740] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.479368 27424 consensus_queue.cc:260] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.483871 27424 ts_tablet_manager.cc:1434] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:27:38.484756 27424 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.493415 27406 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.499413 27406 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.501751 27406 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.509215 27378 raft_consensus.cc:493] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.510248 27378 raft_consensus.cc:515] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.512758 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.513324 27378 leader_election.cc:290] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.516042 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:38.520124 26621 leader_election.cc:336] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.520831 26621 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.521595 27372 raft_consensus.cc:2749] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:27:38.522756 26695 leader_election.cc:336] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.523309 27424 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.523320 26695 leader_election.cc:304] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.524340 27378 raft_consensus.cc:2749] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.524477 27424 tablet_bootstrap.cc:492] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.525295 27424 ts_tablet_manager.cc:1403] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.041s	user 0.025s	sys 0.004s
I20260709 12:27:38.528064 27424 raft_consensus.cc:359] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.528749 27424 raft_consensus.cc:740] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.529419 27424 consensus_queue.cc:260] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.531365 27424 ts_tablet_manager.cc:1434] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:27:38.532182 27424 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.533842 27401 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.534418 27401 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.536300 27390 raft_consensus.cc:493] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.536525 27401 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.536903 27390 raft_consensus.cc:515] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.538503 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.539207 27390 leader_election.cc:290] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.542184 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.549902 27424 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.551625 27424 tablet_bootstrap.cc:492] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.552359 27424 ts_tablet_manager.cc:1403] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:27:38.555369 27424 raft_consensus.cc:359] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.556087 27424 raft_consensus.cc:740] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.556761 27424 consensus_queue.cc:260] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.558738 27379 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.559444 27379 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.561720 27372 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.562022 27379 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.562564 27404 raft_consensus.cc:493] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.558750 27424 ts_tablet_manager.cc:1434] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:27:38.563098 27404 raft_consensus.cc:515] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.563848 27424 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.565177 27404 leader_election.cc:290] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.566704 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.569281 27372 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.569988 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.571424 27372 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:38.578037 26695 leader_election.cc:336] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.578715 26695 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.579530 27404 raft_consensus.cc:493] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.580219 27404 raft_consensus.cc:515] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.579851 27378 raft_consensus.cc:2749] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.585106 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.587697 27404 leader_election.cc:290] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:38.589138 26695 leader_election.cc:336] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.589716 26695 leader_election.cc:304] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.593122 27404 raft_consensus.cc:2749] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.594872 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.604076 27378 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.604621 27378 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.607299 27378 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:38.608281 26695 leader_election.cc:336] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.608862 26695 leader_election.cc:304] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.610651 27378 raft_consensus.cc:2749] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.614727 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:38.616834 26769 leader_election.cc:336] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.617395 26769 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
W20260709 12:27:38.620919 26695 leader_election.cc:336] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.621511 26695 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.622985 27404 raft_consensus.cc:2749] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:27:38.625259 26769 leader_election.cc:336] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.625047 27390 raft_consensus.cc:2749] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.625921 26769 leader_election.cc:304] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:38.626238 27424 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.627182 27390 raft_consensus.cc:2749] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.627544 27424 tablet_bootstrap.cc:492] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.628356 27424 ts_tablet_manager.cc:1403] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.065s	user 0.020s	sys 0.000s
I20260709 12:27:38.631326 27424 raft_consensus.cc:359] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.632135 27424 raft_consensus.cc:740] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.633059 27424 consensus_queue.cc:260] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.635461 27424 ts_tablet_manager.cc:1434] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:27:38.636304 27424 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:38.643950 26621 leader_election.cc:336] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.644652 26621 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.646214 27380 raft_consensus.cc:2749] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.647208 27404 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.662518 27390 raft_consensus.cc:493] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.663177 27390 raft_consensus.cc:515] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.662334 27404 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.666851 27378 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.668185 27378 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.670568 27390 leader_election.cc:290] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.671270 27378 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.671684 27433 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.672247 27433 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.673290 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.674176 26725 raft_consensus.cc:2468] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:38.674918 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:38.675297 27433 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:38.676503 12469 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:27:38.676909 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.345s	user 0.297s	sys 0.017s
I20260709 12:27:38.676743 26653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:38.675388 27401 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.679541 27401 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.682972 27404 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.687271 27390 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.687904 27390 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.689951 27424 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.691160 27424 tablet_bootstrap.cc:492] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.691063 27401 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.692050 27424 ts_tablet_manager.cc:1403] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.020s
I20260709 12:27:38.676638 26768 leader_election.cc:304] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:38.693562 27401 raft_consensus.cc:2804] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.693957 27401 raft_consensus.cc:493] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.694406 27401 raft_consensus.cc:3060] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.695117 27424 raft_consensus.cc:359] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:38.695367 26769 leader_election.cc:336] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.695945 27424 raft_consensus.cc:740] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.696964 27424 consensus_queue.cc:260] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.699172 27424 ts_tablet_manager.cc:1434] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:38.699589 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.700044 27424 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.700328 26725 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
W20260709 12:27:38.701782 26769 leader_election.cc:336] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.708141 27390 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:38.715178 26695 leader_election.cc:336] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.715752 26695 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:38.716262 26769 leader_election.cc:336] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.726795 27404 raft_consensus.cc:2749] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.729359 27433 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.729979 27433 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.731940 27433 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:38.729403 27438 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.732887 27438 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.734344 27424 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.734864 27438 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:38.735800 27424 tablet_bootstrap.cc:492] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.736720 27424 ts_tablet_manager.cc:1403] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.000s
I20260709 12:27:38.736641 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:38.740588 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:38.741068 26769 leader_election.cc:336] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.741250 26802 raft_consensus.cc:2468] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:38.741532 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.746748 27424 raft_consensus.cc:359] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.749209 27424 raft_consensus.cc:740] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.749226 26725 raft_consensus.cc:2468] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:38.750270 27424 consensus_queue.cc:260] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.751808 26768 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.752732 27438 raft_consensus.cc:2804] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.753240 27438 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.753852 27438 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.754652 26768 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.755510 27424 ts_tablet_manager.cc:1434] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 12:27:38.755544 27390 raft_consensus.cc:2804] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.756422 27390 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.756865 27390 raft_consensus.cc:3060] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.757074 27424 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.761739 27380 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.762468 27380 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.765887 27380 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:38.766139 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.767068 26725 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:38.768844 26620 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:38.769778 27380 raft_consensus.cc:2804] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.770423 27380 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.770897 27380 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:27:38.761708 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 46552.2us
W20260709 12:27:38.778728 26695 leader_election.cc:336] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.779151 26695 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.782419 27401 raft_consensus.cc:515] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.777951 27390 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.786863 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df9b6736d82e4e18ba0068b834c114d0" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:38.787575 26725 raft_consensus.cc:3060] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.790308 27424 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.791697 27390 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.791810 27424 tablet_bootstrap.cc:492] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.794938 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:38.796052 26730 raft_consensus.cc:3060] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.796582 27424 ts_tablet_manager.cc:1403] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.040s	user 0.028s	sys 0.004s
I20260709 12:27:38.800570 27424 raft_consensus.cc:359] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.801726 27424 raft_consensus.cc:740] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.802883 27404 raft_consensus.cc:2749] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.803013 27380 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.806058 27380 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:38.807201 27424 consensus_queue.cc:260] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.808948 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:38.809854 26729 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.810691 27424 ts_tablet_manager.cc:1434] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:27:38.811936 27424 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:38.819813 26769 leader_election.cc:336] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
W20260709 12:27:38.820492 26769 leader_election.cc:336] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.822306 27404 raft_consensus.cc:493] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.823221 27404 raft_consensus.cc:515] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.823060 27446 raft_consensus.cc:493] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.842837 27446 raft_consensus.cc:515] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.844491 27404 leader_election.cc:290] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.824971 27390 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.845824 27390 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.846701 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.847821 26727 raft_consensus.cc:2468] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:38.848775 27390 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:38.840337 27438 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.851709 27438 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.852705 26725 raft_consensus.cc:2468] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:38.855857 27401 leader_election.cc:290] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.855160 26695 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:38.860525 26730 raft_consensus.cc:2468] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:38.863356 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:38.864279 26727 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.866955 26729 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 2.
I20260709 12:27:38.868654 26620 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:38.870003 27380 raft_consensus.cc:2804] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:38.870729 27380 raft_consensus.cc:697] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:38.871714 27380 consensus_queue.cc:237] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.878536 27406 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.879271 27406 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.880548 27424 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:38.885921 27424 tablet_bootstrap.cc:492] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:38.881729 27406 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:38.888057 27424 ts_tablet_manager.cc:1403] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.076s	user 0.025s	sys 0.020s
W20260709 12:27:38.880810 26769 leader_election.cc:336] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.883533 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:38.890516 27448 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.892159 26725 raft_consensus.cc:2468] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 1.
I20260709 12:27:38.894652 26727 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:38.868392 26768 leader_election.cc:304] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:38.897578 26620 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:38.898494 27406 raft_consensus.cc:2804] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.899135 27406 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.899528 27406 raft_consensus.cc:3060] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.899791 27448 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:38.901206 27424 raft_consensus.cc:359] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.902505 27424 raft_consensus.cc:740] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:38.903836 27424 consensus_queue.cc:260] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.904748 26768 leader_election.cc:304] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.906474 27450 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.907251 27390 raft_consensus.cc:2804] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.907195 27450 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:38.907781 27390 raft_consensus.cc:493] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.908082 27424 ts_tablet_manager.cc:1434] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 12:27:38.908355 27390 raft_consensus.cc:3060] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.909785 27424 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:38.914382 26768 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:38.916481 26769 leader_election.cc:336] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.918501 27401 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.927695 27401 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.929890 27401 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.915756 27449 raft_consensus.cc:493] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
W20260709 12:27:38.932216 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 77152.1us
I20260709 12:27:38.909900 27450 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:38.933184 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:38.935055 26655 raft_consensus.cc:2468] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:38.933997 27438 raft_consensus.cc:2804] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:38.934155 27378 raft_consensus.cc:2804] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:38.937960 27438 raft_consensus.cc:697] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:38.937868 27378 raft_consensus.cc:493] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:38.938717 27378 raft_consensus.cc:3060] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:38.939854 27448 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:38.941910 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:38.935128 26695 leader_election.cc:336] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.947103 27446 leader_election.cc:290] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.934599 27401 raft_consensus.cc:2804] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:27:38.950629 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 44395us
I20260709 12:27:38.936658 27449 raft_consensus.cc:515] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.953203 27438 consensus_queue.cc:237] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.956220 27449 leader_election.cc:290] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.960493 26768 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:38.965749 26621 leader_election.cc:336] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.964082 27401 raft_consensus.cc:697] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:38.968861 27401 consensus_queue.cc:237] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.969996 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:38.970876 26652 raft_consensus.cc:2468] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
W20260709 12:27:38.973389 26695 leader_election.cc:336] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:38.973958 26695 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:38.977949 27378 raft_consensus.cc:515] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.983472 27390 raft_consensus.cc:515] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:38.984421 27404 raft_consensus.cc:2749] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:38.985263 27378 leader_election.cc:290] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:38.978236 27450 raft_consensus.cc:2804] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:38.987164 27406 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.013270 26477 catalog_manager.cc:5697] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.014873 26478 catalog_manager.cc:5697] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:39.019071 26621 leader_election.cc:336] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.025483 27450 raft_consensus.cc:697] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.027168 27450 consensus_queue.cc:237] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.030156 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:39.030989 26652 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:39.032419 26802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:39.033113 26802 raft_consensus.cc:2468] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
W20260709 12:27:39.037721 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 64364.4us
W20260709 12:27:39.038836 26769 leader_election.cc:336] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.039644 26768 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.042059 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:39.042764 26729 raft_consensus.cc:3060] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.041473 27390 leader_election.cc:290] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:39.045944 27450 raft_consensus.cc:2804] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.046593 27450 raft_consensus.cc:493] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.047019 27450 raft_consensus.cc:3060] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.043955 27406 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:39.046023 26768 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.054162 27453 raft_consensus.cc:2804] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.054629 26729 raft_consensus.cc:2468] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 2.
I20260709 12:27:39.054819 27453 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.055531 27453 raft_consensus.cc:3060] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.056006 26620 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
W20260709 12:27:39.036053 26621 leader_election.cc:336] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.056965 27406 raft_consensus.cc:2804] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.058967 27406 raft_consensus.cc:697] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:39.060103 27406 consensus_queue.cc:237] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:39.063828 26621 leader_election.cc:336] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
W20260709 12:27:39.065267 26769 leader_election.cc:336] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.069669 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:39.070283 26621 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 643057f6f77241529979f572a42ed9b0; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:39.071201 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:39.071866 26725 raft_consensus.cc:2468] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
I20260709 12:27:39.070449 26727 raft_consensus.cc:2468] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 1.
W20260709 12:27:39.073690 26769 leader_election.cc:336] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.068768 27453 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:39.074620 26769 leader_election.cc:336] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.075462 27448 raft_consensus.cc:2749] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:39.078967 27404 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.079661 27404 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.080497 26768 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.084234 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:39.076786 27453 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:39.081988 27404 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:39.087492 26655 raft_consensus.cc:2468] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 1.
I20260709 12:27:39.085000 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c01181f6bd4a92915740e17605f4b9" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:39.090157 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf182133ad942e582a03b114c25daf3" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:39.090955 26799 raft_consensus.cc:3060] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.093592 27424 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.090145 26730 raft_consensus.cc:3060] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.094955 27424 tablet_bootstrap.cc:492] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.095927 27424 ts_tablet_manager.cc:1403] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.187s	user 0.024s	sys 0.001s
I20260709 12:27:39.090972 26694 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:39.098955 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:39.099267 27450 raft_consensus.cc:515] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.099835 26655 raft_consensus.cc:3060] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.101459 27450 leader_election.cc:290] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:39.093626 26695 leader_election.cc:336] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.105160 26799 raft_consensus.cc:2468] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
I20260709 12:27:39.088222 27458 raft_consensus.cc:2804] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.106107 27456 raft_consensus.cc:2804] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.110745 27456 raft_consensus.cc:493] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.110723 27458 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.111222 27456 raft_consensus.cc:3060] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.111858 27458 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.120680 26768 leader_election.cc:304] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.122330 27456 raft_consensus.cc:515] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.125895 26730 raft_consensus.cc:2468] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:39.126641 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faab3b25212c4a0085b5907de0936703" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:39.127704 26652 raft_consensus.cc:3060] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.127883 26695 leader_election.cc:304] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.109619 27424 raft_consensus.cc:359] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.129000 27424 raft_consensus.cc:740] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.127897 26655 raft_consensus.cc:2468] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:39.130820 27404 raft_consensus.cc:2804] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.131282 27404 raft_consensus.cc:493] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.131669 27404 raft_consensus.cc:3060] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:27:39.134577 26769 leader_election.cc:336] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.137272 26652 raft_consensus.cc:2468] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
I20260709 12:27:39.143838 26694 leader_election.cc:304] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:39.144879 27457 raft_consensus.cc:2804] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.145401 27457 raft_consensus.cc:697] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:39.146343 27457 consensus_queue.cc:237] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:39.139930 26695 leader_election.cc:336] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
W20260709 12:27:39.148010 26621 leader_election.cc:336] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.140599 27424 consensus_queue.cc:260] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.149434 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bd7e0d139cb41beb4076b3fe3672c73" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:39.150091 26655 raft_consensus.cc:3060] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.151561 27424 ts_tablet_manager.cc:1434] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 12:27:39.152563 27424 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:39.141011 27458 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.141934 27450 raft_consensus.cc:2804] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.154794 27450 raft_consensus.cc:493] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.155405 27450 raft_consensus.cc:3060] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.156454 27458 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:39.157214 27456 leader_election.cc:290] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Requested vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:39.142386 26768 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.160068 27390 raft_consensus.cc:2804] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:27:39.160472 27390 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting leader election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:39.160876 27390 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.164191 26768 leader_election.cc:304] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.165693 27453 raft_consensus.cc:2804] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:27:39.166138 26695 leader_election.cc:336] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.166225 27453 raft_consensus.cc:697] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.168177 27453 consensus_queue.cc:237] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.171702 26768 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:39.172979 27404 raft_consensus.cc:515] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.176537 26478 catalog_manager.cc:5697] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.184505 27390 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.189225 27390 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:39.173744 27449 raft_consensus.cc:2804] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.189939 27449 raft_consensus.cc:697] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.190917 27449 consensus_queue.cc:237] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.193881 27404 leader_election.cc:290] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:39.185842 26477 catalog_manager.cc:5697] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.195428 26477 catalog_manager.cc:5697] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:39.207697 26695 leader_election.cc:336] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.214290 26695 leader_election.cc:304] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:39.216647 26769 leader_election.cc:336] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.216970 27404 raft_consensus.cc:2804] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.217864 27404 raft_consensus.cc:697] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:39.219089 27404 consensus_queue.cc:237] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.222311 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:39.223068 26729 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:27:39.226842 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:37295, user_credentials={real_user=slave}} blocked reactor thread for 77868.3us
I20260709 12:27:39.225373 26478 catalog_manager.cc:5697] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:39.229084 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95b062d361f340cd9a9202852a196241" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:39.229969 26801 raft_consensus.cc:3060] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.238072 27450 raft_consensus.cc:515] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.240533 27450 leader_election.cc:290] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:39.247430 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9dc5d1308d484c77941c0fc3a0322157" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:39.248070 26479 catalog_manager.cc:5697] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.248217 26730 raft_consensus.cc:3060] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.251550 26479 catalog_manager.cc:5697] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:39.254205 26769 leader_election.cc:336] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
W20260709 12:27:39.254640 26695 leader_election.cc:336] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
W20260709 12:27:39.255131 26769 leader_election.cc:336] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
W20260709 12:27:39.255648 26769 leader_election.cc:336] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.266855 27424 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.268031 27424 tablet_bootstrap.cc:492] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.268750 27424 ts_tablet_manager.cc:1403] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.116s	user 0.013s	sys 0.008s
I20260709 12:27:39.272370 26478 catalog_manager.cc:5697] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.277082 27424 raft_consensus.cc:359] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.277943 27424 raft_consensus.cc:740] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.278723 27424 consensus_queue.cc:260] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.283946 27424 ts_tablet_manager.cc:1434] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:27:39.284829 27424 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:39.287611 26655 raft_consensus.cc:2468] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:39.294400 26730 raft_consensus.cc:2468] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:39.294167 26729 raft_consensus.cc:2468] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:39.298373 26801 raft_consensus.cc:2468] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
W20260709 12:27:39.298965 26769 leader_election.cc:336] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:39.299389 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:39.306805 26655 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:39.309079 26768 leader_election.cc:304] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.315155 27450 raft_consensus.cc:2804] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.315989 27450 raft_consensus.cc:697] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.317057 27450 consensus_queue.cc:237] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.318163 26768 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.319178 27458 raft_consensus.cc:2804] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.319661 27458 raft_consensus.cc:697] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.320544 26695 leader_election.cc:304] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.320662 26655 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f4d278e29fc4ba68bba0aa75a8d81fa in term 2.
I20260709 12:27:39.320566 27458 consensus_queue.cc:237] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.321861 27456 raft_consensus.cc:2804] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.322414 27456 raft_consensus.cc:697] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:39.323292 27456 consensus_queue.cc:237] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.323833 27424 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.325280 27424 tablet_bootstrap.cc:492] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.326210 27424 ts_tablet_manager.cc:1403] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.005s
I20260709 12:27:39.330236 27424 raft_consensus.cc:359] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.331280 27424 raft_consensus.cc:740] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.332234 27424 consensus_queue.cc:260] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.335400 27424 ts_tablet_manager.cc:1434] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:27:39.329298 26768 leader_election.cc:304] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:39.336408 27424 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:39.337312 27390 raft_consensus.cc:2804] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.337836 27390 raft_consensus.cc:697] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.338796 27390 consensus_queue.cc:237] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.364080 26478 catalog_manager.cc:5697] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.368870 26729 raft_consensus.cc:1275] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.377445 26768 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:39.380640 26769 consensus_peers.cc:597] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.379753 26479 catalog_manager.cc:5697] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.381774 27390 raft_consensus.cc:2804] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:39.382464 27390 raft_consensus.cc:697] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Leader. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:39.383805 27390 consensus_queue.cc:237] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.390591 27458 consensus_queue.cc:1048] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:27:39.395740 26621 consensus_peers.cc:597] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.420905 27424 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.422677 27424 tablet_bootstrap.cc:492] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.434316 26730 raft_consensus.cc:1275] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.438568 26725 raft_consensus.cc:1275] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.438855 26478 catalog_manager.cc:5697] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.441285 27372 consensus_queue.cc:1048] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:39.442361 26478 catalog_manager.cc:5697] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.444566 26478 catalog_manager.cc:5697] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa reported cstate change: term changed from 1 to 2, leader changed from cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65) to 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68). New cstate: current_term: 2 leader_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:39.447854 27458 consensus_queue.cc:1048] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:39.497301 27424 ts_tablet_manager.cc:1403] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.161s	user 0.021s	sys 0.008s
W20260709 12:27:39.514710 26769 consensus_peers.cc:597] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.537519 27424 raft_consensus.cc:359] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.549310 27424 raft_consensus.cc:740] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.551550 27424 consensus_queue.cc:260] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.559703 27424 ts_tablet_manager.cc:1434] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:27:39.560652 27424 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:39.594544 26727 raft_consensus.cc:1275] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.596655 27450 consensus_queue.cc:1048] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:39.604770 26730 raft_consensus.cc:1275] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.607879 27449 consensus_queue.cc:1048] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:27:39.610404 26769 consensus_peers.cc:597] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:39.611370 26769 consensus_peers.cc:597] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.634876 27426 raft_consensus.cc:493] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.635798 27426 raft_consensus.cc:515] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:39.643962 26727 raft_consensus.cc:1275] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.650363 27372 consensus_queue.cc:1048] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:39.650846 27426 leader_election.cc:290] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:39.660487 27424 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.662151 27424 tablet_bootstrap.cc:492] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.663240 27424 ts_tablet_manager.cc:1403] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.103s	user 0.017s	sys 0.009s
I20260709 12:27:39.666836 27424 raft_consensus.cc:359] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.667657 27424 raft_consensus.cc:740] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.668601 27424 consensus_queue.cc:260] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:39.671887 26769 consensus_peers.cc:597] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.675177 27424 ts_tablet_manager.cc:1434] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:27:39.678975 27424 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:39.687744 27426 raft_consensus.cc:493] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.688450 27426 raft_consensus.cc:515] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:39.689769 26695 consensus_peers.cc:597] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.690783 27426 leader_election.cc:290] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
W20260709 12:27:39.693032 26695 consensus_peers.cc:597] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:39.710140 26621 consensus_peers.cc:597] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.712049 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:39.720583 26695 consensus_peers.cc:597] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.711769 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:39.736456 26769 consensus_peers.cc:597] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:39.742547 26621 consensus_peers.cc:597] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.748664 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "744dd7e87b4540ad930b1de1885f334a" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:39.751332 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737ddf5e609d480ea8a119e82a057898" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:39.753633 27418 leader_election.cc:304] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:39.755681 27426 raft_consensus.cc:2749] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:39.757452 27418 leader_election.cc:304] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:39.759357 27426 raft_consensus.cc:2749] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:39.744891 26805 raft_consensus.cc:1275] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.763230 26652 raft_consensus.cc:1275] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:27:39.772639 26695 consensus_peers.cc:597] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:39.775583 26621 consensus_peers.cc:597] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.775274 27404 consensus_queue.cc:1048] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:39.794453 26653 raft_consensus.cc:1275] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.794920 27472 consensus_queue.cc:1048] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:39.818058 27450 consensus_queue.cc:1048] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:27:39.827960 26621 consensus_peers.cc:597] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:39.833880 26621 consensus_peers.cc:597] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.844102 27424 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.845810 27424 tablet_bootstrap.cc:492] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.846969 27424 ts_tablet_manager.cc:1403] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.171s	user 0.048s	sys 0.014s
I20260709 12:27:39.850764 27424 raft_consensus.cc:359] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:39.851495 27424 raft_consensus.cc:740] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.852219 27424 consensus_queue.cc:260] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:39.854815 26769 consensus_peers.cc:597] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.866180 26653 raft_consensus.cc:1275] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:27:39.872802 26769 consensus_peers.cc:597] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.881932 27424 ts_tablet_manager.cc:1434] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
I20260709 12:27:39.883459 27424 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:39.885756 26769 consensus_peers.cc:597] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:39.905122 26621 consensus_peers.cc:597] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:39.906018 26695 consensus_peers.cc:597] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.919834 26653 raft_consensus.cc:1275] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.920773 27450 consensus_queue.cc:1048] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:39.934597 27424 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:39.936497 27424 tablet_bootstrap.cc:492] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:39.937557 27424 ts_tablet_manager.cc:1403] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.013s
I20260709 12:27:39.937813 27450 consensus_queue.cc:1048] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:39.941403 27424 raft_consensus.cc:359] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.942770 27424 raft_consensus.cc:740] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:39.944044 27424 consensus_queue.cc:260] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.945286 26800 raft_consensus.cc:1275] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:27:39.947028 26769 consensus_peers.cc:597] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:39.947909 26730 raft_consensus.cc:1275] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.949570 27424 ts_tablet_manager.cc:1434] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.012s	user 0.012s	sys 0.000s
I20260709 12:27:39.950906 27424 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:39.952750 26769 consensus_peers.cc:597] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:39.953342 26769 consensus_peers.cc:597] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:39.963192 26621 consensus_peers.cc:597] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.966473 26730 raft_consensus.cc:1275] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 6f4d278e29fc4ba68bba0aa75a8d81fa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:39.971401 27472 consensus_queue.cc:1048] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:27:39.971164 27446 consensus_queue.cc:1048] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:27:39.976955 27426 raft_consensus.cc:493] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.972635 27449 consensus_queue.cc:1048] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:27:39.977552 27426 raft_consensus.cc:515] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:39.982985 27426 leader_election.cc:290] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:39.992532 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
W20260709 12:27:39.995710 26695 consensus_peers.cc:597] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:39.988966 27426 raft_consensus.cc:493] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.003582 27426 raft_consensus.cc:515] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
W20260709 12:27:40.023665 26621 consensus_peers.cc:597] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.025830 26695 consensus_peers.cc:597] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.094794 26695 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 57945.3us
I20260709 12:27:40.103003 27426 leader_election.cc:290] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:40.117795 27426 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.123538 26729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:40.123373 27426 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:40.130165 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
W20260709 12:27:40.131083 26621 consensus_peers.cc:597] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.132746 27418 leader_election.cc:304] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:40.134791 27426 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:40.135599 27426 raft_consensus.cc:2749] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.137905 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:40.162829 27424 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:27:40.203336 26621 consensus_peers.cc:597] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.204000 27548 raft_consensus.cc:493] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.204573 27548 raft_consensus.cc:515] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:40.213505 27424 tablet_bootstrap.cc:492] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:40.215042 27424 ts_tablet_manager.cc:1403] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.264s	user 0.028s	sys 0.007s
W20260709 12:27:40.238906 26768 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 46218.1us
I20260709 12:27:40.240211 27426 raft_consensus.cc:493] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.265519 26695 consensus_peers.cc:597] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.266439 26695 consensus_peers.cc:597] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.277940 27556 raft_consensus.cc:493] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.302462 26695 consensus_peers.cc:597] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.268924 27426 raft_consensus.cc:515] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:40.290373 27548 leader_election.cc:290] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:40.352439 26621 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 44988.5us
I20260709 12:27:40.355154 27548 raft_consensus.cc:493] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.310837 27556 raft_consensus.cc:515] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:40.371975 26694 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43883, user_credentials={real_user=slave}} blocked reactor thread for 62630.2us
I20260709 12:27:40.373987 27424 raft_consensus.cc:359] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.377198 27424 raft_consensus.cc:740] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:40.382916 27424 consensus_queue.cc:260] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.386081 27424 ts_tablet_manager.cc:1434] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.170s	user 0.003s	sys 0.004s
I20260709 12:27:40.388170 27557 raft_consensus.cc:493] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.393606 27424 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:40.390893 27557 raft_consensus.cc:515] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:40.406677 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:40.416950 27556 leader_election.cc:290] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:40.417654 27556 raft_consensus.cc:493] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.419786 27557 leader_election.cc:290] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:40.420849 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:40.426122 27426 leader_election.cc:290] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:40.426720 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:40.376885 27548 raft_consensus.cc:515] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
W20260709 12:27:40.450871 26769 consensus_peers.cc:597] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.451714 26769 consensus_peers.cc:597] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.452265 26769 consensus_peers.cc:597] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.454866 26769 consensus_peers.cc:597] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.456923 26769 consensus_peers.cc:597] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.459394 26769 consensus_peers.cc:597] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.460352 26769 consensus_peers.cc:597] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.418099 27556 raft_consensus.cc:515] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.481698 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bcd7e1d18334a01bc14e417fbbeaff2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:40.483841 27548 leader_election.cc:290] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:40.486768 26805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0134ff709c4ab3991a5bb56a32f3a1" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:40.488552 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:40.490079 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:40.490849 26653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:40.491042 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37ea7dd7d5f4407abb57b9027b2bf586" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:40.494621 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
W20260709 12:27:40.498523 26769 consensus_peers.cc:597] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.460507 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46ee7ee8e9664d66823351436762d6b2" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "643057f6f77241529979f572a42ed9b0" is_pre_election: true
I20260709 12:27:40.518479 27419 leader_election.cc:304] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:40.522114 27418 leader_election.cc:304] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:40.522150 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee02a1415f74accb6c585fd3556fa3b" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:40.523772 27556 leader_election.cc:290] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:40.524397 27564 raft_consensus.cc:2749] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.526757 26725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f57551729f2f4037a4c769ee1df81853" is_pre_election: true
I20260709 12:27:40.532053 27548 raft_consensus.cc:2749] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.561395 27419 leader_election.cc:304] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
W20260709 12:27:40.561707 26769 consensus_peers.cc:597] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.562866 26769 consensus_peers.cc:597] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.564158 27548 raft_consensus.cc:2749] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.586663 27419 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:40.588395 27419 leader_election.cc:304] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:40.589865 27564 raft_consensus.cc:2749] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.590051 27419 leader_election.cc:304] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
I20260709 12:27:40.590837 27564 raft_consensus.cc:2749] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.591677 27564 raft_consensus.cc:2749] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.592341 27419 leader_election.cc:304] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa
W20260709 12:27:40.628420 26621 consensus_peers.cc:597] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:27:40.635466 26620 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:39937, user_credentials={real_user=slave}} blocked reactor thread for 164056us
W20260709 12:27:40.641556 26695 consensus_peers.cc:597] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:40.650492 27556 raft_consensus.cc:2749] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:27:40.660915 27419 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.68:45475, user_credentials={real_user=slave}} blocked reactor thread for 68665.5us
I20260709 12:27:40.666062 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:40.690573 27419 leader_election.cc:304] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:40.704077 27556 raft_consensus.cc:2749] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:40.790385 27424 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:40.791584 27424 tablet_bootstrap.cc:492] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:40.792321 27424 ts_tablet_manager.cc:1403] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.405s	user 0.022s	sys 0.013s
I20260709 12:27:40.801575 27424 raft_consensus.cc:359] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:40.803241 27424 raft_consensus.cc:740] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:40.807057 27424 consensus_queue.cc:260] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:40.809433 27424 ts_tablet_manager.cc:1434] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.002s
I20260709 12:27:40.810323 27424 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:40.838011 27424 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:40.839452 27584 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:37295 every 8 connection(s)
I20260709 12:27:40.839612 27424 tablet_bootstrap.cc:492] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:40.840054 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:37295
I20260709 12:27:40.845029 27424 ts_tablet_manager.cc:1403] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.035s	user 0.019s	sys 0.004s
I20260709 12:27:40.848951 27424 raft_consensus.cc:359] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.851781 27424 raft_consensus.cc:740] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:40.853394 27424 consensus_queue.cc:260] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.857422 27424 ts_tablet_manager.cc:1434] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:27:40.860831 27424 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:40.919998 27524 raft_consensus.cc:3060] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:40.941563 27521 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:40.943104 27424 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:40.945674 27424 tablet_bootstrap.cc:492] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:40.948501 27424 ts_tablet_manager.cc:1403] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.023s
I20260709 12:27:40.956229 27424 raft_consensus.cc:359] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.957145 27424 raft_consensus.cc:740] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:40.958094 27424 consensus_queue.cc:260] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:40.960582 27424 ts_tablet_manager.cc:1434] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:27:40.961495 27424 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:40.984750 27523 raft_consensus.cc:3060] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:40.987334 27586 heartbeater.cc:344] Connected to a master server at 127.12.45.126:41947
I20260709 12:27:40.987946 27586 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:40.989214 27586 heartbeater.cc:507] Master 127.12.45.126:41947 requested a full tablet report, sending...
I20260709 12:27:41.070958 27424 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.072173 27424 tablet_bootstrap.cc:492] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.073074 27424 ts_tablet_manager.cc:1403] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.009s
I20260709 12:27:41.078996 27424 raft_consensus.cc:359] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.079782 27424 raft_consensus.cc:740] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.080780 27424 consensus_queue.cc:260] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.083446 27424 ts_tablet_manager.cc:1434] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:27:41.085690 27424 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.119719 26478 ts_manager.cc:194] Re-registered known tserver with Master: cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:41.139295 27424 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.140460 27424 tablet_bootstrap.cc:492] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.141199 27424 ts_tablet_manager.cc:1403] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.020s
I20260709 12:27:41.143357 27523 raft_consensus.cc:3060] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.144172 27424 raft_consensus.cc:359] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.145020 27424 raft_consensus.cc:740] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.146040 27424 consensus_queue.cc:260] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.148713 27424 ts_tablet_manager.cc:1434] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:41.149853 27424 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.156520 27521 raft_consensus.cc:3060] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.222450 27424 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.231213 27424 tablet_bootstrap.cc:492] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.232185 27424 ts_tablet_manager.cc:1403] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.083s	user 0.031s	sys 0.028s
I20260709 12:27:41.235581 27424 raft_consensus.cc:359] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.236456 27424 raft_consensus.cc:740] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.237296 27424 consensus_queue.cc:260] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.239768 27424 ts_tablet_manager.cc:1434] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:27:41.240753 27424 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.319707 26478 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48614
I20260709 12:27:41.326503 27586 heartbeater.cc:499] Master 127.12.45.126:41947 was elected leader, sending a full tablet report...
I20260709 12:27:41.331257 27424 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.332523 27424 tablet_bootstrap.cc:492] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.333264 27424 ts_tablet_manager.cc:1403] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.093s	user 0.018s	sys 0.012s
I20260709 12:27:41.338637 27424 raft_consensus.cc:359] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.339514 27424 raft_consensus.cc:740] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.340323 27424 consensus_queue.cc:260] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.342785 27424 ts_tablet_manager.cc:1434] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:27:41.343734 27424 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.372123 27424 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.373724 27424 tablet_bootstrap.cc:492] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.374609 27424 ts_tablet_manager.cc:1403] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.004s
I20260709 12:27:41.377853 27424 raft_consensus.cc:359] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.378681 27424 raft_consensus.cc:740] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.379590 27424 consensus_queue.cc:260] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.381130 27521 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.382294 27424 ts_tablet_manager.cc:1434] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:27:41.383810 27424 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.391467 27523 raft_consensus.cc:3060] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.461805 27523 raft_consensus.cc:3060] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.472311 27424 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.474210 27424 tablet_bootstrap.cc:492] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.474998 27424 ts_tablet_manager.cc:1403] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.003s
I20260709 12:27:41.482965 27424 raft_consensus.cc:359] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.483741 27424 raft_consensus.cc:740] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.484547 27424 consensus_queue.cc:260] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.488574 27424 ts_tablet_manager.cc:1434] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 12:27:41.489537 27424 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.510700 27424 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.512408 27424 tablet_bootstrap.cc:492] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.513502 27424 ts_tablet_manager.cc:1403] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 12:27:41.514223 27523 raft_consensus.cc:3060] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.516489 27424 raft_consensus.cc:359] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.517129 27424 raft_consensus.cc:740] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.517802 27424 consensus_queue.cc:260] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.524986 27424 ts_tablet_manager.cc:1434] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:27:41.526060 27424 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.557883 27424 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.559216 27424 tablet_bootstrap.cc:492] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.560122 27424 ts_tablet_manager.cc:1403] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.034s	user 0.026s	sys 0.003s
I20260709 12:27:41.564960 27424 raft_consensus.cc:359] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.565865 27424 raft_consensus.cc:740] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.566969 27424 consensus_queue.cc:260] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.574807 27424 ts_tablet_manager.cc:1434] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 12:27:41.575896 27424 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.595009 27424 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.596167 27424 tablet_bootstrap.cc:492] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.597975 27424 ts_tablet_manager.cc:1403] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.001s
I20260709 12:27:41.601645 27424 raft_consensus.cc:359] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.602429 27424 raft_consensus.cc:740] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.603160 27424 consensus_queue.cc:260] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.605654 27424 ts_tablet_manager.cc:1434] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 12:27:41.606616 27424 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.634018 27424 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.634939 27424 tablet_bootstrap.cc:492] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.635689 27424 ts_tablet_manager.cc:1403] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.012s
I20260709 12:27:41.638450 27424 raft_consensus.cc:359] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.639153 27424 raft_consensus.cc:740] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.639966 27424 consensus_queue.cc:260] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.642093 27424 ts_tablet_manager.cc:1434] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:27:41.642921 27424 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.666409 27424 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.667452 27424 tablet_bootstrap.cc:492] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.668195 27424 ts_tablet_manager.cc:1403] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.000s
I20260709 12:27:41.671763 27424 raft_consensus.cc:359] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.672472 27424 raft_consensus.cc:740] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.673162 27424 consensus_queue.cc:260] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.675575 27424 ts_tablet_manager.cc:1434] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:27:41.676399 27424 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
W20260709 12:27:41.722529 26621 consensus_peers.cc:597] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:27:41.731009 27424 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.732203 27424 tablet_bootstrap.cc:492] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.733016 27424 ts_tablet_manager.cc:1403] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.057s	user 0.020s	sys 0.013s
I20260709 12:27:41.741003 27424 raft_consensus.cc:359] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.741801 27424 raft_consensus.cc:740] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.742964 27424 consensus_queue.cc:260] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.745218 27424 ts_tablet_manager.cc:1434] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:27:41.746181 27424 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.762297 27424 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.763726 27424 tablet_bootstrap.cc:492] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.764523 27424 ts_tablet_manager.cc:1403] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 12:27:41.767132 27424 raft_consensus.cc:359] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.767858 27424 raft_consensus.cc:740] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.768528 27424 consensus_queue.cc:260] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.771322 27424 ts_tablet_manager.cc:1434] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:27:41.773208 27424 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.795674 27424 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.796727 27424 tablet_bootstrap.cc:492] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.797490 27424 ts_tablet_manager.cc:1403] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.000s
I20260709 12:27:41.801750 27424 raft_consensus.cc:359] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.802491 27424 raft_consensus.cc:740] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.803193 27424 consensus_queue.cc:260] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.806030 27424 ts_tablet_manager.cc:1434] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:27:41.806905 27424 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.843259 27424 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.844339 27424 tablet_bootstrap.cc:492] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.845592 27424 ts_tablet_manager.cc:1403] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.039s	user 0.025s	sys 0.004s
I20260709 12:27:41.848507 27424 raft_consensus.cc:359] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.849251 27424 raft_consensus.cc:740] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.850036 27424 consensus_queue.cc:260] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.851778 27424 ts_tablet_manager.cc:1434] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:27:41.853366 27424 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.868047 27424 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.869127 27424 tablet_bootstrap.cc:492] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.869848 27424 ts_tablet_manager.cc:1403] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.003s
I20260709 12:27:41.872455 27424 raft_consensus.cc:359] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.873338 27424 raft_consensus.cc:740] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.874150 27424 consensus_queue.cc:260] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.876443 27424 ts_tablet_manager.cc:1434] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:27:41.877800 27424 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.894794 27424 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.896142 27424 tablet_bootstrap.cc:492] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.896880 27424 ts_tablet_manager.cc:1403] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.001s
I20260709 12:27:41.899770 27424 raft_consensus.cc:359] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.900501 27424 raft_consensus.cc:740] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.901166 27424 consensus_queue.cc:260] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:41.903167 27424 ts_tablet_manager.cc:1434] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:27:41.904560 27424 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.919188 27424 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.920053 27424 tablet_bootstrap.cc:492] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.920665 27424 ts_tablet_manager.cc:1403] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:27:41.922820 27424 raft_consensus.cc:359] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.923372 27424 raft_consensus.cc:740] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.923923 27424 consensus_queue.cc:260] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:41.925452 27424 ts_tablet_manager.cc:1434] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:27:41.926644 27424 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.930781 27521 raft_consensus.cc:3060] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:41.945407 27424 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.946501 27424 tablet_bootstrap.cc:492] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.947247 27424 ts_tablet_manager.cc:1403] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.006s
I20260709 12:27:41.949965 27424 raft_consensus.cc:359] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.950697 27424 raft_consensus.cc:740] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.951398 27424 consensus_queue.cc:260] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.953804 27424 ts_tablet_manager.cc:1434] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:27:41.954670 27424 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:41.969679 27424 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:41.970647 27424 tablet_bootstrap.cc:492] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:41.971395 27424 ts_tablet_manager.cc:1403] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:27:41.974598 27424 raft_consensus.cc:359] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.975337 27424 raft_consensus.cc:740] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:41.976035 27424 consensus_queue.cc:260] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:41.978194 27424 ts_tablet_manager.cc:1434] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:41.980396 27424 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:42.008347 27424 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:42.009388 27424 tablet_bootstrap.cc:492] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:42.010080 27424 ts_tablet_manager.cc:1403] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.010s
I20260709 12:27:42.012715 27424 raft_consensus.cc:359] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:42.013417 27424 raft_consensus.cc:740] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:42.014170 27424 consensus_queue.cc:260] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:42.016415 27424 ts_tablet_manager.cc:1434] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:42.018776 27424 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:42.044466 27424 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:42.045778 27424 tablet_bootstrap.cc:492] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:42.046614 27424 ts_tablet_manager.cc:1403] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.004s
I20260709 12:27:42.050495 27424 raft_consensus.cc:359] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:42.051116 27424 raft_consensus.cc:740] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:42.053113 27424 consensus_queue.cc:260] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:42.056945 27424 ts_tablet_manager.cc:1434] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:27:42.058627 27424 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap starting.
I20260709 12:27:42.090756 27521 raft_consensus.cc:3060] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:42.098352 27424 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:27:42.100157 27424 tablet_bootstrap.cc:492] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Bootstrap complete.
I20260709 12:27:42.100857 27424 ts_tablet_manager.cc:1403] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.012s
I20260709 12:27:42.107990 27424 raft_consensus.cc:359] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:42.108753 27424 raft_consensus.cc:740] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Initialized, Role: FOLLOWER
I20260709 12:27:42.109469 27424 consensus_queue.cc:260] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:42.111269 27424 ts_tablet_manager.cc:1434] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:27:42.150632 27521 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:42.286108 27523 raft_consensus.cc:3060] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:42.314558 27521 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:50.978477 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:51.045192 26656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:59006
I20260709 12:27:51.046329 26656 raft_consensus.cc:606] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:51.049537 26656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
mode: GRACEFUL
new_leader_uuid: "f57551729f2f4037a4c769ee1df81853"
 from {username='slave'} at 127.0.0.1:59006
I20260709 12:27:51.050069 26656 raft_consensus.cc:606] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Received request to transfer leadership to TS f57551729f2f4037a4c769ee1df81853
I20260709 12:27:51.066583 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:51.067169 26800 raft_consensus.cc:606] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:51.070104 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:51.070611 26800 raft_consensus.cc:606] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:51.073297 26656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:59006
I20260709 12:27:51.074954 26656 raft_consensus.cc:606] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:51.104305 27662 raft_consensus.cc:993] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.105269 27654 raft_consensus.cc:1081] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.107333 26727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931"
dest_uuid: "f57551729f2f4037a4c769ee1df81853"
mode: GRACEFUL
new_leader_uuid: "643057f6f77241529979f572a42ed9b0"
 from {username='slave'} at 127.0.0.1:48624
I20260709 12:27:51.107743 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38688
I20260709 12:27:51.108033 26727 raft_consensus.cc:606] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Received request to transfer leadership to TS 643057f6f77241529979f572a42ed9b0
I20260709 12:27:51.108779 27523 raft_consensus.cc:493] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:51.109270 27523 raft_consensus.cc:3060] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.112828 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:51.115444 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:51.116631 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:51.120476 27523 raft_consensus.cc:515] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:51.128191 27523 leader_election.cc:290] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:51.129726 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:51.130666 26652 raft_consensus.cc:3055] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:51.130925 26652 raft_consensus.cc:740] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:51.132030 26652 consensus_queue.cc:260] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:51.133421 26652 raft_consensus.cc:3060] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.136363 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62bc644ef15433785589f1e01b29a89" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:51.139145 26727 raft_consensus.cc:3060] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.148185 26652 raft_consensus.cc:2468] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:51.150652 27418 leader_election.cc:304] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:51.151749 27662 raft_consensus.cc:993] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.152385 27654 raft_consensus.cc:1081] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.152930 26727 raft_consensus.cc:2468] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:51.155273 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38688
I20260709 12:27:51.155382 27663 raft_consensus.cc:2804] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:51.156085 27523 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:51.156461 27523 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.166204 27523 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:51.168893 27523 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:51.169966 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:51.170917 26652 raft_consensus.cc:3055] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:51.171416 26652 raft_consensus.cc:740] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:51.175349 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a4f3d968244475e805596ce58c213e7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:51.175449 26652 consensus_queue.cc:260] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:51.176252 26730 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.177150 26652 raft_consensus.cc:3060] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.177235 27663 raft_consensus.cc:697] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:51.178484 27663 consensus_queue.cc:237] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:51.194011 26730 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:51.202378 27654 raft_consensus.cc:993] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: : Instructing follower f57551729f2f4037a4c769ee1df81853 to start an election
I20260709 12:27:51.202939 27654 raft_consensus.cc:1081] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Signalling peer f57551729f2f4037a4c769ee1df81853 to start an election
I20260709 12:27:51.205706 26730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696"
dest_uuid: "f57551729f2f4037a4c769ee1df81853"
 from {username='slave'} at 127.0.0.1:45470
I20260709 12:27:51.207989 26730 raft_consensus.cc:493] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:51.208390 26730 raft_consensus.cc:3060] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.205829 27418 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:51.211236 26479 catalog_manager.cc:5697] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 643057f6f77241529979f572a42ed9b0 (127.12.45.66) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:51.212674 27663 raft_consensus.cc:2804] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:51.213433 27663 raft_consensus.cc:697] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:51.214823 27663 consensus_queue.cc:237] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:51.219472 26652 raft_consensus.cc:2468] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:51.217833 26730 raft_consensus.cc:515] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:51.255108 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:51.255791 26652 raft_consensus.cc:3055] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:51.256083 26652 raft_consensus.cc:740] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:51.256665 26730 leader_election.cc:290] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:51.256956 26652 consensus_queue.cc:260] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:51.259241 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b2217c6605e4f229a55e993de7a8696" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:51.260039 26799 raft_consensus.cc:3060] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.258209 26652 raft_consensus.cc:3060] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.284581 26799 raft_consensus.cc:2468] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
I20260709 12:27:51.287631 26695 leader_election.cc:304] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:51.289443 27655 raft_consensus.cc:2804] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:51.290130 27655 raft_consensus.cc:697] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Becoming Leader. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:51.291292 26652 raft_consensus.cc:2468] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57551729f2f4037a4c769ee1df81853 in term 2.
I20260709 12:27:51.291126 27655 consensus_queue.cc:237] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:51.309584 26479 catalog_manager.cc:5697] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 reported cstate change: term changed from 1 to 2, leader changed from 643057f6f77241529979f572a42ed9b0 (127.12.45.66) to f57551729f2f4037a4c769ee1df81853 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "f57551729f2f4037a4c769ee1df81853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:51.313154 26476 catalog_manager.cc:5697] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 2 to 3, leader changed from 643057f6f77241529979f572a42ed9b0 (127.12.45.66) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 3 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:51.453219 27646 raft_consensus.cc:993] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.453969 27646 raft_consensus.cc:1081] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.457773 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:51.459543 27523 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:51.460021 27523 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.466532 27523 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:51.468210 27523 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:51.468950 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:51.469549 26730 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.469542 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "559dcf990edd4760b228f7ecff46a4b7" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:51.470299 26799 raft_consensus.cc:3055] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:51.470618 26799 raft_consensus.cc:740] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:51.471504 26799 consensus_queue.cc:260] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:51.472991 26799 raft_consensus.cc:3060] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.476562 26730 raft_consensus.cc:2468] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:51.477682 27418 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:51.478461 27663 raft_consensus.cc:2804] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:51.478951 27663 raft_consensus.cc:697] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:51.480134 26799 raft_consensus.cc:2468] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:51.479871 27663 consensus_queue.cc:237] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:51.492144 26476 catalog_manager.cc:5697] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 2 to 3, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 3 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:51.500144 27458 raft_consensus.cc:993] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.500754 27646 raft_consensus.cc:1081] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:51.502386 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:51.503000 27523 raft_consensus.cc:493] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:51.503371 27523 raft_consensus.cc:3060] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.510565 27523 raft_consensus.cc:515] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:51.512650 27523 leader_election.cc:290] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:51.513370 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:51.514118 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518c064d1a7740bbbb35c46cfa79a3ff" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:51.514487 26730 raft_consensus.cc:3060] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.514791 26799 raft_consensus.cc:3055] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:51.515148 26799 raft_consensus.cc:740] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:51.515966 26799 consensus_queue.cc:260] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:51.517390 26799 raft_consensus.cc:3060] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:51.524879 26730 raft_consensus.cc:2468] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:51.525468 26799 raft_consensus.cc:2468] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:51.526278 27418 leader_election.cc:304] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:51.527304 27663 raft_consensus.cc:2804] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:51.527863 27663 raft_consensus.cc:697] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:51.528863 27663 consensus_queue.cc:237] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:51.538902 27649 raft_consensus.cc:993] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: : Instructing follower 643057f6f77241529979f572a42ed9b0 to start an election
I20260709 12:27:51.539559 27649 raft_consensus.cc:1081] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Signalling peer 643057f6f77241529979f572a42ed9b0 to start an election
I20260709 12:27:51.541169 26652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
 from {username='slave'} at 127.0.0.1:45064
I20260709 12:27:51.541723 26652 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:51.542039 26652 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.553215 26652 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:51.556537 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:51.556419 26652 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 3 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:51.557267 26730 raft_consensus.cc:3055] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:51.557868 26730 raft_consensus.cc:740] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:51.557296 26476 catalog_manager.cc:5697] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:51.557230 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3ca1a807fb8465785df25e6f3ba9931" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:51.558792 26799 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.558725 26730 consensus_queue.cc:260] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:51.560474 26730 raft_consensus.cc:3060] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:51.566793 26730 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 3.
I20260709 12:27:51.567780 26799 raft_consensus.cc:2468] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 3.
I20260709 12:27:51.568384 26620 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:51.569646 27654 raft_consensus.cc:2804] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:51.570065 27654 raft_consensus.cc:697] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 3 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:51.571038 27654 consensus_queue.cc:237] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:51.584059 26476 catalog_manager.cc:5697] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 2 to 3, leader changed from f57551729f2f4037a4c769ee1df81853 (127.12.45.67) to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 3 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:51.605950 26730 raft_consensus.cc:1275] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:51.607949 27663 consensus_queue.cc:1048] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:51.624042 26652 raft_consensus.cc:1275] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:51.625753 27663 consensus_queue.cc:1048] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:51.691430 26652 raft_consensus.cc:1275] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:51.692970 27657 consensus_queue.cc:1048] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:51.705300 26799 raft_consensus.cc:1275] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer f57551729f2f4037a4c769ee1df81853: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:51.706751 27657 consensus_queue.cc:1048] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:51.726390 26727 raft_consensus.cc:1275] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:51.728008 27678 consensus_queue.cc:1048] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:51.750896 26656 raft_consensus.cc:1275] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:51.755259 27664 consensus_queue.cc:1048] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:51.910543 26799 raft_consensus.cc:1275] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:51.911824 27664 consensus_queue.cc:1048] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:27:51.915083 26727 raft_consensus.cc:1275] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:51.916402 27663 consensus_queue.cc:1048] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:51.930349 26730 raft_consensus.cc:1275] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:51.932104 27664 consensus_queue.cc:1048] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:27:51.944535 26799 raft_consensus.cc:1275] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:51.946655 27678 consensus_queue.cc:1048] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.119702 26800 raft_consensus.cc:1275] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:27:52.121152 27653 consensus_queue.cc:1048] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.135633 26727 raft_consensus.cc:1275] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:27:52.137307 27654 consensus_queue.cc:1048] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.175957 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:52.220849 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:52.221542 26800 raft_consensus.cc:606] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:52.224387 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:52.225006 26800 raft_consensus.cc:606] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:52.227864 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:52.228812 26800 raft_consensus.cc:606] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:52.231673 26727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8"
dest_uuid: "f57551729f2f4037a4c769ee1df81853"
mode: GRACEFUL
new_leader_uuid: "643057f6f77241529979f572a42ed9b0"
 from {username='slave'} at 127.0.0.1:48624
I20260709 12:27:52.232204 26727 raft_consensus.cc:606] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Received request to transfer leadership to TS 643057f6f77241529979f572a42ed9b0
I20260709 12:27:52.235289 26801 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:52.235872 26801 raft_consensus.cc:606] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:52.238781 26652 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:59006
I20260709 12:27:52.239295 26652 raft_consensus.cc:606] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:52.240661 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:52.242635 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:52.243407 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:52.293671 27458 raft_consensus.cc:993] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.294312 27458 raft_consensus.cc:1081] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.296363 27519 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:52.296782 27692 raft_consensus.cc:993] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: : Instructing follower 643057f6f77241529979f572a42ed9b0 to start an election
I20260709 12:27:52.296942 27519 raft_consensus.cc:493] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:52.297396 27519 raft_consensus.cc:3060] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.297443 27649 raft_consensus.cc:1081] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Signalling peer 643057f6f77241529979f572a42ed9b0 to start an election
I20260709 12:27:52.299067 26652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
 from {username='slave'} at 127.0.0.1:45064
I20260709 12:27:52.299656 26652 raft_consensus.cc:493] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:52.299995 26652 raft_consensus.cc:3060] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.306293 27519 raft_consensus.cc:515] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.308882 27519 leader_election.cc:290] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:52.309229 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:52.309625 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f37c0d19f1249cdaa3b380b1c0b01d0" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:52.309883 26801 raft_consensus.cc:3055] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:52.310232 26801 raft_consensus.cc:740] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:52.310242 26656 raft_consensus.cc:3060] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.308295 26652 raft_consensus.cc:515] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:52.311251 26801 consensus_queue.cc:260] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.312844 26801 raft_consensus.cc:3060] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.313041 26652 leader_election.cc:290] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:52.313975 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:52.314602 26730 raft_consensus.cc:3055] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:52.314949 26730 raft_consensus.cc:740] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:52.315838 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "646472eb33fc46e88f978e2aa3fe8ff8" candidate_uuid: "643057f6f77241529979f572a42ed9b0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:52.315776 26730 consensus_queue.cc:260] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:52.316428 26800 raft_consensus.cc:3060] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.317251 26730 raft_consensus.cc:3060] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.319042 26656 raft_consensus.cc:2468] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.321632 27418 leader_election.cc:304] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:52.322571 27664 raft_consensus.cc:2804] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:52.323055 27664 raft_consensus.cc:697] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:52.324008 27664 consensus_queue.cc:237] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.326079 26800 raft_consensus.cc:2468] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 2.
I20260709 12:27:52.328518 26621 leader_election.cc:304] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 
I20260709 12:27:52.329469 27654 raft_consensus.cc:2804] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:52.329990 27654 raft_consensus.cc:697] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Becoming Leader. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:52.330965 27654 consensus_queue.cc:237] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:52.329044 26730 raft_consensus.cc:2468] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643057f6f77241529979f572a42ed9b0 in term 2.
I20260709 12:27:52.342497 26801 raft_consensus.cc:2468] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.349241 26479 catalog_manager.cc:5697] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.352811 26476 catalog_manager.cc:5697] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 reported cstate change: term changed from 1 to 2, leader changed from f57551729f2f4037a4c769ee1df81853 (127.12.45.67) to 643057f6f77241529979f572a42ed9b0 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "643057f6f77241529979f572a42ed9b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.435808 27653 raft_consensus.cc:993] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.436394 27654 raft_consensus.cc:1081] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.437954 27519 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38688
I20260709 12:27:52.438567 27519 raft_consensus.cc:493] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:52.438822 27519 raft_consensus.cc:3060] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.445981 27519 raft_consensus.cc:515] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:52.447772 27519 leader_election.cc:290] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:52.448658 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:52.448864 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dfa515c10414832964dc85f6c82d88b" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:52.449306 26656 raft_consensus.cc:3055] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:52.449456 26730 raft_consensus.cc:3060] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.449612 26656 raft_consensus.cc:740] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:52.450529 26656 consensus_queue.cc:260] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:52.451911 26656 raft_consensus.cc:3060] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.460620 26730 raft_consensus.cc:2468] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.461761 27418 leader_election.cc:304] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:52.462733 27663 raft_consensus.cc:2804] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:52.463253 27663 raft_consensus.cc:697] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:52.464206 27663 consensus_queue.cc:237] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:52.475020 26656 raft_consensus.cc:2468] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.484688 26476 catalog_manager.cc:5697] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 643057f6f77241529979f572a42ed9b0 (127.12.45.66) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.548988 27458 raft_consensus.cc:993] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.549571 27646 raft_consensus.cc:1081] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.551512 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:52.552117 27523 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:52.552575 27523 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:52.559592 27646 raft_consensus.cc:993] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.560212 27646 raft_consensus.cc:1081] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.561944 27521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:52.562520 27521 raft_consensus.cc:493] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:52.562861 27521 raft_consensus.cc:3060] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.564065 27523 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.566542 27523 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:52.567569 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:52.568270 26799 raft_consensus.cc:3055] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:52.568581 26799 raft_consensus.cc:740] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:52.569500 26799 consensus_queue.cc:260] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.570698 26799 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:52.571810 27521 raft_consensus.cc:515] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:52.578810 26799 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:52.579008 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:52.579520 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2564769c4e85479fbfb15bd8a05a5807" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:52.579759 26727 raft_consensus.cc:3060] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.581913 26656 raft_consensus.cc:3060] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:52.586596 27419 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:52.586870 27521 leader_election.cc:290] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:52.587774 27663 raft_consensus.cc:2804] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:52.587947 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51a778606ffa467f9faaadb8412a4e8c" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:52.588332 27663 raft_consensus.cc:697] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:52.588614 26799 raft_consensus.cc:3055] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:52.588987 26799 raft_consensus.cc:740] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:52.589330 27663 consensus_queue.cc:237] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.589965 26799 consensus_queue.cc:260] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:52.591177 26799 raft_consensus.cc:3060] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.592279 26727 raft_consensus.cc:2468] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.604362 27418 leader_election.cc:304] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:52.605198 26656 raft_consensus.cc:2468] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:52.605444 27663 raft_consensus.cc:2804] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:52.605122 26476 catalog_manager.cc:5697] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 2 to 3, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 3 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.606593 27663 raft_consensus.cc:697] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:52.608657 27663 consensus_queue.cc:237] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:52.602622 26799 raft_consensus.cc:2468] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.634781 26476 catalog_manager.cc:5697] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:52.667881 27458 raft_consensus.cc:993] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.668730 27458 raft_consensus.cc:1081] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:52.671106 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:52.671762 27523 raft_consensus.cc:493] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:52.672183 27523 raft_consensus.cc:3060] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.678174 27523 raft_consensus.cc:515] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.680143 27523 leader_election.cc:290] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:52.680832 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:52.682158 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bd8a7d163624a52837a57273d4a7892" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:52.681545 26799 raft_consensus.cc:3055] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:52.683069 26799 raft_consensus.cc:740] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:52.682780 26656 raft_consensus.cc:3060] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.684036 26799 consensus_queue.cc:260] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.685453 26799 raft_consensus.cc:3060] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:52.691169 26656 raft_consensus.cc:2468] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.692596 27418 leader_election.cc:304] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:52.692873 26799 raft_consensus.cc:2468] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:52.693380 27663 raft_consensus.cc:2804] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:52.693876 27663 raft_consensus.cc:697] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:52.694926 27663 consensus_queue.cc:237] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:52.710755 26476 catalog_manager.cc:5697] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:52.730664 26656 raft_consensus.cc:1275] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:52.732014 27664 consensus_queue.cc:1048] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.742305 26727 raft_consensus.cc:1275] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:52.744510 26799 raft_consensus.cc:1275] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:52.746246 27678 consensus_queue.cc:1048] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.749372 27653 consensus_queue.cc:1048] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:52.776974 26801 raft_consensus.cc:1275] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer 643057f6f77241529979f572a42ed9b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:52.787919 27694 consensus_queue.cc:1048] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:53.020016 26727 raft_consensus.cc:1275] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.021538 27664 consensus_queue.cc:1048] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:53.035486 26656 raft_consensus.cc:1275] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.036892 27663 consensus_queue.cc:1048] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:53.047016 26727 raft_consensus.cc:1275] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.048771 27663 consensus_queue.cc:1048] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:53.070437 26799 raft_consensus.cc:1275] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.078555 27718 consensus_queue.cc:1048] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:53.093463 26800 raft_consensus.cc:1275] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.099112 26656 raft_consensus.cc:1275] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:53.099439 27663 consensus_queue.cc:1048] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:53.110699 27678 consensus_queue.cc:1048] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:27:53.135514 26652 raft_consensus.cc:1275] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.136968 27678 consensus_queue.cc:1048] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:53.144099 26800 raft_consensus.cc:1275] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:53.146571 27718 consensus_queue.cc:1048] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:27:53.289697 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:53.329916 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:53.330736 26800 raft_consensus.cc:606] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:53.333587 26800 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "473c572574d044c0adc6366487b343fa"
dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:39156
I20260709 12:27:53.334167 26800 raft_consensus.cc:606] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:53.337755 26727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a"
dest_uuid: "f57551729f2f4037a4c769ee1df81853"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:48624
I20260709 12:27:53.338336 26727 raft_consensus.cc:606] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:53.341069 26727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86"
dest_uuid: "f57551729f2f4037a4c769ee1df81853"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:48624
I20260709 12:27:53.341560 26727 raft_consensus.cc:606] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:53.344825 26656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f"
dest_uuid: "643057f6f77241529979f572a42ed9b0"
mode: GRACEFUL
new_leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:59006
I20260709 12:27:53.345458 26656 raft_consensus.cc:606] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Received request to transfer leadership to TS cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:53.347366 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:53.348707 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:53.349665 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:53.435981 27646 raft_consensus.cc:993] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.436594 27691 raft_consensus.cc:1081] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.438274 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:53.438949 27523 raft_consensus.cc:493] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:53.439353 27523 raft_consensus.cc:3060] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.446918 27523 raft_consensus.cc:515] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:53.448832 27523 leader_election.cc:290] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:53.450107 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:53.450909 26656 raft_consensus.cc:3060] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.450817 26799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cf8b638cc7b4d68a88deeacea18f36d" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:53.451561 26799 raft_consensus.cc:3055] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:53.452023 26799 raft_consensus.cc:740] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:53.452862 26799 consensus_queue.cc:260] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:53.454209 26799 raft_consensus.cc:3060] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.462324 26656 raft_consensus.cc:2468] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.465299 27418 leader_election.cc:304] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:53.466164 27663 raft_consensus.cc:2804] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:53.466687 27663 raft_consensus.cc:697] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:53.467492 26799 raft_consensus.cc:2468] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.467664 27663 consensus_queue.cc:237] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:53.484187 26479 catalog_manager.cc:5697] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:53.612578 27649 raft_consensus.cc:993] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.613124 27704 raft_consensus.cc:1081] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.615101 27521 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38674
I20260709 12:27:53.615747 27521 raft_consensus.cc:493] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:53.616075 27521 raft_consensus.cc:3060] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.621173 27649 raft_consensus.cc:993] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.621793 27704 raft_consensus.cc:1081] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.622946 27521 raft_consensus.cc:515] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:53.623693 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38674
I20260709 12:27:53.624696 27523 raft_consensus.cc:493] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:53.625068 27523 raft_consensus.cc:3060] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.625279 27521 leader_election.cc:290] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:53.626435 26727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:53.627089 26727 raft_consensus.cc:3055] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:53.627411 26727 raft_consensus.cc:740] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:53.628404 26727 consensus_queue.cc:260] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:53.630144 26655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83ac1836c91d4280be584799f8bb0c2a" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:53.631381 26655 raft_consensus.cc:3060] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.630457 26727 raft_consensus.cc:3060] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.633535 27523 raft_consensus.cc:515] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:53.635614 27523 leader_election.cc:290] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:53.637091 27654 raft_consensus.cc:993] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.637575 27654 raft_consensus.cc:1081] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.637712 26652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:53.638406 26652 raft_consensus.cc:3060] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.638652 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86583bdc7b4a453f9d4215b64a0d9a86" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:53.639233 26730 raft_consensus.cc:3055] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:53.639529 26730 raft_consensus.cc:740] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f57551729f2f4037a4c769ee1df81853, State: Running, Role: LEADER
I20260709 12:27:53.640291 26730 consensus_queue.cc:260] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:53.641623 26730 raft_consensus.cc:3060] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.643496 26727 raft_consensus.cc:2468] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.645058 27418 leader_election.cc:304] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:53.651656 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38688
I20260709 12:27:53.652287 27523 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:53.652666 27523 raft_consensus.cc:3060] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.653297 27663 raft_consensus.cc:2804] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:53.653849 27663 raft_consensus.cc:697] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:53.651007 26730 raft_consensus.cc:2468] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.654846 27663 consensus_queue.cc:237] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:53.666730 27523 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:53.667326 27418 leader_election.cc:304] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:53.668139 27678 raft_consensus.cc:2804] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:53.669679 27523 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Requested vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:53.669955 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:53.670593 26800 raft_consensus.cc:3060] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.672722 26656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b647e01f42114926b7c04cd1ebf9e23f" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "643057f6f77241529979f572a42ed9b0"
I20260709 12:27:53.673362 26656 raft_consensus.cc:3055] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:27:53.673651 26656 raft_consensus.cc:740] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 643057f6f77241529979f572a42ed9b0, State: Running, Role: LEADER
I20260709 12:27:53.674558 26656 consensus_queue.cc:260] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:53.674947 27678 raft_consensus.cc:697] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:53.676011 26656 raft_consensus.cc:3060] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:27:53.676142 27678 consensus_queue.cc:237] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:53.673633 26652 raft_consensus.cc:2468] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.685541 26655 raft_consensus.cc:2468] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.698993 26656 raft_consensus.cc:2468] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.699095 26800 raft_consensus.cc:2468] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 2.
I20260709 12:27:53.700238 27418 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f; no voters: 
I20260709 12:27:53.700712 26479 catalog_manager.cc:5697] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from f57551729f2f4037a4c769ee1df81853 (127.12.45.67) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:53.701957 27678 raft_consensus.cc:2804] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:27:53.702663 27678 raft_consensus.cc:697] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:53.703564 27678 consensus_queue.cc:237] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:53.731894 26479 catalog_manager.cc:5697] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from f57551729f2f4037a4c769ee1df81853 (127.12.45.67) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:53.733867 26479 catalog_manager.cc:5697] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 1 to 2, leader changed from 643057f6f77241529979f572a42ed9b0 (127.12.45.66) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:53.846778 27691 raft_consensus.cc:993] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: : Instructing follower cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.848068 27691 raft_consensus.cc:1081] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Signalling peer cb56388a4ff14b5a9f8f1196dc94461f to start an election
I20260709 12:27:53.850167 27523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "473c572574d044c0adc6366487b343fa"
dest_uuid: "cb56388a4ff14b5a9f8f1196dc94461f"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:27:53.850904 27523 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:53.851275 27523 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:53.859102 27523 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:53.862898 27523 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Requested vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475)
I20260709 12:27:53.863483 26730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f57551729f2f4037a4c769ee1df81853"
I20260709 12:27:53.864149 26730 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:53.864452 26800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa"
I20260709 12:27:53.864988 26800 raft_consensus.cc:3055] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:27:53.865276 26800 raft_consensus.cc:740] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6f4d278e29fc4ba68bba0aa75a8d81fa, State: Running, Role: LEADER
I20260709 12:27:53.865994 26800 consensus_queue.cc:260] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:53.867159 26800 raft_consensus.cc:3060] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:27:53.870231 26730 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:53.872069 27418 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853; no voters: 
I20260709 12:27:53.872694 26800 raft_consensus.cc:2468] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb56388a4ff14b5a9f8f1196dc94461f in term 3.
I20260709 12:27:53.873019 27678 raft_consensus.cc:2804] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:27:53.873869 27678 raft_consensus.cc:697] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Becoming Leader. State: Replica: cb56388a4ff14b5a9f8f1196dc94461f, State: Running, Role: LEADER
I20260709 12:27:53.875030 27678 consensus_queue.cc:237] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:53.879684 26652 raft_consensus.cc:1275] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.881158 27678 consensus_queue.cc:1048] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:53.886704 26479 catalog_manager.cc:5697] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f reported cstate change: term changed from 2 to 3, leader changed from 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68) to cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65). New cstate: current_term: 3 leader_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:53.905290 26800 raft_consensus.cc:1275] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:53.907191 27718 consensus_queue.cc:1048] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:54.053187 26730 raft_consensus.cc:1275] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.054693 27718 consensus_queue.cc:1048] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:54.074479 26652 raft_consensus.cc:1275] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.080332 27664 consensus_queue.cc:1048] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:27:54.151474 26727 raft_consensus.cc:1275] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.152992 27718 consensus_queue.cc:1048] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:54.165129 26656 raft_consensus.cc:1275] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.167104 27740 consensus_queue.cc:1048] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:54.172703 26652 raft_consensus.cc:1275] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.174949 27664 consensus_queue.cc:1048] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:54.194413 26800 raft_consensus.cc:1275] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:27:54.196326 27740 consensus_queue.cc:1048] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:27:54.298195 26801 raft_consensus.cc:1275] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:54.299809 27740 consensus_queue.cc:1048] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:27:54.314090 26727 raft_consensus.cc:1275] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Refusing update from remote peer cb56388a4ff14b5a9f8f1196dc94461f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:27:54.317950 27678 consensus_queue.cc:1048] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:27:54.402809 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:54.446827 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:27:54.448153 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:54.448766 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:54.497715 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:54.552416 12469 tablet_server.cc:179] TabletServer@127.12.45.65:37295 shutting down...
W20260709 12:27:54.565837 26695 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 295 similar messages]
W20260709 12:27:54.594194 26621 consensus_peers.cc:597] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.602571 26621 consensus_peers.cc:597] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.605499 26769 consensus_peers.cc:597] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.608071 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:54.617403 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.618885 12469 raft_consensus.cc:2243] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.619498 12469 raft_consensus.cc:2272] T fdc9b665b7de4e61ae5821f49d7459ec P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.623451 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.624963 12469 raft_consensus.cc:2243] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.626044 12469 raft_consensus.cc:2272] T b2e5dc8624444d368676b1a1b3fa878e P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.630172 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.631491 12469 raft_consensus.cc:2243] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.632155 12469 raft_consensus.cc:2272] T 5bf5c3113a22464e81d4cb8462e8ac67 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.635957 12469 tablet_replica.cc:333] stopping tablet replica
W20260709 12:27:54.636933 26695 consensus_peers.cc:597] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.637128 12469 raft_consensus.cc:2243] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.638443 12469 raft_consensus.cc:2272] T 5150550259ec4e3fac2af6e10b6ca9b5 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.642560 12469 tablet_replica.cc:333] stopping tablet replica
W20260709 12:27:54.643544 26769 consensus_peers.cc:597] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.643728 12469 raft_consensus.cc:2243] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.644567 12469 raft_consensus.cc:2272] T 95b062d361f340cd9a9202852a196241 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:54.644722 26695 consensus_peers.cc:597] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.647535 26621 consensus_peers.cc:597] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.649217 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.650738 12469 raft_consensus.cc:2243] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:54.651047 26621 consensus_peers.cc:597] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.651536 12469 raft_consensus.cc:2272] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.655786 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.656776 12469 raft_consensus.cc:2243] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.658314 12469 raft_consensus.cc:2272] T e62bc644ef15433785589f1e01b29a89 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.662217 12469 tablet_replica.cc:333] stopping tablet replica
W20260709 12:27:54.662480 26769 consensus_peers.cc:597] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.663226 12469 raft_consensus.cc:2243] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.664516 12469 raft_consensus.cc:2272] T 559dcf990edd4760b228f7ecff46a4b7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.668246 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.669210 12469 raft_consensus.cc:2243] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.669783 12469 raft_consensus.cc:2272] T dc7a25d1e78a4401b1347ed2f07e858d P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.673401 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.674430 12469 raft_consensus.cc:2243] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.675724 12469 raft_consensus.cc:2272] T 3cf8b638cc7b4d68a88deeacea18f36d P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.679411 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.680433 12469 raft_consensus.cc:2243] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.681073 12469 raft_consensus.cc:2272] T 6f355a7f12b14432844a7a4508117120 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.684274 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.685096 12469 raft_consensus.cc:2243] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.685984 12469 raft_consensus.cc:2272] T 3a4f3d968244475e805596ce58c213e7 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.690837 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.691951 12469 raft_consensus.cc:2243] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.692677 12469 raft_consensus.cc:2272] T 9dc5d1308d484c77941c0fc3a0322157 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.696834 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.697726 12469 raft_consensus.cc:2243] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.698228 12469 raft_consensus.cc:2272] T 03c01181f6bd4a92915740e17605f4b9 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.701159 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.702145 12469 raft_consensus.cc:2243] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.702698 12469 raft_consensus.cc:2272] T 8437e96f55e2448e883f6f8bcd057be2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.706184 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.706874 12469 raft_consensus.cc:2243] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.707379 12469 raft_consensus.cc:2272] T d89d07b1f88649de8fe8c262f31a0ac6 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.710000 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.710677 12469 raft_consensus.cc:2243] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.711536 12469 raft_consensus.cc:2272] T 518c064d1a7740bbbb35c46cfa79a3ff P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.714422 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.715274 12469 raft_consensus.cc:2243] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.715816 12469 raft_consensus.cc:2272] T 6cde6c3890534a20956dd0622e8cb1dd P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.718834 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.719583 12469 raft_consensus.cc:2243] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.720074 12469 raft_consensus.cc:2272] T c629cb845a5247089d3810e037fe2035 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.722539 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.723193 12469 raft_consensus.cc:2243] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.723678 12469 raft_consensus.cc:2272] T 5d0fc2016a9b4f2390b55edc0d8289f0 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.726423 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.727068 12469 raft_consensus.cc:2243] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.728097 12469 raft_consensus.cc:2272] T b647e01f42114926b7c04cd1ebf9e23f P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.732133 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.733124 12469 raft_consensus.cc:2243] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.733758 12469 raft_consensus.cc:2272] T 37ea7dd7d5f4407abb57b9027b2bf586 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.736307 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.736884 12469 raft_consensus.cc:2243] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.737287 12469 raft_consensus.cc:2272] T 737ddf5e609d480ea8a119e82a057898 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.740077 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.740619 12469 raft_consensus.cc:2243] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.741122 12469 raft_consensus.cc:2272] T 46ee7ee8e9664d66823351436762d6b2 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.745800 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.746737 12469 raft_consensus.cc:2243] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.747334 12469 raft_consensus.cc:2272] T ff4761da8e934968ae1169a27136254a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.750638 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.751323 12469 raft_consensus.cc:2243] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.751849 12469 raft_consensus.cc:2272] T 0ee02a1415f74accb6c585fd3556fa3b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.754956 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.755807 12469 raft_consensus.cc:2243] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.756305 12469 raft_consensus.cc:2272] T 293707805bab498a9449cd938d3b1b6b P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:54.756762 26621 consensus_peers.cc:597] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.759956 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.760948 12469 raft_consensus.cc:2243] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:54.761118 26769 consensus_peers.cc:597] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.761396 12469 raft_consensus.cc:2272] T 6147d09bb6fe4ec5be65fa9239d10536 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.764608 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.765513 12469 raft_consensus.cc:2243] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.766606 12469 raft_consensus.cc:2272] T 2564769c4e85479fbfb15bd8a05a5807 P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:54.766804 26769 consensus_peers.cc:597] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.769027 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.769889 12469 raft_consensus.cc:2243] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.770736 12469 raft_consensus.cc:2272] T 2bcd7e1d18334a01bc14e417fbbeaff2 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.773176 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.773829 12469 raft_consensus.cc:2243] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.774501 12469 raft_consensus.cc:2272] T faab3b25212c4a0085b5907de0936703 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.777231 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.777863 12469 raft_consensus.cc:2243] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:27:54.778246 26695 consensus_peers.cc:597] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.778856 12469 raft_consensus.cc:2272] T 86583bdc7b4a453f9d4215b64a0d9a86 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.782341 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.783224 12469 raft_consensus.cc:2243] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.784355 12469 raft_consensus.cc:2272] T 7dfa515c10414832964dc85f6c82d88b P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.787595 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.788427 12469 raft_consensus.cc:2243] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.789626 12469 raft_consensus.cc:2272] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.793133 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.794044 12469 raft_consensus.cc:2243] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.794709 12469 raft_consensus.cc:2272] T cbf182133ad942e582a03b114c25daf3 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.798224 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.799149 12469 raft_consensus.cc:2243] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.800423 12469 raft_consensus.cc:2272] T 473c572574d044c0adc6366487b343fa P cb56388a4ff14b5a9f8f1196dc94461f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.803637 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.804358 12469 raft_consensus.cc:2243] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.804874 12469 raft_consensus.cc:2272] T 673d53dde2dc42e1b7feb31f4e28bb3c P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.807909 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.808748 12469 raft_consensus.cc:2243] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.809335 12469 raft_consensus.cc:2272] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.812568 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.813247 12469 raft_consensus.cc:2243] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.813832 12469 raft_consensus.cc:2272] T df9b6736d82e4e18ba0068b834c114d0 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:54.814011 26769 consensus_peers.cc:597] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.817086 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.818002 12469 raft_consensus.cc:2243] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.819104 12469 raft_consensus.cc:2272] T 7bd8a7d163624a52837a57273d4a7892 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.821466 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.822086 12469 raft_consensus.cc:2243] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.822953 12469 raft_consensus.cc:2272] T 83ac1836c91d4280be584799f8bb0c2a P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.826403 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.827384 12469 raft_consensus.cc:2243] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.827981 12469 raft_consensus.cc:2272] T 8bd7e0d139cb41beb4076b3fe3672c73 P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:54.828719 26695 consensus_peers.cc:597] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.831820 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.833101 12469 raft_consensus.cc:2243] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:54.833536 26769 consensus_peers.cc:597] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.833951 12469 raft_consensus.cc:2272] T cf0134ff709c4ab3991a5bb56a32f3a1 P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.836993 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.837881 12469 raft_consensus.cc:2243] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:54.838835 12469 raft_consensus.cc:2272] T 51a778606ffa467f9faaadb8412a4e8c P cb56388a4ff14b5a9f8f1196dc94461f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:54.841881 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:27:54.842567 12469 raft_consensus.cc:2243] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:54.843009 12469 raft_consensus.cc:2272] T 744dd7e87b4540ad930b1de1885f334a P cb56388a4ff14b5a9f8f1196dc94461f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:54.864455 26769 consensus_peers.cc:597] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.870553 26695 consensus_peers.cc:597] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.882658 26769 consensus_peers.cc:597] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.892103 26769 consensus_peers.cc:597] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.897871 26769 consensus_peers.cc:597] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:54.949394 12469 tablet_server.cc:196] TabletServer@127.12.45.65:37295 shutdown complete.
W20260709 12:27:54.951260 26695 consensus_peers.cc:597] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.963693 26621 consensus_peers.cc:597] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.972723 26695 consensus_peers.cc:597] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.990455 26695 consensus_peers.cc:597] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:54.997718 26769 consensus_peers.cc:597] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.002215 26769 consensus_peers.cc:597] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.010429 26621 consensus_peers.cc:597] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.023609 26769 consensus_peers.cc:597] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.068184 26695 consensus_peers.cc:597] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.095120 26621 consensus_peers.cc:597] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 -> Peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Couldn't send request to peer cb56388a4ff14b5a9f8f1196dc94461f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.129746 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
W20260709 12:27:55.182499 26768 consensus_peers.cc:597] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.195503 26694 consensus_peers.cc:597] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.200651 26768 consensus_peers.cc:597] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.205766 26694 consensus_peers.cc:597] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.219316 26694 consensus_peers.cc:597] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.239642 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:55.241289 12469 tablet_replica.cc:333] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.242204 12469 raft_consensus.cc:2243] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.242837 12469 raft_consensus.cc:2272] T b2e5dc8624444d368676b1a1b3fa878e P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.245369 12469 tablet_replica.cc:333] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.246019 12469 raft_consensus.cc:2243] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.246511 12469 raft_consensus.cc:2272] T 2ceb8aabf0a94f9385121858268a3169 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.248822 12469 tablet_replica.cc:333] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.249374 12469 raft_consensus.cc:2243] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.249815 12469 raft_consensus.cc:2272] T 2564769c4e85479fbfb15bd8a05a5807 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.252619 12469 tablet_replica.cc:333] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.253501 12469 raft_consensus.cc:2243] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.254057 12469 raft_consensus.cc:2272] T e62bc644ef15433785589f1e01b29a89 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.256398 12469 tablet_replica.cc:333] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.257030 12469 raft_consensus.cc:2243] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.258200 12469 raft_consensus.cc:2272] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.260782 12469 tablet_replica.cc:333] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.261703 12469 raft_consensus.cc:2243] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.262310 12469 raft_consensus.cc:2272] T 37ea7dd7d5f4407abb57b9027b2bf586 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.265262 12469 tablet_replica.cc:333] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.266338 12469 raft_consensus.cc:2243] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.267514 12469 raft_consensus.cc:2272] T 673d53dde2dc42e1b7feb31f4e28bb3c P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.270574 12469 tablet_replica.cc:333] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.271576 12469 raft_consensus.cc:2243] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.272541 12469 raft_consensus.cc:2272] T dc276f88554143c3bd0a5b4f74143b2f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.275044 12469 tablet_replica.cc:333] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.275694 12469 raft_consensus.cc:2243] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.276275 12469 raft_consensus.cc:2272] T 6e6beba6a6d747ebbcd858a8b61ad68b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.279057 12469 tablet_replica.cc:333] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.279771 12469 raft_consensus.cc:2243] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.280289 12469 raft_consensus.cc:2272] T d89d07b1f88649de8fe8c262f31a0ac6 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.282961 12469 tablet_replica.cc:333] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.283706 12469 raft_consensus.cc:2243] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.284832 12469 raft_consensus.cc:2272] T 737ddf5e609d480ea8a119e82a057898 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.288080 12469 tablet_replica.cc:333] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.288754 12469 raft_consensus.cc:2243] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.289839 12469 raft_consensus.cc:2272] T 646472eb33fc46e88f978e2aa3fe8ff8 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.292574 26768 consensus_peers.cc:597] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.293915 12469 tablet_replica.cc:333] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.294975 12469 raft_consensus.cc:2243] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.295634 12469 raft_consensus.cc:2272] T 2cb970d065d54c94be1a04a340a5821c P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.298641 12469 tablet_replica.cc:333] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.299405 12469 raft_consensus.cc:2243] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.300390 12469 raft_consensus.cc:2272] T 6f355a7f12b14432844a7a4508117120 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.304085 12469 tablet_replica.cc:333] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.304972 12469 raft_consensus.cc:2243] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.305576 12469 raft_consensus.cc:2272] T 3a4f3d968244475e805596ce58c213e7 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.308640 12469 tablet_replica.cc:333] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.309319 12469 raft_consensus.cc:2243] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.309896 12469 raft_consensus.cc:2272] T 5150550259ec4e3fac2af6e10b6ca9b5 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.312676 12469 tablet_replica.cc:333] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.313398 12469 raft_consensus.cc:2243] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.313898 12469 raft_consensus.cc:2272] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.317268 12469 tablet_replica.cc:333] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.318351 12469 raft_consensus.cc:2243] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.318993 12469 raft_consensus.cc:2272] T 8bd7e0d139cb41beb4076b3fe3672c73 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.321990 12469 tablet_replica.cc:333] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.322805 12469 raft_consensus.cc:2243] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:55.323143 26768 consensus_peers.cc:597] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.323473 12469 raft_consensus.cc:2272] T 7bd8a7d163624a52837a57273d4a7892 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.326141 12469 tablet_replica.cc:333] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.327216 12469 raft_consensus.cc:2243] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.327896 12469 raft_consensus.cc:2272] T cf0134ff709c4ab3991a5bb56a32f3a1 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.332719 12469 tablet_replica.cc:333] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.333630 12469 raft_consensus.cc:2243] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.334360 12469 raft_consensus.cc:2272] T 83ac1836c91d4280be584799f8bb0c2a P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.337534 12469 tablet_replica.cc:333] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.342891 12469 raft_consensus.cc:2243] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.343562 12469 raft_consensus.cc:2272] T 744dd7e87b4540ad930b1de1885f334a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.346956 12469 tablet_replica.cc:333] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.347886 12469 raft_consensus.cc:2243] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.349061 12469 raft_consensus.cc:2272] T b1d43a3098bd4f5d80095bc30638c181 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.352581 12469 tablet_replica.cc:333] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.353394 12469 raft_consensus.cc:2243] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.354507 12469 raft_consensus.cc:2272] T fdc9b665b7de4e61ae5821f49d7459ec P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.358023 12469 tablet_replica.cc:333] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.358944 12469 raft_consensus.cc:2243] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.359586 12469 raft_consensus.cc:2272] T 3cf8b638cc7b4d68a88deeacea18f36d P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.362736 12469 tablet_replica.cc:333] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.363425 12469 raft_consensus.cc:2243] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.363906 12469 raft_consensus.cc:2272] T 4b2217c6605e4f229a55e993de7a8696 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.366734 12469 tablet_replica.cc:333] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.367780 12469 raft_consensus.cc:2243] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.368623 12469 raft_consensus.cc:2272] T b647e01f42114926b7c04cd1ebf9e23f P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.371388 12469 tablet_replica.cc:333] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.372103 12469 raft_consensus.cc:2243] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.373225 12469 raft_consensus.cc:2272] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.377046 12469 tablet_replica.cc:333] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.377930 12469 raft_consensus.cc:2243] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.378532 12469 raft_consensus.cc:2272] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.384714 12469 tablet_replica.cc:333] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.385612 12469 raft_consensus.cc:2243] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.386821 12469 raft_consensus.cc:2272] T 6cb4745cc61545b0a28693865420f43f P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.390195 12469 tablet_replica.cc:333] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.391320 12469 raft_consensus.cc:2243] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.392263 12469 raft_consensus.cc:2272] T bba92ba095b54d48914151dcac5033d1 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.395244 12469 tablet_replica.cc:333] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.395947 12469 raft_consensus.cc:2243] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.396842 12469 raft_consensus.cc:2272] T 46ee7ee8e9664d66823351436762d6b2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.399181 12469 tablet_replica.cc:333] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.399735 12469 raft_consensus.cc:2243] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.401130 12469 raft_consensus.cc:2272] T f3ca1a807fb8465785df25e6f3ba9931 P 643057f6f77241529979f572a42ed9b0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.404126 12469 tablet_replica.cc:333] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.404961 12469 raft_consensus.cc:2243] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.405560 12469 raft_consensus.cc:2272] T 0ee02a1415f74accb6c585fd3556fa3b P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.408769 12469 tablet_replica.cc:333] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.409579 12469 raft_consensus.cc:2243] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.410163 12469 raft_consensus.cc:2272] T ff4761da8e934968ae1169a27136254a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.413304 12469 tablet_replica.cc:333] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
W20260709 12:27:55.413496 26768 consensus_peers.cc:597] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.414350 12469 raft_consensus.cc:2243] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.415081 12469 raft_consensus.cc:2272] T c629cb845a5247089d3810e037fe2035 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.418610 12469 tablet_replica.cc:333] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.419521 12469 raft_consensus.cc:2243] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.420163 12469 raft_consensus.cc:2272] T d187e1b239c246cea5ae376f42a72d6a P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.420619 26694 consensus_peers.cc:597] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.423204 12469 tablet_replica.cc:333] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.424010 12469 raft_consensus.cc:2243] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.424965 12469 raft_consensus.cc:2272] T a60b354295114347b828b0acc3386536 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.428251 12469 tablet_replica.cc:333] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.429124 12469 raft_consensus.cc:2243] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.429742 12469 raft_consensus.cc:2272] T 7dfa515c10414832964dc85f6c82d88b P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.432888 12469 tablet_replica.cc:333] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.433770 12469 raft_consensus.cc:2243] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.435144 12469 raft_consensus.cc:2272] T 8437e96f55e2448e883f6f8bcd057be2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.438374 26768 consensus_peers.cc:597] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.439033 12469 tablet_replica.cc:333] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.439996 12469 raft_consensus.cc:2243] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.440721 12469 raft_consensus.cc:2272] T 413d45f35e654998908cd4c1c6b8a0f2 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.443704 26694 consensus_peers.cc:597] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.444005 12469 tablet_replica.cc:333] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.444893 12469 raft_consensus.cc:2243] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.445524 12469 raft_consensus.cc:2272] T 86583bdc7b4a453f9d4215b64a0d9a86 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.448697 12469 tablet_replica.cc:333] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.449457 12469 raft_consensus.cc:2243] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.450076 12469 raft_consensus.cc:2272] T f229d58a8c304ca1b723a4954aed76f4 P 643057f6f77241529979f572a42ed9b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.453699 12469 tablet_replica.cc:333] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0: stopping tablet replica
I20260709 12:27:55.454553 12469 raft_consensus.cc:2243] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.455212 12469 raft_consensus.cc:2272] T faab3b25212c4a0085b5907de0936703 P 643057f6f77241529979f572a42ed9b0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.485827 26694 consensus_peers.cc:597] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.527642 26768 consensus_peers.cc:597] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.542922 26768 consensus_peers.cc:597] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.549932 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
W20260709 12:27:55.564049 26694 consensus_peers.cc:597] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.578024 26768 consensus_peers.cc:597] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.598305 26694 consensus_peers.cc:597] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.621786 26694 consensus_peers.cc:597] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.646418 26694 consensus_peers.cc:597] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 -> Peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Couldn't send request to peer 643057f6f77241529979f572a42ed9b0. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.676775 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
I20260709 12:27:55.693186 27758 raft_consensus.cc:493] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.694042 27758 raft_consensus.cc:515] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:55.700137 27758 leader_election.cc:290] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:55.700652 27738 raft_consensus.cc:493] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.701370 27738 raft_consensus.cc:515] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
W20260709 12:27:55.702206 26768 leader_election.cc:336] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:55.705912 27738 leader_election.cc:290] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:55.708858 26769 leader_election.cc:336] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.709370 26769 leader_election.cc:304] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:55.710379 27738 raft_consensus.cc:2749] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:27:55.716548 26768 leader_election.cc:336] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
W20260709 12:27:55.719571 26769 leader_election.cc:336] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.720149 26769 leader_election.cc:304] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:55.720850 27738 raft_consensus.cc:2749] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:27:55.733036 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:27:55.734287 12469 tablet_replica.cc:333] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.735400 12469 raft_consensus.cc:2243] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.736060 12469 raft_consensus.cc:2272] T 2bcd7e1d18334a01bc14e417fbbeaff2 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.737685 26768 consensus_peers.cc:597] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.739403 27762 raft_consensus.cc:493] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.739737 12469 tablet_replica.cc:333] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.739962 27762 raft_consensus.cc:515] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:55.740617 12469 raft_consensus.cc:2243] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.741142 12469 raft_consensus.cc:2272] T dc7a25d1e78a4401b1347ed2f07e858d P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.742125 27762 leader_election.cc:290] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:55.744393 12469 tablet_replica.cc:333] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.745589 12469 raft_consensus.cc:2243] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.746347 12469 raft_consensus.cc:2272] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.750083 12469 tablet_replica.cc:333] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
W20260709 12:27:55.750000 26694 leader_election.cc:336] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
W20260709 12:27:55.750775 26695 leader_election.cc:336] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.751271 12469 raft_consensus.cc:2243] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.751242 26695 leader_election.cc:304] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:55.752542 26695 raft_consensus.cc:2686] T e62bc644ef15433785589f1e01b29a89 P f57551729f2f4037a4c769ee1df81853: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:55.753144 12469 raft_consensus.cc:2272] T 2ceb8aabf0a94f9385121858268a3169 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.756562 12469 tablet_replica.cc:333] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.757375 12469 raft_consensus.cc:2243] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.757912 12469 raft_consensus.cc:2272] T 673d53dde2dc42e1b7feb31f4e28bb3c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.761013 12469 tablet_replica.cc:333] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.761758 12469 raft_consensus.cc:2243] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.762336 12469 raft_consensus.cc:2272] T dc276f88554143c3bd0a5b4f74143b2f P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.765131 12469 tablet_replica.cc:333] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.766007 12469 raft_consensus.cc:2243] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.766739 12469 raft_consensus.cc:2272] T 47d7e2ee9fa8426f9ce60485d6e4bb7a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.769690 12469 tablet_replica.cc:333] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.770622 12469 raft_consensus.cc:2243] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.771596 12469 raft_consensus.cc:2272] T 737ddf5e609d480ea8a119e82a057898 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.775763 12469 tablet_replica.cc:333] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.777004 12469 raft_consensus.cc:2243] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.777716 12469 raft_consensus.cc:2272] T 03c01181f6bd4a92915740e17605f4b9 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.781577 12469 tablet_replica.cc:333] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.782318 12469 raft_consensus.cc:2243] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.783258 12469 raft_consensus.cc:2272] T 6e6beba6a6d747ebbcd858a8b61ad68b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.786216 12469 tablet_replica.cc:333] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.786980 12469 raft_consensus.cc:2243] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.787568 12469 raft_consensus.cc:2272] T 6cb4745cc61545b0a28693865420f43f P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.789978 12469 tablet_replica.cc:333] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.790934 12469 raft_consensus.cc:2243] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.791529 12469 raft_consensus.cc:2272] T 51a778606ffa467f9faaadb8412a4e8c P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.794173 12469 tablet_replica.cc:333] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.795048 27738 raft_consensus.cc:493] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.795073 12469 raft_consensus.cc:2243] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.795812 27738 raft_consensus.cc:515] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:55.796197 12469 raft_consensus.cc:2272] T 646472eb33fc46e88f978e2aa3fe8ff8 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.799588 12469 tablet_replica.cc:333] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.800864 12469 raft_consensus.cc:2243] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.802011 12469 raft_consensus.cc:2272] T 2cb970d065d54c94be1a04a340a5821c P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.803851 27738 leader_election.cc:290] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:55.805361 12469 tablet_replica.cc:333] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.806739 12469 raft_consensus.cc:2243] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.807566 12469 raft_consensus.cc:2272] T df9b6736d82e4e18ba0068b834c114d0 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.810791 26768 leader_election.cc:336] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
I20260709 12:27:55.811060 12469 tablet_replica.cc:333] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.812201 12469 raft_consensus.cc:2243] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.812892 12469 raft_consensus.cc:2272] T 83ac1836c91d4280be584799f8bb0c2a P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.816259 12469 tablet_replica.cc:333] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
W20260709 12:27:55.816676 26769 leader_election.cc:336] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.817281 12469 raft_consensus.cc:2243] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.817238 26769 leader_election.cc:304] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:55.818634 12469 raft_consensus.cc:2272] T 95b062d361f340cd9a9202852a196241 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.818825 27765 raft_consensus.cc:2749] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:27:55.822203 27762 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.822433 12469 tablet_replica.cc:333] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.822814 27762 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:55.823801 12469 raft_consensus.cc:2243] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.824359 12469 raft_consensus.cc:2272] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.825093 27762 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6f4d278e29fc4ba68bba0aa75a8d81fa (127.12.45.68:45475), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:55.826087 26801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "473c572574d044c0adc6366487b343fa" candidate_uuid: "f57551729f2f4037a4c769ee1df81853" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" is_pre_election: true
I20260709 12:27:55.829267 12469 tablet_replica.cc:333] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
W20260709 12:27:55.830467 26695 leader_election.cc:336] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.830642 12469 raft_consensus.cc:2243] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.831043 26695 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 6f4d278e29fc4ba68bba0aa75a8d81fa, cb56388a4ff14b5a9f8f1196dc94461f
W20260709 12:27:55.831664 26695 raft_consensus.cc:2686] T 473c572574d044c0adc6366487b343fa P f57551729f2f4037a4c769ee1df81853: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:27:55.832108 12469 raft_consensus.cc:2272] T cbf182133ad942e582a03b114c25daf3 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.835870 12469 tablet_replica.cc:333] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.836952 12469 raft_consensus.cc:2243] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.837970 12469 raft_consensus.cc:2272] T 5bf5c3113a22464e81d4cb8462e8ac67 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.838020 26768 consensus_peers.cc:597] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.840904 12469 tablet_replica.cc:333] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.840884 27765 raft_consensus.cc:493] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.841974 12469 raft_consensus.cc:2243] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.841600 27765 raft_consensus.cc:515] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:55.842631 12469 raft_consensus.cc:2272] T bba92ba095b54d48914151dcac5033d1 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.844024 26768 consensus_peers.cc:597] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:55.845947 12469 tablet_replica.cc:333] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.844578 27765 leader_election.cc:290] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:55.846956 12469 raft_consensus.cc:2243] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.847602 12469 raft_consensus.cc:2272] T 86583bdc7b4a453f9d4215b64a0d9a86 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.850744 12469 tablet_replica.cc:333] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
W20260709 12:27:55.851305 26769 leader_election.cc:336] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.851667 12469 raft_consensus.cc:2243] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.852424 12469 raft_consensus.cc:2272] T f229d58a8c304ca1b723a4954aed76f4 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.853675 26768 consensus_peers.cc:597] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.854375 26768 leader_election.cc:336] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
I20260709 12:27:55.854947 26768 leader_election.cc:304] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:55.855724 27758 raft_consensus.cc:2749] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:27:55.855931 12469 tablet_replica.cc:333] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.856896 12469 raft_consensus.cc:2243] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.857849 12469 raft_consensus.cc:2272] T faab3b25212c4a0085b5907de0936703 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.861063 12469 tablet_replica.cc:333] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.861778 12469 raft_consensus.cc:2243] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.862982 12469 raft_consensus.cc:2272] T 744dd7e87b4540ad930b1de1885f334a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.863126 27762 raft_consensus.cc:493] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.863749 27762 raft_consensus.cc:515] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } }
I20260709 12:27:55.865842 27762 leader_election.cc:290] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
I20260709 12:27:55.866387 12469 tablet_replica.cc:333] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.867249 12469 raft_consensus.cc:2243] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.867785 12469 raft_consensus.cc:2272] T b1d43a3098bd4f5d80095bc30638c181 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.871173 26694 leader_election.cc:336] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
I20260709 12:27:55.871628 12469 tablet_replica.cc:333] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.872812 12469 raft_consensus.cc:2243] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:55.872907 26695 leader_election.cc:336] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.873466 26695 leader_election.cc:304] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f57551729f2f4037a4c769ee1df81853; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:55.873606 12469 raft_consensus.cc:2272] T fdc9b665b7de4e61ae5821f49d7459ec P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.874171 27762 raft_consensus.cc:2749] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:27:55.876444 12469 tablet_replica.cc:333] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.877197 12469 raft_consensus.cc:2243] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.878181 12469 raft_consensus.cc:2272] T 293707805bab498a9449cd938d3b1b6b P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.880690 12469 tablet_replica.cc:333] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.881323 12469 raft_consensus.cc:2243] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.882223 12469 raft_consensus.cc:2272] T 6147d09bb6fe4ec5be65fa9239d10536 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.884423 12469 tablet_replica.cc:333] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.885035 12469 raft_consensus.cc:2243] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.885953 12469 raft_consensus.cc:2272] T c629cb845a5247089d3810e037fe2035 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.888264 12469 tablet_replica.cc:333] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.888855 12469 raft_consensus.cc:2243] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.889858 12469 raft_consensus.cc:2272] T d187e1b239c246cea5ae376f42a72d6a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.892719 12469 tablet_replica.cc:333] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.893368 12469 raft_consensus.cc:2243] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.893880 12469 raft_consensus.cc:2272] T 6cde6c3890534a20956dd0622e8cb1dd P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.896859 12469 tablet_replica.cc:333] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.897449 12469 raft_consensus.cc:2243] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.897950 12469 raft_consensus.cc:2272] T a60b354295114347b828b0acc3386536 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.900388 12469 tablet_replica.cc:333] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.900997 12469 raft_consensus.cc:2243] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.901485 12469 raft_consensus.cc:2272] T 518c064d1a7740bbbb35c46cfa79a3ff P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.903612 12469 tablet_replica.cc:333] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.904551 12469 raft_consensus.cc:2243] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.905164 12469 raft_consensus.cc:2272] T 46ee7ee8e9664d66823351436762d6b2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.907614 12469 tablet_replica.cc:333] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.908231 12469 raft_consensus.cc:2243] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.908727 12469 raft_consensus.cc:2272] T 7dfa515c10414832964dc85f6c82d88b P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.911226 12469 tablet_replica.cc:333] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.911892 12469 raft_consensus.cc:2243] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.912814 12469 raft_consensus.cc:2272] T 413d45f35e654998908cd4c1c6b8a0f2 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.915242 12469 tablet_replica.cc:333] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.915799 12469 raft_consensus.cc:2243] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.916281 12469 raft_consensus.cc:2272] T f3ca1a807fb8465785df25e6f3ba9931 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.919057 12469 tablet_replica.cc:333] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.919740 12469 raft_consensus.cc:2243] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.920820 12469 raft_consensus.cc:2272] T ff4761da8e934968ae1169a27136254a P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.923588 12469 tablet_replica.cc:333] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.924175 12469 raft_consensus.cc:2243] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.924615 12469 raft_consensus.cc:2272] T 559dcf990edd4760b228f7ecff46a4b7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.926905 12469 tablet_replica.cc:333] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.927497 12469 raft_consensus.cc:2243] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:55.928452 12469 raft_consensus.cc:2272] T 4b2217c6605e4f229a55e993de7a8696 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.931236 12469 tablet_replica.cc:333] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.931994 12469 raft_consensus.cc:2243] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.932528 12469 raft_consensus.cc:2272] T 6f355a7f12b14432844a7a4508117120 P f57551729f2f4037a4c769ee1df81853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.934435 27758 raft_consensus.cc:493] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.935322 27758 raft_consensus.cc:515] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:55.935657 12469 tablet_replica.cc:333] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
I20260709 12:27:55.936365 12469 raft_consensus.cc:2243] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.936879 12469 raft_consensus.cc:2272] T 3a4f3d968244475e805596ce58c213e7 P f57551729f2f4037a4c769ee1df81853 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:55.938000 27758 leader_election.cc:290] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:55.940268 12469 tablet_replica.cc:333] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853: stopping tablet replica
W20260709 12:27:55.940506 26768 leader_election.cc:336] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:27:55.941134 12469 raft_consensus.cc:2243] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:55.941740 12469 raft_consensus.cc:2272] T 9dc5d1308d484c77941c0fc3a0322157 P f57551729f2f4037a4c769ee1df81853 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:55.942621 26769 leader_election.cc:336] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:55.943073 26769 leader_election.cc:304] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:55.943704 27758 raft_consensus.cc:2749] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:27:55.992431 27758 raft_consensus.cc:493] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:55.992952 27758 raft_consensus.cc:515] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:55.996691 27758 leader_election.cc:290] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:56.051102 26768 leader_election.cc:336] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
W20260709 12:27:56.052819 26769 leader_election.cc:336] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:56.059934 26769 leader_election.cc:304] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:56.061887 27758 raft_consensus.cc:2749] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:27:56.117925 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
W20260709 12:27:56.134909 26768 consensus_peers.cc:597] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:56.160763 26768 consensus_peers.cc:597] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:56.203926 27758 raft_consensus.cc:493] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:56.204115 27765 raft_consensus.cc:493] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:56.206606 27765 raft_consensus.cc:515] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:56.210991 27758 raft_consensus.cc:515] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } } peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } }
I20260709 12:27:56.220513 27765 leader_election.cc:290] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
I20260709 12:27:56.222635 27758 leader_election.cc:290] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295), 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883)
W20260709 12:27:56.228487 26768 leader_election.cc:336] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111)
W20260709 12:27:56.229720 26768 leader_election.cc:336] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
W20260709 12:27:56.232535 26769 leader_election.cc:336] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:56.233242 26769 leader_election.cc:304] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, cb56388a4ff14b5a9f8f1196dc94461f
I20260709 12:27:56.234251 27765 raft_consensus.cc:2749] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:27:56.235440 26769 leader_election.cc:336] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:56.235877 26769 leader_election.cc:304] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:56.236685 27765 raft_consensus.cc:2749] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:27:56.237722 26768 consensus_peers.cc:597] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa -> Peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Couldn't send request to peer f57551729f2f4037a4c769ee1df81853. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:56.244653 27765 raft_consensus.cc:493] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cb56388a4ff14b5a9f8f1196dc94461f)
I20260709 12:27:56.245565 27765 raft_consensus.cc:515] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } } peers { permanent_uuid: "cb56388a4ff14b5a9f8f1196dc94461f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37295 } }
I20260709 12:27:56.247960 27765 leader_election.cc:290] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937), cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295)
W20260709 12:27:56.255822 26768 leader_election.cc:336] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111)
W20260709 12:27:56.256834 26769 leader_election.cc:336] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cb56388a4ff14b5a9f8f1196dc94461f (127.12.45.65:37295): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37295: connect: Connection refused (error 111)
I20260709 12:27:56.257371 26769 leader_election.cc:304] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: cb56388a4ff14b5a9f8f1196dc94461f, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:56.258492 27765 raft_consensus.cc:2749] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:27:56.325029 12469 tablet_server.cc:179] TabletServer@127.12.45.68:0 shutting down...
I20260709 12:27:56.387619 27758 raft_consensus.cc:493] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 643057f6f77241529979f572a42ed9b0)
I20260709 12:27:56.388509 27758 raft_consensus.cc:515] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:56.391191 27758 leader_election.cc:290] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
W20260709 12:27:56.396157 26768 leader_election.cc:336] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
I20260709 12:27:56.396522 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:27:56.397161 26768 leader_election.cc:336] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111)
I20260709 12:27:56.397807 12469 tablet_replica.cc:333] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.397676 26768 leader_election.cc:304] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:56.398689 27758 raft_consensus.cc:2749] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:56.399353 12469 raft_consensus.cc:2243] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.400990 12469 raft_consensus.cc:2272] T b2e5dc8624444d368676b1a1b3fa878e P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.404681 12469 tablet_replica.cc:333] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.405853 12469 raft_consensus.cc:2243] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.407100 12469 raft_consensus.cc:2272] T f229d58a8c304ca1b723a4954aed76f4 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.409952 12469 tablet_replica.cc:333] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.410813 12469 raft_consensus.cc:2243] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.412369 12469 raft_consensus.cc:2272] T dc7a25d1e78a4401b1347ed2f07e858d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.416446 12469 tablet_replica.cc:333] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.417608 12469 raft_consensus.cc:2243] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.418678 12469 raft_consensus.cc:2272] T 2bcd7e1d18334a01bc14e417fbbeaff2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.421346 12469 tablet_replica.cc:333] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.422029 12469 raft_consensus.cc:2243] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.422849 12469 raft_consensus.cc:2272] T 2ceb8aabf0a94f9385121858268a3169 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.425312 12469 tablet_replica.cc:333] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.426008 12469 raft_consensus.cc:2243] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.426704 12469 raft_consensus.cc:2272] T dc276f88554143c3bd0a5b4f74143b2f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.429239 12469 tablet_replica.cc:333] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.429883 12469 raft_consensus.cc:2243] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.431000 12469 raft_consensus.cc:2272] T 03c01181f6bd4a92915740e17605f4b9 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.433871 12469 tablet_replica.cc:333] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.434718 12469 raft_consensus.cc:2243] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.435412 27758 raft_consensus.cc:493] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 643057f6f77241529979f572a42ed9b0)
I20260709 12:27:56.435532 12469 raft_consensus.cc:2272] T 6e6beba6a6d747ebbcd858a8b61ad68b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.436270 27758 raft_consensus.cc:515] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:56.438583 27758 leader_election.cc:290] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:56.439087 12469 tablet_replica.cc:333] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.440361 12469 raft_consensus.cc:2243] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.442236 12469 raft_consensus.cc:2272] T 37ea7dd7d5f4407abb57b9027b2bf586 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:27:56.443568 26768 leader_election.cc:336] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
W20260709 12:27:56.444342 26768 leader_election.cc:336] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111)
I20260709 12:27:56.444666 26768 leader_election.cc:304] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:56.445359 27758 raft_consensus.cc:2749] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:27:56.445480 12469 tablet_replica.cc:333] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.446468 12469 raft_consensus.cc:2243] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.447512 12469 raft_consensus.cc:2272] T d89d07b1f88649de8fe8c262f31a0ac6 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.450079 12469 tablet_replica.cc:333] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.450883 12469 raft_consensus.cc:2243] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.451463 12469 raft_consensus.cc:2272] T 51a778606ffa467f9faaadb8412a4e8c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.454198 12469 tablet_replica.cc:333] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.455039 12469 raft_consensus.cc:2243] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.455626 12469 raft_consensus.cc:2272] T 646472eb33fc46e88f978e2aa3fe8ff8 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.457952 12469 tablet_replica.cc:333] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.458637 12469 raft_consensus.cc:2243] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.459199 12469 raft_consensus.cc:2272] T 2cb970d065d54c94be1a04a340a5821c P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.461794 12469 tablet_replica.cc:333] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.462539 12469 raft_consensus.cc:2243] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.463696 12469 raft_consensus.cc:2272] T 3d4e1d3b6a6a41bc81db8ef63b2cc50a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.466190 12469 tablet_replica.cc:333] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.466877 12469 raft_consensus.cc:2243] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.467970 12469 raft_consensus.cc:2272] T df9b6736d82e4e18ba0068b834c114d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.470620 12469 tablet_replica.cc:333] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.471302 12469 raft_consensus.cc:2243] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.472671 12469 raft_consensus.cc:2272] T cf0134ff709c4ab3991a5bb56a32f3a1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.476442 12469 tablet_replica.cc:333] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.477170 12469 raft_consensus.cc:2243] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.478446 12469 raft_consensus.cc:2272] T 8bd7e0d139cb41beb4076b3fe3672c73 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.480872 12469 tablet_replica.cc:333] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.481525 12469 raft_consensus.cc:2243] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.482044 12469 raft_consensus.cc:2272] T 7bd8a7d163624a52837a57273d4a7892 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.484380 12469 tablet_replica.cc:333] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.485019 12469 raft_consensus.cc:2243] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.485675 12469 raft_consensus.cc:2272] T b1d43a3098bd4f5d80095bc30638c181 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.488435 12469 tablet_replica.cc:333] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.489116 12469 raft_consensus.cc:2243] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.490366 12469 raft_consensus.cc:2272] T 5150550259ec4e3fac2af6e10b6ca9b5 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.492753 12469 tablet_replica.cc:333] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.493517 12469 raft_consensus.cc:2243] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.494240 12469 raft_consensus.cc:2272] T 95b062d361f340cd9a9202852a196241 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.496719 12469 tablet_replica.cc:333] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.497428 12469 raft_consensus.cc:2243] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.497978 12469 raft_consensus.cc:2272] T 473c572574d044c0adc6366487b343fa P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.500450 12469 tablet_replica.cc:333] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.501066 12469 raft_consensus.cc:2243] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.501557 12469 raft_consensus.cc:2272] T cbf182133ad942e582a03b114c25daf3 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.503840 12469 tablet_replica.cc:333] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.504522 12469 raft_consensus.cc:2243] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.505059 12469 raft_consensus.cc:2272] T 8437e96f55e2448e883f6f8bcd057be2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.507550 12469 tablet_replica.cc:333] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.508193 12469 raft_consensus.cc:2243] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.509100 12469 raft_consensus.cc:2272] T 413d45f35e654998908cd4c1c6b8a0f2 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.512454 12469 tablet_replica.cc:333] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.513257 12469 raft_consensus.cc:2243] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.513797 12469 raft_consensus.cc:2272] T d187e1b239c246cea5ae376f42a72d6a P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.516407 12469 tablet_replica.cc:333] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.517012 12469 raft_consensus.cc:2243] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.518056 12469 raft_consensus.cc:2272] T 6cde6c3890534a20956dd0622e8cb1dd P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.520670 12469 tablet_replica.cc:333] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.521322 12469 raft_consensus.cc:2243] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.521874 12469 raft_consensus.cc:2272] T a60b354295114347b828b0acc3386536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.524547 12469 tablet_replica.cc:333] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.525365 12469 raft_consensus.cc:2243] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.525956 12469 raft_consensus.cc:2272] T 518c064d1a7740bbbb35c46cfa79a3ff P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.528770 12469 tablet_replica.cc:333] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.529644 12469 raft_consensus.cc:2243] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.530655 12469 raft_consensus.cc:2272] T 4b2217c6605e4f229a55e993de7a8696 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.533622 12469 tablet_replica.cc:333] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.534538 12469 raft_consensus.cc:2243] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.535214 12469 raft_consensus.cc:2272] T 3cf8b638cc7b4d68a88deeacea18f36d P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.537858 12469 tablet_replica.cc:333] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.538527 12469 raft_consensus.cc:2243] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.539211 12469 raft_consensus.cc:2272] T 2564769c4e85479fbfb15bd8a05a5807 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.541687 12469 tablet_replica.cc:333] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.542335 12469 raft_consensus.cc:2243] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.542933 12469 raft_consensus.cc:2272] T 293707805bab498a9449cd938d3b1b6b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.545361 12469 tablet_replica.cc:333] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.546386 12469 raft_consensus.cc:2243] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.546161 27758 raft_consensus.cc:493] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 643057f6f77241529979f572a42ed9b0)
I20260709 12:27:56.547762 27758 raft_consensus.cc:515] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "643057f6f77241529979f572a42ed9b0" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43883 } } peers { permanent_uuid: "f57551729f2f4037a4c769ee1df81853" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 39937 } } peers { permanent_uuid: "6f4d278e29fc4ba68bba0aa75a8d81fa" member_type: VOTER last_known_addr { host: "127.12.45.68" port: 45475 } }
I20260709 12:27:56.547884 12469 raft_consensus.cc:2272] T 6147d09bb6fe4ec5be65fa9239d10536 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.549978 27758 leader_election.cc:290] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883), f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937)
I20260709 12:27:56.554536 12469 tablet_replica.cc:333] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
W20260709 12:27:56.555496 26768 leader_election.cc:336] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 643057f6f77241529979f572a42ed9b0 (127.12.45.66:43883): Network error: Client connection negotiation failed: client connection to 127.12.45.66:43883: connect: Connection refused (error 111)
I20260709 12:27:56.555755 12469 raft_consensus.cc:2243] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:27:56.556337 26768 leader_election.cc:336] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f57551729f2f4037a4c769ee1df81853 (127.12.45.67:39937): Network error: Client connection negotiation failed: client connection to 127.12.45.67:39937: connect: Connection refused (error 111)
I20260709 12:27:56.556710 26768 leader_election.cc:304] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f4d278e29fc4ba68bba0aa75a8d81fa; no voters: 643057f6f77241529979f572a42ed9b0, f57551729f2f4037a4c769ee1df81853
I20260709 12:27:56.556739 12469 raft_consensus.cc:2272] T 5d0fc2016a9b4f2390b55edc0d8289f0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.557382 27758 raft_consensus.cc:2749] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:27:56.559913 12469 tablet_replica.cc:333] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.560618 12469 raft_consensus.cc:2243] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.561182 12469 raft_consensus.cc:2272] T b647e01f42114926b7c04cd1ebf9e23f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.564342 12469 tablet_replica.cc:333] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.564885 12469 raft_consensus.cc:2243] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.565393 12469 raft_consensus.cc:2272] T 3f37c0d19f1249cdaa3b380b1c0b01d0 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.567813 12469 tablet_replica.cc:333] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.568334 12469 raft_consensus.cc:2243] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.568781 12469 raft_consensus.cc:2272] T 6cb4745cc61545b0a28693865420f43f P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.571056 12469 tablet_replica.cc:333] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.571568 12469 raft_consensus.cc:2243] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.572058 12469 raft_consensus.cc:2272] T 5bf5c3113a22464e81d4cb8462e8ac67 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.574906 12469 tablet_replica.cc:333] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.575510 12469 raft_consensus.cc:2243] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.576006 12469 raft_consensus.cc:2272] T bba92ba095b54d48914151dcac5033d1 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.578271 12469 tablet_replica.cc:333] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.578879 12469 raft_consensus.cc:2243] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.579428 12469 raft_consensus.cc:2272] T f3ca1a807fb8465785df25e6f3ba9931 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.582222 12469 tablet_replica.cc:333] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.582819 12469 raft_consensus.cc:2243] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:27:56.583350 12469 raft_consensus.cc:2272] T 559dcf990edd4760b228f7ecff46a4b7 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.586845 12469 tablet_replica.cc:333] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.587528 12469 raft_consensus.cc:2243] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.588747 12469 raft_consensus.cc:2272] T 0ee02a1415f74accb6c585fd3556fa3b P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.591169 12469 tablet_replica.cc:333] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa: stopping tablet replica
I20260709 12:27:56.591634 12469 raft_consensus.cc:2243] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.592780 12469 raft_consensus.cc:2272] T 9dc5d1308d484c77941c0fc3a0322157 P 6f4d278e29fc4ba68bba0aa75a8d81fa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.700098 12469 tablet_server.cc:196] TabletServer@127.12.45.68:0 shutdown complete.
I20260709 12:27:56.792811 12469 master.cc:562] Master@127.12.45.126:41947 shutting down...
I20260709 12:27:56.825440 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:27:56.826318 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:27:56.826701 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8a50c935c236466db6d2ca6b4a823a7c: stopping tablet replica
I20260709 12:27:56.852012 12469 master.cc:584] Master@127.12.45.126:41947 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (34832 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:27:56.967172 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:36311
I20260709 12:27:56.968669 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:56.974931 27768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:56.975085 27769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:56.976428 27771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:56.979033 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:56.980448 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:56.980649 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:56.980847 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600076980829 us; error 0 us; skew 500 ppm
I20260709 12:27:56.981499 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:56.984578 12469 webserver.cc:533] Webserver started at http://127.12.45.126:42875/ using document root <none> and password file <none>
I20260709 12:27:56.985180 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:56.985395 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:56.985729 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:56.987248 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "0c8b1f39d3034f8b938af1581a0dc76f"
format_stamp: "Formatted at 2026-07-09 12:27:56 on dist-test-slave-90bm"
I20260709 12:27:56.992254 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:27:56.996285 27776 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:56.997190 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:27:56.997582 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "0c8b1f39d3034f8b938af1581a0dc76f"
format_stamp: "Formatted at 2026-07-09 12:27:56 on dist-test-slave-90bm"
I20260709 12:27:56.997939 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:57.020452 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:57.022003 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:57.072180 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:36311
I20260709 12:27:57.072285 27827 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:36311 every 8 connection(s)
I20260709 12:27:57.077414 27828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.098543 27828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f: Bootstrap starting.
I20260709 12:27:57.105603 27828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:57.114302 27828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f: No bootstrap required, opened a new log
I20260709 12:27:57.117848 27828 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" member_type: VOTER }
I20260709 12:27:57.118403 27828 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:57.118592 27828 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8b1f39d3034f8b938af1581a0dc76f, State: Initialized, Role: FOLLOWER
I20260709 12:27:57.119186 27828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" member_type: VOTER }
I20260709 12:27:57.119640 27828 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:27:57.119846 27828 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:27:57.120075 27828 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:57.126024 27828 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" member_type: VOTER }
I20260709 12:27:57.127044 27828 leader_election.cc:304] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8b1f39d3034f8b938af1581a0dc76f; no voters: 
I20260709 12:27:57.128407 27828 leader_election.cc:290] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:27:57.129027 27831 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:57.130831 27831 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 1 LEADER]: Becoming Leader. State: Replica: 0c8b1f39d3034f8b938af1581a0dc76f, State: Running, Role: LEADER
I20260709 12:27:57.131798 27831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" member_type: VOTER }
I20260709 12:27:57.131937 27828 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:27:57.137200 27832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" member_type: VOTER } }
I20260709 12:27:57.137956 27832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [sys.catalog]: This master's current role is: LEADER
I20260709 12:27:57.138249 27833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8b1f39d3034f8b938af1581a0dc76f. Latest consensus state: current_term: 1 leader_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8b1f39d3034f8b938af1581a0dc76f" member_type: VOTER } }
I20260709 12:27:57.139251 27833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [sys.catalog]: This master's current role is: LEADER
I20260709 12:27:57.141450 27838 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:27:57.147415 27838 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:27:57.152432 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:27:57.158059 27838 catalog_manager.cc:1383] Generated new cluster ID: dda7eba10bd5472c8106d375674d74ff
I20260709 12:27:57.158437 27838 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:27:57.183753 27838 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:27:57.185369 27838 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:27:57.199747 27838 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f: Generated new TSK 0
I20260709 12:27:57.200660 27838 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:27:57.220388 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:57.228099 27849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:57.230307 27850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:57.232558 12469 server_base.cc:1061] running on GCE node
W20260709 12:27:57.233172 27852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:57.240139 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:57.240482 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:57.240697 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600077240678 us; error 0 us; skew 500 ppm
I20260709 12:27:57.241339 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:57.244546 12469 webserver.cc:533] Webserver started at http://127.12.45.65:43431/ using document root <none> and password file <none>
I20260709 12:27:57.245146 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:57.245381 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:57.245671 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:57.246986 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "2ab6b96c5e65444a80252670af0f1d24"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:27:57.252324 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:27:57.256552 27857 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.257858 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:27:57.258298 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "2ab6b96c5e65444a80252670af0f1d24"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:27:57.258664 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:57.276981 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:57.278614 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:57.280937 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:57.283865 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:57.284068 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.284369 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:57.284597 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.337896 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:37903
I20260709 12:27:57.338093 27919 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:37903 every 8 connection(s)
I20260709 12:27:57.344290 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:57.352154 27924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:57.352463 27925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:57.355755 27928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:57.356235 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:57.358019 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:57.358338 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:57.358557 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600077358537 us; error 0 us; skew 500 ppm
I20260709 12:27:57.359346 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:57.361123 27920 heartbeater.cc:344] Connected to a master server at 127.12.45.126:36311
I20260709 12:27:57.361541 27920 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:57.362428 12469 webserver.cc:533] Webserver started at http://127.12.45.66:39709/ using document root <none> and password file <none>
I20260709 12:27:57.362605 27920 heartbeater.cc:507] Master 127.12.45.126:36311 requested a full tablet report, sending...
I20260709 12:27:57.363252 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:57.363526 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:57.363890 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:57.365315 27793 ts_manager.cc:194] Registered new tserver with Master: 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903)
I20260709 12:27:57.365698 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "663c28d0e3c9414486153d7d2c36a1ae"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:27:57.368232 27793 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52044
I20260709 12:27:57.372407 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:27:57.376468 27932 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.377375 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:27:57.377660 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "663c28d0e3c9414486153d7d2c36a1ae"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:27:57.377974 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:57.405464 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:57.406888 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:57.408522 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:57.411286 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:57.411525 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.411758 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:57.411980 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.465013 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:45613
I20260709 12:27:57.465454 27994 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:45613 every 8 connection(s)
I20260709 12:27:57.470744 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:27:57.480504 27998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:27:57.484696 27999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:57.489161 27995 heartbeater.cc:344] Connected to a master server at 127.12.45.126:36311
W20260709 12:27:57.489729 28001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:27:57.489880 27995 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:57.490168 12469 server_base.cc:1061] running on GCE node
I20260709 12:27:57.491222 27995 heartbeater.cc:507] Master 127.12.45.126:36311 requested a full tablet report, sending...
I20260709 12:27:57.491516 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:27:57.491850 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:27:57.492086 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600077492065 us; error 0 us; skew 500 ppm
I20260709 12:27:57.492821 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:27:57.494776 27793 ts_manager.cc:194] Registered new tserver with Master: 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:27:57.496863 12469 webserver.cc:533] Webserver started at http://127.12.45.67:32907/ using document root <none> and password file <none>
I20260709 12:27:57.497336 27793 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52048
I20260709 12:27:57.497830 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:27:57.498183 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:27:57.498564 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:27:57.500669 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "579731fab2d44e00b7f200e96d5f4bb9"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:27:57.506743 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 12:27:57.511277 28006 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.512497 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:27:57.512822 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "579731fab2d44e00b7f200e96d5f4bb9"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:27:57.513098 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:27:57.568336 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:27:57.569756 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:27:57.571398 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:27:57.574020 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:27:57.574225 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.574570 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:27:57.574795 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:27:57.631472 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:42399
I20260709 12:27:57.631577 28068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:42399 every 8 connection(s)
I20260709 12:27:57.650992 28069 heartbeater.cc:344] Connected to a master server at 127.12.45.126:36311
I20260709 12:27:57.651602 28069 heartbeater.cc:461] Registering TS with master...
I20260709 12:27:57.652691 28069 heartbeater.cc:507] Master 127.12.45.126:36311 requested a full tablet report, sending...
I20260709 12:27:57.655356 27793 ts_manager.cc:194] Registered new tserver with Master: 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:57.656018 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019372129s
I20260709 12:27:57.657531 27793 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52056
I20260709 12:27:57.688935 27793 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52072:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:27:57.692260 27793 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:27:57.783370 27960 tablet_service.cc:1511] Processing CreateTablet for tablet bc46b610261c4908a191697b0d4e7471 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:27:57.786242 27960 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc46b610261c4908a191697b0d4e7471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.788408 27959 tablet_service.cc:1511] Processing CreateTablet for tablet e0bfd3dd40104d25bb9148497df3913a (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:27:57.791723 27958 tablet_service.cc:1511] Processing CreateTablet for tablet d794190450c94c91b65193a6013ff9c3 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:27:57.793437 27959 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bfd3dd40104d25bb9148497df3913a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.798710 27958 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d794190450c94c91b65193a6013ff9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.803759 27957 tablet_service.cc:1511] Processing CreateTablet for tablet 87d48713787d4f3094b63dd3059d2642 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:27:57.805696 27957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d48713787d4f3094b63dd3059d2642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.818415 27956 tablet_service.cc:1511] Processing CreateTablet for tablet 6829a4c5fd1f46b0a3d38b1defe419af (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:27:57.818734 27885 tablet_service.cc:1511] Processing CreateTablet for tablet bc46b610261c4908a191697b0d4e7471 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:27:57.820389 27885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc46b610261c4908a191697b0d4e7471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.823189 27955 tablet_service.cc:1511] Processing CreateTablet for tablet 15849d102a27427684b28905bc02f55d (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:27:57.820369 27956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6829a4c5fd1f46b0a3d38b1defe419af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.817821 27884 tablet_service.cc:1511] Processing CreateTablet for tablet e0bfd3dd40104d25bb9148497df3913a (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:27:57.827235 27955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15849d102a27427684b28905bc02f55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.827744 27884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bfd3dd40104d25bb9148497df3913a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.831513 27954 tablet_service.cc:1511] Processing CreateTablet for tablet 93a0ab731e4a4eef8cc308a4637fb636 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:27:57.833235 27954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a0ab731e4a4eef8cc308a4637fb636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.836194 28034 tablet_service.cc:1511] Processing CreateTablet for tablet bc46b610261c4908a191697b0d4e7471 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:27:57.837962 28034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc46b610261c4908a191697b0d4e7471. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.838027 27953 tablet_service.cc:1511] Processing CreateTablet for tablet 869da3fc7b8849c898130dafc04ff652 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:27:57.839565 27882 tablet_service.cc:1511] Processing CreateTablet for tablet 87d48713787d4f3094b63dd3059d2642 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:27:57.840481 27953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869da3fc7b8849c898130dafc04ff652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.840919 27879 tablet_service.cc:1511] Processing CreateTablet for tablet 93a0ab731e4a4eef8cc308a4637fb636 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:27:57.838104 27883 tablet_service.cc:1511] Processing CreateTablet for tablet d794190450c94c91b65193a6013ff9c3 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:27:57.842667 27881 tablet_service.cc:1511] Processing CreateTablet for tablet 6829a4c5fd1f46b0a3d38b1defe419af (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:27:57.843549 27878 tablet_service.cc:1511] Processing CreateTablet for tablet 869da3fc7b8849c898130dafc04ff652 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:27:57.839578 27880 tablet_service.cc:1511] Processing CreateTablet for tablet 15849d102a27427684b28905bc02f55d (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:27:57.855180 27876 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7bb407bc6d4d939d9d03602f825637 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:27:57.849694 27877 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebd577c08a746f49df51dd2dbfbbd1e (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:27:57.857558 27951 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7bb407bc6d4d939d9d03602f825637 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:27:57.841187 27882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d48713787d4f3094b63dd3059d2642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.859462 27951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7bb407bc6d4d939d9d03602f825637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.862187 27879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a0ab731e4a4eef8cc308a4637fb636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.851725 27952 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebd577c08a746f49df51dd2dbfbbd1e (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:27:57.865660 27883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d794190450c94c91b65193a6013ff9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.865971 27952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebd577c08a746f49df51dd2dbfbbd1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.866888 27876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7bb407bc6d4d939d9d03602f825637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.874852 27878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869da3fc7b8849c898130dafc04ff652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.879936 27877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebd577c08a746f49df51dd2dbfbbd1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.881147 27880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15849d102a27427684b28905bc02f55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.887732 27881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6829a4c5fd1f46b0a3d38b1defe419af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.887782 28033 tablet_service.cc:1511] Processing CreateTablet for tablet e0bfd3dd40104d25bb9148497df3913a (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:27:57.889619 28033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bfd3dd40104d25bb9148497df3913a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.892951 28032 tablet_service.cc:1511] Processing CreateTablet for tablet d794190450c94c91b65193a6013ff9c3 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:27:57.896620 28032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d794190450c94c91b65193a6013ff9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.896303 28031 tablet_service.cc:1511] Processing CreateTablet for tablet 87d48713787d4f3094b63dd3059d2642 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:27:57.898736 28031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87d48713787d4f3094b63dd3059d2642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.911912 28030 tablet_service.cc:1511] Processing CreateTablet for tablet 6829a4c5fd1f46b0a3d38b1defe419af (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:27:57.913908 28030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6829a4c5fd1f46b0a3d38b1defe419af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.963929 28032 tablet_service.cc:1511] Processing CreateTablet for tablet 93a0ab731e4a4eef8cc308a4637fb636 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:27:57.967751 28029 tablet_service.cc:1511] Processing CreateTablet for tablet 869da3fc7b8849c898130dafc04ff652 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:27:57.971374 28028 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebd577c08a746f49df51dd2dbfbbd1e (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:27:57.978884 28032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a0ab731e4a4eef8cc308a4637fb636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.978382 28027 tablet_service.cc:1511] Processing CreateTablet for tablet 0f7bb407bc6d4d939d9d03602f825637 (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:27:57.986927 28088 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:57.991001 28087 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:57.992297 28029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869da3fc7b8849c898130dafc04ff652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.993212 28028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebd577c08a746f49df51dd2dbfbbd1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.994652 28027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f7bb407bc6d4d939d9d03602f825637. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:57.994670 28034 tablet_service.cc:1511] Processing CreateTablet for tablet 15849d102a27427684b28905bc02f55d (DEFAULT_TABLE table=test-workload [id=db6062a8214542f5a01dd2ed67f970f8]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:27:58.000366 28034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15849d102a27427684b28905bc02f55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:58.004993 28089 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.020392 28087 tablet_bootstrap.cc:654] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.034950 28088 tablet_bootstrap.cc:654] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.024540 28089 tablet_bootstrap.cc:654] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.052690 28088 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.053232 28088 ts_tablet_manager.cc:1403] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.067s	user 0.002s	sys 0.013s
I20260709 12:27:58.056780 28088 raft_consensus.cc:359] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.057583 28088 raft_consensus.cc:385] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.057907 28088 raft_consensus.cc:740] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.059631 28088 consensus_queue.cc:260] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.086377 28089 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.088110 28088 ts_tablet_manager.cc:1434] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.035s	user 0.012s	sys 0.001s
I20260709 12:27:58.088725 28089 ts_tablet_manager.cc:1403] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.084s	user 0.024s	sys 0.002s
I20260709 12:27:58.090521 27995 heartbeater.cc:499] Master 127.12.45.126:36311 was elected leader, sending a full tablet report...
I20260709 12:27:58.090814 28088 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.086454 28094 raft_consensus.cc:493] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.098769 28094 raft_consensus.cc:515] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.107771 28088 tablet_bootstrap.cc:654] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.108786 28087 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.109441 28087 ts_tablet_manager.cc:1403] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.119s	user 0.023s	sys 0.021s
I20260709 12:27:58.111336 28089 raft_consensus.cc:359] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.112630 28089 raft_consensus.cc:385] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.113062 28089 raft_consensus.cc:740] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.116218 28094 leader_election.cc:290] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.112849 28087 raft_consensus.cc:359] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.120460 28087 raft_consensus.cc:385] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.120872 28087 raft_consensus.cc:740] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.121763 28087 consensus_queue.cc:260] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.114477 28089 consensus_queue.cc:260] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.129133 27920 heartbeater.cc:499] Master 127.12.45.126:36311 was elected leader, sending a full tablet report...
I20260709 12:27:58.131314 28087 ts_tablet_manager.cc:1434] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:27:58.132715 28087 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.145073 28089 ts_tablet_manager.cc:1434] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.056s	user 0.012s	sys 0.003s
I20260709 12:27:58.146234 28069 heartbeater.cc:499] Master 127.12.45.126:36311 was elected leader, sending a full tablet report...
I20260709 12:27:58.146587 28089 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.153489 28088 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.154213 28088 ts_tablet_manager.cc:1403] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.064s	user 0.033s	sys 0.022s
I20260709 12:27:58.157708 28088 raft_consensus.cc:359] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.158560 28088 raft_consensus.cc:385] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.158854 28088 raft_consensus.cc:740] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.159373 28087 tablet_bootstrap.cc:654] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.160383 28088 consensus_queue.cc:260] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.165758 28089 tablet_bootstrap.cc:654] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.165614 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
W20260709 12:27:58.167994 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:58.169325 28099 raft_consensus.cc:493] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.169893 28099 raft_consensus.cc:515] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.173204 28099 leader_election.cc:290] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
W20260709 12:27:58.169325 27936 leader_election.cc:343] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.181258 28088 ts_tablet_manager.cc:1434] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:27:58.183794 28088 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.194217 28088 tablet_bootstrap.cc:654] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.208478 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
W20260709 12:27:58.210561 27936 leader_election.cc:343] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.211267 27936 leader_election.cc:304] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:27:58.212881 28094 raft_consensus.cc:2749] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.218780 28088 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.220111 28088 ts_tablet_manager.cc:1403] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.019s
W20260709 12:27:58.220525 27996 tablet.cc:2406] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:27:58.223886 28088 raft_consensus.cc:359] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.224725 28088 raft_consensus.cc:385] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.225216 28088 raft_consensus.cc:740] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.226430 28088 consensus_queue.cc:260] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.227558 28089 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.228333 28089 ts_tablet_manager.cc:1403] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.082s	user 0.031s	sys 0.024s
I20260709 12:27:58.229468 28088 ts_tablet_manager.cc:1434] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:27:58.230849 28088 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.238364 28088 tablet_bootstrap.cc:654] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.240629 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
W20260709 12:27:58.243206 28010 leader_election.cc:343] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.250813 28087 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.250834 28088 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.251835 28088 ts_tablet_manager.cc:1403] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.014s
I20260709 12:27:58.253130 28089 raft_consensus.cc:359] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.253996 28089 raft_consensus.cc:385] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.254325 28089 raft_consensus.cc:740] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.255414 28089 consensus_queue.cc:260] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.255746 28088 raft_consensus.cc:359] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.256747 28088 raft_consensus.cc:385] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.257155 28088 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.257747 28087 ts_tablet_manager.cc:1403] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.125s	user 0.023s	sys 0.009s
I20260709 12:27:58.258530 27970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.258343 28088 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.261466 28088 ts_tablet_manager.cc:1434] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:27:58.261794 28087 raft_consensus.cc:359] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.262598 28087 raft_consensus.cc:385] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.262787 28088 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.262993 28087 raft_consensus.cc:740] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.264464 28089 ts_tablet_manager.cc:1434] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
W20260709 12:27:58.264696 28007 leader_election.cc:343] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.265298 28007 leader_election.cc:304] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9; no voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.265116 28087 consensus_queue.cc:260] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.267675 28087 ts_tablet_manager.cc:1434] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.006s
I20260709 12:27:58.268479 28098 raft_consensus.cc:2749] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.270862 28089 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.271482 28088 tablet_bootstrap.cc:654] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.271653 28087 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.280063 28089 tablet_bootstrap.cc:654] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.282226 28088 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.282996 28088 ts_tablet_manager.cc:1403] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 12:27:58.286137 28087 tablet_bootstrap.cc:654] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.290577 28088 raft_consensus.cc:359] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.291489 28088 raft_consensus.cc:385] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.292001 28088 raft_consensus.cc:740] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.292418 28089 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.293038 28089 ts_tablet_manager.cc:1403] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:27:58.293038 28088 consensus_queue.cc:260] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.296972 28089 raft_consensus.cc:359] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.297825 28089 raft_consensus.cc:385] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.298141 28089 raft_consensus.cc:740] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.298673 28088 ts_tablet_manager.cc:1434] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:27:58.299347 28089 consensus_queue.cc:260] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.300165 28088 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.303524 28089 ts_tablet_manager.cc:1434] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:27:58.303678 28087 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.304469 28087 ts_tablet_manager.cc:1403] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.008s
I20260709 12:27:58.304872 28089 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.308698 28088 tablet_bootstrap.cc:654] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.308993 28087 raft_consensus.cc:359] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.311075 28087 raft_consensus.cc:385] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.311616 28087 raft_consensus.cc:740] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.312795 28087 consensus_queue.cc:260] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.313587 28089 tablet_bootstrap.cc:654] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.318372 28087 ts_tablet_manager.cc:1434] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:27:58.319993 28087 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.322872 28088 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.323859 28088 ts_tablet_manager.cc:1403] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.010s
I20260709 12:27:58.327275 28087 tablet_bootstrap.cc:654] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.328472 28089 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.327973 28088 raft_consensus.cc:359] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.329173 28088 raft_consensus.cc:385] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.329793 28088 raft_consensus.cc:740] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.329227 28089 ts_tablet_manager.cc:1403] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.012s
I20260709 12:27:58.331930 28088 consensus_queue.cc:260] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.334569 28089 raft_consensus.cc:359] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.335364 28088 ts_tablet_manager.cc:1434] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 12:27:58.335778 28089 raft_consensus.cc:385] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.336025 28094 raft_consensus.cc:493] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.336261 28089 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.336618 28094 raft_consensus.cc:515] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.337040 28088 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.337759 28089 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.339129 28094 leader_election.cc:290] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.340828 28089 ts_tablet_manager.cc:1434] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:27:58.341372 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869da3fc7b8849c898130dafc04ff652" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.342336 28089 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.343154 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869da3fc7b8849c898130dafc04ff652" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.343710 28098 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.344290 28098 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
W20260709 12:27:58.345031 27936 leader_election.cc:343] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.346823 28098 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
W20260709 12:27:58.348723 27936 leader_election.cc:343] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.349354 27936 leader_election.cc:304] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:27:58.349850 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.350203 27970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.350463 28094 raft_consensus.cc:2749] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.350564 28089 tablet_bootstrap.cc:654] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
W20260709 12:27:58.351611 28010 leader_election.cc:343] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:27:58.352370 28007 leader_election.cc:343] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.352861 28007 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9; no voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.354233 28098 raft_consensus.cc:2749] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.354797 28088 tablet_bootstrap.cc:654] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.354965 28094 raft_consensus.cc:493] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.355757 28094 raft_consensus.cc:515] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.359304 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.360001 27895 raft_consensus.cc:2468] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.361167 28087 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.361876 28087 ts_tablet_manager.cc:1403] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.042s	user 0.009s	sys 0.006s
I20260709 12:27:58.362674 27936 leader_election.cc:304] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.363826 28093 raft_consensus.cc:2804] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.364233 28093 raft_consensus.cc:493] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.364586 28093 raft_consensus.cc:3060] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.365513 28087 raft_consensus.cc:359] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.365914 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.366389 28087 raft_consensus.cc:385] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.366631 28044 raft_consensus.cc:2468] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.367365 28094 leader_election.cc:290] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.366719 28087 raft_consensus.cc:740] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.369668 28087 consensus_queue.cc:260] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.380671 28087 ts_tablet_manager.cc:1434] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:27:58.376859 28093 raft_consensus.cc:515] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.382094 28087 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.384387 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.385218 27895 raft_consensus.cc:3060] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.387230 28088 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.387897 28088 ts_tablet_manager.cc:1403] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.051s	user 0.010s	sys 0.012s
I20260709 12:27:58.390329 28087 tablet_bootstrap.cc:654] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.390805 28089 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.391376 28089 ts_tablet_manager.cc:1403] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.018s
I20260709 12:27:58.391330 28088 raft_consensus.cc:359] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.392357 28088 raft_consensus.cc:385] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.392673 28088 raft_consensus.cc:740] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.393663 28088 consensus_queue.cc:260] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.395218 28089 raft_consensus.cc:359] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.395994 28089 raft_consensus.cc:385] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.396313 28089 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.397133 28089 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.399560 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.399729 28089 ts_tablet_manager.cc:1434] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:27:58.400202 28044 raft_consensus.cc:3060] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.401084 28089 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.408499 28088 ts_tablet_manager.cc:1434] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:27:58.409055 28093 leader_election.cc:290] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.409659 28089 tablet_bootstrap.cc:654] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.412068 28087 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.412660 28087 ts_tablet_manager.cc:1403] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.000s
I20260709 12:27:58.415534 27895 raft_consensus.cc:2468] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:58.416481 28087 raft_consensus.cc:359] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.417250 28087 raft_consensus.cc:385] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.417557 28087 raft_consensus.cc:740] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.417841 27936 leader_election.cc:304] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.416322 28044 raft_consensus.cc:2468] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:58.418905 28093 raft_consensus.cc:2804] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.418516 28087 consensus_queue.cc:260] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.423399 28093 raft_consensus.cc:697] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:27:58.423732 28089 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.424353 28089 ts_tablet_manager.cc:1403] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.012s
I20260709 12:27:58.424409 28093 consensus_queue.cc:237] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.425210 28087 ts_tablet_manager.cc:1434] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:27:58.426486 28087 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.428021 28097 raft_consensus.cc:493] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.428109 28089 raft_consensus.cc:359] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.428783 28097 raft_consensus.cc:515] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.429147 28089 raft_consensus.cc:385] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.430158 28089 raft_consensus.cc:740] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.422407 28088 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.431384 28089 consensus_queue.cc:260] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.436338 28087 tablet_bootstrap.cc:654] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.437628 28089 ts_tablet_manager.cc:1434] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:27:58.439030 28089 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.444384 28088 tablet_bootstrap.cc:654] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.450034 28089 tablet_bootstrap.cc:654] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.452623 27793 catalog_manager.cc:5697] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 1 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.473740 28097 leader_election.cc:290] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.475792 28089 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.476748 28089 ts_tablet_manager.cc:1403] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.011s
I20260709 12:27:58.485170 28089 raft_consensus.cc:359] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.486393 28089 raft_consensus.cc:385] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.486794 28089 raft_consensus.cc:740] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.493567 28089 consensus_queue.cc:260] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.501075 28089 ts_tablet_manager.cc:1434] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 12:27:58.502862 28089 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.519764 28089 tablet_bootstrap.cc:654] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.519739 28093 raft_consensus.cc:493] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.520835 28093 raft_consensus.cc:515] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.525358 28093 leader_election.cc:290] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.527693 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.528410 28087 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.528514 28044 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.529049 28087 ts_tablet_manager.cc:1403] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.103s	user 0.020s	sys 0.054s
I20260709 12:27:58.529978 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.530128 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.531716 27936 leader_election.cc:304] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
W20260709 12:27:58.532881 27936 leader_election.cc:343] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.530812 28043 raft_consensus.cc:2468] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:58.535218 28093 raft_consensus.cc:2804] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.535667 28093 raft_consensus.cc:493] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.536725 28093 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.535571 27861 leader_election.cc:304] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:27:58.538595 28097 raft_consensus.cc:2804] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.539269 28097 raft_consensus.cc:493] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.539870 28097 raft_consensus.cc:3060] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.532944 28087 raft_consensus.cc:359] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.541297 28087 raft_consensus.cc:385] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.541677 28087 raft_consensus.cc:740] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.543320 28087 consensus_queue.cc:260] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.548427 27970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.538553 28088 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.549305 27970 raft_consensus.cc:2468] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:58.549697 28088 ts_tablet_manager.cc:1403] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.128s	user 0.050s	sys 0.029s
I20260709 12:27:58.551281 28093 raft_consensus.cc:515] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.554988 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.556350 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.557806 28093 leader_election.cc:290] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
W20260709 12:27:58.558084 27936 leader_election.cc:343] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.558248 28043 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.560810 28089 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.561313 28087 ts_tablet_manager.cc:1434] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:27:58.561565 28089 ts_tablet_manager.cc:1403] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.059s	user 0.030s	sys 0.008s
I20260709 12:27:58.562522 28087 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.564857 28089 raft_consensus.cc:359] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.565606 28089 raft_consensus.cc:385] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.565897 28089 raft_consensus.cc:740] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.566649 28089 consensus_queue.cc:260] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.554414 28097 raft_consensus.cc:515] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.571614 28089 ts_tablet_manager.cc:1434] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:27:58.572463 28087 tablet_bootstrap.cc:654] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.572710 28089 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.573092 27970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:27:58.573668 27970 raft_consensus.cc:3060] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.579739 28089 tablet_bootstrap.cc:654] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.556371 28088 raft_consensus.cc:359] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.582741 28088 raft_consensus.cc:385] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.584517 28088 raft_consensus.cc:740] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.569383 28043 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:58.583842 27970 raft_consensus.cc:2468] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 1.
I20260709 12:27:58.587996 27936 leader_election.cc:304] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24
I20260709 12:27:58.587795 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.588436 28088 consensus_queue.cc:260] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.588683 28097 leader_election.cc:290] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Requested vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.589556 28093 raft_consensus.cc:2804] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.590569 28093 raft_consensus.cc:697] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:27:58.589419 28044 raft_consensus.cc:3060] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.591610 28093 consensus_queue.cc:237] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.593178 27858 leader_election.cc:304] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.594720 28126 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.595249 28126 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.601140 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.602123 28126 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.603199 28126 raft_consensus.cc:2804] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.605178 28126 raft_consensus.cc:697] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Running, Role: LEADER
I20260709 12:27:58.605508 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.606364 28126 consensus_queue.cc:237] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
W20260709 12:27:58.607445 27858 leader_election.cc:343] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.610507 28043 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:58.612442 27861 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.618809 28126 raft_consensus.cc:2804] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.619685 28126 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.620319 28126 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.621039 28088 ts_tablet_manager.cc:1434] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.005s
I20260709 12:27:58.621980 27792 catalog_manager.cc:5697] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "2ab6b96c5e65444a80252670af0f1d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.623395 28088 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.628360 28044 raft_consensus.cc:2468] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 1.
I20260709 12:27:58.631976 28089 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.632938 28089 ts_tablet_manager.cc:1403] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.004s
I20260709 12:27:58.636905 28089 raft_consensus.cc:359] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.637662 28089 raft_consensus.cc:385] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.637822 28087 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.638671 28087 ts_tablet_manager.cc:1403] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.076s	user 0.026s	sys 0.042s
I20260709 12:27:58.641498 28088 tablet_bootstrap.cc:654] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.642511 28087 raft_consensus.cc:359] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.643249 28087 raft_consensus.cc:385] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.643605 28087 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.642370 27789 catalog_manager.cc:5697] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 1 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.637945 28089 raft_consensus.cc:740] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.646134 28089 consensus_queue.cc:260] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.649027 28089 ts_tablet_manager.cc:1434] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:27:58.649405 28087 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.650203 28089 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9: Bootstrap starting.
I20260709 12:27:58.650791 28126 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.651616 28088 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.652217 28088 ts_tablet_manager.cc:1403] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.001s
I20260709 12:27:58.654503 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.653333 28087 ts_tablet_manager.cc:1434] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:27:58.655212 28044 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.656798 27970 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:27:58.657220 28088 raft_consensus.cc:359] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.658856 28088 raft_consensus.cc:385] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.659200 28088 raft_consensus.cc:740] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.657816 28087 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.659987 28097 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.660552 28097 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.663178 28097 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.664124 28044 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 1.
I20260709 12:27:58.664451 28098 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.664924 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.665096 28098 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.665537 27861 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:27:58.666230 28043 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:58.666574 28099 raft_consensus.cc:493] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.667265 28099 raft_consensus.cc:515] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.668226 27861 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:27:58.670231 28097 raft_consensus.cc:2804] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.672163 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869da3fc7b8849c898130dafc04ff652" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.672884 27969 raft_consensus.cc:2468] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 0.
I20260709 12:27:58.672734 28088 consensus_queue.cc:260] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.675215 27970 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.671366 28093 raft_consensus.cc:493] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.676874 28093 raft_consensus.cc:515] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.677191 28088 ts_tablet_manager.cc:1434] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:27:58.678689 28088 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: Bootstrap starting.
I20260709 12:27:58.670632 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869da3fc7b8849c898130dafc04ff652" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.679926 27895 raft_consensus.cc:2468] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 0.
I20260709 12:27:58.682566 28087 tablet_bootstrap.cc:654] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.683753 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.671705 28126 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Requested vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.686062 28010 leader_election.cc:304] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:27:58.686863 28126 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.687299 28126 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.688925 28126 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.671361 28099 leader_election.cc:290] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:27:58.682803 28093 leader_election.cc:290] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.695776 28129 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.696128 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.685948 28044 raft_consensus.cc:2468] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.699513 28099 raft_consensus.cc:493] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.701503 28099 raft_consensus.cc:515] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.703825 28099 leader_election.cc:290] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:27:58.704375 28099 raft_consensus.cc:2804] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.704711 28099 raft_consensus.cc:493] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.705042 28099 raft_consensus.cc:3060] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.706601 28129 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.708061 27969 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:58.708190 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.709637 28129 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.709975 28098 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
W20260709 12:27:58.710690 28007 leader_election.cc:343] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.710706 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.712924 28129 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.716326 28093 raft_consensus.cc:493] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.726424 27894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.730878 27936 leader_election.cc:304] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.732712 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.733829 27968 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 0.
I20260709 12:27:58.733804 28094 raft_consensus.cc:2804] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.734061 28093 raft_consensus.cc:515] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.735105 28094 raft_consensus.cc:493] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.735606 28094 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.737336 28093 leader_election.cc:290] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
W20260709 12:27:58.739498 27936 leader_election.cc:343] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.739673 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.740636 27895 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.743270 27936 leader_election.cc:304] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.745705 28094 raft_consensus.cc:515] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.745774 28093 raft_consensus.cc:2804] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.747305 28089 tablet_bootstrap.cc:654] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.752336 28088 tablet_bootstrap.cc:654] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.711179 28097 raft_consensus.cc:697] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Running, Role: LEADER
I20260709 12:27:58.747411 28093 raft_consensus.cc:493] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.724350 28129 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.760217 28093 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.758762 27970 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 1.
I20260709 12:27:58.759094 28094 leader_election.cc:290] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.764040 28007 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.766223 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.767472 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.769490 28087 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
W20260709 12:27:58.772150 28010 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.65:37903, user_credentials={real_user=slave}} blocked reactor thread for 86244.4us
I20260709 12:27:58.772444 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.773386 28043 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.772152 28087 ts_tablet_manager.cc:1403] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.115s	user 0.013s	sys 0.013s
I20260709 12:27:58.774487 28044 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:58.776706 28126 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.777419 27892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.777920 27894 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 663c28d0e3c9414486153d7d2c36a1ae in current term 1: Already voted for candidate 2ab6b96c5e65444a80252670af0f1d24 in this term.
I20260709 12:27:58.774463 27893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.754045 28097 consensus_queue.cc:237] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.781684 28126 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Requested vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.783717 28135 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.787002 28044 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.787817 28044 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.788017 28129 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.789654 28093 raft_consensus.cc:515] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
W20260709 12:27:58.791270 27936 leader_election.cc:343] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.791539 27894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:58.799875 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.792173 28135 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.799362 27936 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24
I20260709 12:27:58.801005 28135 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:27:58.801589 28010 leader_election.cc:343] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.792666 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.792932 28093 leader_election.cc:290] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.782919 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:27:58.802491 28010 leader_election.cc:304] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9; no voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.802474 28129 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:58.805486 28129 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.806195 28129 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.806893 27894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.807077 28099 raft_consensus.cc:515] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.807655 27894 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.809530 28099 leader_election.cc:290] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:27:58.811419 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869da3fc7b8849c898130dafc04ff652" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:27:58.812374 27968 raft_consensus.cc:3060] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.810226 28099 raft_consensus.cc:2749] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.820982 28089 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9: No bootstrap required, opened a new log
I20260709 12:27:58.823239 28089 ts_tablet_manager.cc:1403] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9: Time spent bootstrapping tablet: real 0.173s	user 0.009s	sys 0.006s
I20260709 12:27:58.822721 27789 catalog_manager.cc:5697] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "2ab6b96c5e65444a80252670af0f1d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.829130 28044 raft_consensus.cc:2468] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
W20260709 12:27:58.822439 27936 leader_election.cc:343] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.829581 28088 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: No bootstrap required, opened a new log
I20260709 12:27:58.830184 28088 ts_tablet_manager.cc:1403] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.014s
W20260709 12:27:58.830329 27936 leader_election.cc:343] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:27:58.832988 28089 raft_consensus.cc:359] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.834203 28089 raft_consensus.cc:385] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.834590 28089 raft_consensus.cc:740] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.835270 28088 raft_consensus.cc:359] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.835417 28089 consensus_queue.cc:260] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.836113 28088 raft_consensus.cc:385] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.836467 28088 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.826844 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.837212 28088 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.837713 28043 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.838043 28089 ts_tablet_manager.cc:1434] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:27:58.830973 27936 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:27:58.827080 27894 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:58.845219 28093 raft_consensus.cc:2749] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.845973 28129 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.824911 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869da3fc7b8849c898130dafc04ff652" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.847088 27895 raft_consensus.cc:3060] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.847445 27969 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ab6b96c5e65444a80252670af0f1d24 in current term 1: Already voted for candidate 663c28d0e3c9414486153d7d2c36a1ae in this term.
I20260709 12:27:58.848040 28043 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:58.833546 28135 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.805549 28087 raft_consensus.cc:359] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.850994 28087 raft_consensus.cc:385] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.851317 28087 raft_consensus.cc:740] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.851619 28042 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2ab6b96c5e65444a80252670af0f1d24 in current term 1: Already voted for candidate 579731fab2d44e00b7f200e96d5f4bb9 in this term.
I20260709 12:27:58.852144 28087 consensus_queue.cc:260] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.854113 27861 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24; no voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.855827 27936 leader_election.cc:304] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24
I20260709 12:27:58.856772 28093 raft_consensus.cc:2804] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.857283 28093 raft_consensus.cc:697] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:27:58.857468 27893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.858407 27893 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 579731fab2d44e00b7f200e96d5f4bb9 in current term 1: Already voted for candidate 2ab6b96c5e65444a80252670af0f1d24 in this term.
I20260709 12:27:58.858233 28093 consensus_queue.cc:237] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.859267 27895 raft_consensus.cc:2468] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 1.
I20260709 12:27:58.860162 28097 raft_consensus.cc:2749] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.860845 28010 leader_election.cc:304] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:27:58.861995 28136 raft_consensus.cc:2804] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.862908 28136 raft_consensus.cc:697] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:27:58.863339 27936 leader_election.cc:304] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:58.863684 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:27:58.864609 27969 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 579731fab2d44e00b7f200e96d5f4bb9 in current term 1: Already voted for candidate 663c28d0e3c9414486153d7d2c36a1ae in this term.
I20260709 12:27:58.864027 28136 consensus_queue.cc:237] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.866307 28007 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9; no voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.866497 28094 raft_consensus.cc:2804] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:58.867425 28097 raft_consensus.cc:493] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.868001 28099 raft_consensus.cc:2749] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.868407 28097 raft_consensus.cc:515] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.868911 28135 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:27:58.872061 28129 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.872718 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:58.875958 28097 leader_election.cc:290] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:58.868700 28087 ts_tablet_manager.cc:1434] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.001s
I20260709 12:27:58.876739 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:58.824836 27968 raft_consensus.cc:2468] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 1.
I20260709 12:27:58.877560 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:58.878464 28043 raft_consensus.cc:2393] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ab6b96c5e65444a80252670af0f1d24 in current term 1: Already voted for candidate 663c28d0e3c9414486153d7d2c36a1ae in this term.
I20260709 12:27:58.877810 27895 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 663c28d0e3c9414486153d7d2c36a1ae in current term 1: Already voted for candidate 2ab6b96c5e65444a80252670af0f1d24 in this term.
I20260709 12:27:58.879575 27861 leader_election.cc:304] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24; no voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:58.879709 28129 consensus_queue.cc:1048] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:58.881263 28129 consensus_queue.cc:1048] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:58.881735 28097 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:58.883934 28087 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.886528 28094 raft_consensus.cc:697] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:27:58.888273 28094 consensus_queue.cc:237] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.893143 28087 tablet_bootstrap.cc:654] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.883854 28088 ts_tablet_manager.cc:1434] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: Time spent starting tablet: real 0.053s	user 0.004s	sys 0.002s
I20260709 12:27:58.907876 27789 catalog_manager.cc:5697] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:58.912258 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:58.913252 28043 raft_consensus.cc:2393] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 663c28d0e3c9414486153d7d2c36a1ae in current term 1: Already voted for candidate 579731fab2d44e00b7f200e96d5f4bb9 in this term.
I20260709 12:27:58.913110 28097 raft_consensus.cc:2749] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.918426 27936 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:27:58.919277 28129 raft_consensus.cc:2749] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:27:58.921015 27792 catalog_manager.cc:5697] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 1 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.923110 27792 catalog_manager.cc:5697] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 1 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.937371 28087 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.938053 28087 ts_tablet_manager.cc:1403] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.014s
I20260709 12:27:58.941510 28087 raft_consensus.cc:359] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.942185 28087 raft_consensus.cc:385] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.942494 28087 raft_consensus.cc:740] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.943392 28087 consensus_queue.cc:260] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.945328 28087 ts_tablet_manager.cc:1434] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:27:58.946353 28087 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:27:58.952915 28087 tablet_bootstrap.cc:654] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Neither blocks nor log segments found. Creating new log.
I20260709 12:27:58.958462 28087 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: No bootstrap required, opened a new log
I20260709 12:27:58.958952 28087 ts_tablet_manager.cc:1403] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 12:27:58.961602 28087 raft_consensus.cc:359] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.962376 28087 raft_consensus.cc:385] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:27:58.962658 28087 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:27:58.963364 28087 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:58.965592 28087 ts_tablet_manager.cc:1434] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:27:59.044623 28129 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:59.047490 28129 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.055665 28097 consensus_queue.cc:1048] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.074662 28134 consensus_queue.cc:1048] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:59.077108 28126 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.077682 28126 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.081514 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:59.081961 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:59.082343 27968 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:59.082976 28042 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 0.
I20260709 12:27:59.083767 27858 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:59.083937 28126 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:59.084779 28134 raft_consensus.cc:2804] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:59.085224 28134 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.085562 28134 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.095082 28134 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.097967 28134 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Requested vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:59.098145 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:27:59.098711 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:59.099054 27968 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.099416 28043 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.108485 27968 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 1.
I20260709 12:27:59.108485 28043 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab6b96c5e65444a80252670af0f1d24 in term 1.
I20260709 12:27:59.110553 27861 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:59.111636 28134 raft_consensus.cc:2804] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:59.112272 28134 raft_consensus.cc:697] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Running, Role: LEADER
I20260709 12:27:59.113139 28134 consensus_queue.cc:237] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.121894 27793 catalog_manager.cc:5697] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "2ab6b96c5e65444a80252670af0f1d24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:59.170804 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:27:59.204074 28093 raft_consensus.cc:493] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.204813 28093 raft_consensus.cc:515] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.206991 28093 leader_election.cc:290] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:59.208102 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:27:59.208272 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:59.209055 27895 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:59.209056 28043 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 0.
I20260709 12:27:59.210619 27936 leader_election.cc:304] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:59.211392 28093 raft_consensus.cc:2804] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:27:59.211790 28093 raft_consensus.cc:493] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.212107 28093 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.216369 28134 consensus_queue.cc:1048] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.219537 28093 raft_consensus.cc:515] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.222160 28093 leader_election.cc:290] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:59.222978 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:27:59.223744 27895 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.224900 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:27:59.225582 28043 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.232851 28134 consensus_queue.cc:1048] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:59.242051 27895 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:59.243675 27936 leader_election.cc:304] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:27:59.244747 28093 raft_consensus.cc:2804] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:27:59.244865 28043 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:27:59.245445 28093 raft_consensus.cc:697] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:27:59.246517 28093 consensus_queue.cc:237] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.256199 27793 catalog_manager.cc:5697] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 1 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:59.259416 28134 raft_consensus.cc:493] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.260080 28134 raft_consensus.cc:515] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:27:59.262252 28134 leader_election.cc:290] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:27:59.263527 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:27:59.264778 28043 raft_consensus.cc:2393] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ab6b96c5e65444a80252670af0f1d24 in current term 1: Already voted for candidate 663c28d0e3c9414486153d7d2c36a1ae in this term.
I20260709 12:27:59.265666 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "2ab6b96c5e65444a80252670af0f1d24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:27:59.267999 27858 leader_election.cc:304] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24; no voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:27:59.268999 28134 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:27:59.275588 28134 raft_consensus.cc:2749] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:27:59.322628 28136 consensus_queue.cc:1048] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.335949 28161 consensus_queue.cc:1048] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:59.372164 28093 consensus_queue.cc:1048] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.386895 28093 consensus_queue.cc:1048] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.466048 28093 consensus_queue.cc:1048] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.474949 28142 consensus_queue.cc:1048] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:59.625331 28134 consensus_queue.cc:1048] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.639920 28133 consensus_queue.cc:1048] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:27:59.655328 28093 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:27:59.670146 28094 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:00.173671 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:00.481499 28094 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.482097 28094 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:00.483960 28094 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:00.484980 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24" is_pre_election: true
I20260709 12:28:00.485548 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:28:00.485662 27895 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:28:00.486336 28042 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:28:00.487506 27936 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:00.488466 28094 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:00.488807 28094 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.489357 28094 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:00.496227 28094 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:00.498160 28094 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:00.499228 27895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:00.499689 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:00.499926 27895 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:00.500422 28042 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:00.506119 27895 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 2.
I20260709 12:28:00.506772 28042 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 2.
I20260709 12:28:00.507571 27936 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:00.508445 28094 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:00.508984 28094 raft_consensus.cc:697] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:00.509819 28094 consensus_queue.cc:237] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:00.518606 27793 catalog_manager.cc:5697] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 0 to 2, leader changed from <none> to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 2 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:00.974709 27895 raft_consensus.cc:1275] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:00.975997 28042 raft_consensus.cc:1275] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:00.976503 28094 consensus_queue.cc:1048] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:00.977780 28093 consensus_queue.cc:1048] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:01.156231 12469 ts_manager.cc:295] Set tserver state for 2ab6b96c5e65444a80252670af0f1d24 to MAINTENANCE_MODE
I20260709 12:28:01.157197 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
W20260709 12:28:01.167687 27936 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 166 similar messages]
W20260709 12:28:01.173770 27936 consensus_peers.cc:597] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:01.174832 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:01.186815 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:01.187896 12469 tablet_replica.cc:333] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.188818 12469 raft_consensus.cc:2243] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.189455 12469 raft_consensus.cc:2272] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:01.191926 28010 consensus_peers.cc:597] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:01.192075 12469 tablet_replica.cc:333] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.192937 12469 raft_consensus.cc:2243] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:01.193804 12469 raft_consensus.cc:2272] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.196069 12469 tablet_replica.cc:333] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.196652 12469 raft_consensus.cc:2243] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.197047 12469 raft_consensus.cc:2272] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.199206 12469 tablet_replica.cc:333] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.199765 12469 raft_consensus.cc:2243] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.200245 12469 raft_consensus.cc:2272] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.202518 12469 tablet_replica.cc:333] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.203063 12469 raft_consensus.cc:2243] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:01.203904 12469 raft_consensus.cc:2272] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.205942 12469 tablet_replica.cc:333] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.206594 12469 raft_consensus.cc:2243] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.207005 12469 raft_consensus.cc:2272] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.209065 12469 tablet_replica.cc:333] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.209642 12469 raft_consensus.cc:2243] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.210038 12469 raft_consensus.cc:2272] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.212656 12469 tablet_replica.cc:333] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.213222 12469 raft_consensus.cc:2243] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.213620 12469 raft_consensus.cc:2272] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.215852 12469 tablet_replica.cc:333] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.216336 12469 raft_consensus.cc:2243] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:01.216719 12469 raft_consensus.cc:2272] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:01.219211 12469 tablet_replica.cc:333] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: stopping tablet replica
I20260709 12:28:01.219718 12469 raft_consensus.cc:2243] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:01.220649 12469 raft_consensus.cc:2272] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:01.234097 27936 consensus_peers.cc:597] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:01.243734 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
W20260709 12:28:01.248039 27936 consensus_peers.cc:597] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:01.456609 27936 consensus_peers.cc:597] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:01.553864 27936 consensus_peers.cc:597] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:01.586735 27936 consensus_peers.cc:597] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:02.175963 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:02.208607 28191 raft_consensus.cc:493] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.209430 28191 raft_consensus.cc:515] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.211527 28191 leader_election.cc:290] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:02.213222 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
W20260709 12:28:02.215902 27936 leader_election.cc:336] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.216450 27936 leader_election.cc:304] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:02.217252 28191 raft_consensus.cc:2749] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:02.373427 28191 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.374075 28191 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.376259 28191 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:02.378336 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
W20260709 12:28:02.380757 27936 leader_election.cc:336] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.381188 27936 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:02.381875 28191 raft_consensus.cc:2749] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:02.516918 28194 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.517617 28194 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.519634 28194 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:02.520954 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:28:02.521826 27968 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 1.
I20260709 12:28:02.523077 28007 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
W20260709 12:28:02.523610 28010 leader_election.cc:336] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.523856 28194 raft_consensus.cc:2804] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:02.524302 28194 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.524689 28194 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:02.529978 28200 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.530707 28200 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.532773 28200 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:02.532980 28194 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.534053 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:28:02.535455 28194 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:02.536726 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
W20260709 12:28:02.537192 28010 leader_election.cc:336] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.537462 27968 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:02.537748 28010 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9; no voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:02.538749 28194 raft_consensus.cc:2749] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:28:02.541627 28010 leader_election.cc:336] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.545076 27968 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:02.546476 28007 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24
I20260709 12:28:02.547233 28194 raft_consensus.cc:2804] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:02.547777 28194 raft_consensus.cc:697] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:02.548512 28194 consensus_queue.cc:237] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.556957 27793 catalog_manager.cc:5697] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.583696 28191 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.584342 28191 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.586623 28191 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:02.588061 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9" is_pre_election: true
I20260709 12:28:02.588809 28042 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 1.
I20260709 12:28:02.590030 27936 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
W20260709 12:28:02.590873 27936 leader_election.cc:336] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.590883 28191 raft_consensus.cc:2804] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:02.591466 28191 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.591758 28191 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:02.598496 28191 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.600251 28191 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:02.601614 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:02.602298 28042 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:02.605389 27936 leader_election.cc:336] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.608456 28042 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 2.
I20260709 12:28:02.609634 27936 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24
I20260709 12:28:02.610455 28191 raft_consensus.cc:2804] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:02.610914 28191 raft_consensus.cc:697] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:02.611652 28191 consensus_queue.cc:237] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.624405 27793 catalog_manager.cc:5697] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 1 to 2, leader changed from 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 2 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:02.661901 28194 raft_consensus.cc:493] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
I20260709 12:28:02.662436 28194 raft_consensus.cc:515] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.664399 28194 leader_election.cc:290] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:02.665937 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae" is_pre_election: true
I20260709 12:28:02.666998 27968 raft_consensus.cc:2468] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 1.
I20260709 12:28:02.668376 28007 leader_election.cc:304] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:02.669299 28194 raft_consensus.cc:2804] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:02.669725 28194 raft_consensus.cc:493] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2ab6b96c5e65444a80252670af0f1d24)
W20260709 12:28:02.669924 28010 leader_election.cc:336] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.670137 28194 raft_consensus.cc:3060] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:02.676465 28194 raft_consensus.cc:515] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.678248 28194 leader_election.cc:290] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:02.679674 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f7bb407bc6d4d939d9d03602f825637" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:02.680511 27968 raft_consensus.cc:3060] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:02.682651 28010 leader_election.cc:336] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111)
I20260709 12:28:02.686517 27968 raft_consensus.cc:2468] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:02.687774 28007 leader_election.cc:304] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 2ab6b96c5e65444a80252670af0f1d24
I20260709 12:28:02.688686 28194 raft_consensus.cc:2804] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:02.689179 28194 raft_consensus.cc:697] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:02.690076 28194 consensus_queue.cc:237] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:02.700997 27793 catalog_manager.cc:5697] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.944582 27968 raft_consensus.cc:1275] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:02.946091 28194 consensus_queue.cc:1048] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:02.955500 28010 consensus_peers.cc:597] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:03.083014 27936 consensus_peers.cc:597] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:03.171926 28042 raft_consensus.cc:1275] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:03.173524 28191 consensus_queue.cc:1048] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:03.177026 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:03.276952 27968 raft_consensus.cc:1275] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:03.278621 28194 consensus_queue.cc:1048] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:03.293257 28010 consensus_peers.cc:597] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:03.526814 28010 consensus_peers.cc:597] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:03.540377 27936 consensus_peers.cc:597] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:03.619556 27936 consensus_peers.cc:597] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:03.879746 27936 consensus_peers.cc:597] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:03.919412 27936 consensus_peers.cc:597] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:04.069406 27936 consensus_peers.cc:597] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:04.113550 27936 consensus_peers.cc:597] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:04.178401 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:05.179978 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:05.433503 28010 consensus_peers.cc:597] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.576957 27936 consensus_peers.cc:597] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.704425 28010 consensus_peers.cc:597] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:05.936647 28010 consensus_peers.cc:597] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:05.942234 27936 consensus_peers.cc:597] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:06.114619 27936 consensus_peers.cc:597] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:06.181677 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:28:06.214733 27936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111) [suppressed 102 similar messages]
W20260709 12:28:06.217368 27936 consensus_peers.cc:597] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:06.266129 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:06.275919 28221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:06.276319 28222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:06.278932 28224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:06.280130 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:06.281807 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:06.282085 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:06.282317 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600086282295 us; error 0 us; skew 500 ppm
I20260709 12:28:06.283107 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:06.286377 12469 webserver.cc:533] Webserver started at http://127.12.45.65:43431/ using document root <none> and password file <none>
I20260709 12:28:06.287091 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:06.287343 12469 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:28:06.291949 12469 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 12:28:06.296840 28229 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:06.297979 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:28:06.298417 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "2ab6b96c5e65444a80252670af0f1d24"
format_stamp: "Formatted at 2026-07-09 12:27:57 on dist-test-slave-90bm"
I20260709 12:28:06.298909 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:06.314368 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:06.315982 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:06.317716 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:06.322047 28236 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:28:06.362911 27936 consensus_peers.cc:597] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:06.393050 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:28:06.393649 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.073s	user 0.002s	sys 0.002s
I20260709 12:28:06.394079 12469 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:28:06.401628 28236 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.418802 28236 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.419801 28236 tablet_bootstrap.cc:492] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.420481 28236 ts_tablet_manager.cc:1403] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:28:06.423226 28236 raft_consensus.cc:359] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.424116 28236 raft_consensus.cc:740] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.424880 28236 consensus_queue.cc:260] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.428028 28236 ts_tablet_manager.cc:1434] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.008s
I20260709 12:28:06.428819 28236 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.444485 28236 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.445497 28236 tablet_bootstrap.cc:492] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.446158 28236 ts_tablet_manager.cc:1403] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:28:06.448859 12469 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:28:06.449283 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.055s	user 0.048s	sys 0.005s
I20260709 12:28:06.448868 28236 raft_consensus.cc:359] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.449901 28236 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.450788 28236 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.452584 28236 ts_tablet_manager.cc:1434] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:06.453367 28236 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.470515 28236 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.471648 28236 tablet_bootstrap.cc:492] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.472307 28236 ts_tablet_manager.cc:1403] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.019s	user 0.004s	sys 0.012s
I20260709 12:28:06.475251 28236 raft_consensus.cc:359] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.475898 28236 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.476564 28236 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.478247 28236 ts_tablet_manager.cc:1434] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:06.479084 28236 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.494635 28236 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.495600 28236 tablet_bootstrap.cc:492] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.496238 28236 ts_tablet_manager.cc:1403] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.008s
I20260709 12:28:06.499380 28236 raft_consensus.cc:359] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.500231 28236 raft_consensus.cc:740] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.501507 28236 consensus_queue.cc:260] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.503934 28236 ts_tablet_manager.cc:1434] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:06.504827 28236 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.528599 28236 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.530009 28236 tablet_bootstrap.cc:492] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.530759 28236 ts_tablet_manager.cc:1403] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.003s
I20260709 12:28:06.533375 28236 raft_consensus.cc:359] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.534036 28236 raft_consensus.cc:740] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.535261 28236 consensus_queue.cc:260] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.537390 28236 ts_tablet_manager.cc:1434] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:28:06.537483 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:37903
I20260709 12:28:06.537593 28298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:37903 every 8 connection(s)
I20260709 12:28:06.538477 28236 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.543956 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:06.560039 28236 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.561467 28236 tablet_bootstrap.cc:492] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.562415 28236 ts_tablet_manager.cc:1403] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.003s
I20260709 12:28:06.565857 28236 raft_consensus.cc:359] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.566684 28236 raft_consensus.cc:740] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.567513 28236 consensus_queue.cc:260] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.569888 28236 ts_tablet_manager.cc:1434] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:06.570866 28236 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.570791 28299 heartbeater.cc:344] Connected to a master server at 127.12.45.126:36311
I20260709 12:28:06.571635 28299 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:06.572681 28299 heartbeater.cc:507] Master 127.12.45.126:36311 requested a full tablet report, sending...
I20260709 12:28:06.583262 27968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:06.583705 27790 ts_manager.cc:194] Re-registered known tserver with Master: 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903)
I20260709 12:28:06.584431 27968 raft_consensus.cc:606] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:06.588296 27968 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:06.589071 27968 raft_consensus.cc:606] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:06.590909 28219 raft_consensus.cc:993] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:06.591845 28220 raft_consensus.cc:1081] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:06.595124 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:06.595140 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:06.595983 27969 raft_consensus.cc:606] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:06.596127 28042 raft_consensus.cc:493] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.596729 28042 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.598977 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:06.599941 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:06.601097 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:06.603654 28236 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.604952 28236 tablet_bootstrap.cc:492] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.605870 28236 ts_tablet_manager.cc:1403] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.035s	user 0.021s	sys 0.008s
I20260709 12:28:06.609158 28236 raft_consensus.cc:359] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.609892 28236 raft_consensus.cc:740] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.610853 28236 consensus_queue.cc:260] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.610955 28042 raft_consensus.cc:515] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.613298 28236 ts_tablet_manager.cc:1434] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:06.614244 28236 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.615734 27790 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36120
I20260709 12:28:06.618950 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:06.619436 28042 leader_election.cc:290] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:06.619880 27969 raft_consensus.cc:3055] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:06.620239 27969 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:06.621109 28299 heartbeater.cc:499] Master 127.12.45.126:36311 was elected leader, sending a full tablet report...
I20260709 12:28:06.621449 27969 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.625231 27969 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.637362 28219 raft_consensus.cc:993] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:06.637809 28236 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.638221 28220 raft_consensus.cc:1081] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:06.639191 28236 tablet_bootstrap.cc:492] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.640020 28236 ts_tablet_manager.cc:1403] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.011s
I20260709 12:28:06.640677 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:06.641366 28042 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.641803 28042 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:06.645596 27969 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:06.645874 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:06.646780 28273 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.647912 28007 leader_election.cc:304] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:06.649606 28194 raft_consensus.cc:2804] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:06.643177 28236 raft_consensus.cc:359] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.651770 28236 raft_consensus.cc:740] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.652843 28236 consensus_queue.cc:260] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.655582 28236 ts_tablet_manager.cc:1434] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:28:06.657270 28273 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:06.659492 28042 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.661742 28194 raft_consensus.cc:697] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:06.662976 28042 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:06.664605 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:06.664443 28194 consensus_queue.cc:237] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.665228 27969 raft_consensus.cc:3055] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:06.665650 27969 raft_consensus.cc:740] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:06.666872 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:06.666648 27969 consensus_queue.cc:260] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.667369 28236 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.667531 28273 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:06.669008 27969 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:06.680371 27969 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 3.
I20260709 12:28:06.680378 28273 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 3.
I20260709 12:28:06.682700 28272 raft_consensus.cc:3060] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.685731 28007 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579731fab2d44e00b7f200e96d5f4bb9, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:06.688843 27790 catalog_manager.cc:5697] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:06.694146 28309 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:06.703950 28309 raft_consensus.cc:697] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:06.707396 28309 consensus_queue.cc:237] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.712826 28236 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.714030 28236 tablet_bootstrap.cc:492] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.714924 28236 ts_tablet_manager.cc:1403] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.031s
I20260709 12:28:06.719120 28236 raft_consensus.cc:359] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.720367 28236 raft_consensus.cc:740] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.721827 28236 consensus_queue.cc:260] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.725608 28236 ts_tablet_manager.cc:1434] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:28:06.726892 28236 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap starting.
I20260709 12:28:06.739406 27790 catalog_manager.cc:5697] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 2 to 3, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:06.745752 28236 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:06.747197 28236 tablet_bootstrap.cc:492] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Bootstrap complete.
I20260709 12:28:06.747975 28236 ts_tablet_manager.cc:1403] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.002s
I20260709 12:28:06.750995 28236 raft_consensus.cc:359] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.751632 28236 raft_consensus.cc:740] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab6b96c5e65444a80252670af0f1d24, State: Initialized, Role: FOLLOWER
I20260709 12:28:06.752254 28236 consensus_queue.cc:260] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:06.754076 28236 ts_tablet_manager.cc:1434] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:28:06.925648 28272 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.928800 28273 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:06.990149 28220 raft_consensus.cc:993] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:06.990649 28220 raft_consensus.cc:1081] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:06.992635 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:06.993338 28042 raft_consensus.cc:493] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:06.993623 28042 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.000844 28042 raft_consensus.cc:515] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.002600 28042 leader_election.cc:290] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:07.003512 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:07.003648 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:07.004288 28273 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.004493 27969 raft_consensus.cc:3055] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:07.004921 27969 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:07.005815 27969 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.007122 27969 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.010538 28273 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:07.012898 28010 leader_election.cc:304] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:07.013576 27969 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:07.013792 28309 raft_consensus.cc:2804] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:07.014565 28309 raft_consensus.cc:697] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:07.015707 28309 consensus_queue.cc:237] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.023094 27790 catalog_manager.cc:5697] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.054158 27969 raft_consensus.cc:1275] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.056445 28309 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:07.069437 28273 raft_consensus.cc:1275] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.075320 28309 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:07.131099 27969 raft_consensus.cc:1275] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:07.132694 28309 consensus_queue.cc:1048] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:07.144052 28272 raft_consensus.cc:1275] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:07.147058 28309 consensus_queue.cc:1048] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:07.183036 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:07.439745 28273 raft_consensus.cc:1275] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.441463 28313 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.443398 27969 raft_consensus.cc:1275] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:07.445021 28313 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:07.644917 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:07.665817 28042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
mode: GRACEFUL
new_leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:60646
I20260709 12:28:07.666600 28042 raft_consensus.cc:606] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Received request to transfer leadership to TS 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:07.670043 28042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
mode: GRACEFUL
new_leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:60646
I20260709 12:28:07.670627 28042 raft_consensus.cc:606] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Received request to transfer leadership to TS 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:07.673846 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:07.674435 27969 raft_consensus.cc:606] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:07.675877 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:07.676513 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:07.677201 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:07.711050 28313 raft_consensus.cc:993] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: : Instructing follower 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:07.711635 28309 raft_consensus.cc:1081] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Signalling peer 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:07.713557 27969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:28:07.714198 27969 raft_consensus.cc:493] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:07.714589 27969 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:07.717895 28220 raft_consensus.cc:993] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:07.718564 28328 raft_consensus.cc:1081] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:07.720232 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:07.720709 28042 raft_consensus.cc:493] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:07.721014 28042 raft_consensus.cc:3060] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.721386 27969 raft_consensus.cc:515] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.723541 27969 leader_election.cc:290] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:07.724517 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:07.725291 28273 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:07.725570 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6829a4c5fd1f46b0a3d38b1defe419af" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:07.726317 28043 raft_consensus.cc:3055] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:07.726673 28043 raft_consensus.cc:740] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:07.727761 28043 consensus_queue.cc:260] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.729128 28042 raft_consensus.cc:515] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.729539 28043 raft_consensus.cc:3060] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:07.731474 28042 leader_election.cc:290] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613), 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903)
I20260709 12:28:07.732651 28272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:07.732823 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc46b610261c4908a191697b0d4e7471" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:07.733397 28272 raft_consensus.cc:3060] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.733589 27969 raft_consensus.cc:3055] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:07.734390 27969 raft_consensus.cc:740] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:07.735193 28273 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:07.735562 27969 consensus_queue.cc:260] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.737115 27936 leader_election.cc:304] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:07.737200 27969 raft_consensus.cc:3060] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:07.738233 28328 raft_consensus.cc:2804] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:07.738802 28328 raft_consensus.cc:697] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:07.740391 28328 consensus_queue.cc:237] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.741099 28043 raft_consensus.cc:2468] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:07.742528 28272 raft_consensus.cc:2468] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:07.743882 28010 leader_election.cc:304] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:07.744854 28309 raft_consensus.cc:2804] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:07.745726 28309 raft_consensus.cc:697] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:07.748726 27969 raft_consensus.cc:2468] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:07.749015 28309 consensus_queue.cc:237] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:07.751230 27790 catalog_manager.cc:5697] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 2 to 3, leader changed from 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 3 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:07.760675 27793 catalog_manager.cc:5697] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:07.997411 28313 raft_consensus.cc:993] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: : Instructing follower 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:07.997956 28313 raft_consensus.cc:1081] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Signalling peer 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:07.999846 27969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:28:08.000768 27969 raft_consensus.cc:493] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:08.001183 27969 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:08.008249 27969 raft_consensus.cc:515] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:08.010202 27969 leader_election.cc:290] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:08.011162 28272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:08.011518 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:08.011984 28272 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:08.012197 28043 raft_consensus.cc:3055] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:08.012645 28043 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:08.013545 28043 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:08.014760 28043 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:08.020479 28272 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:08.021692 27936 leader_election.cc:304] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:08.022444 28043 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:08.022516 28328 raft_consensus.cc:2804] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:08.023238 28328 raft_consensus.cc:697] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:08.024060 28328 consensus_queue.cc:237] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:08.030633 27793 catalog_manager.cc:5697] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 2 to 3, leader changed from 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 3 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:08.176385 28043 raft_consensus.cc:1275] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:08.177974 28328 consensus_queue.cc:1048] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:28:08.184087 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:08.191392 28272 raft_consensus.cc:1275] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:08.193480 28342 consensus_queue.cc:1048] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:08.236272 27969 raft_consensus.cc:1275] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.237775 28309 consensus_queue.cc:1048] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.250319 28272 raft_consensus.cc:1275] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:08.252084 28313 consensus_queue.cc:1048] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:08.463438 28273 raft_consensus.cc:1275] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:08.464985 28328 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:08.475324 28043 raft_consensus.cc:1275] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:08.476918 28220 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:08.684386 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:08.692423 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:08.693097 27969 raft_consensus.cc:606] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:08.696416 28043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
mode: GRACEFUL
new_leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:60646
I20260709 12:28:08.697077 28043 raft_consensus.cc:606] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Received request to transfer leadership to TS 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:08.699890 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:08.700480 27969 raft_consensus.cc:606] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:08.701687 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:08.702337 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:08.702924 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:08.899362 28342 raft_consensus.cc:993] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:08.899888 28353 raft_consensus.cc:1081] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:08.901748 28043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:08.902457 28043 raft_consensus.cc:493] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:08.902776 28043 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:08.908777 28043 raft_consensus.cc:515] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:08.910665 28043 leader_election.cc:290] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 4 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:08.911834 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:08.912108 27968 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87d48713787d4f3094b63dd3059d2642" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:08.912545 28273 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:08.912766 27968 raft_consensus.cc:3055] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:08.913133 27968 raft_consensus.cc:740] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:08.913944 27968 consensus_queue.cc:260] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:08.915705 27968 raft_consensus.cc:3060] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:08.921252 28273 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 4.
I20260709 12:28:08.923060 27968 raft_consensus.cc:2468] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 4.
I20260709 12:28:08.923260 28010 leader_election.cc:304] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:08.924311 28313 raft_consensus.cc:2804] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:08.924889 28313 raft_consensus.cc:697] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:08.925941 28313 consensus_queue.cc:237] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:08.932999 27793 catalog_manager.cc:5697] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 3 to 4, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 4 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:09.123931 28352 raft_consensus.cc:993] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: : Instructing follower 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:09.124583 28313 raft_consensus.cc:1081] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Signalling peer 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:09.126489 27968 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:28:09.127074 27968 raft_consensus.cc:493] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:09.127399 27968 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:09.134215 27968 raft_consensus.cc:515] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.136251 27968 leader_election.cc:290] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:09.137575 28043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:09.137276 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:09.138554 28043 raft_consensus.cc:3055] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:09.138645 28273 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:09.139066 28043 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:09.140148 28043 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.141506 28043 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:09.147486 28273 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:09.149009 27936 leader_election.cc:304] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:09.149950 28353 raft_consensus.cc:2804] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:09.149960 28043 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:09.150619 28353 raft_consensus.cc:697] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:09.151868 28353 consensus_queue.cc:237] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.160679 27793 catalog_manager.cc:5697] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 2 to 3, leader changed from 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 3 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:09.185366 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:09.228205 28342 raft_consensus.cc:993] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:09.228865 28353 raft_consensus.cc:1081] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:09.230926 28043 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:09.231601 28043 raft_consensus.cc:493] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:09.231989 28043 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:09.238843 28043 raft_consensus.cc:515] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.240818 28043 leader_election.cc:290] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:09.241559 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:09.242195 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:09.242753 28273 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:09.242895 27969 raft_consensus.cc:3055] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:09.243363 27969 raft_consensus.cc:740] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:09.244165 27969 consensus_queue.cc:260] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.245287 27969 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:09.250021 28273 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:09.251343 27969 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:09.251574 28010 leader_election.cc:304] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:09.252588 28313 raft_consensus.cc:2804] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:09.253098 28313 raft_consensus.cc:697] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:09.254034 28313 consensus_queue.cc:237] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.261729 27793 catalog_manager.cc:5697] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:09.504204 28273 raft_consensus.cc:1275] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:09.505887 28313 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:28:09.514166 27969 raft_consensus.cc:1275] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:09.517321 28352 consensus_queue.cc:1048] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:28:09.700965 27969 raft_consensus.cc:1275] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:09.702512 28352 consensus_queue.cc:1048] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:09.705534 28273 raft_consensus.cc:1275] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:09.707373 28353 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:09.709847 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:09.715363 28272 raft_consensus.cc:1275] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:09.717327 28352 consensus_queue.cc:1048] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:09.726675 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15849d102a27427684b28905bc02f55d"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:09.728482 27969 raft_consensus.cc:606] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:09.734485 28043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
mode: GRACEFUL
new_leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:60646
I20260709 12:28:09.735227 28043 raft_consensus.cc:606] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Received request to transfer leadership to TS 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:09.738436 28043 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
mode: GRACEFUL
new_leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:60646
I20260709 12:28:09.739190 28043 raft_consensus.cc:606] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Received request to transfer leadership to TS 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:09.741997 28042 raft_consensus.cc:1275] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:09.742010 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:09.743680 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:09.744561 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:09.751502 28369 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:09.802104 28313 raft_consensus.cc:993] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: : Instructing follower 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:09.802767 28352 raft_consensus.cc:1081] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Signalling peer 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:09.804816 27969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:28:09.805488 27969 raft_consensus.cc:493] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:09.805848 27969 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:09.813613 27969 raft_consensus.cc:515] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.816044 27969 leader_election.cc:290] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 4 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:09.818624 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:09.819398 28273 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:09.821038 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0bfd3dd40104d25bb9148497df3913a" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:09.821697 28042 raft_consensus.cc:3055] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:09.822036 28042 raft_consensus.cc:740] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:09.822918 28042 consensus_queue.cc:260] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.824295 28042 raft_consensus.cc:3060] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:09.829540 28273 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 4.
I20260709 12:28:09.830920 27936 leader_election.cc:304] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:09.832314 28370 raft_consensus.cc:2804] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:09.832967 28370 raft_consensus.cc:697] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 4 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:09.834532 28370 consensus_queue.cc:237] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.836598 28042 raft_consensus.cc:2468] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 4.
I20260709 12:28:09.844995 27790 catalog_manager.cc:5697] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 3 to 4, leader changed from 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 4 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:09.907025 28353 raft_consensus.cc:993] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:09.907702 28370 raft_consensus.cc:1081] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:09.909516 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15849d102a27427684b28905bc02f55d"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:09.910099 28042 raft_consensus.cc:493] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:09.910554 28042 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:09.917171 28042 raft_consensus.cc:515] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.918893 28042 leader_election.cc:290] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:09.920055 28272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:09.920058 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:09.920810 28272 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:09.920902 27969 raft_consensus.cc:3055] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:09.921386 27969 raft_consensus.cc:740] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:09.922111 27969 consensus_queue.cc:260] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.923358 27969 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:09.927501 28272 raft_consensus.cc:2468] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:09.929194 27969 raft_consensus.cc:2468] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 2.
I20260709 12:28:09.929316 28010 leader_election.cc:304] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:09.930485 28352 raft_consensus.cc:2804] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:09.930998 28352 raft_consensus.cc:697] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:09.931968 28352 consensus_queue.cc:237] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:09.942481 27790 catalog_manager.cc:5697] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 1 to 2, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:10.105947 28367 raft_consensus.cc:993] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: : Instructing follower 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:10.106587 28352 raft_consensus.cc:1081] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Signalling peer 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:10.108549 27969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:28:10.109241 27969 raft_consensus.cc:493] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:10.109619 27969 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:10.115406 27969 raft_consensus.cc:515] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:10.117149 27969 leader_election.cc:290] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:10.118211 28272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:10.118347 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d794190450c94c91b65193a6013ff9c3" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:10.119004 28272 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:10.119035 28042 raft_consensus.cc:3055] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:10.119478 28042 raft_consensus.cc:740] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:10.120244 28042 consensus_queue.cc:260] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:10.121227 28042 raft_consensus.cc:3060] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:10.125512 28272 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:10.126386 28042 raft_consensus.cc:2468] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:10.126930 27936 leader_election.cc:304] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:10.127687 28370 raft_consensus.cc:2804] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:10.128111 28370 raft_consensus.cc:697] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:10.128909 28370 consensus_queue.cc:237] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:10.137030 27790 catalog_manager.cc:5697] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 2 to 3, leader changed from 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 3 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:10.186668 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:10.221248 28272 raft_consensus.cc:1275] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:28:10.223116 28370 consensus_queue.cc:1048] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:10.233968 28042 raft_consensus.cc:1275] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:28:10.236299 28353 consensus_queue.cc:1048] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:10.325814 28272 raft_consensus.cc:1275] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:10.327436 28352 consensus_queue.cc:1048] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:10.338399 27969 raft_consensus.cc:1275] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:10.340114 28352 consensus_queue.cc:1048] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:10.511268 28042 raft_consensus.cc:1275] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:10.512692 28353 consensus_queue.cc:1048] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:10.523697 28272 raft_consensus.cc:1275] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:10.525159 28353 consensus_queue.cc:1048] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:10.762786 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:10.771687 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:10.772302 27969 raft_consensus.cc:606] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:10.775827 28042 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15849d102a27427684b28905bc02f55d"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
mode: GRACEFUL
new_leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:60646
I20260709 12:28:10.776365 28042 raft_consensus.cc:606] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Received request to transfer leadership to TS 663c28d0e3c9414486153d7d2c36a1ae
I20260709 12:28:10.779167 27969 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
mode: GRACEFUL
new_leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:57078
I20260709 12:28:10.779663 27969 raft_consensus.cc:606] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Received request to transfer leadership to TS 579731fab2d44e00b7f200e96d5f4bb9
I20260709 12:28:10.780855 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:28:10.781440 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:10.782111 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:10.916291 28382 raft_consensus.cc:993] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: : Instructing follower 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:10.917021 28383 raft_consensus.cc:1081] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Signalling peer 663c28d0e3c9414486153d7d2c36a1ae to start an election
I20260709 12:28:10.919274 27969 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15849d102a27427684b28905bc02f55d"
dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
 from {username='slave'} at 127.0.0.1:49730
I20260709 12:28:10.919840 27969 raft_consensus.cc:493] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:10.920189 27969 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:10.925787 27969 raft_consensus.cc:515] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:10.927380 27969 leader_election.cc:290] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67:42399)
I20260709 12:28:10.928448 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:10.928690 28042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15849d102a27427684b28905bc02f55d" candidate_uuid: "663c28d0e3c9414486153d7d2c36a1ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
I20260709 12:28:10.929180 28273 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:10.929212 28042 raft_consensus.cc:3055] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:10.929744 28042 raft_consensus.cc:740] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:10.930617 28042 consensus_queue.cc:260] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:10.931887 28042 raft_consensus.cc:3060] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:10.937338 28273 raft_consensus.cc:2468] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:10.939028 27936 leader_election.cc:304] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 663c28d0e3c9414486153d7d2c36a1ae; no voters: 
I20260709 12:28:10.939534 28042 raft_consensus.cc:2468] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 663c28d0e3c9414486153d7d2c36a1ae in term 3.
I20260709 12:28:10.940023 28353 raft_consensus.cc:2804] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:10.940596 28353 raft_consensus.cc:697] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Leader. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:10.941622 28353 consensus_queue.cc:237] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:10.949769 27790 catalog_manager.cc:5697] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae reported cstate change: term changed from 2 to 3, leader changed from 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67) to 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66). New cstate: current_term: 3 leader_uuid: "663c28d0e3c9414486153d7d2c36a1ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:11.096762 28371 raft_consensus.cc:993] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:11.097411 28353 raft_consensus.cc:1081] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:11.099164 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:11.099701 28042 raft_consensus.cc:493] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:11.100071 28042 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:11.106875 28042 raft_consensus.cc:515] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:11.108856 28042 leader_election.cc:290] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 3 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:11.109772 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:11.109916 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebd577c08a746f49df51dd2dbfbbd1e" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:11.110472 28273 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:11.110634 27969 raft_consensus.cc:3055] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:11.111081 27969 raft_consensus.cc:740] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:11.111824 27969 consensus_queue.cc:260] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:11.112872 27969 raft_consensus.cc:3060] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:11.116595 28273 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 3.
I20260709 12:28:11.117966 28010 leader_election.cc:304] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:11.119107 28383 raft_consensus.cc:2804] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:11.119624 28383 raft_consensus.cc:697] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:11.119720 27969 raft_consensus.cc:2468] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 3.
I20260709 12:28:11.120752 28383 consensus_queue.cc:237] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:11.129619 27790 catalog_manager.cc:5697] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 2 to 3, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:11.187954 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:11.203270 28371 raft_consensus.cc:993] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: : Instructing follower 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:11.203913 28371 raft_consensus.cc:1081] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Signalling peer 579731fab2d44e00b7f200e96d5f4bb9 to start an election
I20260709 12:28:11.205492 28042 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636"
dest_uuid: "579731fab2d44e00b7f200e96d5f4bb9"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:28:11.206017 28042 raft_consensus.cc:493] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:11.206390 28042 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:11.211877 28042 raft_consensus.cc:515] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:11.213541 28042 leader_election.cc:290] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 4 election: Requested vote from peers 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903), 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613)
I20260709 12:28:11.214699 27969 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "663c28d0e3c9414486153d7d2c36a1ae"
I20260709 12:28:11.214669 28273 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a0ab731e4a4eef8cc308a4637fb636" candidate_uuid: "579731fab2d44e00b7f200e96d5f4bb9" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2ab6b96c5e65444a80252670af0f1d24"
I20260709 12:28:11.215582 27969 raft_consensus.cc:3055] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:28:11.215600 28273 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:11.216106 27969 raft_consensus.cc:740] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 663c28d0e3c9414486153d7d2c36a1ae, State: Running, Role: LEADER
I20260709 12:28:11.216951 27969 consensus_queue.cc:260] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:11.218240 27969 raft_consensus.cc:3060] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:11.223510 28273 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 4.
I20260709 12:28:11.224888 28010 leader_election.cc:304] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab6b96c5e65444a80252670af0f1d24, 579731fab2d44e00b7f200e96d5f4bb9; no voters: 
I20260709 12:28:11.225507 27969 raft_consensus.cc:2468] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579731fab2d44e00b7f200e96d5f4bb9 in term 4.
I20260709 12:28:11.225651 28383 raft_consensus.cc:2804] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:11.226209 28383 raft_consensus.cc:697] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 LEADER]: Becoming Leader. State: Replica: 579731fab2d44e00b7f200e96d5f4bb9, State: Running, Role: LEADER
I20260709 12:28:11.227169 28383 consensus_queue.cc:237] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } }
I20260709 12:28:11.235134 27790 catalog_manager.cc:5697] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 reported cstate change: term changed from 3 to 4, leader changed from 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66) to 579731fab2d44e00b7f200e96d5f4bb9 (127.12.45.67). New cstate: current_term: 4 leader_uuid: "579731fab2d44e00b7f200e96d5f4bb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.495609 28273 raft_consensus.cc:1275] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:11.497301 28371 consensus_queue.cc:1048] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:11.512130 28042 raft_consensus.cc:1275] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Refusing update from remote peer 663c28d0e3c9414486153d7d2c36a1ae: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:11.513659 28353 consensus_queue.cc:1048] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "579731fab2d44e00b7f200e96d5f4bb9" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 42399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:11.638304 28273 raft_consensus.cc:1275] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:11.639828 28383 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:11.643107 28272 raft_consensus.cc:1275] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:11.644992 28382 consensus_queue.cc:1048] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab6b96c5e65444a80252670af0f1d24" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 37903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:11.650729 27969 raft_consensus.cc:1275] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:28:11.652804 28396 consensus_queue.cc:1048] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:28:11.666313 27969 raft_consensus.cc:1275] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Refusing update from remote peer 579731fab2d44e00b7f200e96d5f4bb9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:11.668438 28383 consensus_queue.cc:1048] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "663c28d0e3c9414486153d7d2c36a1ae" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:11.788743 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:11.819821 12469 tablet_server.cc:179] TabletServer@127.12.45.65:37903 shutting down...
W20260709 12:28:11.830171 27936 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 12:28:11.835089 27936 consensus_peers.cc:597] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:11.855159 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:11.856287 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.857062 12469 raft_consensus.cc:2243] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.857590 12469 raft_consensus.cc:2272] T 6829a4c5fd1f46b0a3d38b1defe419af P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.859970 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.860674 12469 raft_consensus.cc:2243] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.861303 12469 raft_consensus.cc:2272] T 0f7bb407bc6d4d939d9d03602f825637 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.863601 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.864224 12469 raft_consensus.cc:2243] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.864697 12469 raft_consensus.cc:2272] T bc46b610261c4908a191697b0d4e7471 P 2ab6b96c5e65444a80252670af0f1d24 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.866885 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.867480 12469 raft_consensus.cc:2243] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.867957 12469 raft_consensus.cc:2272] T d794190450c94c91b65193a6013ff9c3 P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.870100 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.870736 12469 raft_consensus.cc:2243] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.871197 12469 raft_consensus.cc:2272] T e0bfd3dd40104d25bb9148497df3913a P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.873306 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.873945 12469 raft_consensus.cc:2243] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.874449 12469 raft_consensus.cc:2272] T 0ebd577c08a746f49df51dd2dbfbbd1e P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.876822 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.877439 12469 raft_consensus.cc:2243] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.877944 12469 raft_consensus.cc:2272] T 87d48713787d4f3094b63dd3059d2642 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.880156 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.880745 12469 raft_consensus.cc:2243] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.881371 12469 raft_consensus.cc:2272] T 93a0ab731e4a4eef8cc308a4637fb636 P 2ab6b96c5e65444a80252670af0f1d24 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.883944 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.884510 12469 raft_consensus.cc:2243] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.884976 12469 raft_consensus.cc:2272] T 15849d102a27427684b28905bc02f55d P 2ab6b96c5e65444a80252670af0f1d24 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.887523 12469 tablet_replica.cc:333] stopping tablet replica
I20260709 12:28:11.888072 12469 raft_consensus.cc:2243] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.888722 12469 raft_consensus.cc:2272] T 869da3fc7b8849c898130dafc04ff652 P 2ab6b96c5e65444a80252670af0f1d24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.921509 12469 tablet_server.cc:196] TabletServer@127.12.45.65:37903 shutdown complete.
W20260709 12:28:11.947750 27936 consensus_peers.cc:597] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:11.947861 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
W20260709 12:28:11.959160 27936 consensus_peers.cc:597] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:11.990947 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:11.992178 12469 tablet_replica.cc:333] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
W20260709 12:28:11.993127 28007 consensus_peers.cc:597] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Couldn't send request to peer 663c28d0e3c9414486153d7d2c36a1ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:11.993498 12469 raft_consensus.cc:2243] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:11.994204 12469 raft_consensus.cc:2272] T 93a0ab731e4a4eef8cc308a4637fb636 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:11.998564 12469 tablet_replica.cc:333] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
W20260709 12:28:11.997735 28007 consensus_peers.cc:597] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Couldn't send request to peer 663c28d0e3c9414486153d7d2c36a1ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.000581 12469 raft_consensus.cc:2243] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.001446 12469 raft_consensus.cc:2272] T 6829a4c5fd1f46b0a3d38b1defe419af P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.003839 12469 tablet_replica.cc:333] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.004492 12469 raft_consensus.cc:2243] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.004957 12469 raft_consensus.cc:2272] T 0ebd577c08a746f49df51dd2dbfbbd1e P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.008654 12469 tablet_replica.cc:333] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.009485 12469 raft_consensus.cc:2243] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.010701 12469 raft_consensus.cc:2272] T e0bfd3dd40104d25bb9148497df3913a P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.013469 12469 tablet_replica.cc:333] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.014326 12469 raft_consensus.cc:2243] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.015391 12469 raft_consensus.cc:2272] T d794190450c94c91b65193a6013ff9c3 P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:12.017539 28007 consensus_peers.cc:597] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Couldn't send request to peer 663c28d0e3c9414486153d7d2c36a1ae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.018508 12469 tablet_replica.cc:333] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.019147 12469 raft_consensus.cc:2243] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.019969 12469 raft_consensus.cc:2272] T 15849d102a27427684b28905bc02f55d P 663c28d0e3c9414486153d7d2c36a1ae [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.024088 12469 tablet_replica.cc:333] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.024734 12469 raft_consensus.cc:2243] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.025218 12469 raft_consensus.cc:2272] T 869da3fc7b8849c898130dafc04ff652 P 663c28d0e3c9414486153d7d2c36a1ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.028101 12469 tablet_replica.cc:333] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.028719 12469 raft_consensus.cc:2243] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.029296 12469 raft_consensus.cc:2272] T 0f7bb407bc6d4d939d9d03602f825637 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.032401 12469 tablet_replica.cc:333] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.033031 12469 raft_consensus.cc:2243] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.033679 12469 raft_consensus.cc:2272] T bc46b610261c4908a191697b0d4e7471 P 663c28d0e3c9414486153d7d2c36a1ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.036376 12469 tablet_replica.cc:333] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae: stopping tablet replica
I20260709 12:28:12.037266 12469 raft_consensus.cc:2243] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.037883 12469 raft_consensus.cc:2272] T 87d48713787d4f3094b63dd3059d2642 P 663c28d0e3c9414486153d7d2c36a1ae [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.068348 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:28:12.103014 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
W20260709 12:28:12.121116 28010 consensus_peers.cc:597] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.121774 28010 consensus_peers.cc:597] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.129393 28007 consensus_peers.cc:597] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 663c28d0e3c9414486153d7d2c36a1ae (127.12.45.66:45613): Couldn't send request to peer 663c28d0e3c9414486153d7d2c36a1ae. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.66:45613: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:12.136984 28010 consensus_peers.cc:597] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 -> Peer 2ab6b96c5e65444a80252670af0f1d24 (127.12.45.65:37903): Couldn't send request to peer 2ab6b96c5e65444a80252670af0f1d24. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:37903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:12.150107 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:12.151541 12469 tablet_replica.cc:333] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.152630 12469 raft_consensus.cc:2243] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.153637 12469 raft_consensus.cc:2272] T 0ebd577c08a746f49df51dd2dbfbbd1e P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.157527 12469 tablet_replica.cc:333] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.158339 12469 raft_consensus.cc:2243] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.158826 12469 raft_consensus.cc:2272] T 15849d102a27427684b28905bc02f55d P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.162053 12469 tablet_replica.cc:333] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.162756 12469 raft_consensus.cc:2243] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.163702 12469 raft_consensus.cc:2272] T 869da3fc7b8849c898130dafc04ff652 P 579731fab2d44e00b7f200e96d5f4bb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.166318 12469 tablet_replica.cc:333] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.166983 12469 raft_consensus.cc:2243] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.168180 12469 raft_consensus.cc:2272] T 87d48713787d4f3094b63dd3059d2642 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.170992 12469 tablet_replica.cc:333] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.171638 12469 raft_consensus.cc:2243] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.172613 12469 raft_consensus.cc:2272] T 93a0ab731e4a4eef8cc308a4637fb636 P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.175657 12469 tablet_replica.cc:333] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.176673 12469 raft_consensus.cc:2243] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.177359 12469 raft_consensus.cc:2272] T 6829a4c5fd1f46b0a3d38b1defe419af P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.180361 12469 tablet_replica.cc:333] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.181061 12469 raft_consensus.cc:2243] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.181959 12469 raft_consensus.cc:2272] T 0f7bb407bc6d4d939d9d03602f825637 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.184886 12469 tablet_replica.cc:333] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.185570 12469 raft_consensus.cc:2243] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.186585 12469 raft_consensus.cc:2272] T bc46b610261c4908a191697b0d4e7471 P 579731fab2d44e00b7f200e96d5f4bb9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:12.189198 27841 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:28:12.189388 12469 tablet_replica.cc:333] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.190371 12469 raft_consensus.cc:2243] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.190851 12469 raft_consensus.cc:2272] T e0bfd3dd40104d25bb9148497df3913a P 579731fab2d44e00b7f200e96d5f4bb9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.193326 12469 tablet_replica.cc:333] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9: stopping tablet replica
I20260709 12:28:12.193809 12469 raft_consensus.cc:2243] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:12.194242 12469 raft_consensus.cc:2272] T d794190450c94c91b65193a6013ff9c3 P 579731fab2d44e00b7f200e96d5f4bb9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.229775 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:28:12.260279 12469 master.cc:562] Master@127.12.45.126:36311 shutting down...
I20260709 12:28:12.290477 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:12.291181 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:12.291536 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0c8b1f39d3034f8b938af1581a0dc76f: stopping tablet replica
I20260709 12:28:12.313057 12469 master.cc:584] Master@127.12.45.126:36311 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (15396 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:28:12.362733 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:35799
I20260709 12:28:12.364043 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:12.369800 28407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:12.369845 28408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.372102 12469 server_base.cc:1061] running on GCE node
W20260709 12:28:12.372190 28410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.373363 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:12.373615 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:12.373793 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600092373769 us; error 0 us; skew 500 ppm
I20260709 12:28:12.374399 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:12.377064 12469 webserver.cc:533] Webserver started at http://127.12.45.126:36565/ using document root <none> and password file <none>
I20260709 12:28:12.377636 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:12.377866 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:12.378176 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:12.379520 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "3b0668bd56094db9bdfdf94eca61808e"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.384487 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:28:12.388527 28415 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.389523 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:28:12.389864 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "3b0668bd56094db9bdfdf94eca61808e"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.390197 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:12.402908 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:12.404289 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:12.451678 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:35799
I20260709 12:28:12.451798 28466 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:35799 every 8 connection(s)
I20260709 12:28:12.456130 28467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:12.470865 28467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e: Bootstrap starting.
I20260709 12:28:12.477479 28467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:12.482590 28467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e: No bootstrap required, opened a new log
I20260709 12:28:12.485159 28467 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0668bd56094db9bdfdf94eca61808e" member_type: VOTER }
I20260709 12:28:12.485592 28467 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:12.485860 28467 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b0668bd56094db9bdfdf94eca61808e, State: Initialized, Role: FOLLOWER
I20260709 12:28:12.486737 28467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0668bd56094db9bdfdf94eca61808e" member_type: VOTER }
I20260709 12:28:12.487272 28467 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:12.487534 28467 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:12.487815 28467 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:12.495604 28467 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0668bd56094db9bdfdf94eca61808e" member_type: VOTER }
I20260709 12:28:12.496250 28467 leader_election.cc:304] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b0668bd56094db9bdfdf94eca61808e; no voters: 
I20260709 12:28:12.497716 28467 leader_election.cc:290] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:12.498389 28470 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:12.499907 28470 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 1 LEADER]: Becoming Leader. State: Replica: 3b0668bd56094db9bdfdf94eca61808e, State: Running, Role: LEADER
I20260709 12:28:12.500756 28470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0668bd56094db9bdfdf94eca61808e" member_type: VOTER }
I20260709 12:28:12.501940 28467 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:12.505713 28472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b0668bd56094db9bdfdf94eca61808e. Latest consensus state: current_term: 1 leader_uuid: "3b0668bd56094db9bdfdf94eca61808e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0668bd56094db9bdfdf94eca61808e" member_type: VOTER } }
I20260709 12:28:12.505923 28471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b0668bd56094db9bdfdf94eca61808e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b0668bd56094db9bdfdf94eca61808e" member_type: VOTER } }
I20260709 12:28:12.506549 28472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:12.506981 28471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:12.510591 28476 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:12.519574 28476 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:12.525599 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:12.531787 28476 catalog_manager.cc:1383] Generated new cluster ID: e1344db016f24da287c0fcbf637e8963
I20260709 12:28:12.532183 28476 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:12.555083 28476 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:12.556694 28476 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:12.573949 28476 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e: Generated new TSK 0
I20260709 12:28:12.574858 28476 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:12.593667 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:12.601603 28488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:12.602595 28489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.606768 12469 server_base.cc:1061] running on GCE node
W20260709 12:28:12.608049 28491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.612445 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:12.612787 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:12.613031 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600092613003 us; error 0 us; skew 500 ppm
I20260709 12:28:12.613639 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:12.616817 12469 webserver.cc:533] Webserver started at http://127.12.45.65:38405/ using document root <none> and password file <none>
I20260709 12:28:12.617430 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:12.617662 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:12.617972 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:12.619462 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "04df0112d5274a11aa3a03f904c0a6c2"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.624614 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:28:12.629298 28496 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.630442 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:12.630736 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "04df0112d5274a11aa3a03f904c0a6c2"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.631029 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:12.645004 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:12.646523 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:12.648126 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:12.650869 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:12.651104 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.651410 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:12.651577 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.703714 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:44017
I20260709 12:28:12.703814 28558 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:44017 every 8 connection(s)
I20260709 12:28:12.708943 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:12.718045 28563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:12.719491 28564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.721686 12469 server_base.cc:1061] running on GCE node
W20260709 12:28:12.722792 28566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.723953 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:12.724211 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:12.724723 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600092724690 us; error 0 us; skew 500 ppm
I20260709 12:28:12.725653 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:12.725786 28559 heartbeater.cc:344] Connected to a master server at 127.12.45.126:35799
I20260709 12:28:12.726447 28559 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:12.727656 28559 heartbeater.cc:507] Master 127.12.45.126:35799 requested a full tablet report, sending...
I20260709 12:28:12.729244 12469 webserver.cc:533] Webserver started at http://127.12.45.66:40907/ using document root <none> and password file <none>
I20260709 12:28:12.730124 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:12.730453 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:12.730407 28432 ts_manager.cc:194] Registered new tserver with Master: 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017)
I20260709 12:28:12.731062 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:12.732837 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "6553532c440e4941a468fb249a0f92db"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.733389 28432 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57630
I20260709 12:28:12.739439 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:28:12.744115 28571 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.745252 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:28:12.745661 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "6553532c440e4941a468fb249a0f92db"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.745945 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:12.781010 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:12.782485 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:12.784600 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:12.787621 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:12.787894 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.788162 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:12.788347 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.840751 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:36835
I20260709 12:28:12.840862 28633 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:36835 every 8 connection(s)
I20260709 12:28:12.846740 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:12.855849 28637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:12.858541 28638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:12.860103 28640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:12.862424 28634 heartbeater.cc:344] Connected to a master server at 127.12.45.126:35799
I20260709 12:28:12.863029 28634 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:12.863664 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:12.864161 28634 heartbeater.cc:507] Master 127.12.45.126:35799 requested a full tablet report, sending...
I20260709 12:28:12.865069 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:12.865384 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:12.865619 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600092865601 us; error 0 us; skew 500 ppm
I20260709 12:28:12.866493 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:12.867018 28432 ts_manager.cc:194] Registered new tserver with Master: 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835)
I20260709 12:28:12.868872 28432 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57646
I20260709 12:28:12.869964 12469 webserver.cc:533] Webserver started at http://127.12.45.67:33467/ using document root <none> and password file <none>
I20260709 12:28:12.870774 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:12.870982 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:12.871275 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:12.872546 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "e45a64d8394f4e78a582f7865a076cea"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.877782 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:12.882043 28645 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.883064 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:12.883572 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "e45a64d8394f4e78a582f7865a076cea"
format_stamp: "Formatted at 2026-07-09 12:28:12 on dist-test-slave-90bm"
I20260709 12:28:12.883898 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:12.915138 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:12.916497 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:12.918155 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:12.920830 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:12.921034 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.921357 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:12.921531 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:12.971285 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:34275
I20260709 12:28:12.971405 28707 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:34275 every 8 connection(s)
I20260709 12:28:12.988374 28708 heartbeater.cc:344] Connected to a master server at 127.12.45.126:35799
I20260709 12:28:12.988924 28708 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:12.989940 28708 heartbeater.cc:507] Master 127.12.45.126:35799 requested a full tablet report, sending...
I20260709 12:28:12.992565 28432 ts_manager.cc:194] Registered new tserver with Master: e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:12.993191 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018227266s
I20260709 12:28:12.994215 28432 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57650
I20260709 12:28:13.025265 28432 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57654:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:13.028087 28432 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:13.086453 28673 tablet_service.cc:1511] Processing CreateTablet for tablet 612ab577495546f8a7546437177d99b2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=83a3cef0e5ad402b8cc5600ac58d54cb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:13.087903 28673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612ab577495546f8a7546437177d99b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.087337 28598 tablet_service.cc:1511] Processing CreateTablet for tablet 84ed240160c14244b4c90e05d0800ef3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=83a3cef0e5ad402b8cc5600ac58d54cb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:13.089385 28598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ed240160c14244b4c90e05d0800ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.091522 28599 tablet_service.cc:1511] Processing CreateTablet for tablet 612ab577495546f8a7546437177d99b2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=83a3cef0e5ad402b8cc5600ac58d54cb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:13.093381 28599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612ab577495546f8a7546437177d99b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.095624 28672 tablet_service.cc:1511] Processing CreateTablet for tablet 84ed240160c14244b4c90e05d0800ef3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=83a3cef0e5ad402b8cc5600ac58d54cb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:13.097505 28672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ed240160c14244b4c90e05d0800ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.105935 28524 tablet_service.cc:1511] Processing CreateTablet for tablet 612ab577495546f8a7546437177d99b2 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=83a3cef0e5ad402b8cc5600ac58d54cb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:13.107632 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612ab577495546f8a7546437177d99b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.117131 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 84ed240160c14244b4c90e05d0800ef3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=83a3cef0e5ad402b8cc5600ac58d54cb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:13.119072 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ed240160c14244b4c90e05d0800ef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.125967 28723 tablet_bootstrap.cc:492] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea: Bootstrap starting.
I20260709 12:28:13.135710 28723 tablet_bootstrap.cc:654] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.141343 28725 tablet_bootstrap.cc:492] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db: Bootstrap starting.
I20260709 12:28:13.148459 28723 tablet_bootstrap.cc:492] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea: No bootstrap required, opened a new log
I20260709 12:28:13.149271 28723 ts_tablet_manager.cc:1403] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.004s
I20260709 12:28:13.153565 28725 tablet_bootstrap.cc:654] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.153308 28723 raft_consensus.cc:359] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.154206 28723 raft_consensus.cc:385] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.154716 28723 raft_consensus.cc:740] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.157405 28723 consensus_queue.cc:260] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.158079 28726 tablet_bootstrap.cc:492] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2: Bootstrap starting.
I20260709 12:28:13.165781 28726 tablet_bootstrap.cc:654] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.166600 28708 heartbeater.cc:499] Master 127.12.45.126:35799 was elected leader, sending a full tablet report...
I20260709 12:28:13.168174 28723 ts_tablet_manager.cc:1434] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.008s
I20260709 12:28:13.169724 28723 tablet_bootstrap.cc:492] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea: Bootstrap starting.
I20260709 12:28:13.180831 28723 tablet_bootstrap.cc:654] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.182403 28725 tablet_bootstrap.cc:492] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db: No bootstrap required, opened a new log
I20260709 12:28:13.183105 28725 ts_tablet_manager.cc:1403] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.010s
I20260709 12:28:13.187280 28726 tablet_bootstrap.cc:492] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2: No bootstrap required, opened a new log
I20260709 12:28:13.187060 28725 raft_consensus.cc:359] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.187924 28726 ts_tablet_manager.cc:1403] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.007s
I20260709 12:28:13.188115 28723 tablet_bootstrap.cc:492] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea: No bootstrap required, opened a new log
I20260709 12:28:13.187961 28725 raft_consensus.cc:385] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.188634 28725 raft_consensus.cc:740] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.188807 28723 ts_tablet_manager.cc:1403] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.003s
I20260709 12:28:13.189603 28725 consensus_queue.cc:260] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.191711 28726 raft_consensus.cc:359] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.192586 28726 raft_consensus.cc:385] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.192662 28634 heartbeater.cc:499] Master 127.12.45.126:35799 was elected leader, sending a full tablet report...
I20260709 12:28:13.193009 28726 raft_consensus.cc:740] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.193116 28725 ts_tablet_manager.cc:1434] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:28:13.192737 28723 raft_consensus.cc:359] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.193742 28723 raft_consensus.cc:385] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.194056 28723 raft_consensus.cc:740] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.194442 28725 tablet_bootstrap.cc:492] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db: Bootstrap starting.
I20260709 12:28:13.194195 28726 consensus_queue.cc:260] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.195031 28723 consensus_queue.cc:260] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.200688 28723 ts_tablet_manager.cc:1434] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:28:13.205361 28726 ts_tablet_manager.cc:1434] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent starting tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 12:28:13.205622 28725 tablet_bootstrap.cc:654] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.206295 28559 heartbeater.cc:499] Master 127.12.45.126:35799 was elected leader, sending a full tablet report...
I20260709 12:28:13.207437 28726 tablet_bootstrap.cc:492] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: Bootstrap starting.
I20260709 12:28:13.217772 28726 tablet_bootstrap.cc:654] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.221342 28725 tablet_bootstrap.cc:492] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db: No bootstrap required, opened a new log
I20260709 12:28:13.222030 28725 ts_tablet_manager.cc:1403] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.002s
I20260709 12:28:13.223615 28726 tablet_bootstrap.cc:492] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: No bootstrap required, opened a new log
I20260709 12:28:13.224246 28726 ts_tablet_manager.cc:1403] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:28:13.225391 28725 raft_consensus.cc:359] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.226392 28725 raft_consensus.cc:385] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.226585 28725 raft_consensus.cc:740] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.227283 28726 raft_consensus.cc:359] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.227382 28725 consensus_queue.cc:260] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.228016 28726 raft_consensus.cc:385] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.228346 28726 raft_consensus.cc:740] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.229482 28725 ts_tablet_manager.cc:1434] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:13.229871 28726 consensus_queue.cc:260] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.232256 28726 ts_tablet_manager.cc:1434] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:13.297920 28732 raft_consensus.cc:493] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.298741 28732 raft_consensus.cc:515] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.301278 28732 leader_election.cc:290] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:13.315558 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612ab577495546f8a7546437177d99b2" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6553532c440e4941a468fb249a0f92db" is_pre_election: true
I20260709 12:28:13.315968 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612ab577495546f8a7546437177d99b2" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45a64d8394f4e78a582f7865a076cea" is_pre_election: true
I20260709 12:28:13.316396 28609 raft_consensus.cc:2468] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 0.
I20260709 12:28:13.316737 28683 raft_consensus.cc:2468] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 0.
I20260709 12:28:13.317804 28498 leader_election.cc:304] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, 6553532c440e4941a468fb249a0f92db; no voters: 
I20260709 12:28:13.318780 28732 raft_consensus.cc:2804] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:13.319180 28732 raft_consensus.cc:493] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.319475 28732 raft_consensus.cc:3060] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.325555 28732 raft_consensus.cc:515] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.327538 28732 leader_election.cc:290] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:13.328584 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612ab577495546f8a7546437177d99b2" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6553532c440e4941a468fb249a0f92db"
I20260709 12:28:13.328598 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612ab577495546f8a7546437177d99b2" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
I20260709 12:28:13.329295 28609 raft_consensus.cc:3060] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.329525 28683 raft_consensus.cc:3060] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.336122 28609 raft_consensus.cc:2468] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 1.
I20260709 12:28:13.336159 28683 raft_consensus.cc:2468] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 1.
I20260709 12:28:13.337857 28499 leader_election.cc:304] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, 6553532c440e4941a468fb249a0f92db, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:13.338852 28732 raft_consensus.cc:2804] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:13.340159 28732 raft_consensus.cc:697] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Running, Role: LEADER
I20260709 12:28:13.340938 28732 consensus_queue.cc:237] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } }
I20260709 12:28:13.351322 28431 catalog_manager.cc:5697] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "04df0112d5274a11aa3a03f904c0a6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:13.411075 28732 raft_consensus.cc:493] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.411799 28732 raft_consensus.cc:515] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.414654 28732 leader_election.cc:290] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:13.415764 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6553532c440e4941a468fb249a0f92db" is_pre_election: true
I20260709 12:28:13.415764 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45a64d8394f4e78a582f7865a076cea" is_pre_election: true
I20260709 12:28:13.416944 28683 raft_consensus.cc:2468] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 0.
I20260709 12:28:13.417101 28609 raft_consensus.cc:2468] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 0.
I20260709 12:28:13.418694 28499 leader_election.cc:304] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:13.419639 28732 raft_consensus.cc:2804] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:13.420185 28732 raft_consensus.cc:493] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.420563 28732 raft_consensus.cc:3060] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.426817 28732 raft_consensus.cc:515] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.428606 28732 leader_election.cc:290] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:13.429698 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6553532c440e4941a468fb249a0f92db"
I20260709 12:28:13.429754 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
I20260709 12:28:13.430533 28683 raft_consensus.cc:3060] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.430511 28609 raft_consensus.cc:3060] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.437161 28683 raft_consensus.cc:2468] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 1.
I20260709 12:28:13.437381 28609 raft_consensus.cc:2468] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 1.
I20260709 12:28:13.438663 28499 leader_election.cc:304] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:13.439669 28732 raft_consensus.cc:2804] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:13.440111 28732 raft_consensus.cc:697] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Running, Role: LEADER
I20260709 12:28:13.440826 28732 consensus_queue.cc:237] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.449324 28431 catalog_manager.cc:5697] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "04df0112d5274a11aa3a03f904c0a6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:13.510628 28431 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57662:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:13.513479 28431 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:13.542122 28523 tablet_service.cc:1511] Processing CreateTablet for tablet 069a624baf5248eb98242658df9f7fa7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=8942103afbd1458e8716d332d1e94e09]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:13.543519 28523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 069a624baf5248eb98242658df9f7fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.543478 28599 tablet_service.cc:1511] Processing CreateTablet for tablet 069a624baf5248eb98242658df9f7fa7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=8942103afbd1458e8716d332d1e94e09]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:13.545123 28599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 069a624baf5248eb98242658df9f7fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.546876 28672 tablet_service.cc:1511] Processing CreateTablet for tablet 069a624baf5248eb98242658df9f7fa7 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=8942103afbd1458e8716d332d1e94e09]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:28:13.548336 28672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 069a624baf5248eb98242658df9f7fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.549984 28598 tablet_service.cc:1511] Processing CreateTablet for tablet c26dd77d4b614dce8a4e6cd41ba70369 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=8942103afbd1458e8716d332d1e94e09]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:13.551427 28598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26dd77d4b614dce8a4e6cd41ba70369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.552461 28673 tablet_service.cc:1511] Processing CreateTablet for tablet c26dd77d4b614dce8a4e6cd41ba70369 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=8942103afbd1458e8716d332d1e94e09]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:13.553797 28673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26dd77d4b614dce8a4e6cd41ba70369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.556739 28524 tablet_service.cc:1511] Processing CreateTablet for tablet c26dd77d4b614dce8a4e6cd41ba70369 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=8942103afbd1458e8716d332d1e94e09]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:28:13.558813 28524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26dd77d4b614dce8a4e6cd41ba70369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:13.566653 28725 tablet_bootstrap.cc:492] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: Bootstrap starting.
I20260709 12:28:13.573875 28725 tablet_bootstrap.cc:654] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.575368 28723 tablet_bootstrap.cc:492] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: Bootstrap starting.
I20260709 12:28:13.584908 28723 tablet_bootstrap.cc:654] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.589927 28725 tablet_bootstrap.cc:492] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: No bootstrap required, opened a new log
I20260709 12:28:13.590566 28725 ts_tablet_manager.cc:1403] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.005s
I20260709 12:28:13.594452 28726 tablet_bootstrap.cc:492] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2: Bootstrap starting.
I20260709 12:28:13.594041 28725 raft_consensus.cc:359] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.595418 28725 raft_consensus.cc:385] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.595734 28725 raft_consensus.cc:740] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.596757 28725 consensus_queue.cc:260] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.598738 28723 tablet_bootstrap.cc:492] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: No bootstrap required, opened a new log
I20260709 12:28:13.599355 28723 ts_tablet_manager.cc:1403] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.002s
I20260709 12:28:13.602490 28725 ts_tablet_manager.cc:1434] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:28:13.602931 28723 raft_consensus.cc:359] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.603817 28723 raft_consensus.cc:385] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.604125 28723 raft_consensus.cc:740] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.604696 28725 tablet_bootstrap.cc:492] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db: Bootstrap starting.
I20260709 12:28:13.605226 28726 tablet_bootstrap.cc:654] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.605226 28723 consensus_queue.cc:260] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.607986 28723 ts_tablet_manager.cc:1434] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:28:13.609122 28723 tablet_bootstrap.cc:492] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea: Bootstrap starting.
I20260709 12:28:13.630870 28723 tablet_bootstrap.cc:654] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.633988 28725 tablet_bootstrap.cc:654] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.638130 28726 tablet_bootstrap.cc:492] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2: No bootstrap required, opened a new log
I20260709 12:28:13.638223 28723 tablet_bootstrap.cc:492] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea: No bootstrap required, opened a new log
I20260709 12:28:13.638854 28726 ts_tablet_manager.cc:1403] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.009s
I20260709 12:28:13.638909 28723 ts_tablet_manager.cc:1403] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea: Time spent bootstrapping tablet: real 0.030s	user 0.025s	sys 0.004s
I20260709 12:28:13.639681 28725 tablet_bootstrap.cc:492] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db: No bootstrap required, opened a new log
I20260709 12:28:13.640190 28725 ts_tablet_manager.cc:1403] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.006s
I20260709 12:28:13.642910 28725 raft_consensus.cc:359] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.642553 28726 raft_consensus.cc:359] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.642804 28723 raft_consensus.cc:359] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.643626 28725 raft_consensus.cc:385] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.643707 28726 raft_consensus.cc:385] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.643839 28723 raft_consensus.cc:385] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.644090 28725 raft_consensus.cc:740] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.644197 28726 raft_consensus.cc:740] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.644585 28723 raft_consensus.cc:740] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.645298 28726 consensus_queue.cc:260] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.645392 28725 consensus_queue.cc:260] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.645632 28723 consensus_queue.cc:260] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.650171 28726 ts_tablet_manager.cc:1434] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 12:28:13.651314 28723 ts_tablet_manager.cc:1434] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:13.652835 28725 ts_tablet_manager.cc:1434] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:13.651396 28726 tablet_bootstrap.cc:492] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: Bootstrap starting.
I20260709 12:28:13.663156 28726 tablet_bootstrap.cc:654] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:13.668776 28726 tablet_bootstrap.cc:492] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: No bootstrap required, opened a new log
I20260709 12:28:13.669291 28726 ts_tablet_manager.cc:1403] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.002s
I20260709 12:28:13.671974 28726 raft_consensus.cc:359] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.672572 28726 raft_consensus.cc:385] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:13.672804 28726 raft_consensus.cc:740] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Initialized, Role: FOLLOWER
I20260709 12:28:13.673592 28726 consensus_queue.cc:260] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.675438 28726 ts_tablet_manager.cc:1434] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:28:13.774566 28731 raft_consensus.cc:493] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.775242 28731 raft_consensus.cc:515] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.777540 28731 leader_election.cc:290] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:13.791513 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7" candidate_uuid: "6553532c440e4941a468fb249a0f92db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45a64d8394f4e78a582f7865a076cea" is_pre_election: true
I20260709 12:28:13.791513 28534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7" candidate_uuid: "6553532c440e4941a468fb249a0f92db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2" is_pre_election: true
I20260709 12:28:13.792732 28534 raft_consensus.cc:2468] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6553532c440e4941a468fb249a0f92db in term 0.
I20260709 12:28:13.792732 28683 raft_consensus.cc:2468] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6553532c440e4941a468fb249a0f92db in term 0.
I20260709 12:28:13.794389 28575 leader_election.cc:304] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, 6553532c440e4941a468fb249a0f92db; no voters: 
I20260709 12:28:13.795241 28731 raft_consensus.cc:2804] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:13.795697 28731 raft_consensus.cc:493] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.795996 28731 raft_consensus.cc:3060] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.802989 28731 raft_consensus.cc:515] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.805260 28731 leader_election.cc:290] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [CANDIDATE]: Term 1 election: Requested vote from peers 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:13.806118 28534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7" candidate_uuid: "6553532c440e4941a468fb249a0f92db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
I20260709 12:28:13.806623 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7" candidate_uuid: "6553532c440e4941a468fb249a0f92db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
I20260709 12:28:13.806895 28534 raft_consensus.cc:3060] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.807364 28683 raft_consensus.cc:3060] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.816780 28683 raft_consensus.cc:2468] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6553532c440e4941a468fb249a0f92db in term 1.
I20260709 12:28:13.817693 28534 raft_consensus.cc:2468] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6553532c440e4941a468fb249a0f92db in term 1.
I20260709 12:28:13.818135 28574 leader_election.cc:304] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6553532c440e4941a468fb249a0f92db, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:13.819116 28731 raft_consensus.cc:2804] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:13.820160 28731 raft_consensus.cc:697] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 LEADER]: Becoming Leader. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Running, Role: LEADER
I20260709 12:28:13.820930 28731 consensus_queue.cc:237] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.828706 28429 catalog_manager.cc:5697] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db reported cstate change: term changed from 0 to 1, leader changed from <none> to 6553532c440e4941a468fb249a0f92db (127.12.45.66). New cstate: current_term: 1 leader_uuid: "6553532c440e4941a468fb249a0f92db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:13.836282 28732 consensus_queue.cc:1048] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:13.848359 28732 consensus_queue.cc:1048] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:13.856482 28737 consensus_queue.cc:1048] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:13.872514 28732 consensus_queue.cc:1048] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:13.884078 28727 raft_consensus.cc:493] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.884689 28727 raft_consensus.cc:515] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.887765 28727 leader_election.cc:290] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017), 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835)
I20260709 12:28:13.907474 28534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "e45a64d8394f4e78a582f7865a076cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2" is_pre_election: true
I20260709 12:28:13.908277 28534 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e45a64d8394f4e78a582f7865a076cea in term 0.
I20260709 12:28:13.909300 28649 leader_election.cc:304] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:13.909873 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "e45a64d8394f4e78a582f7865a076cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6553532c440e4941a468fb249a0f92db" is_pre_election: true
I20260709 12:28:13.910071 28727 raft_consensus.cc:2804] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:13.910575 28727 raft_consensus.cc:493] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.910611 28609 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e45a64d8394f4e78a582f7865a076cea in term 0.
I20260709 12:28:13.910888 28727 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.918145 28727 raft_consensus.cc:515] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.920181 28727 leader_election.cc:290] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [CANDIDATE]: Term 1 election: Requested vote from peers 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017), 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835)
I20260709 12:28:13.921053 28534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "e45a64d8394f4e78a582f7865a076cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
I20260709 12:28:13.921747 28534 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.921708 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "e45a64d8394f4e78a582f7865a076cea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6553532c440e4941a468fb249a0f92db"
I20260709 12:28:13.922585 28609 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:13.930832 28609 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e45a64d8394f4e78a582f7865a076cea in term 1.
I20260709 12:28:13.930832 28534 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e45a64d8394f4e78a582f7865a076cea in term 1.
I20260709 12:28:13.932536 28649 leader_election.cc:304] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:13.933434 28727 raft_consensus.cc:2804] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:13.934773 28727 raft_consensus.cc:697] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 LEADER]: Becoming Leader. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Running, Role: LEADER
I20260709 12:28:13.935602 28727 consensus_queue.cc:237] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:13.944139 28430 catalog_manager.cc:5697] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea reported cstate change: term changed from 0 to 1, leader changed from <none> to e45a64d8394f4e78a582f7865a076cea (127.12.45.67). New cstate: current_term: 1 leader_uuid: "e45a64d8394f4e78a582f7865a076cea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.001843 28534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3"
dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
mode: GRACEFUL
new_leader_uuid: "6553532c440e4941a468fb249a0f92db"
 from {username='slave'} at 127.0.0.1:52990
I20260709 12:28:14.002545 28534 raft_consensus.cc:606] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Received request to transfer leadership to TS 6553532c440e4941a468fb249a0f92db
I20260709 12:28:14.019629 28609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7"
dest_uuid: "6553532c440e4941a468fb249a0f92db"
mode: GRACEFUL
new_leader_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
 from {username='slave'} at 127.0.0.1:39288
I20260709 12:28:14.020380 28609 raft_consensus.cc:606] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 LEADER]: Received request to transfer leadership to TS 04df0112d5274a11aa3a03f904c0a6c2
I20260709 12:28:14.037618 28683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369"
dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
mode: GRACEFUL
new_leader_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
 from {username='slave'} at 127.0.0.1:38596
I20260709 12:28:14.038372 28683 raft_consensus.cc:606] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 LEADER]: Received request to transfer leadership to TS 04df0112d5274a11aa3a03f904c0a6c2
I20260709 12:28:14.280226 28731 consensus_queue.cc:1048] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:14.294054 28754 consensus_queue.cc:1048] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [LEADER]: Connected to new peer: Peer: permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:14.294541 28737 raft_consensus.cc:993] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: : Instructing follower 6553532c440e4941a468fb249a0f92db to start an election
I20260709 12:28:14.295598 28737 raft_consensus.cc:1081] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Signalling peer 6553532c440e4941a468fb249a0f92db to start an election
I20260709 12:28:14.298357 28609 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3"
dest_uuid: "6553532c440e4941a468fb249a0f92db"
 from {username='slave'} at 127.0.0.1:39278
I20260709 12:28:14.298967 28609 raft_consensus.cc:493] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:14.299391 28609 raft_consensus.cc:3060] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.307878 28754 raft_consensus.cc:993] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: : Instructing follower 04df0112d5274a11aa3a03f904c0a6c2 to start an election
I20260709 12:28:14.308591 28773 raft_consensus.cc:1081] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 LEADER]: Signalling peer 04df0112d5274a11aa3a03f904c0a6c2 to start an election
I20260709 12:28:14.308954 28609 raft_consensus.cc:515] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.310756 28534 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7"
dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
 from {username='slave'} at 127.0.0.1:52972
I20260709 12:28:14.311540 28534 raft_consensus.cc:493] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:14.311942 28534 raft_consensus.cc:3060] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.312191 28609 leader_election.cc:290] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [CANDIDATE]: Term 2 election: Requested vote from peers 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:14.312285 28533 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3" candidate_uuid: "6553532c440e4941a468fb249a0f92db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
I20260709 12:28:14.312430 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84ed240160c14244b4c90e05d0800ef3" candidate_uuid: "6553532c440e4941a468fb249a0f92db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
I20260709 12:28:14.313311 28533 raft_consensus.cc:3055] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:14.313745 28683 raft_consensus.cc:3060] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.313720 28533 raft_consensus.cc:740] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Running, Role: LEADER
I20260709 12:28:14.315062 28533 consensus_queue.cc:260] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.316491 28533 raft_consensus.cc:3060] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.321184 28683 raft_consensus.cc:2468] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6553532c440e4941a468fb249a0f92db in term 2.
I20260709 12:28:14.321451 28534 raft_consensus.cc:515] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.322506 28574 leader_election.cc:304] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6553532c440e4941a468fb249a0f92db, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:14.323284 28773 raft_consensus.cc:2804] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:14.323827 28773 raft_consensus.cc:697] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 2 LEADER]: Becoming Leader. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Running, Role: LEADER
I20260709 12:28:14.324728 28534 leader_election.cc:290] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 2 election: Requested vote from peers 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:14.325568 28533 raft_consensus.cc:2468] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6553532c440e4941a468fb249a0f92db in term 2.
I20260709 12:28:14.325006 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
I20260709 12:28:14.324745 28773 consensus_queue.cc:237] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.326933 28683 raft_consensus.cc:3060] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.326076 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "069a624baf5248eb98242658df9f7fa7" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6553532c440e4941a468fb249a0f92db"
I20260709 12:28:14.328671 28609 raft_consensus.cc:3055] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:14.329141 28609 raft_consensus.cc:740] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6553532c440e4941a468fb249a0f92db, State: Running, Role: LEADER
I20260709 12:28:14.330116 28609 consensus_queue.cc:260] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.331673 28609 raft_consensus.cc:3060] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.337641 28683 raft_consensus.cc:2468] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 2.
I20260709 12:28:14.337751 28429 catalog_manager.cc:5697] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db reported cstate change: term changed from 1 to 2, leader changed from 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65) to 6553532c440e4941a468fb249a0f92db (127.12.45.66). New cstate: current_term: 2 leader_uuid: "6553532c440e4941a468fb249a0f92db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:14.339375 28499 leader_election.cc:304] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:14.340273 28737 raft_consensus.cc:2804] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:14.340794 28737 raft_consensus.cc:697] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Becoming Leader. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Running, Role: LEADER
I20260709 12:28:14.342041 28609 raft_consensus.cc:2468] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 2.
I20260709 12:28:14.341709 28737 consensus_queue.cc:237] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.349419 28430 catalog_manager.cc:5697] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 reported cstate change: term changed from 1 to 2, leader changed from 6553532c440e4941a468fb249a0f92db (127.12.45.66) to 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "04df0112d5274a11aa3a03f904c0a6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:14.492314 28727 consensus_queue.cc:1048] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [LEADER]: Connected to new peer: Peer: permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:14.500509 28776 raft_consensus.cc:993] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: : Instructing follower 04df0112d5274a11aa3a03f904c0a6c2 to start an election
I20260709 12:28:14.501549 28727 raft_consensus.cc:1081] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 LEADER]: Signalling peer 04df0112d5274a11aa3a03f904c0a6c2 to start an election
I20260709 12:28:14.504978 28534 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369"
dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
 from {username='slave'} at 127.0.0.1:52988
I20260709 12:28:14.506136 28534 raft_consensus.cc:493] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:14.512751 28534 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.512859 28776 consensus_queue.cc:1048] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [LEADER]: Connected to new peer: Peer: permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:14.523862 28534 raft_consensus.cc:515] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.527874 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6553532c440e4941a468fb249a0f92db"
I20260709 12:28:14.528286 28683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "04df0112d5274a11aa3a03f904c0a6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
I20260709 12:28:14.528569 28609 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.529018 28683 raft_consensus.cc:3055] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:14.529402 28683 raft_consensus.cc:740] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Running, Role: LEADER
I20260709 12:28:14.530304 28683 consensus_queue.cc:260] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.531476 28683 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:14.533959 28534 leader_election.cc:290] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 2 election: Requested vote from peers 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835), e45a64d8394f4e78a582f7865a076cea (127.12.45.67:34275)
I20260709 12:28:14.540406 28609 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 2.
I20260709 12:28:14.541498 28683 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04df0112d5274a11aa3a03f904c0a6c2 in term 2.
I20260709 12:28:14.541904 28498 leader_election.cc:304] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04df0112d5274a11aa3a03f904c0a6c2, 6553532c440e4941a468fb249a0f92db; no voters: 
I20260709 12:28:14.543051 28737 raft_consensus.cc:2804] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:14.543503 28737 raft_consensus.cc:697] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Becoming Leader. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Running, Role: LEADER
I20260709 12:28:14.544274 28737 consensus_queue.cc:237] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:14.551496 28432 catalog_manager.cc:5697] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 reported cstate change: term changed from 1 to 2, leader changed from e45a64d8394f4e78a582f7865a076cea (127.12.45.67) to 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "04df0112d5274a11aa3a03f904c0a6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:14.822294 12469 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:28:14.823944 12469 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:28:14.824342 12469 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:28:14.844996 28534 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369"
dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
mode: GRACEFUL
new_leader_uuid: "e45a64d8394f4e78a582f7865a076cea"
 from {username='slave'} at 127.0.0.1:44086
I20260709 12:28:14.845749 28534 raft_consensus.cc:606] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Received request to transfer leadership to TS e45a64d8394f4e78a582f7865a076cea
I20260709 12:28:14.847327 12469 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:28:14.869582 28683 raft_consensus.cc:1275] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Refusing update from remote peer 6553532c440e4941a468fb249a0f92db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:14.871377 28773 consensus_queue.cc:1048] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:14.883061 28534 raft_consensus.cc:1275] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Refusing update from remote peer 6553532c440e4941a468fb249a0f92db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:14.884860 28773 consensus_queue.cc:1048] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [LEADER]: Connected to new peer: Peer: permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:14.936254 28609 raft_consensus.cc:1275] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Refusing update from remote peer 04df0112d5274a11aa3a03f904c0a6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:14.938062 28737 consensus_queue.cc:1048] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:14.942353 28683 raft_consensus.cc:1275] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Refusing update from remote peer 04df0112d5274a11aa3a03f904c0a6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:14.943926 28737 consensus_queue.cc:1048] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:15.140048 28609 raft_consensus.cc:1275] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Refusing update from remote peer 04df0112d5274a11aa3a03f904c0a6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:15.141510 28783 consensus_queue.cc:1048] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:15.150871 28683 raft_consensus.cc:1275] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Refusing update from remote peer 04df0112d5274a11aa3a03f904c0a6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:15.152588 28737 consensus_queue.cc:1048] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:15.159559 28783 raft_consensus.cc:993] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: : Instructing follower e45a64d8394f4e78a582f7865a076cea to start an election
I20260709 12:28:15.160111 28737 raft_consensus.cc:1081] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Signalling peer e45a64d8394f4e78a582f7865a076cea to start an election
I20260709 12:28:15.161765 28683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369"
dest_uuid: "e45a64d8394f4e78a582f7865a076cea"
 from {username='slave'} at 127.0.0.1:38574
I20260709 12:28:15.162364 28683 raft_consensus.cc:493] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:15.162777 28683 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:15.168871 28683 raft_consensus.cc:515] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:15.170914 28683 leader_election.cc:290] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [CANDIDATE]: Term 3 election: Requested vote from peers 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65:44017), 6553532c440e4941a468fb249a0f92db (127.12.45.66:36835)
I20260709 12:28:15.171813 28534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "e45a64d8394f4e78a582f7865a076cea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "04df0112d5274a11aa3a03f904c0a6c2"
I20260709 12:28:15.172211 28609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26dd77d4b614dce8a4e6cd41ba70369" candidate_uuid: "e45a64d8394f4e78a582f7865a076cea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6553532c440e4941a468fb249a0f92db"
I20260709 12:28:15.172645 28534 raft_consensus.cc:3055] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:15.172919 28609 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:15.173166 28534 raft_consensus.cc:740] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 04df0112d5274a11aa3a03f904c0a6c2, State: Running, Role: LEADER
I20260709 12:28:15.174299 28534 consensus_queue.cc:260] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:15.175745 28534 raft_consensus.cc:3060] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:15.180102 28609 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e45a64d8394f4e78a582f7865a076cea in term 3.
I20260709 12:28:15.181578 28534 raft_consensus.cc:2468] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e45a64d8394f4e78a582f7865a076cea in term 3.
I20260709 12:28:15.181622 28647 leader_election.cc:304] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6553532c440e4941a468fb249a0f92db, e45a64d8394f4e78a582f7865a076cea; no voters: 
I20260709 12:28:15.182924 28776 raft_consensus.cc:2804] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:15.183894 28776 raft_consensus.cc:697] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 3 LEADER]: Becoming Leader. State: Replica: e45a64d8394f4e78a582f7865a076cea, State: Running, Role: LEADER
I20260709 12:28:15.184777 28776 consensus_queue.cc:237] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } }
I20260709 12:28:15.191773 28429 catalog_manager.cc:5697] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea reported cstate change: term changed from 2 to 3, leader changed from 04df0112d5274a11aa3a03f904c0a6c2 (127.12.45.65) to e45a64d8394f4e78a582f7865a076cea (127.12.45.67). New cstate: current_term: 3 leader_uuid: "e45a64d8394f4e78a582f7865a076cea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04df0112d5274a11aa3a03f904c0a6c2" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 44017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6553532c440e4941a468fb249a0f92db" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e45a64d8394f4e78a582f7865a076cea" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 34275 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:15.234303 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
I20260709 12:28:15.259593 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:15.260347 12469 tablet_replica.cc:333] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2: stopping tablet replica
I20260709 12:28:15.261114 12469 raft_consensus.cc:2243] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.261678 12469 raft_consensus.cc:2272] T c26dd77d4b614dce8a4e6cd41ba70369 P 04df0112d5274a11aa3a03f904c0a6c2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.264382 12469 tablet_replica.cc:333] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2: stopping tablet replica
I20260709 12:28:15.264983 12469 raft_consensus.cc:2243] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.265848 12469 raft_consensus.cc:2272] T 069a624baf5248eb98242658df9f7fa7 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.268800 12469 tablet_replica.cc:333] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2: stopping tablet replica
I20260709 12:28:15.269515 12469 raft_consensus.cc:2243] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.270072 12469 raft_consensus.cc:2272] T 84ed240160c14244b4c90e05d0800ef3 P 04df0112d5274a11aa3a03f904c0a6c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.273092 12469 tablet_replica.cc:333] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2: stopping tablet replica
I20260709 12:28:15.273706 12469 raft_consensus.cc:2243] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.274716 12469 raft_consensus.cc:2272] T 612ab577495546f8a7546437177d99b2 P 04df0112d5274a11aa3a03f904c0a6c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.299973 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
I20260709 12:28:15.313488 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
I20260709 12:28:15.336383 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:15.337229 12469 tablet_replica.cc:333] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db: stopping tablet replica
I20260709 12:28:15.337821 12469 raft_consensus.cc:2243] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.338385 12469 raft_consensus.cc:2272] T c26dd77d4b614dce8a4e6cd41ba70369 P 6553532c440e4941a468fb249a0f92db [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.340761 12469 tablet_replica.cc:333] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db: stopping tablet replica
I20260709 12:28:15.341372 12469 raft_consensus.cc:2243] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.341849 12469 raft_consensus.cc:2272] T 612ab577495546f8a7546437177d99b2 P 6553532c440e4941a468fb249a0f92db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.343825 12469 tablet_replica.cc:333] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db: stopping tablet replica
I20260709 12:28:15.344360 12469 raft_consensus.cc:2243] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.344813 12469 raft_consensus.cc:2272] T 069a624baf5248eb98242658df9f7fa7 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.347323 12469 tablet_replica.cc:333] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db: stopping tablet replica
I20260709 12:28:15.347893 12469 raft_consensus.cc:2243] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.348716 12469 raft_consensus.cc:2272] T 84ed240160c14244b4c90e05d0800ef3 P 6553532c440e4941a468fb249a0f92db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.365396 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:28:15.380453 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
I20260709 12:28:15.405556 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:15.406344 12469 tablet_replica.cc:333] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: stopping tablet replica
I20260709 12:28:15.406992 12469 raft_consensus.cc:2243] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.407696 12469 pending_rounds.cc:70] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: Trying to abort 1 pending ops.
I20260709 12:28:15.407866 12469 pending_rounds.cc:77] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305625989884583936 op_type: NO_OP noop_request { }
I20260709 12:28:15.408139 12469 raft_consensus.cc:2889] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:15.408394 12469 raft_consensus.cc:2272] T c26dd77d4b614dce8a4e6cd41ba70369 P e45a64d8394f4e78a582f7865a076cea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.410971 12469 tablet_replica.cc:333] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea: stopping tablet replica
I20260709 12:28:15.411527 12469 raft_consensus.cc:2243] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.412227 12469 raft_consensus.cc:2272] T 069a624baf5248eb98242658df9f7fa7 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.415228 12469 tablet_replica.cc:333] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea: stopping tablet replica
I20260709 12:28:15.415820 12469 raft_consensus.cc:2243] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.416294 12469 raft_consensus.cc:2272] T 84ed240160c14244b4c90e05d0800ef3 P e45a64d8394f4e78a582f7865a076cea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.419404 12469 tablet_replica.cc:333] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea: stopping tablet replica
I20260709 12:28:15.420433 12469 raft_consensus.cc:2243] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.420938 12469 raft_consensus.cc:2272] T 612ab577495546f8a7546437177d99b2 P e45a64d8394f4e78a582f7865a076cea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.445813 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:28:15.461438 12469 master.cc:562] Master@127.12.45.126:35799 shutting down...
I20260709 12:28:15.480041 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.480641 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.480988 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3b0668bd56094db9bdfdf94eca61808e: stopping tablet replica
I20260709 12:28:15.501407 12469 master.cc:584] Master@127.12.45.126:35799 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3169 ms)
[----------] 6 tests from LeaderRebalancerTest (521067 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:28:15.531356 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:46075
I20260709 12:28:15.532505 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:15.538717 28789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.539150 28790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.540782 28792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:15.541693 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:15.542932 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:15.543150 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:15.543354 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600095543340 us; error 0 us; skew 500 ppm
I20260709 12:28:15.543990 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:15.547271 12469 webserver.cc:533] Webserver started at http://127.12.45.126:33051/ using document root <none> and password file <none>
I20260709 12:28:15.547813 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:15.548020 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:15.548329 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:15.549490 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "46594ded80ce47f899e03c48072b90eb"
format_stamp: "Formatted at 2026-07-09 12:28:15 on dist-test-slave-90bm"
I20260709 12:28:15.554747 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 12:28:15.558858 28797 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.560102 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:28:15.560482 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "46594ded80ce47f899e03c48072b90eb"
format_stamp: "Formatted at 2026-07-09 12:28:15 on dist-test-slave-90bm"
I20260709 12:28:15.560802 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:15.571159 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:15.572185 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:15.615563 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:46075
I20260709 12:28:15.615720 28848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:46075 every 8 connection(s)
I20260709 12:28:15.620184 28849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:15.634328 28849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb: Bootstrap starting.
I20260709 12:28:15.639881 28849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:15.644762 28849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb: No bootstrap required, opened a new log
I20260709 12:28:15.647276 28849 raft_consensus.cc:359] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46594ded80ce47f899e03c48072b90eb" member_type: VOTER }
I20260709 12:28:15.647703 28849 raft_consensus.cc:385] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:15.647886 28849 raft_consensus.cc:740] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46594ded80ce47f899e03c48072b90eb, State: Initialized, Role: FOLLOWER
I20260709 12:28:15.648579 28849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46594ded80ce47f899e03c48072b90eb" member_type: VOTER }
I20260709 12:28:15.649001 28849 raft_consensus.cc:399] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:15.649195 28849 raft_consensus.cc:493] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:15.649400 28849 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:15.656126 28849 raft_consensus.cc:515] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46594ded80ce47f899e03c48072b90eb" member_type: VOTER }
I20260709 12:28:15.656810 28849 leader_election.cc:304] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46594ded80ce47f899e03c48072b90eb; no voters: 
I20260709 12:28:15.658120 28849 leader_election.cc:290] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:15.658684 28852 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:15.660223 28852 raft_consensus.cc:697] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 1 LEADER]: Becoming Leader. State: Replica: 46594ded80ce47f899e03c48072b90eb, State: Running, Role: LEADER
I20260709 12:28:15.661324 28852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46594ded80ce47f899e03c48072b90eb" member_type: VOTER }
I20260709 12:28:15.661494 28849 sys_catalog.cc:565] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:15.664783 28853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46594ded80ce47f899e03c48072b90eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46594ded80ce47f899e03c48072b90eb" member_type: VOTER } }
I20260709 12:28:15.665576 28853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:15.664783 28854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46594ded80ce47f899e03c48072b90eb. Latest consensus state: current_term: 1 leader_uuid: "46594ded80ce47f899e03c48072b90eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46594ded80ce47f899e03c48072b90eb" member_type: VOTER } }
I20260709 12:28:15.667520 28854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:15.671518 28857 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:15.676208 28857 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:15.683475 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:15.686837 28857 catalog_manager.cc:1383] Generated new cluster ID: e83dae27366948a98ceb794f0ffae9a0
I20260709 12:28:15.687140 28857 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:15.714035 28857 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:15.715559 28857 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:15.733664 28857 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb: Generated new TSK 0
I20260709 12:28:15.734493 28857 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:15.755523 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:15.762349 28870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.763111 28871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.765416 28873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:15.765676 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:15.766817 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:15.767064 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:15.767258 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600095767236 us; error 0 us; skew 500 ppm
I20260709 12:28:15.767823 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:15.770865 12469 webserver.cc:533] Webserver started at http://127.12.45.65:39203/ using document root <none> and password file <none>
I20260709 12:28:15.771410 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:15.771656 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:15.771960 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:15.773339 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "012f61c51f47429ca57af12350fe42f4"
format_stamp: "Formatted at 2026-07-09 12:28:15 on dist-test-slave-90bm"
I20260709 12:28:15.779001 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:28:15.782785 28878 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.783663 12469 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:28:15.783990 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "012f61c51f47429ca57af12350fe42f4"
format_stamp: "Formatted at 2026-07-09 12:28:15 on dist-test-slave-90bm"
I20260709 12:28:15.784323 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:15.798945 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:15.800240 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:15.801864 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:15.804584 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:15.804836 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.805084 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:15.805265 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.859315 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:45029
I20260709 12:28:15.859465 28940 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:45029 every 8 connection(s)
I20260709 12:28:15.865228 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:15.873219 28945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.878484 28946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:15.879279 12469 server_base.cc:1061] running on GCE node
W20260709 12:28:15.880846 28948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:15.882174 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:15.882606 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:15.882894 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600095882849 us; error 0 us; skew 500 ppm
I20260709 12:28:15.883605 28941 heartbeater.cc:344] Connected to a master server at 127.12.45.126:46075
I20260709 12:28:15.883715 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:15.884181 28941 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:15.885370 28941 heartbeater.cc:507] Master 127.12.45.126:46075 requested a full tablet report, sending...
I20260709 12:28:15.887331 12469 webserver.cc:533] Webserver started at http://127.12.45.66:38827/ using document root <none> and password file <none>
I20260709 12:28:15.888036 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:15.888248 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:15.888212 28814 ts_manager.cc:194] Registered new tserver with Master: 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029)
I20260709 12:28:15.888778 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:15.890172 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
format_stamp: "Formatted at 2026-07-09 12:28:15 on dist-test-slave-90bm"
I20260709 12:28:15.890573 28814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60866
I20260709 12:28:15.896275 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:28:15.900627 28953 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.901785 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:28:15.902163 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
format_stamp: "Formatted at 2026-07-09 12:28:15 on dist-test-slave-90bm"
I20260709 12:28:15.902611 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:15.924678 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:15.925949 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:15.927785 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:15.930431 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:15.930716 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.930996 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:15.931177 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:15.979883 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:46859
I20260709 12:28:15.980034 29015 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:46859 every 8 connection(s)
I20260709 12:28:15.985415 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:15.993248 29019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.993579 29020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:15.998085 29023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:15.998883 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:16.000459 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:16.000810 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:16.001001 29016 heartbeater.cc:344] Connected to a master server at 127.12.45.126:46075
I20260709 12:28:16.001042 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600096001018 us; error 0 us; skew 500 ppm
I20260709 12:28:16.001708 29016 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:16.002321 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:16.002797 29016 heartbeater.cc:507] Master 127.12.45.126:46075 requested a full tablet report, sending...
I20260709 12:28:16.005474 12469 webserver.cc:533] Webserver started at http://127.12.45.67:33589/ using document root <none> and password file <none>
I20260709 12:28:16.005816 28814 ts_manager.cc:194] Registered new tserver with Master: e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:16.006364 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:16.006700 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:16.007045 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:16.007913 28814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60868
I20260709 12:28:16.008697 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "09474a5ec3194a23a04d99c316b86f09"
format_stamp: "Formatted at 2026-07-09 12:28:16 on dist-test-slave-90bm"
I20260709 12:28:16.014436 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:28:16.019006 29027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:16.020015 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:16.020377 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "09474a5ec3194a23a04d99c316b86f09"
format_stamp: "Formatted at 2026-07-09 12:28:16 on dist-test-slave-90bm"
I20260709 12:28:16.020711 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:16.035990 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:16.037312 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:16.039055 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:16.041666 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:16.041944 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:16.042207 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:16.042439 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:28:16.092664 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:44819
I20260709 12:28:16.092839 29089 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:44819 every 8 connection(s)
I20260709 12:28:16.109606 29090 heartbeater.cc:344] Connected to a master server at 127.12.45.126:46075
I20260709 12:28:16.110376 29090 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:16.111297 29090 heartbeater.cc:507] Master 127.12.45.126:46075 requested a full tablet report, sending...
I20260709 12:28:16.113695 28814 ts_manager.cc:194] Registered new tserver with Master: 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.114162 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017738318s
I20260709 12:28:16.115670 28814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60880
I20260709 12:28:16.199527 28814 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:16.203326 28814 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:16.302538 28905 tablet_service.cc:1511] Processing CreateTablet for tablet 4a8b2f17137b433296a99c7bc6971f12 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:16.303737 28906 tablet_service.cc:1511] Processing CreateTablet for tablet c7514edcd3054478ae21b23d56d58360 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:16.305032 28905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a8b2f17137b433296a99c7bc6971f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.305473 28903 tablet_service.cc:1511] Processing CreateTablet for tablet 1d117af0c3f24c4a8fd44303277d126a (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:16.306464 28906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7514edcd3054478ae21b23d56d58360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.303829 28904 tablet_service.cc:1511] Processing CreateTablet for tablet a5515f4981eb4fc3951f316b5915f12c (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:16.308830 28904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5515f4981eb4fc3951f316b5915f12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.310930 28903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d117af0c3f24c4a8fd44303277d126a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.312863 28902 tablet_service.cc:1511] Processing CreateTablet for tablet 897444c2b13e4cb69911b6de6d34b6fe (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:16.317374 28902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897444c2b13e4cb69911b6de6d34b6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.322567 28901 tablet_service.cc:1511] Processing CreateTablet for tablet 869187ef93bc49c0aad674b6b04d6574 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:16.324404 28901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869187ef93bc49c0aad674b6b04d6574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.330032 29054 tablet_service.cc:1511] Processing CreateTablet for tablet 4a8b2f17137b433296a99c7bc6971f12 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:16.331815 28898 tablet_service.cc:1511] Processing CreateTablet for tablet f3575a1ff5a34e659f8921eb7bab27ab (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:16.332232 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a8b2f17137b433296a99c7bc6971f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.333040 28899 tablet_service.cc:1511] Processing CreateTablet for tablet 368b8c8ea73c4ec8a51ff11fb9598522 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:16.335765 29055 tablet_service.cc:1511] Processing CreateTablet for tablet c7514edcd3054478ae21b23d56d58360 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:16.337414 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7514edcd3054478ae21b23d56d58360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.339912 29053 tablet_service.cc:1511] Processing CreateTablet for tablet a5515f4981eb4fc3951f316b5915f12c (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:16.341645 29053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5515f4981eb4fc3951f316b5915f12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.343722 29052 tablet_service.cc:1511] Processing CreateTablet for tablet 1d117af0c3f24c4a8fd44303277d126a (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:16.345506 29052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d117af0c3f24c4a8fd44303277d126a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.348037 28900 tablet_service.cc:1511] Processing CreateTablet for tablet 05682701dab0458fba871736c6f6a74a (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:16.350492 29051 tablet_service.cc:1511] Processing CreateTablet for tablet 897444c2b13e4cb69911b6de6d34b6fe (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:16.352069 29051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897444c2b13e4cb69911b6de6d34b6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.356432 29050 tablet_service.cc:1511] Processing CreateTablet for tablet 869187ef93bc49c0aad674b6b04d6574 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:16.358078 29050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869187ef93bc49c0aad674b6b04d6574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.374800 28974 tablet_service.cc:1511] Processing CreateTablet for tablet 368b8c8ea73c4ec8a51ff11fb9598522 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:16.360402 29049 tablet_service.cc:1511] Processing CreateTablet for tablet 05682701dab0458fba871736c6f6a74a (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:16.376976 29049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05682701dab0458fba871736c6f6a74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.377507 28974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 368b8c8ea73c4ec8a51ff11fb9598522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.363133 29048 tablet_service.cc:1511] Processing CreateTablet for tablet 368b8c8ea73c4ec8a51ff11fb9598522 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:16.365489 28981 tablet_service.cc:1511] Processing CreateTablet for tablet c7514edcd3054478ae21b23d56d58360 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:16.366503 28978 tablet_service.cc:1511] Processing CreateTablet for tablet 1d117af0c3f24c4a8fd44303277d126a (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:16.367589 28979 tablet_service.cc:1511] Processing CreateTablet for tablet a5515f4981eb4fc3951f316b5915f12c (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:16.368084 28976 tablet_service.cc:1511] Processing CreateTablet for tablet 869187ef93bc49c0aad674b6b04d6574 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:16.369519 28980 tablet_service.cc:1511] Processing CreateTablet for tablet 4a8b2f17137b433296a99c7bc6971f12 (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:16.369557 28977 tablet_service.cc:1511] Processing CreateTablet for tablet 897444c2b13e4cb69911b6de6d34b6fe (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:16.372148 29047 tablet_service.cc:1511] Processing CreateTablet for tablet f3575a1ff5a34e659f8921eb7bab27ab (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:16.372728 28973 tablet_service.cc:1511] Processing CreateTablet for tablet f3575a1ff5a34e659f8921eb7bab27ab (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:16.372951 28975 tablet_service.cc:1511] Processing CreateTablet for tablet 05682701dab0458fba871736c6f6a74a (DEFAULT_TABLE table=test-workload [id=8134ecc469c94b07a96e0cc66a71b681]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:16.396425 29048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 368b8c8ea73c4ec8a51ff11fb9598522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.399113 29047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3575a1ff5a34e659f8921eb7bab27ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.333701 28898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3575a1ff5a34e659f8921eb7bab27ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.406143 28900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05682701dab0458fba871736c6f6a74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.384850 28981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7514edcd3054478ae21b23d56d58360. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.410339 28976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 869187ef93bc49c0aad674b6b04d6574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.414108 28973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3575a1ff5a34e659f8921eb7bab27ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.418725 28978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d117af0c3f24c4a8fd44303277d126a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.422817 28977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 897444c2b13e4cb69911b6de6d34b6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.430874 28979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5515f4981eb4fc3951f316b5915f12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.432345 28899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 368b8c8ea73c4ec8a51ff11fb9598522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.432739 28975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05682701dab0458fba871736c6f6a74a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.434484 28980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a8b2f17137b433296a99c7bc6971f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:16.444137 29107 tablet_bootstrap.cc:492] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:16.481209 29108 tablet_bootstrap.cc:492] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.487373 29107 tablet_bootstrap.cc:654] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.508503 29106 tablet_bootstrap.cc:492] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:16.530706 29106 tablet_bootstrap.cc:654] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.532168 29108 tablet_bootstrap.cc:654] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.587494 29106 tablet_bootstrap.cc:492] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:16.588131 29106 ts_tablet_manager.cc:1403] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.080s	user 0.016s	sys 0.030s
I20260709 12:28:16.592057 29106 raft_consensus.cc:359] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.593152 29106 raft_consensus.cc:385] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.593508 29106 raft_consensus.cc:740] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.594079 29108 tablet_bootstrap.cc:492] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:16.594751 29108 ts_tablet_manager.cc:1403] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.114s	user 0.020s	sys 0.013s
I20260709 12:28:16.594444 29106 consensus_queue.cc:260] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.597823 28941 heartbeater.cc:499] Master 127.12.45.126:46075 was elected leader, sending a full tablet report...
I20260709 12:28:16.598208 29106 ts_tablet_manager.cc:1434] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:28:16.599439 29106 tablet_bootstrap.cc:492] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:16.608450 29106 tablet_bootstrap.cc:654] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.611049 29108 raft_consensus.cc:359] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.612049 29108 raft_consensus.cc:385] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.612666 29108 raft_consensus.cc:740] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.614709 29108 consensus_queue.cc:260] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.631184 29107 tablet_bootstrap.cc:492] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:16.631955 29107 ts_tablet_manager.cc:1403] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.188s	user 0.018s	sys 0.021s
I20260709 12:28:16.635818 29107 raft_consensus.cc:359] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.636718 29107 raft_consensus.cc:385] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.637040 29107 raft_consensus.cc:740] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.637982 29107 consensus_queue.cc:260] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.641373 29108 ts_tablet_manager.cc:1434] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.006s
I20260709 12:28:16.643280 29090 heartbeater.cc:499] Master 127.12.45.126:46075 was elected leader, sending a full tablet report...
I20260709 12:28:16.645056 29016 heartbeater.cc:499] Master 127.12.45.126:46075 was elected leader, sending a full tablet report...
I20260709 12:28:16.645660 29106 tablet_bootstrap.cc:492] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:16.646291 29106 ts_tablet_manager.cc:1403] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.022s
I20260709 12:28:16.649923 29107 ts_tablet_manager.cc:1434] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:28:16.649930 29106 raft_consensus.cc:359] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.651136 29106 raft_consensus.cc:385] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.651417 29107 tablet_bootstrap.cc:492] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:16.651463 29114 raft_consensus.cc:493] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.652338 29114 raft_consensus.cc:515] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.654969 29108 tablet_bootstrap.cc:492] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.660641 29107 tablet_bootstrap.cc:654] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.651484 29106 raft_consensus.cc:740] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.663975 29106 consensus_queue.cc:260] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.666870 29108 tablet_bootstrap.cc:654] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.669175 29106 ts_tablet_manager.cc:1434] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:28:16.670529 29106 tablet_bootstrap.cc:492] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
W20260709 12:28:16.678165 28862 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:16.685209 29108 tablet_bootstrap.cc:492] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:16.685303 29114 leader_election.cc:290] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.686367 29108 ts_tablet_manager.cc:1403] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.007s
I20260709 12:28:16.691192 29108 raft_consensus.cc:359] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.692283 29108 raft_consensus.cc:385] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.692617 29108 raft_consensus.cc:740] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.693583 29108 consensus_queue.cc:260] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.696023 29108 ts_tablet_manager.cc:1434] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:16.697207 29108 tablet_bootstrap.cc:492] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.705942 29108 tablet_bootstrap.cc:654] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.712692 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
W20260709 12:28:16.715726 28955 leader_election.cc:343] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:16.722239 29106 tablet_bootstrap.cc:654] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.738436 29108 tablet_bootstrap.cc:492] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:16.739071 29108 ts_tablet_manager.cc:1403] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.021s
I20260709 12:28:16.742827 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:16.742771 29108 raft_consensus.cc:359] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.743726 29108 raft_consensus.cc:385] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.744007 29107 tablet_bootstrap.cc:492] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:16.744170 29106 tablet_bootstrap.cc:492] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:16.744053 29108 raft_consensus.cc:740] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.744745 29107 ts_tablet_manager.cc:1403] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.094s	user 0.047s	sys 0.043s
I20260709 12:28:16.744935 29106 ts_tablet_manager.cc:1403] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.005s
W20260709 12:28:16.745150 28957 leader_election.cc:343] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:16.745427 29108 consensus_queue.cc:260] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.745774 28957 leader_election.cc:304] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:16.746799 29114 raft_consensus.cc:2749] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:16.748530 29107 raft_consensus.cc:359] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.749356 29107 raft_consensus.cc:385] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.749635 29107 raft_consensus.cc:740] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.749730 29108 ts_tablet_manager.cc:1434] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:28:16.749730 29106 raft_consensus.cc:359] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.750897 29106 raft_consensus.cc:385] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.750818 29107 consensus_queue.cc:260] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.751338 29106 raft_consensus.cc:740] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.752398 29106 consensus_queue.cc:260] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.753206 29108 tablet_bootstrap.cc:492] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.753212 29107 ts_tablet_manager.cc:1434] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 12:28:16.755823 29112 raft_consensus.cc:493] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.756721 29107 tablet_bootstrap.cc:492] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:16.756577 29112 raft_consensus.cc:515] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.762905 29108 tablet_bootstrap.cc:654] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.762836 29112 leader_election.cc:290] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.766505 29107 tablet_bootstrap.cc:654] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.754850 29106 ts_tablet_manager.cc:1434] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:28:16.774597 29106 tablet_bootstrap.cc:492] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:16.793628 29106 tablet_bootstrap.cc:654] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.801172 29108 tablet_bootstrap.cc:492] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:16.802124 29108 ts_tablet_manager.cc:1403] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.049s	user 0.028s	sys 0.016s
I20260709 12:28:16.805851 29108 raft_consensus.cc:359] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.806757 29108 raft_consensus.cc:385] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.807049 29108 raft_consensus.cc:740] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.810468 29108 consensus_queue.cc:260] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.814427 29108 ts_tablet_manager.cc:1434] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:28:16.814975 29107 tablet_bootstrap.cc:492] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:16.815598 29107 ts_tablet_manager.cc:1403] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.059s	user 0.021s	sys 0.034s
I20260709 12:28:16.815800 29108 tablet_bootstrap.cc:492] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.819028 29107 raft_consensus.cc:359] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.819890 29107 raft_consensus.cc:385] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.820277 29107 raft_consensus.cc:740] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.821465 29107 consensus_queue.cc:260] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.824357 29107 ts_tablet_manager.cc:1434] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 12:28:16.824961 29108 tablet_bootstrap.cc:654] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.825519 29107 tablet_bootstrap.cc:492] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:16.835361 29106 tablet_bootstrap.cc:492] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:16.836057 29106 ts_tablet_manager.cc:1403] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.003s
I20260709 12:28:16.836308 29107 tablet_bootstrap.cc:654] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.844712 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5515f4981eb4fc3951f316b5915f12c" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:16.845597 28991 raft_consensus.cc:2468] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 0.
I20260709 12:28:16.847431 28882 leader_election.cc:304] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:16.847565 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5515f4981eb4fc3951f316b5915f12c" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:16.848532 29112 raft_consensus.cc:2804] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:16.848954 29112 raft_consensus.cc:493] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:16.849164 28882 leader_election.cc:343] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:16.849328 29112 raft_consensus.cc:3060] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.849412 29108 tablet_bootstrap.cc:492] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:16.849458 29107 tablet_bootstrap.cc:492] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:16.850435 29108 ts_tablet_manager.cc:1403] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.014s
I20260709 12:28:16.850440 29107 ts_tablet_manager.cc:1403] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.006s
I20260709 12:28:16.854609 29108 raft_consensus.cc:359] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.854805 29107 raft_consensus.cc:359] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.855567 29108 raft_consensus.cc:385] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.855624 29107 raft_consensus.cc:385] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.856036 29108 raft_consensus.cc:740] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.856078 29107 raft_consensus.cc:740] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.857069 29107 consensus_queue.cc:260] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.857069 29108 consensus_queue.cc:260] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.839656 29106 raft_consensus.cc:359] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.858525 29106 raft_consensus.cc:385] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.858845 29106 raft_consensus.cc:740] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.857981 29112 raft_consensus.cc:515] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.859947 29108 ts_tablet_manager.cc:1434] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:28:16.862699 29108 tablet_bootstrap.cc:492] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.862546 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5515f4981eb4fc3951f316b5915f12c" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:16.863233 28991 raft_consensus.cc:3060] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.863519 29106 consensus_queue.cc:260] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.865636 29112 leader_election.cc:290] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.867775 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5515f4981eb4fc3951f316b5915f12c" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:16.868496 29065 raft_consensus.cc:3060] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.869637 29107 ts_tablet_manager.cc:1434] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.003s
I20260709 12:28:16.871510 29107 tablet_bootstrap.cc:492] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:16.873292 29114 raft_consensus.cc:493] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.873894 29114 raft_consensus.cc:515] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.876688 28991 raft_consensus.cc:2468] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:16.878731 29106 ts_tablet_manager.cc:1434] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.042s	user 0.010s	sys 0.003s
I20260709 12:28:16.878660 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:16.879611 29064 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:16.879830 29107 tablet_bootstrap.cc:654] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.881119 28957 leader_election.cc:304] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:16.882231 29115 raft_consensus.cc:2804] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:16.882637 29115 raft_consensus.cc:493] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.882963 29115 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.884431 29108 tablet_bootstrap.cc:654] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.887657 29106 tablet_bootstrap.cc:492] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:16.887778 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:16.888753 28916 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:16.894935 28882 leader_election.cc:304] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:16.895680 29114 leader_election.cc:290] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.898223 29112 raft_consensus.cc:493] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.898751 29112 raft_consensus.cc:515] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.901072 29112 leader_election.cc:290] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.901589 29112 raft_consensus.cc:2804] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:16.902027 29112 raft_consensus.cc:697] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:16.905243 29112 consensus_queue.cc:237] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.905896 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897444c2b13e4cb69911b6de6d34b6fe" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:16.906198 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897444c2b13e4cb69911b6de6d34b6fe" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
W20260709 12:28:16.909175 28882 leader_election.cc:343] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:16.909950 28882 leader_election.cc:343] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:16.910385 28882 leader_election.cc:304] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4; no voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:16.911365 29130 raft_consensus.cc:2749] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:16.914845 29106 tablet_bootstrap.cc:654] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.916409 29115 raft_consensus.cc:515] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.925019 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:16.925709 28916 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.927140 28812 catalog_manager.cc:5697] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:16.921922 29065 raft_consensus.cc:2468] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:16.943454 28916 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:16.949333 28955 leader_election.cc:304] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:16.951054 29114 raft_consensus.cc:2804] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:16.946247 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:16.955016 29065 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.960700 29106 tablet_bootstrap.cc:492] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:16.947086 29115 leader_election.cc:290] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:16.961419 29106 ts_tablet_manager.cc:1403] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.074s	user 0.015s	sys 0.007s
I20260709 12:28:16.966132 29106 raft_consensus.cc:359] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.968415 29106 raft_consensus.cc:385] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.969165 29106 raft_consensus.cc:740] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.970523 29108 tablet_bootstrap.cc:492] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:16.971203 29108 ts_tablet_manager.cc:1403] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.109s	user 0.024s	sys 0.061s
I20260709 12:28:16.970746 29106 consensus_queue.cc:260] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.971818 29114 raft_consensus.cc:697] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:16.972817 29114 consensus_queue.cc:237] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.971493 29107 tablet_bootstrap.cc:492] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:16.973984 29107 ts_tablet_manager.cc:1403] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.103s	user 0.040s	sys 0.055s
I20260709 12:28:16.974511 29108 raft_consensus.cc:359] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.975567 29108 raft_consensus.cc:385] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.975971 29108 raft_consensus.cc:740] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.976994 29112 raft_consensus.cc:493] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.976982 29108 consensus_queue.cc:260] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.977659 29112 raft_consensus.cc:515] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.978819 29107 raft_consensus.cc:359] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:16.974506 29106 ts_tablet_manager.cc:1434] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:28:16.980520 29106 tablet_bootstrap.cc:492] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:16.981097 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7514edcd3054478ae21b23d56d58360" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:16.981585 29065 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:16.981851 28991 raft_consensus.cc:2468] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 0.
I20260709 12:28:16.983678 29108 ts_tablet_manager.cc:1434] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:28:16.984880 29108 tablet_bootstrap.cc:492] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:16.984609 28882 leader_election.cc:304] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:16.985946 29130 raft_consensus.cc:2804] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:16.986377 29130 raft_consensus.cc:493] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.986701 29130 raft_consensus.cc:3060] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:16.993588 29108 tablet_bootstrap.cc:654] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:16.992774 28810 catalog_manager.cc:5697] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:16.996145 29107 raft_consensus.cc:385] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:16.996690 29107 raft_consensus.cc:740] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:16.997798 29107 consensus_queue.cc:260] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.000634 29107 ts_tablet_manager.cc:1434] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:28:17.001825 29107 tablet_bootstrap.cc:492] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:17.003448 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7514edcd3054478ae21b23d56d58360" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
W20260709 12:28:17.006079 28882 leader_election.cc:343] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:17.007193 29114 raft_consensus.cc:493] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.008028 29114 raft_consensus.cc:515] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.009802 29107 tablet_bootstrap.cc:654] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.021286 29112 leader_election.cc:290] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.029142 29114 leader_election.cc:290] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.035903 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:17.036239 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:17.037269 29065 raft_consensus.cc:2468] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
W20260709 12:28:17.037665 28955 leader_election.cc:343] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:17.035964 29108 tablet_bootstrap.cc:492] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:17.038743 28957 leader_election.cc:304] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:17.039187 29108 ts_tablet_manager.cc:1403] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.055s	user 0.034s	sys 0.012s
I20260709 12:28:17.039369 29130 raft_consensus.cc:515] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.042429 29114 raft_consensus.cc:2804] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:17.042927 29114 raft_consensus.cc:493] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.043268 29114 raft_consensus.cc:3060] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.043680 29130 leader_election.cc:290] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.043926 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7514edcd3054478ae21b23d56d58360" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
W20260709 12:28:17.045763 28882 leader_election.cc:343] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:17.047405 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7514edcd3054478ae21b23d56d58360" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:17.048135 28991 raft_consensus.cc:3060] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.049446 29106 tablet_bootstrap.cc:654] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.056641 29114 raft_consensus.cc:515] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.058997 28991 raft_consensus.cc:2468] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:17.062165 28882 leader_election.cc:304] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:17.063170 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.064064 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.064138 29130 raft_consensus.cc:2804] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:17.064714 28955 leader_election.cc:343] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:17.064808 29065 raft_consensus.cc:3060] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.064983 29130 raft_consensus.cc:697] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:17.066217 29130 consensus_queue.cc:237] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.059365 29108 raft_consensus.cc:359] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.069867 29114 leader_election.cc:290] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.073410 29107 tablet_bootstrap.cc:492] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:17.074045 29107 ts_tablet_manager.cc:1403] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.036s
I20260709 12:28:17.069674 29108 raft_consensus.cc:385] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.075831 29108 raft_consensus.cc:740] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.077106 29065 raft_consensus.cc:2468] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.077934 29107 raft_consensus.cc:359] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.077273 29108 consensus_queue.cc:260] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.079172 29107 raft_consensus.cc:385] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.079159 28957 leader_election.cc:304] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:17.080030 29107 raft_consensus.cc:740] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.081218 29114 raft_consensus.cc:2804] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:17.080938 29107 consensus_queue.cc:260] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.081794 29114 raft_consensus.cc:697] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.082191 29108 ts_tablet_manager.cc:1434] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.003s
I20260709 12:28:17.082793 29114 consensus_queue.cc:237] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.083433 29108 tablet_bootstrap.cc:492] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:17.083786 29107 ts_tablet_manager.cc:1434] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:28:17.085258 29107 tablet_bootstrap.cc:492] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:17.093063 29108 tablet_bootstrap.cc:654] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.096753 29106 tablet_bootstrap.cc:492] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:17.097463 29106 ts_tablet_manager.cc:1403] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.117s	user 0.026s	sys 0.009s
I20260709 12:28:17.105456 29106 raft_consensus.cc:359] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.110805 28813 catalog_manager.cc:5697] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.113036 29107 tablet_bootstrap.cc:654] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.116016 29106 raft_consensus.cc:385] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.116636 29106 raft_consensus.cc:740] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.117799 29106 consensus_queue.cc:260] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.118151 28810 catalog_manager.cc:5697] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.121732 29106 ts_tablet_manager.cc:1434] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:28:17.124166 29106 tablet_bootstrap.cc:492] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:17.130302 29108 tablet_bootstrap.cc:492] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:17.131177 29108 ts_tablet_manager.cc:1403] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.020s
I20260709 12:28:17.134640 29106 tablet_bootstrap.cc:654] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.135006 29108 raft_consensus.cc:359] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.135859 29108 raft_consensus.cc:385] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.136207 29108 raft_consensus.cc:740] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.137229 29108 consensus_queue.cc:260] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.140347 29108 ts_tablet_manager.cc:1434] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:28:17.141781 29108 tablet_bootstrap.cc:492] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:17.147246 29114 raft_consensus.cc:493] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.147944 29114 raft_consensus.cc:515] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.149909 29108 tablet_bootstrap.cc:654] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.153573 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:17.154501 28916 raft_consensus.cc:2468] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:17.155999 28955 leader_election.cc:304] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.156965 29137 raft_consensus.cc:2804] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:17.157399 29137 raft_consensus.cc:493] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.157253 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:17.158004 29137 raft_consensus.cc:3060] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.158443 29065 raft_consensus.cc:2468] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:17.160553 29114 leader_election.cc:290] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.163265 29106 tablet_bootstrap.cc:492] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:17.163905 29106 ts_tablet_manager.cc:1403] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.010s
I20260709 12:28:17.165933 29108 tablet_bootstrap.cc:492] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:17.166599 29108 ts_tablet_manager.cc:1403] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.003s
I20260709 12:28:17.167382 29137 raft_consensus.cc:515] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.167804 29106 raft_consensus.cc:359] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.168784 29106 raft_consensus.cc:385] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.169227 29106 raft_consensus.cc:740] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.170445 29108 raft_consensus.cc:359] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.170372 29106 consensus_queue.cc:260] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.171208 29108 raft_consensus.cc:385] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.171525 29108 raft_consensus.cc:740] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.171479 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.172349 28916 raft_consensus.cc:3060] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.173024 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.172660 29108 consensus_queue.cc:260] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.173627 29065 raft_consensus.cc:3060] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.175567 29108 ts_tablet_manager.cc:1434] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:28:17.178027 29106 ts_tablet_manager.cc:1434] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:28:17.178414 29137 leader_election.cc:290] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.179136 29106 tablet_bootstrap.cc:492] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:17.181666 29107 tablet_bootstrap.cc:492] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:17.182230 29107 ts_tablet_manager.cc:1403] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.097s	user 0.019s	sys 0.023s
I20260709 12:28:17.183270 29065 raft_consensus.cc:2468] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.184784 28957 leader_election.cc:304] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.184942 29137 raft_consensus.cc:493] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.185561 29137 raft_consensus.cc:515] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.185762 29114 raft_consensus.cc:2804] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:17.186839 29114 raft_consensus.cc:697] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.188014 29106 tablet_bootstrap.cc:654] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.187738 29114 consensus_queue.cc:237] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.188177 29137 leader_election.cc:290] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.193274 29106 tablet_bootstrap.cc:492] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:17.193526 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:17.193805 29106 ts_tablet_manager.cc:1403] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 12:28:17.194394 29064 raft_consensus.cc:2468] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:17.195783 28957 leader_election.cc:304] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.196372 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:17.196810 29115 raft_consensus.cc:2804] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:17.197247 29115 raft_consensus.cc:493] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.197608 29115 raft_consensus.cc:3060] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.197644 29106 raft_consensus.cc:359] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
W20260709 12:28:17.198168 28955 leader_election.cc:343] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:17.198498 29106 raft_consensus.cc:385] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.198843 29106 raft_consensus.cc:740] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.199730 29106 consensus_queue.cc:260] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.200300 28916 raft_consensus.cc:2468] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.194406 29107 raft_consensus.cc:359] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.202167 29107 raft_consensus.cc:385] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.203280 29106 ts_tablet_manager.cc:1434] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:17.203822 29107 raft_consensus.cc:740] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.204509 29106 tablet_bootstrap.cc:492] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:17.205003 29107 consensus_queue.cc:260] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.206796 29115 raft_consensus.cc:515] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.209832 29107 ts_tablet_manager.cc:1434] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 12:28:17.211328 29107 tablet_bootstrap.cc:492] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:17.212494 29106 tablet_bootstrap.cc:654] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.226912 29115 leader_election.cc:290] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.227210 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.228071 28916 raft_consensus.cc:3060] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.229099 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.229856 29065 raft_consensus.cc:3060] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.239455 28916 raft_consensus.cc:2468] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.240185 29106 tablet_bootstrap.cc:492] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:17.240293 29065 raft_consensus.cc:2468] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.240990 29106 ts_tablet_manager.cc:1403] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.014s
I20260709 12:28:17.241187 28955 leader_election.cc:304] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.242781 29115 raft_consensus.cc:2804] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:17.244920 29115 raft_consensus.cc:697] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.246232 29115 consensus_queue.cc:237] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.246737 29106 raft_consensus.cc:359] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.247506 29106 raft_consensus.cc:385] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.247783 29106 raft_consensus.cc:740] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.247644 29107 tablet_bootstrap.cc:654] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.248867 29106 consensus_queue.cc:260] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.251519 29106 ts_tablet_manager.cc:1434] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.006s
I20260709 12:28:17.259896 28813 catalog_manager.cc:5697] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.262213 29107 tablet_bootstrap.cc:492] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:17.262974 29107 ts_tablet_manager.cc:1403] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.016s
I20260709 12:28:17.265707 29107 raft_consensus.cc:359] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.266518 29107 raft_consensus.cc:385] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.267287 29107 raft_consensus.cc:740] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.268105 29107 consensus_queue.cc:260] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.270484 29107 ts_tablet_manager.cc:1434] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:17.271636 29107 tablet_bootstrap.cc:492] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:17.272588 29130 raft_consensus.cc:493] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.273231 29130 raft_consensus.cc:515] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.275646 29130 leader_election.cc:290] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.278437 29107 tablet_bootstrap.cc:654] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:17.278241 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:17.279429 28991 raft_consensus.cc:2468] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 0.
I20260709 12:28:17.278334 28813 catalog_manager.cc:5697] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.282609 29116 raft_consensus.cc:493] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.283193 29116 raft_consensus.cc:515] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.283667 29107 tablet_bootstrap.cc:492] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:17.284215 29107 ts_tablet_manager.cc:1403] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.001s
I20260709 12:28:17.287176 29107 raft_consensus.cc:359] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.287865 29107 raft_consensus.cc:385] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:17.288161 29107 raft_consensus.cc:740] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:17.288800 29107 consensus_queue.cc:260] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.290280 28882 leader_election.cc:304] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.291551 29107 ts_tablet_manager.cc:1434] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:28:17.278734 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:17.294023 29065 raft_consensus.cc:2468] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 0.
I20260709 12:28:17.295013 29130 raft_consensus.cc:2804] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:17.295610 29116 leader_election.cc:290] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:17.296378 29130 raft_consensus.cc:493] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.297096 29130 raft_consensus.cc:3060] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.309950 29130 raft_consensus.cc:515] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.310993 29112 raft_consensus.cc:493] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.311614 29112 raft_consensus.cc:515] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.313813 29112 leader_election.cc:290] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.315634 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:17.316542 28991 raft_consensus.cc:3060] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.316664 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.317408 29064 raft_consensus.cc:3060] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.320386 29130 leader_election.cc:290] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.320583 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:17.321148 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05682701dab0458fba871736c6f6a74a" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:17.321517 29065 raft_consensus.cc:2393] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 012f61c51f47429ca57af12350fe42f4 in current term 1: Already voted for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in this term.
I20260709 12:28:17.321923 28916 raft_consensus.cc:2468] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 0.
I20260709 12:28:17.323473 28990 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:17.323444 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05682701dab0458fba871736c6f6a74a" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:17.323999 29029 leader_election.cc:304] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:17.324405 28989 raft_consensus.cc:2468] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 0.
I20260709 12:28:17.325031 29116 raft_consensus.cc:2804] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:17.325441 29116 raft_consensus.cc:493] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.325788 29116 raft_consensus.cc:3060] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.326309 28882 leader_election.cc:304] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4; no voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:17.328547 29130 raft_consensus.cc:3060] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.330761 29064 raft_consensus.cc:2468] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:17.330761 28991 raft_consensus.cc:2468] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:17.332309 28882 leader_election.cc:304] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:17.333065 29112 raft_consensus.cc:2804] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:17.334816 29112 raft_consensus.cc:697] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:17.334867 29116 raft_consensus.cc:515] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.336763 29112 consensus_queue.cc:237] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.337756 29130 raft_consensus.cc:2749] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:17.338649 29116 leader_election.cc:290] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:17.339715 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05682701dab0458fba871736c6f6a74a" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.339844 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05682701dab0458fba871736c6f6a74a" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:17.340312 28916 raft_consensus.cc:3060] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.340674 28991 raft_consensus.cc:3060] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.349002 28991 raft_consensus.cc:2468] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 1.
I20260709 12:28:17.349094 28916 raft_consensus.cc:2468] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 1.
I20260709 12:28:17.348320 28814 catalog_manager.cc:5697] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.350847 29031 leader_election.cc:304] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.351716 29116 raft_consensus.cc:2804] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:17.352193 29116 raft_consensus.cc:697] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:17.353058 29116 consensus_queue.cc:237] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.362082 28814 catalog_manager.cc:5697] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.363961 29115 raft_consensus.cc:493] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.364583 29115 raft_consensus.cc:515] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.366559 29115 leader_election.cc:290] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.367544 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897444c2b13e4cb69911b6de6d34b6fe" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:17.368244 28916 raft_consensus.cc:2468] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:17.368402 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897444c2b13e4cb69911b6de6d34b6fe" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:17.369112 29064 raft_consensus.cc:2468] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:17.369288 28955 leader_election.cc:304] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.370440 29115 raft_consensus.cc:2804] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:17.370819 29115 raft_consensus.cc:493] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.371304 29115 raft_consensus.cc:3060] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.377864 29115 raft_consensus.cc:515] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.379881 29115 leader_election.cc:290] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.380725 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897444c2b13e4cb69911b6de6d34b6fe" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.380972 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "897444c2b13e4cb69911b6de6d34b6fe" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.381397 28916 raft_consensus.cc:3060] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.381608 29064 raft_consensus.cc:3060] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.387562 29064 raft_consensus.cc:2468] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.387650 28916 raft_consensus.cc:2468] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:17.388868 28957 leader_election.cc:304] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.389671 29115 raft_consensus.cc:2804] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:17.390137 29115 raft_consensus.cc:697] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.391206 29115 consensus_queue.cc:237] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.400143 28814 catalog_manager.cc:5697] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.491339 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:28:17.492127 28991 raft_consensus.cc:606] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:17.492717 29112 consensus_queue.cc:1048] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.503233 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:28:17.503968 28991 raft_consensus.cc:606] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:17.510013 29130 consensus_queue.cc:1048] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.520095 29130 consensus_queue.cc:1048] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.540958 29064 raft_consensus.cc:3060] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:17.547607 29065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05682701dab0458fba871736c6f6a74a"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
mode: GRACEFUL
new_leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
 from {username='slave'} at 127.0.0.1:34412
I20260709 12:28:17.547999 29114 consensus_queue.cc:1048] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.548432 29065 raft_consensus.cc:606] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Received request to transfer leadership to TS e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:17.552738 29159 consensus_queue.cc:1048] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.563386 29115 raft_consensus.cc:993] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:17.563985 29115 raft_consensus.cc:1081] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:17.566329 28915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:36762
I20260709 12:28:17.568209 28915 raft_consensus.cc:493] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.568789 28915 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.577558 29137 consensus_queue.cc:1048] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.579730 28915 raft_consensus.cc:515] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.580843 28916 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
mode: GRACEFUL
new_leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
 from {username='slave'} at 127.0.0.1:36778
I20260709 12:28:17.581583 28916 raft_consensus.cc:606] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Received request to transfer leadership to TS e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:17.588441 28915 leader_election.cc:290] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.590081 28989 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:28:17.590866 28989 raft_consensus.cc:606] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:17.591940 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:17.592721 28991 raft_consensus.cc:3055] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.593302 28991 raft_consensus.cc:740] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.594749 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.595849 29064 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.594450 28991 consensus_queue.cc:260] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.597119 28991 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.608634 28991 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:17.608632 29064 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:17.610467 28882 leader_election.cc:304] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.611367 29159 raft_consensus.cc:2804] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.611958 29159 raft_consensus.cc:697] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:17.612947 29159 consensus_queue.cc:237] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.621063 28814 catalog_manager.cc:5697] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.676616 29114 consensus_queue.cc:1048] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.680032 29114 consensus_queue.cc:1048] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.716396 29114 consensus_queue.cc:1048] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.724785 29137 raft_consensus.cc:993] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:17.725389 29115 raft_consensus.cc:1081] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:17.733251 29137 consensus_queue.cc:1048] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.733970 29064 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:34386
I20260709 12:28:17.737438 29064 raft_consensus.cc:493] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.737857 29064 raft_consensus.cc:3060] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.745842 29064 raft_consensus.cc:515] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.746170 29159 consensus_queue.cc:1048] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.748276 29064 leader_election.cc:290] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:17.749692 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:17.750496 28991 raft_consensus.cc:3055] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.751252 28991 raft_consensus.cc:740] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.758916 29137 consensus_queue.cc:1048] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.758435 28991 consensus_queue.cc:260] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.763751 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3575a1ff5a34e659f8921eb7bab27ab" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.764434 28916 raft_consensus.cc:3060] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.764554 28991 raft_consensus.cc:3060] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.767261 29112 consensus_queue.cc:1048] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.781756 29137 consensus_queue.cc:1048] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.785272 29114 raft_consensus.cc:993] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:17.789644 29114 raft_consensus.cc:1081] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:17.800101 29114 consensus_queue.cc:1048] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.786199 28916 raft_consensus.cc:2468] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:17.810163 28915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:36762
I20260709 12:28:17.811194 28915 raft_consensus.cc:493] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.811728 28915 raft_consensus.cc:3060] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.817972 29029 leader_election.cc:304] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:17.819730 29116 raft_consensus.cc:2804] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.820369 29116 raft_consensus.cc:697] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:17.821331 29116 consensus_queue.cc:237] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.824074 29115 consensus_queue.cc:1048] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.829064 29112 raft_consensus.cc:993] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: : Instructing follower e59dbf606f6c4bebafd56cbe0e7d4ece to start an election
I20260709 12:28:17.828908 28991 raft_consensus.cc:2468] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:17.829648 29112 raft_consensus.cc:1081] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Signalling peer e59dbf606f6c4bebafd56cbe0e7d4ece to start an election
I20260709 12:28:17.832212 28991 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
 from {username='slave'} at 127.0.0.1:38672
I20260709 12:28:17.833947 28991 raft_consensus.cc:493] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.834400 28991 raft_consensus.cc:3060] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.847285 28915 raft_consensus.cc:515] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.850687 28814 catalog_manager.cc:5697] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.853463 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:17.854285 28989 raft_consensus.cc:3055] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.854624 28989 raft_consensus.cc:740] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.854861 28915 leader_election.cc:290] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.855681 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d117af0c3f24c4a8fd44303277d126a" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.855573 28989 consensus_queue.cc:260] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.863230 28989 raft_consensus.cc:3060] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.866586 29065 raft_consensus.cc:3060] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.858163 28991 raft_consensus.cc:515] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.872350 28989 raft_consensus.cc:2468] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:17.877562 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.877893 29065 raft_consensus.cc:2468] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:17.878335 28916 raft_consensus.cc:3055] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.879431 28916 raft_consensus.cc:740] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:17.879498 28882 leader_election.cc:304] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:17.880535 28916 consensus_queue.cc:260] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.881484 29130 raft_consensus.cc:2804] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.882062 29130 raft_consensus.cc:697] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:17.883013 29130 consensus_queue.cc:237] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.893249 28916 raft_consensus.cc:3060] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.894136 29116 consensus_queue.cc:1048] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:17.893586 28810 catalog_manager.cc:5697] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.895009 28991 leader_election.cc:290] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.895547 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "368b8c8ea73c4ec8a51ff11fb9598522" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.896194 29065 raft_consensus.cc:3060] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.905131 29065 raft_consensus.cc:2468] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 2.
I20260709 12:28:17.906348 28957 leader_election.cc:304] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.907606 29181 raft_consensus.cc:2804] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.906210 28916 raft_consensus.cc:2468] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 2.
I20260709 12:28:17.908358 29181 raft_consensus.cc:697] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.909741 29181 consensus_queue.cc:237] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.917778 29116 consensus_queue.cc:1048] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:17.925251 28812 catalog_manager.cc:5697] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 1 to 2, leader changed from 012f61c51f47429ca57af12350fe42f4 (127.12.45.65) to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 2 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.929823 29158 raft_consensus.cc:993] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: : Instructing follower e59dbf606f6c4bebafd56cbe0e7d4ece to start an election
I20260709 12:28:17.930437 29116 raft_consensus.cc:1081] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Signalling peer e59dbf606f6c4bebafd56cbe0e7d4ece to start an election
I20260709 12:28:17.934787 28991 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05682701dab0458fba871736c6f6a74a"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
 from {username='slave'} at 127.0.0.1:38676
I20260709 12:28:17.935470 28991 raft_consensus.cc:493] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:17.935873 28991 raft_consensus.cc:3060] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.943557 28991 raft_consensus.cc:515] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.945753 28991 leader_election.cc:290] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:17.946839 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05682701dab0458fba871736c6f6a74a" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:17.947008 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05682701dab0458fba871736c6f6a74a" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:17.947726 28915 raft_consensus.cc:3060] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.947794 29065 raft_consensus.cc:3055] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:17.948273 29065 raft_consensus.cc:740] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:17.949195 29065 consensus_queue.cc:260] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.950748 29065 raft_consensus.cc:3060] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:17.954825 28915 raft_consensus.cc:2468] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 2.
I20260709 12:28:17.956269 28955 leader_election.cc:304] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:17.957296 29065 raft_consensus.cc:2468] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 2.
I20260709 12:28:17.957285 29181 raft_consensus.cc:2804] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:17.958108 29181 raft_consensus.cc:697] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:17.958974 29181 consensus_queue.cc:237] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:17.966359 28812 catalog_manager.cc:5697] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 1 to 2, leader changed from 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67) to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 2 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:18.036582 29065 raft_consensus.cc:1275] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.038218 29112 consensus_queue.cc:1048] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.047783 28991 raft_consensus.cc:1275] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.050300 29159 consensus_queue.cc:1048] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.322117 29065 raft_consensus.cc:1275] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer e59dbf606f6c4bebafd56cbe0e7d4ece: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.323673 29181 consensus_queue.cc:1048] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.328753 28991 raft_consensus.cc:1275] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.330904 28915 raft_consensus.cc:1275] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer e59dbf606f6c4bebafd56cbe0e7d4ece: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.332739 29114 consensus_queue.cc:1048] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.333393 29159 consensus_queue.cc:1048] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:18.343961 28991 raft_consensus.cc:1275] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.346946 29116 consensus_queue.cc:1048] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.352005 29065 raft_consensus.cc:1275] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.357846 29190 consensus_queue.cc:1048] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:18.360280 28915 raft_consensus.cc:1275] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.367313 29116 consensus_queue.cc:1048] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.417026 29065 raft_consensus.cc:1275] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer e59dbf606f6c4bebafd56cbe0e7d4ece: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.418684 29114 consensus_queue.cc:1048] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:18.429229 28915 raft_consensus.cc:1275] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer e59dbf606f6c4bebafd56cbe0e7d4ece: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:18.431821 29181 consensus_queue.cc:1048] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:20.628638 28812 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60896:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:20.631057 28812 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:20.689967 28901 tablet_service.cc:1511] Processing CreateTablet for tablet 59b9ca5890804906b936ecc5c501a374 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:20.690982 28973 tablet_service.cc:1511] Processing CreateTablet for tablet 59b9ca5890804906b936ecc5c501a374 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:20.691725 28901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b9ca5890804906b936ecc5c501a374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.692430 28973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b9ca5890804906b936ecc5c501a374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.692677 29048 tablet_service.cc:1511] Processing CreateTablet for tablet 59b9ca5890804906b936ecc5c501a374 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:20.694047 29048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b9ca5890804906b936ecc5c501a374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.696180 28898 tablet_service.cc:1511] Processing CreateTablet for tablet b10d048fcf7d45bb92c4ccea6da3a342 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:20.697599 28898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10d048fcf7d45bb92c4ccea6da3a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.699656 29055 tablet_service.cc:1511] Processing CreateTablet for tablet b10d048fcf7d45bb92c4ccea6da3a342 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:20.699934 29051 tablet_service.cc:1511] Processing CreateTablet for tablet 0635e538e32f4cd98e440ff04cf0fd55 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:20.701310 29055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10d048fcf7d45bb92c4ccea6da3a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.701678 28977 tablet_service.cc:1511] Processing CreateTablet for tablet 37436ff925f944399a08784cf485f0a3 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:20.703248 28977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37436ff925f944399a08784cf485f0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.703420 28975 tablet_service.cc:1511] Processing CreateTablet for tablet 0635e538e32f4cd98e440ff04cf0fd55 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:20.704718 28975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0635e538e32f4cd98e440ff04cf0fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.701787 28978 tablet_service.cc:1511] Processing CreateTablet for tablet b10d048fcf7d45bb92c4ccea6da3a342 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:20.709141 28978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10d048fcf7d45bb92c4ccea6da3a342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.708973 28899 tablet_service.cc:1511] Processing CreateTablet for tablet 0635e538e32f4cd98e440ff04cf0fd55 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:20.710536 28899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0635e538e32f4cd98e440ff04cf0fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.709161 28900 tablet_service.cc:1511] Processing CreateTablet for tablet 37436ff925f944399a08784cf485f0a3 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:20.711755 28900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37436ff925f944399a08784cf485f0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.715016 29051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0635e538e32f4cd98e440ff04cf0fd55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.720991 29052 tablet_service.cc:1511] Processing CreateTablet for tablet 37436ff925f944399a08784cf485f0a3 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:20.723511 29052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37436ff925f944399a08784cf485f0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.723909 28976 tablet_service.cc:1511] Processing CreateTablet for tablet 7bcbc267390d464ab001c38efea39c00 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:20.725498 28976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bcbc267390d464ab001c38efea39c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.727469 28906 tablet_service.cc:1511] Processing CreateTablet for tablet 7bcbc267390d464ab001c38efea39c00 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:20.728307 28903 tablet_service.cc:1511] Processing CreateTablet for tablet 3f676b06f1c74c848307d5a41962ce5e (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:20.729035 28906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bcbc267390d464ab001c38efea39c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.733417 28905 tablet_service.cc:1511] Processing CreateTablet for tablet f8e9159ea91642abb4fcb73bb9f79f93 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:20.733855 28904 tablet_service.cc:1511] Processing CreateTablet for tablet 393b2cf0f8e7449da9cde08f89185150 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:20.735320 29053 tablet_service.cc:1511] Processing CreateTablet for tablet f8e9159ea91642abb4fcb73bb9f79f93 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:20.739763 28905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8e9159ea91642abb4fcb73bb9f79f93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.739969 29053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8e9159ea91642abb4fcb73bb9f79f93. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:20.740355 28862 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:20.735044 29047 tablet_service.cc:1511] Processing CreateTablet for tablet 7bcbc267390d464ab001c38efea39c00 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:20.742192 29047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bcbc267390d464ab001c38efea39c00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.744181 28904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 393b2cf0f8e7449da9cde08f89185150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.738332 28981 tablet_service.cc:1511] Processing CreateTablet for tablet 3f676b06f1c74c848307d5a41962ce5e (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:20.734117 29049 tablet_service.cc:1511] Processing CreateTablet for tablet 3f676b06f1c74c848307d5a41962ce5e (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:20.745383 28981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f676b06f1c74c848307d5a41962ce5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.745898 29049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f676b06f1c74c848307d5a41962ce5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.737605 28979 tablet_service.cc:1511] Processing CreateTablet for tablet f8e9159ea91642abb4fcb73bb9f79f93 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:20.747557 28979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8e9159ea91642abb4fcb73bb9f79f93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.750743 28903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f676b06f1c74c848307d5a41962ce5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.768602 28980 tablet_service.cc:1511] Processing CreateTablet for tablet 393b2cf0f8e7449da9cde08f89185150 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:20.770316 28980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 393b2cf0f8e7449da9cde08f89185150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.776752 29204 tablet_bootstrap.cc:492] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:20.779116 29050 tablet_service.cc:1511] Processing CreateTablet for tablet 393b2cf0f8e7449da9cde08f89185150 (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:20.780799 29050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 393b2cf0f8e7449da9cde08f89185150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.782299 28902 tablet_service.cc:1511] Processing CreateTablet for tablet 910c29c25fbd4ba1a51b3aeef4f79c0d (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:20.783761 28902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910c29c25fbd4ba1a51b3aeef4f79c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.787842 29054 tablet_service.cc:1511] Processing CreateTablet for tablet 910c29c25fbd4ba1a51b3aeef4f79c0d (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:20.787974 28977 tablet_service.cc:1511] Processing CreateTablet for tablet 910c29c25fbd4ba1a51b3aeef4f79c0d (DEFAULT_TABLE table=soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:20.789546 29054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910c29c25fbd4ba1a51b3aeef4f79c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.790496 28977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 910c29c25fbd4ba1a51b3aeef4f79c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.857771 29204 tablet_bootstrap.cc:654] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:20.874475 28882 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:46859, user_credentials={real_user=slave}} blocked reactor thread for 111501us
I20260709 12:28:20.904209 29205 tablet_bootstrap.cc:492] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:20.916810 29206 tablet_bootstrap.cc:492] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:20.926091 29206 tablet_bootstrap.cc:654] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:20.936254 29204 tablet_bootstrap.cc:492] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:20.936849 29204 ts_tablet_manager.cc:1403] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.160s	user 0.018s	sys 0.000s
I20260709 12:28:20.938916 29205 tablet_bootstrap.cc:654] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:20.939983 29204 raft_consensus.cc:359] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.940689 29204 raft_consensus.cc:385] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:20.940953 29204 raft_consensus.cc:740] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:20.941669 29204 consensus_queue.cc:260] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.947767 29204 ts_tablet_manager.cc:1434] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 12:28:20.948956 29204 tablet_bootstrap.cc:492] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:20.953315 29205 tablet_bootstrap.cc:492] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:20.953951 29205 ts_tablet_manager.cc:1403] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.000s
I20260709 12:28:20.956980 29204 tablet_bootstrap.cc:654] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:20.960309 29205 raft_consensus.cc:359] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.961202 29205 raft_consensus.cc:385] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:20.961514 29205 raft_consensus.cc:740] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:20.962355 29205 consensus_queue.cc:260] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.967088 29206 tablet_bootstrap.cc:492] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:20.968070 29205 ts_tablet_manager.cc:1434] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:28:20.968292 29206 ts_tablet_manager.cc:1403] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.007s
I20260709 12:28:20.969281 29205 tablet_bootstrap.cc:492] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:20.971195 29204 tablet_bootstrap.cc:492] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:20.971797 29204 ts_tablet_manager.cc:1403] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.009s
I20260709 12:28:20.972818 29206 raft_consensus.cc:359] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.973699 29206 raft_consensus.cc:385] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:20.974082 29206 raft_consensus.cc:740] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:20.975126 29206 consensus_queue.cc:260] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.975423 29204 raft_consensus.cc:359] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.976178 29204 raft_consensus.cc:385] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:20.976507 29204 raft_consensus.cc:740] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:20.977335 29204 consensus_queue.cc:260] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:20.978075 29205 tablet_bootstrap.cc:654] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:20.978289 29206 ts_tablet_manager.cc:1434] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:28:20.979668 29204 ts_tablet_manager.cc:1434] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:28:20.980754 29204 tablet_bootstrap.cc:492] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:20.988536 29204 tablet_bootstrap.cc:654] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:20.988955 29206 tablet_bootstrap.cc:492] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:20.995954 29205 tablet_bootstrap.cc:492] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:20.996901 29205 ts_tablet_manager.cc:1403] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.004s
I20260709 12:28:20.998862 29206 tablet_bootstrap.cc:654] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.000727 29205 raft_consensus.cc:359] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.001645 29205 raft_consensus.cc:385] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.001966 29205 raft_consensus.cc:740] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.007058 29205 consensus_queue.cc:260] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.008204 29204 tablet_bootstrap.cc:492] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.008850 29204 ts_tablet_manager.cc:1403] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.001s
I20260709 12:28:21.010506 29205 ts_tablet_manager.cc:1434] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:28:21.011718 29205 tablet_bootstrap.cc:492] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.012516 29204 raft_consensus.cc:359] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.013763 29206 tablet_bootstrap.cc:492] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.018024 29204 raft_consensus.cc:385] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.018388 29206 ts_tablet_manager.cc:1403] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.009s
I20260709 12:28:21.018441 29204 raft_consensus.cc:740] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.021495 29204 consensus_queue.cc:260] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.021947 29206 raft_consensus.cc:359] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.022991 29206 raft_consensus.cc:385] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.023387 29206 raft_consensus.cc:740] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.024274 29206 consensus_queue.cc:260] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.025018 29204 ts_tablet_manager.cc:1434] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 12:28:21.026321 29204 tablet_bootstrap.cc:492] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:21.027022 29206 ts_tablet_manager.cc:1434] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:28:21.027066 29205 tablet_bootstrap.cc:654] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.028592 29206 tablet_bootstrap.cc:492] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.033867 29204 tablet_bootstrap.cc:654] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.041523 29206 tablet_bootstrap.cc:654] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.054713 29203 raft_consensus.cc:493] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.055327 29203 raft_consensus.cc:515] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.056273 29205 tablet_bootstrap.cc:492] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.056988 29205 ts_tablet_manager.cc:1403] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.046s	user 0.023s	sys 0.014s
I20260709 12:28:21.057821 29116 raft_consensus.cc:493] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.058452 29116 raft_consensus.cc:515] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.060775 29116 leader_election.cc:290] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:21.061045 29205 raft_consensus.cc:359] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.061606 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.061988 29205 raft_consensus.cc:385] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.062395 29205 raft_consensus.cc:740] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.063381 29205 consensus_queue.cc:260] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.063913 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.064345 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
W20260709 12:28:21.066129 28882 leader_election.cc:343] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.060606 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
W20260709 12:28:21.066344 29029 leader_election.cc:343] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:21.067981 29031 leader_election.cc:343] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.068459 29031 leader_election.cc:304] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09; no voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:21.069758 29116 raft_consensus.cc:2749] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:21.070462 29205 ts_tablet_manager.cc:1434] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
W20260709 12:28:21.070786 28882 leader_election.cc:343] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.071310 28882 leader_election.cc:304] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4; no voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:21.072151 29190 raft_consensus.cc:2749] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:21.057979 29203 leader_election.cc:290] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.072968 29205 tablet_bootstrap.cc:492] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.078568 29204 tablet_bootstrap.cc:492] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.079239 29204 ts_tablet_manager.cc:1403] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.053s	user 0.038s	sys 0.004s
I20260709 12:28:21.080761 29206 tablet_bootstrap.cc:492] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.081362 29206 ts_tablet_manager.cc:1403] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.004s
I20260709 12:28:21.082964 29204 raft_consensus.cc:359] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.083904 29204 raft_consensus.cc:385] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.084268 29204 raft_consensus.cc:740] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.084647 29206 raft_consensus.cc:359] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.085484 29206 raft_consensus.cc:385] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.085106 29204 consensus_queue.cc:260] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.085839 29206 raft_consensus.cc:740] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.086903 29192 raft_consensus.cc:493] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.087000 29206 consensus_queue.cc:260] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.087426 29192 raft_consensus.cc:515] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.088270 29205 tablet_bootstrap.cc:654] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.091351 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.092720 29206 ts_tablet_manager.cc:1434] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:28:21.093094 29204 ts_tablet_manager.cc:1434] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
W20260709 12:28:21.093334 28955 leader_election.cc:343] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.093870 29206 tablet_bootstrap.cc:492] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.094395 29204 tablet_bootstrap.cc:492] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:21.094439 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
W20260709 12:28:21.097754 28957 leader_election.cc:343] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.098330 28957 leader_election.cc:304] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:21.099422 29192 leader_election.cc:290] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.101027 29192 raft_consensus.cc:2749] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:21.103292 29206 tablet_bootstrap.cc:654] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.106308 29204 tablet_bootstrap.cc:654] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.115134 29205 tablet_bootstrap.cc:492] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.115782 29205 ts_tablet_manager.cc:1403] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.044s	user 0.006s	sys 0.017s
I20260709 12:28:21.120313 29205 raft_consensus.cc:359] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.121460 29205 raft_consensus.cc:385] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.121824 29205 raft_consensus.cc:740] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.123039 29205 consensus_queue.cc:260] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.132467 29205 ts_tablet_manager.cc:1434] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.002s
I20260709 12:28:21.133824 29205 tablet_bootstrap.cc:492] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.145622 29205 tablet_bootstrap.cc:654] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:21.148927 28957 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:44819, user_credentials={real_user=slave}} blocked reactor thread for 51209.6us
I20260709 12:28:21.155457 29204 tablet_bootstrap.cc:492] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.155879 29206 tablet_bootstrap.cc:492] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.156422 29204 ts_tablet_manager.cc:1403] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.026s
I20260709 12:28:21.156510 29206 ts_tablet_manager.cc:1403] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.063s	user 0.033s	sys 0.025s
I20260709 12:28:21.159430 29205 tablet_bootstrap.cc:492] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.160144 29205 ts_tablet_manager.cc:1403] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.010s
I20260709 12:28:21.160198 29204 raft_consensus.cc:359] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.161170 29206 raft_consensus.cc:359] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.161182 29204 raft_consensus.cc:385] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.162031 29206 raft_consensus.cc:385] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.162212 29204 raft_consensus.cc:740] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.162463 29206 raft_consensus.cc:740] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.163336 29206 consensus_queue.cc:260] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.163236 29204 consensus_queue.cc:260] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.164273 29205 raft_consensus.cc:359] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.168124 29205 raft_consensus.cc:385] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.168417 29205 raft_consensus.cc:740] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.169227 29205 consensus_queue.cc:260] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.165926 29206 ts_tablet_manager.cc:1434] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 12:28:21.166009 29204 ts_tablet_manager.cc:1434] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:28:21.174892 29204 tablet_bootstrap.cc:492] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:21.175429 29205 ts_tablet_manager.cc:1434] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:28:21.176632 29205 tablet_bootstrap.cc:492] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.173573 29206 tablet_bootstrap.cc:492] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.184882 29206 tablet_bootstrap.cc:654] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.189026 29205 tablet_bootstrap.cc:654] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.193899 29204 tablet_bootstrap.cc:654] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.195782 29205 tablet_bootstrap.cc:492] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.196491 29205 ts_tablet_manager.cc:1403] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.006s
I20260709 12:28:21.199590 29206 tablet_bootstrap.cc:492] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.200279 29206 ts_tablet_manager.cc:1403] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.009s
I20260709 12:28:21.200114 29205 raft_consensus.cc:359] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.200961 29205 raft_consensus.cc:385] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.201318 29205 raft_consensus.cc:740] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.201574 29204 tablet_bootstrap.cc:492] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.202132 29204 ts_tablet_manager.cc:1403] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.002s
I20260709 12:28:21.202139 29205 consensus_queue.cc:260] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.203858 29206 raft_consensus.cc:359] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.204659 29206 raft_consensus.cc:385] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.204965 29206 raft_consensus.cc:740] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.205150 29205 ts_tablet_manager.cc:1434] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:28:21.205713 29204 raft_consensus.cc:359] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.206390 29205 tablet_bootstrap.cc:492] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.205790 29206 consensus_queue.cc:260] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.206636 29204 raft_consensus.cc:385] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.209916 29204 raft_consensus.cc:740] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.210417 29203 raft_consensus.cc:493] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.210937 29203 raft_consensus.cc:515] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.210901 29204 consensus_queue.cc:260] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.213332 29203 leader_election.cc:290] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.208665 29206 ts_tablet_manager.cc:1434] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:28:21.215171 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:21.215914 28991 raft_consensus.cc:2468] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 0.
I20260709 12:28:21.216455 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.217267 28882 leader_election.cc:304] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.214882 29204 ts_tablet_manager.cc:1434] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
W20260709 12:28:21.218310 28882 leader_election.cc:343] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.219239 29204 tablet_bootstrap.cc:492] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:21.222210 29203 raft_consensus.cc:2804] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.222698 29203 raft_consensus.cc:493] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.223042 29203 raft_consensus.cc:3060] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.223501 29206 tablet_bootstrap.cc:492] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.227407 29205 tablet_bootstrap.cc:654] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.232409 29206 tablet_bootstrap.cc:654] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.236627 29204 tablet_bootstrap.cc:654] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.241693 29205 tablet_bootstrap.cc:492] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.244573 29206 tablet_bootstrap.cc:492] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.245225 29206 ts_tablet_manager.cc:1403] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.001s
I20260709 12:28:21.242396 29205 ts_tablet_manager.cc:1403] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.011s
I20260709 12:28:21.247771 29203 raft_consensus.cc:515] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.248765 29206 raft_consensus.cc:359] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.249614 29206 raft_consensus.cc:385] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.249303 29205 raft_consensus.cc:359] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.249951 29206 raft_consensus.cc:740] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.250100 29205 raft_consensus.cc:385] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.250679 29205 raft_consensus.cc:740] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.251042 29206 consensus_queue.cc:260] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.251464 29205 consensus_queue.cc:260] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.252563 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:21.253376 28991 raft_consensus.cc:3060] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.253885 29206 ts_tablet_manager.cc:1434] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:28:21.254104 29205 ts_tablet_manager.cc:1434] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.003s
I20260709 12:28:21.255239 29206 tablet_bootstrap.cc:492] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.260864 29205 tablet_bootstrap.cc:492] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.262811 29206 tablet_bootstrap.cc:654] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.263564 28991 raft_consensus.cc:2468] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:21.265048 28882 leader_election.cc:304] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.266036 29190 raft_consensus.cc:2804] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.279215 29203 leader_election.cc:290] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.280468 29218 raft_consensus.cc:493] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.280794 29205 tablet_bootstrap.cc:654] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.281064 29218 raft_consensus.cc:515] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.283358 29218 leader_election.cc:290] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.284265 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.285040 28916 raft_consensus.cc:2468] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.285207 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.285998 29064 raft_consensus.cc:2468] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.287065 28955 leader_election.cc:304] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.287986 29218 raft_consensus.cc:2804] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.288318 29205 tablet_bootstrap.cc:492] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.288405 29218 raft_consensus.cc:493] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.288318 29206 tablet_bootstrap.cc:492] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.289186 29205 ts_tablet_manager.cc:1403] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.005s
I20260709 12:28:21.289776 29206 ts_tablet_manager.cc:1403] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.018s
I20260709 12:28:21.289143 29218 raft_consensus.cc:3060] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.290344 29204 tablet_bootstrap.cc:492] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.290971 29204 ts_tablet_manager.cc:1403] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.010s
I20260709 12:28:21.293260 29205 raft_consensus.cc:359] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.294178 29205 raft_consensus.cc:385] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.294528 29205 raft_consensus.cc:740] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.294559 29204 raft_consensus.cc:359] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.295557 29204 raft_consensus.cc:385] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.296028 29204 raft_consensus.cc:740] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.295593 29205 consensus_queue.cc:260] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.297299 29116 raft_consensus.cc:493] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.293804 29206 raft_consensus.cc:359] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.298218 29206 raft_consensus.cc:385] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.297942 29116 raft_consensus.cc:515] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.298602 29206 raft_consensus.cc:740] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.298974 29205 ts_tablet_manager.cc:1434] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:28:21.300340 29205 tablet_bootstrap.cc:492] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: Bootstrap starting.
I20260709 12:28:21.301600 29218 raft_consensus.cc:515] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.301987 29206 consensus_queue.cc:260] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.302888 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.303689 29116 leader_election.cc:290] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:21.305225 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.305845 28915 raft_consensus.cc:3060] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.308512 29205 tablet_bootstrap.cc:654] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.309619 29190 raft_consensus.cc:697] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:21.312036 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.312740 29064 raft_consensus.cc:3060] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.310817 29190 consensus_queue.cc:237] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.318130 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.318935 29065 raft_consensus.cc:3060] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.304490 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:21.322043 28991 raft_consensus.cc:2393] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09474a5ec3194a23a04d99c316b86f09 in current term 1: Already voted for candidate 012f61c51f47429ca57af12350fe42f4 in this term.
I20260709 12:28:21.327631 29218 leader_election.cc:290] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.330081 29064 raft_consensus.cc:2468] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:21.330811 28915 raft_consensus.cc:2468] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.296885 29204 consensus_queue.cc:260] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.332274 28955 leader_election.cc:304] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.333545 29218 raft_consensus.cc:2804] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.334065 29204 ts_tablet_manager.cc:1434] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:28:21.334095 29218 raft_consensus.cc:697] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:21.335330 29218 consensus_queue.cc:237] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.329540 29065 raft_consensus.cc:2468] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.335335 29204 tablet_bootstrap.cc:492] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:21.344506 29029 leader_election.cc:304] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09; no voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:21.346014 29116 raft_consensus.cc:2749] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:21.346892 28814 catalog_manager.cc:5697] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.350065 29206 ts_tablet_manager.cc:1434] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.002s
I20260709 12:28:21.356297 29204 tablet_bootstrap.cc:654] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.373044 29205 tablet_bootstrap.cc:492] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: No bootstrap required, opened a new log
I20260709 12:28:21.374322 29205 ts_tablet_manager.cc:1403] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: Time spent bootstrapping tablet: real 0.074s	user 0.022s	sys 0.020s
I20260709 12:28:21.375614 29192 raft_consensus.cc:493] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.376463 29192 raft_consensus.cc:515] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.378898 29205 raft_consensus.cc:359] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.380013 29205 raft_consensus.cc:385] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.380393 29205 raft_consensus.cc:740] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.367635 28812 catalog_manager.cc:5697] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.367635 29218 raft_consensus.cc:493] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.393179 29192 leader_election.cc:290] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.394855 29206 tablet_bootstrap.cc:492] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.395479 29205 consensus_queue.cc:260] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.395653 29218 raft_consensus.cc:515] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.399272 29205 ts_tablet_manager.cc:1434] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.003s
I20260709 12:28:21.399484 29239 raft_consensus.cc:493] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.400570 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcbc267390d464ab001c38efea39c00" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.401399 29064 raft_consensus.cc:2468] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.400593 29239 raft_consensus.cc:515] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.403605 29239 leader_election.cc:290] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.403518 28957 leader_election.cc:304] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.404562 29239 raft_consensus.cc:2804] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.404991 29239 raft_consensus.cc:493] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.405359 29239 raft_consensus.cc:3060] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.405597 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f676b06f1c74c848307d5a41962ce5e" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.405974 29204 tablet_bootstrap.cc:492] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.406590 29204 ts_tablet_manager.cc:1403] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.071s	user 0.022s	sys 0.023s
W20260709 12:28:21.407351 28957 leader_election.cc:343] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.409986 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcbc267390d464ab001c38efea39c00" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.410800 28916 raft_consensus.cc:2468] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.412770 29218 leader_election.cc:290] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.414671 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.415458 29064 raft_consensus.cc:2468] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.416213 29204 raft_consensus.cc:359] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.417142 29190 raft_consensus.cc:493] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.417896 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f676b06f1c74c848307d5a41962ce5e" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.418637 28916 raft_consensus.cc:2468] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.419281 29239 raft_consensus.cc:515] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.417716 29190 raft_consensus.cc:515] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.422360 29239 leader_election.cc:290] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.417222 29204 raft_consensus.cc:385] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.423331 29204 raft_consensus.cc:740] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.423880 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.424631 29065 raft_consensus.cc:2468] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 0.
I20260709 12:28:21.424507 29204 consensus_queue.cc:260] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.426178 28882 leader_election.cc:304] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:21.427105 29190 leader_election.cc:290] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.427138 29229 raft_consensus.cc:2804] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.416256 29206 tablet_bootstrap.cc:654] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.427743 29229 raft_consensus.cc:493] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.428305 29229 raft_consensus.cc:3060] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.428684 29204 ts_tablet_manager.cc:1434] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 12:28:21.429886 29204 tablet_bootstrap.cc:492] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: Bootstrap starting.
I20260709 12:28:21.435205 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.436663 28957 leader_election.cc:304] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.442579 29206 tablet_bootstrap.cc:492] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.443264 29206 ts_tablet_manager.cc:1403] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.002s
I20260709 12:28:21.443596 28955 leader_election.cc:304] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:21.444898 29239 raft_consensus.cc:2804] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.445523 29239 raft_consensus.cc:493] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.445636 29192 raft_consensus.cc:2804] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.446038 29239 raft_consensus.cc:3060] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.446168 29192 raft_consensus.cc:493] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.446835 29192 raft_consensus.cc:3060] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.448963 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcbc267390d464ab001c38efea39c00" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.449666 28915 raft_consensus.cc:3060] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.453599 29229 raft_consensus.cc:515] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.454893 29238 raft_consensus.cc:493] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.455137 28916 raft_consensus.cc:2393] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e59dbf606f6c4bebafd56cbe0e7d4ece in current term 1: Already voted for candidate 012f61c51f47429ca57af12350fe42f4 in this term.
I20260709 12:28:21.455552 29238 raft_consensus.cc:515] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.457445 29229 leader_election.cc:290] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.460994 28915 raft_consensus.cc:2468] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.459319 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.461851 29065 raft_consensus.cc:3060] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.462337 29064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bcbc267390d464ab001c38efea39c00" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.462990 29064 raft_consensus.cc:3060] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.464620 29218 raft_consensus.cc:493] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.467176 29232 raft_consensus.cc:493] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.460994 29192 raft_consensus.cc:515] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.467685 29232 raft_consensus.cc:515] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.469388 29192 leader_election.cc:290] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.470505 29206 raft_consensus.cc:359] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.466701 29218 raft_consensus.cc:515] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.471359 29206 raft_consensus.cc:385] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.471683 29206 raft_consensus.cc:740] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.472432 29206 consensus_queue.cc:260] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.473093 29218 leader_election.cc:290] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.470028 29232 leader_election.cc:290] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:21.461334 29238 leader_election.cc:290] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.473603 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.474692 28916 raft_consensus.cc:2468] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 0.
I20260709 12:28:21.472474 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:21.482409 28991 raft_consensus.cc:2468] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 0.
I20260709 12:28:21.483311 29206 ts_tablet_manager.cc:1434] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:28:21.482874 29065 raft_consensus.cc:2468] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 1.
I20260709 12:28:21.476483 29029 leader_election.cc:304] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:21.484524 29206 tablet_bootstrap.cc:492] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: Bootstrap starting.
I20260709 12:28:21.486053 29204 tablet_bootstrap.cc:654] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:21.486655 29232 raft_consensus.cc:2804] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.487378 29232 raft_consensus.cc:493] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.487756 29232 raft_consensus.cc:3060] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.489956 28882 leader_election.cc:304] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:21.491766 29229 raft_consensus.cc:2804] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.492401 29229 raft_consensus.cc:697] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:21.492450 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:21.495105 28989 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:21.498817 29063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f676b06f1c74c848307d5a41962ce5e" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.495167 29229 consensus_queue.cc:237] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.492707 29064 raft_consensus.cc:2468] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.500505 29062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.501302 29062 raft_consensus.cc:2468] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.502086 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.502821 28916 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.503129 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.503880 29065 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.500547 29206 tablet_bootstrap.cc:654] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:21.506801 28957 leader_election.cc:343] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:21.511979 29239 raft_consensus.cc:515] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.514211 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.514461 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f676b06f1c74c848307d5a41962ce5e" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.526257 29204 tablet_bootstrap.cc:492] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: No bootstrap required, opened a new log
I20260709 12:28:21.517712 28957 leader_election.cc:304] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.527667 28991 raft_consensus.cc:2393] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 012f61c51f47429ca57af12350fe42f4 in current term 1: Already voted for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in this term.
I20260709 12:28:21.527765 29204 ts_tablet_manager.cc:1403] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent bootstrapping tablet: real 0.098s	user 0.014s	sys 0.000s
I20260709 12:28:21.527889 28915 raft_consensus.cc:2468] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.528374 29218 raft_consensus.cc:2804] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.527274 28916 raft_consensus.cc:3060] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.529042 29218 raft_consensus.cc:697] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:21.522536 29232 raft_consensus.cc:515] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.530524 29218 consensus_queue.cc:237] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.536365 28955 leader_election.cc:304] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.537842 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:21.538548 28991 raft_consensus.cc:3060] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.538632 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.539314 28915 raft_consensus.cc:3060] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.540366 29232 leader_election.cc:290] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:21.548638 28991 raft_consensus.cc:2468] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 1.
I20260709 12:28:21.550225 29031 leader_election.cc:304] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.531550 29204 raft_consensus.cc:359] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.532533 29239 leader_election.cc:290] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.551440 29204 raft_consensus.cc:385] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.542793 29206 tablet_bootstrap.cc:492] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: No bootstrap required, opened a new log
I20260709 12:28:21.551851 29204 raft_consensus.cc:740] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.551034 28916 raft_consensus.cc:2468] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.552512 29206 ts_tablet_manager.cc:1403] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.003s
I20260709 12:28:21.552640 29204 consensus_queue.cc:260] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.531651 28957 leader_election.cc:304] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.556733 29204 ts_tablet_manager.cc:1434] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:28:21.556919 29239 raft_consensus.cc:2804] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.557467 29238 raft_consensus.cc:2804] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.558559 29238 raft_consensus.cc:493] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.558497 29239 raft_consensus.cc:493] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.559180 29238 raft_consensus.cc:3060] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.551241 29232 raft_consensus.cc:2804] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.559326 29239 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.559926 29232 raft_consensus.cc:697] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:21.557834 28955 leader_election.cc:304] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:21.560926 29232 consensus_queue.cc:237] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.562036 29192 raft_consensus.cc:2804] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.563755 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.564713 29065 raft_consensus.cc:2393] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e59dbf606f6c4bebafd56cbe0e7d4ece in current term 1: Already voted for candidate 012f61c51f47429ca57af12350fe42f4 in this term.
I20260709 12:28:21.565866 28914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d048fcf7d45bb92c4ccea6da3a342" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.558830 29206 raft_consensus.cc:359] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.566916 29206 raft_consensus.cc:385] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:21.567241 29206 raft_consensus.cc:740] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Initialized, Role: FOLLOWER
I20260709 12:28:21.568018 29206 consensus_queue.cc:260] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.572219 29206 ts_tablet_manager.cc:1434] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:28:21.573302 28915 raft_consensus.cc:2468] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 1.
I20260709 12:28:21.600672 28957 leader_election.cc:304] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:21.600086 28814 catalog_manager.cc:5697] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 1 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.602425 28812 catalog_manager.cc:5697] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:21.600261 29192 raft_consensus.cc:697] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:21.608494 29192 consensus_queue.cc:237] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.612160 29239 raft_consensus.cc:515] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.613060 29218 raft_consensus.cc:2749] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:21.624768 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.626884 29239 leader_election.cc:290] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.627391 28915 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.622673 29238 raft_consensus.cc:515] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.624234 28813 catalog_manager.cc:5697] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.626407 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.630683 29238 leader_election.cc:290] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.631006 29065 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.631721 29062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.632282 29062 raft_consensus.cc:3060] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.634793 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.635504 28916 raft_consensus.cc:3060] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.639140 29065 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.640275 29062 raft_consensus.cc:2468] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.640578 28957 leader_election.cc:304] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.641505 29238 raft_consensus.cc:2804] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.642016 28957 leader_election.cc:304] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.641976 29238 raft_consensus.cc:697] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:21.642999 29239 raft_consensus.cc:2804] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.643613 29239 raft_consensus.cc:697] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:21.644467 29239 consensus_queue.cc:237] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.643286 29238 consensus_queue.cc:237] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.650218 28915 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.652526 28916 raft_consensus.cc:2468] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.659782 28813 catalog_manager.cc:5697] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.661094 28813 catalog_manager.cc:5697] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.662180 28813 catalog_manager.cc:5697] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.720305 29232 raft_consensus.cc:493] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.721073 29232 raft_consensus.cc:515] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.724053 29232 leader_election.cc:290] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:21.724665 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f676b06f1c74c848307d5a41962ce5e" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" is_pre_election: true
I20260709 12:28:21.724635 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f676b06f1c74c848307d5a41962ce5e" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.725804 28916 raft_consensus.cc:2393] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09474a5ec3194a23a04d99c316b86f09 in current term 1: Already voted for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in this term.
I20260709 12:28:21.726819 29029 leader_election.cc:304] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09; no voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:21.727614 29232 raft_consensus.cc:3060] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.735252 29232 raft_consensus.cc:2749] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:21.749490 28862 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:21.805552 29238 consensus_queue.cc:1048] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:21.821043 29229 consensus_queue.cc:1048] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:21.827262 29192 consensus_queue.cc:1048] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:21.849494 29230 consensus_queue.cc:1048] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:21.883435 29229 consensus_queue.cc:1048] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:21.894829 29190 consensus_queue.cc:1048] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:21.930140 29192 raft_consensus.cc:493] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.930709 29192 raft_consensus.cc:515] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.932686 29192 leader_election.cc:290] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.934034 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09" is_pre_election: true
I20260709 12:28:21.933868 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4" is_pre_election: true
I20260709 12:28:21.934823 29065 raft_consensus.cc:2468] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.934913 28916 raft_consensus.cc:2468] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 0.
I20260709 12:28:21.936152 28957 leader_election.cc:304] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.936916 29192 raft_consensus.cc:2804] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:21.937307 29192 raft_consensus.cc:493] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.937641 29192 raft_consensus.cc:3060] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.945943 29192 raft_consensus.cc:515] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.947870 29192 leader_election.cc:290] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:21.948927 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:21.948969 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:21.949615 28916 raft_consensus.cc:3060] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.949708 29065 raft_consensus.cc:3060] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:21.956060 29065 raft_consensus.cc:2468] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.956451 28916 raft_consensus.cc:2468] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 1.
I20260709 12:28:21.957320 28957 leader_election.cc:304] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:21.958354 29192 raft_consensus.cc:2804] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:21.959035 29192 raft_consensus.cc:697] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:21.959854 29192 consensus_queue.cc:237] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:21.968111 28812 catalog_manager.cc:5697] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 0 to 1, leader changed from <none> to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 1 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:21.993623 29192 consensus_queue.cc:1048] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:21.994568 29239 consensus_queue.cc:1048] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.032438 29232 consensus_queue.cc:1048] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.047102 29269 consensus_queue.cc:1048] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:22.076535 29251 consensus_queue.cc:1048] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.088164 29238 consensus_queue.cc:1048] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.155331 29238 consensus_queue.cc:1048] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.161912 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:28:22.162679 28991 raft_consensus.cc:606] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:22.171684 29238 consensus_queue.cc:1048] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.174642 29062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
mode: GRACEFUL
new_leader_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:34412
I20260709 12:28:22.175343 29062 raft_consensus.cc:606] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Received request to transfer leadership to TS 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:22.182812 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37436ff925f944399a08784cf485f0a3"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:28:22.183346 28991 raft_consensus.cc:606] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 012f61c51f47429ca57af12350fe42f4
I20260709 12:28:22.194533 28915 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
mode: GRACEFUL
new_leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
 from {username='slave'} at 127.0.0.1:36778
I20260709 12:28:22.195199 28915 raft_consensus.cc:606] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Received request to transfer leadership to TS e59dbf606f6c4bebafd56cbe0e7d4ece
I20260709 12:28:22.205051 28991 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:38684
I20260709 12:28:22.205641 28991 raft_consensus.cc:606] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:22.210152 29238 consensus_queue.cc:1048] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.213867 29249 consensus_queue.cc:1048] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:22.246414 29249 raft_consensus.cc:993] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:22.246939 29249 raft_consensus.cc:1081] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:22.248591 28915 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:36762
I20260709 12:28:22.249123 28915 raft_consensus.cc:493] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.249537 28915 raft_consensus.cc:3060] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.256366 28915 raft_consensus.cc:515] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.258467 28915 leader_election.cc:290] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:22.259415 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:22.259759 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ca5890804906b936ecc5c501a374" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:22.260169 28991 raft_consensus.cc:3055] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:22.260301 29065 raft_consensus.cc:3060] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.260564 28991 raft_consensus.cc:740] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:22.261435 28991 consensus_queue.cc:260] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.262853 28991 raft_consensus.cc:3060] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.268324 29065 raft_consensus.cc:2468] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:22.269972 28882 leader_election.cc:304] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:22.270284 28991 raft_consensus.cc:2468] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:22.270789 29190 raft_consensus.cc:2804] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:22.271323 29190 raft_consensus.cc:697] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:22.272149 29190 consensus_queue.cc:237] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.279987 28812 catalog_manager.cc:5697] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.403903 29261 raft_consensus.cc:993] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: : Instructing follower e59dbf606f6c4bebafd56cbe0e7d4ece to start an election
I20260709 12:28:22.404575 29261 raft_consensus.cc:1081] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Signalling peer e59dbf606f6c4bebafd56cbe0e7d4ece to start an election
I20260709 12:28:22.406124 28991 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
 from {username='slave'} at 127.0.0.1:38672
I20260709 12:28:22.406795 28991 raft_consensus.cc:493] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.407140 28991 raft_consensus.cc:3060] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.413134 28991 raft_consensus.cc:515] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.415091 28991 leader_election.cc:290] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:22.416257 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:22.417004 28916 raft_consensus.cc:3055] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:22.417327 28916 raft_consensus.cc:740] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:22.416179 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8e9159ea91642abb4fcb73bb9f79f93" candidate_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:22.417883 29065 raft_consensus.cc:3060] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.418157 28916 consensus_queue.cc:260] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.419507 28916 raft_consensus.cc:3060] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.424050 29065 raft_consensus.cc:2468] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 2.
I20260709 12:28:22.425134 28916 raft_consensus.cc:2468] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e59dbf606f6c4bebafd56cbe0e7d4ece in term 2.
I20260709 12:28:22.425201 28957 leader_election.cc:304] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:22.426189 29249 raft_consensus.cc:2804] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:22.426671 29249 raft_consensus.cc:697] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 LEADER]: Becoming Leader. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:22.427481 29249 consensus_queue.cc:237] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.434836 28812 catalog_manager.cc:5697] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece reported cstate change: term changed from 1 to 2, leader changed from 012f61c51f47429ca57af12350fe42f4 (127.12.45.65) to e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66). New cstate: current_term: 2 leader_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.468201 29249 consensus_queue.cc:1048] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.482460 29239 consensus_queue.cc:1048] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:22.490963 29239 raft_consensus.cc:993] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:22.491608 29238 raft_consensus.cc:1081] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:22.493424 29065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:34386
I20260709 12:28:22.494036 29065 raft_consensus.cc:493] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.494419 29065 raft_consensus.cc:3060] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.502399 29065 raft_consensus.cc:515] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.505023 29065 leader_election.cc:290] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:22.505874 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:22.506212 28916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "910c29c25fbd4ba1a51b3aeef4f79c0d" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:22.506651 28991 raft_consensus.cc:3055] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:22.506788 28916 raft_consensus.cc:3060] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.507025 28991 raft_consensus.cc:740] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:22.507881 28991 consensus_queue.cc:260] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.509229 28991 raft_consensus.cc:3060] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.515785 28916 raft_consensus.cc:2468] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:22.516563 28991 raft_consensus.cc:2468] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:22.517081 29029 leader_election.cc:304] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:22.518013 29269 raft_consensus.cc:2804] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:22.518492 29269 raft_consensus.cc:697] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:22.519161 29269 consensus_queue.cc:237] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.526087 28812 catalog_manager.cc:5697] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.605626 29116 raft_consensus.cc:993] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: : Instructing follower 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:22.606230 29279 raft_consensus.cc:1081] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Signalling peer 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:22.607813 28916 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:36772
I20260709 12:28:22.608402 28916 raft_consensus.cc:493] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.608747 28916 raft_consensus.cc:3060] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.615453 28916 raft_consensus.cc:515] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.617153 28916 leader_election.cc:290] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:22.618172 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:22.618486 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0635e538e32f4cd98e440ff04cf0fd55" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:22.618875 28991 raft_consensus.cc:3060] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.619117 29065 raft_consensus.cc:3055] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:22.619444 29065 raft_consensus.cc:740] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:22.620105 29065 consensus_queue.cc:260] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.621068 29065 raft_consensus.cc:3060] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.624887 28991 raft_consensus.cc:2468] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:22.626287 29065 raft_consensus.cc:2468] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:22.626672 28882 leader_election.cc:304] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:22.627599 29261 raft_consensus.cc:2804] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:22.628053 29261 raft_consensus.cc:697] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:22.628755 29261 consensus_queue.cc:237] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.636144 28812 catalog_manager.cc:5697] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 1 to 2, leader changed from 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67) to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.657019 28991 raft_consensus.cc:1275] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:22.658855 29261 consensus_queue.cc:1048] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:22.669965 29065 raft_consensus.cc:1275] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:22.671514 29261 consensus_queue.cc:1048] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:22.751624 29251 raft_consensus.cc:993] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:22.752220 29238 raft_consensus.cc:1081] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 012f61c51f47429ca57af12350fe42f4 to start an election
I20260709 12:28:22.754209 28916 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37436ff925f944399a08784cf485f0a3"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
 from {username='slave'} at 127.0.0.1:36762
I20260709 12:28:22.754877 28916 raft_consensus.cc:493] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.755316 28916 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.764384 28916 raft_consensus.cc:515] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.766742 28916 leader_election.cc:290] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Requested vote from peers e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859), 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67:44819)
I20260709 12:28:22.768076 28991 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:22.768160 29065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "012f61c51f47429ca57af12350fe42f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
I20260709 12:28:22.768944 28991 raft_consensus.cc:3055] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:22.769044 29065 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.769438 28991 raft_consensus.cc:740] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:22.770346 28991 consensus_queue.cc:260] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.771226 28991 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:22.777396 28991 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:22.778299 29065 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012f61c51f47429ca57af12350fe42f4 in term 2.
I20260709 12:28:22.778601 28882 leader_election.cc:304] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:22.779616 29190 raft_consensus.cc:2804] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:22.780125 29190 raft_consensus.cc:697] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Leader. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:22.781185 29190 consensus_queue.cc:237] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:22.789093 28812 catalog_manager.cc:5697] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 012f61c51f47429ca57af12350fe42f4 (127.12.45.65). New cstate: current_term: 2 leader_uuid: "012f61c51f47429ca57af12350fe42f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:22.973098 28916 raft_consensus.cc:1275] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:22.974575 29279 consensus_queue.cc:1048] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:22.983423 28991 raft_consensus.cc:1275] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:22.985663 29280 consensus_queue.cc:1048] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:23.006525 28916 raft_consensus.cc:1275] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer e59dbf606f6c4bebafd56cbe0e7d4ece: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:23.007925 29251 consensus_queue.cc:1048] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:23.018738 29064 raft_consensus.cc:1275] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer e59dbf606f6c4bebafd56cbe0e7d4ece: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:23.020627 29238 consensus_queue.cc:1048] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:23.139501 29064 raft_consensus.cc:1275] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:23.140838 29261 consensus_queue.cc:1048] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:23.150665 28991 raft_consensus.cc:1275] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:23.152241 29229 consensus_queue.cc:1048] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:23.360244 28989 raft_consensus.cc:1275] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:23.361716 29190 consensus_queue.cc:1048] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:23.371140 29064 raft_consensus.cc:1275] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Refusing update from remote peer 012f61c51f47429ca57af12350fe42f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:23.372687 29229 consensus_queue.cc:1048] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:25.209390 28812 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60896:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:28:25.232774 28812 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60896:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:28:25.275537 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:28:25.528664 29306 log.cc:927] Time spent T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:28:25.590178 29323 log.cc:927] Time spent T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:28:25.578483 29330 tablet.cc:1724] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:25.599061 28942 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:b10d048fcf7d45bb92c4ccea6da3a342)
W20260709 12:28:25.613641 28882 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:44819, user_credentials={real_user=slave}} blocked reactor thread for 107851us
W20260709 12:28:25.612596 28942 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:b10d048fcf7d45bb92c4ccea6da3a342)
W20260709 12:28:25.621483 28942 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:b10d048fcf7d45bb92c4ccea6da3a342)
W20260709 12:28:25.654776 29329 log.cc:927] Time spent T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 12:28:25.662727 29325 log.cc:927] Time spent T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:28:25.666011 29324 log.cc:927] Time spent T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:28:25.667291 28911 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37436ff925f944399a08784cf485f0a3"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
mode: GRACEFUL
new_leader_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:43554
I20260709 12:28:25.670439 28911 raft_consensus.cc:606] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Received request to transfer leadership to TS 09474a5ec3194a23a04d99c316b86f09
W20260709 12:28:25.671649 29327 log.cc:927] Time spent T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:28:25.686947 29346 tablet.cc:1724] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.691493 29342 tablet.cc:1724] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:25.700304 28882 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:46859, user_credentials={real_user=slave}} blocked reactor thread for 50118.1us
I20260709 12:28:25.697923 29337 tablet.cc:1724] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.645462 29334 tablet.cc:1724] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:25.706832 29341 log.cc:927] Time spent T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:28:25.712379 29190 raft_consensus.cc:993] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: : Instructing follower 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:25.715744 29190 raft_consensus.cc:1081] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Signalling peer 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:25.779630 29065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37436ff925f944399a08784cf485f0a3"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:34400
I20260709 12:28:25.788223 29065 raft_consensus.cc:493] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:25.788838 29065 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:25.804159 29344 tablet.cc:1724] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.805009 29330 tablet.cc:1724] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.813941 29347 tablet.cc:1724] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.846323 29065 raft_consensus.cc:515] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:25.862979 29357 tablet.cc:1724] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.866456 29330 tablet.cc:1724] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.875609 29354 tablet.cc:1724] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.876348 29344 tablet.cc:1724] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.881562 29337 tablet.cc:1724] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.885331 28915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:25.886054 28915 raft_consensus.cc:3055] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:25.886422 28915 raft_consensus.cc:740] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:25.887269 28915 consensus_queue.cc:260] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:25.888180 28915 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:25.869874 29347 tablet.cc:1724] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.897873 29065 leader_election.cc:290] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 3 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:25.911051 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37436ff925f944399a08784cf485f0a3" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:25.912292 28986 raft_consensus.cc:3060] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:25.915041 29354 tablet.cc:1724] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.918589 29350 tablet.cc:1724] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.931655 28915 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 3.
I20260709 12:28:25.936955 29029 leader_election.cc:304] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:25.939520 29288 raft_consensus.cc:2804] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:25.943078 29288 raft_consensus.cc:697] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 3 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:25.944428 29288 consensus_queue.cc:237] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
W20260709 12:28:25.948390 29348 log.cc:927] Time spent T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:28:25.951901 28957 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:44819, user_credentials={real_user=slave}} blocked reactor thread for 52701us
I20260709 12:28:25.958098 29349 tablet.cc:1724] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.963335 29330 tablet.cc:1724] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.963862 29355 tablet.cc:1724] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.967027 28987 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:36644
I20260709 12:28:25.967803 28987 raft_consensus.cc:606] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:25.971043 29342 tablet.cc:1724] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.981911 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:28:25.982865 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:25.992501 28812 catalog_manager.cc:5697] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 2 to 3, leader changed from 012f61c51f47429ca57af12350fe42f4 (127.12.45.65) to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:25.993753 29354 tablet.cc:1724] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.995939 28986 raft_consensus.cc:2468] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 3.
I20260709 12:28:25.997573 28913 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12"
dest_uuid: "012f61c51f47429ca57af12350fe42f4"
mode: GRACEFUL
new_leader_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:43554
I20260709 12:28:25.999810 28913 raft_consensus.cc:606] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Received request to transfer leadership to TS 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:25.997493 29346 tablet.cc:1724] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:26.005223 28987 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574"
dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
mode: GRACEFUL
new_leader_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:36644
I20260709 12:28:26.005911 28987 raft_consensus.cc:606] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Received request to transfer leadership to TS 09474a5ec3194a23a04d99c316b86f09
I20260709 12:28:26.008744 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:26.009922 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:26.013011 29354 tablet.cc:1724] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:25.933490 29337 tablet.cc:1724] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:26.028769 29346 tablet.cc:1724] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:26.047209 29350 tablet.cc:1724] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:26.066859 29338 raft_consensus.cc:993] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: : Instructing follower 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:26.067601 29343 raft_consensus.cc:1081] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Signalling peer 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:26.069425 29059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:34400
I20260709 12:28:26.070575 29059 raft_consensus.cc:493] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:26.071162 29059 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:26.082955 29059 raft_consensus.cc:515] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.086148 28913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:26.084287 28813 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb: soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe] alter complete (version 1)
I20260709 12:28:26.087276 28913 raft_consensus.cc:3055] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:26.087662 28913 raft_consensus.cc:740] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 012f61c51f47429ca57af12350fe42f4, State: Running, Role: LEADER
I20260709 12:28:26.088541 28913 consensus_queue.cc:260] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.089758 28913 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:26.094874 29059 leader_election.cc:290] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 3 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:26.096446 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8b2f17137b433296a99c7bc6971f12" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:26.097257 28986 raft_consensus.cc:3060] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:26.100313 28913 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 3.
I20260709 12:28:26.101468 29029 leader_election.cc:304] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:26.102327 29288 raft_consensus.cc:2804] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:26.102792 29288 raft_consensus.cc:697] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 3 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:26.103686 28986 raft_consensus.cc:2468] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 3.
I20260709 12:28:26.103637 29288 consensus_queue.cc:237] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.113770 28813 catalog_manager.cc:5697] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 2 to 3, leader changed from 012f61c51f47429ca57af12350fe42f4 (127.12.45.65) to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:26.165105 29339 raft_consensus.cc:993] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:26.165678 29238 raft_consensus.cc:1081] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:26.167639 29059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:34386
I20260709 12:28:26.168476 29059 raft_consensus.cc:493] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:26.169127 29059 raft_consensus.cc:3060] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:26.176221 29059 raft_consensus.cc:515] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.178402 29059 leader_election.cc:290] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:26.179525 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:26.179530 28913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "869187ef93bc49c0aad674b6b04d6574" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:26.180384 28986 raft_consensus.cc:3055] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:26.180433 28913 raft_consensus.cc:3060] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:26.180907 28986 raft_consensus.cc:740] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:26.181936 28986 consensus_queue.cc:260] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.183262 28986 raft_consensus.cc:3060] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:26.252599 28986 raft_consensus.cc:2468] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:26.255859 29031 leader_election.cc:304] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09474a5ec3194a23a04d99c316b86f09, e59dbf606f6c4bebafd56cbe0e7d4ece; no voters: 
I20260709 12:28:26.257862 29314 raft_consensus.cc:2804] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:26.258574 29314 raft_consensus.cc:697] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:26.260380 29314 consensus_queue.cc:237] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.264926 28913 raft_consensus.cc:2468] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:26.266532 28915 raft_consensus.cc:1240] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Rejecting Update request from peer e59dbf606f6c4bebafd56cbe0e7d4ece for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:26.285763 28813 catalog_manager.cc:5697] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:26.371397 28986 raft_consensus.cc:1275] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:28:26.373056 29314 consensus_queue.cc:1048] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:26.377828 29354 tablet.cc:1724] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:26.390292 28913 raft_consensus.cc:1275] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:28:26.393558 29288 consensus_queue.cc:1048] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:28:26.396579 28813 catalog_manager.cc:5789] TS e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859) does not have the latest schema for tablet 37436ff925f944399a08784cf485f0a3 (table soft_deleted_table [id=4a5a246436bd4956a46d15448ddf3dfe]). Expected version 1 got 0
I20260709 12:28:26.441746 29289 raft_consensus.cc:993] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: : Instructing follower 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:26.442476 29358 raft_consensus.cc:1081] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Signalling peer 09474a5ec3194a23a04d99c316b86f09 to start an election
I20260709 12:28:26.444051 29065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150"
dest_uuid: "09474a5ec3194a23a04d99c316b86f09"
 from {username='slave'} at 127.0.0.1:34386
I20260709 12:28:26.444602 29065 raft_consensus.cc:493] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:26.444859 29065 raft_consensus.cc:3060] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:26.453123 29065 raft_consensus.cc:515] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.454834 29065 leader_election.cc:290] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Requested vote from peers 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029), e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859)
I20260709 12:28:26.456142 28986 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece"
I20260709 12:28:26.456112 28913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "393b2cf0f8e7449da9cde08f89185150" candidate_uuid: "09474a5ec3194a23a04d99c316b86f09" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "012f61c51f47429ca57af12350fe42f4"
I20260709 12:28:26.457197 28986 raft_consensus.cc:3055] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:26.457206 28913 raft_consensus.cc:3060] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:26.457741 28986 raft_consensus.cc:740] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e59dbf606f6c4bebafd56cbe0e7d4ece, State: Running, Role: LEADER
I20260709 12:28:26.459084 28986 consensus_queue.cc:260] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.460844 28986 raft_consensus.cc:3060] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:26.467314 28913 raft_consensus.cc:2468] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:26.469210 29029 leader_election.cc:304] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012f61c51f47429ca57af12350fe42f4, 09474a5ec3194a23a04d99c316b86f09; no voters: 
I20260709 12:28:26.469992 28986 raft_consensus.cc:2468] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09474a5ec3194a23a04d99c316b86f09 in term 2.
I20260709 12:28:26.470175 29288 raft_consensus.cc:2804] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:26.470837 29288 raft_consensus.cc:697] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Becoming Leader. State: Replica: 09474a5ec3194a23a04d99c316b86f09, State: Running, Role: LEADER
I20260709 12:28:26.472034 29288 consensus_queue.cc:237] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } }
I20260709 12:28:26.480127 28813 catalog_manager.cc:5697] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 reported cstate change: term changed from 1 to 2, leader changed from e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66) to 09474a5ec3194a23a04d99c316b86f09 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "09474a5ec3194a23a04d99c316b86f09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09474a5ec3194a23a04d99c316b86f09" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 44819 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:26.496788 28913 raft_consensus.cc:1275] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:26.498227 29288 consensus_queue.cc:1048] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:26.503659 28986 raft_consensus.cc:1275] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:26.505493 29288 consensus_queue.cc:1048] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:26.712777 28913 raft_consensus.cc:1275] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:26.714442 29369 consensus_queue.cc:1048] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:26.726359 28986 raft_consensus.cc:1275] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:26.728302 29288 consensus_queue.cc:1048] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:26.909335 28915 raft_consensus.cc:1275] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:28:26.910883 29288 consensus_queue.cc:1048] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "012f61c51f47429ca57af12350fe42f4" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:26.921728 28986 raft_consensus.cc:1275] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Refusing update from remote peer 09474a5ec3194a23a04d99c316b86f09: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:28:26.924084 29288 consensus_queue.cc:1048] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e59dbf606f6c4bebafd56cbe0e7d4ece" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:27.011453 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:27.018222 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:27.018857 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:27.024101 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:27.024748 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:28.026335 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:28.032351 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:28.032995 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:28.038528 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:28.039018 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:29.040091 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:29.045533 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:29.046456 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:29.052026 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:29.052459 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:30.053673 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:30.060212 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:30.060909 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:30.066664 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:30.067163 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:31.068378 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:31.073797 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:31.074412 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:31.080490 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:31.080984 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:32.082306 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:32.091557 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:32.092588 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:32.101255 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:32.102375 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:33.103658 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:33.110363 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:33.110831 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:33.115980 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:33.116365 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:34.117668 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:34.124392 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:34.124975 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:34.129812 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:34.130295 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:35.131520 12469 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:28:35.136881 12469 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:28:35.137462 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:35.142859 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:35.143471 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:36.214074 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
I20260709 12:28:36.296456 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:36.296988 29029 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
I20260709 12:28:36.298885 12469 tablet_replica.cc:333] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.301587 12469 raft_consensus.cc:2243] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:36.301955 29029 consensus_peers.cc:597] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.302989 29029 consensus_peers.cc:597] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.303388 12469 raft_consensus.cc:2272] T 37436ff925f944399a08784cf485f0a3 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.307732 12469 tablet_replica.cc:333] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.309016 12469 raft_consensus.cc:2243] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.311185 12469 raft_consensus.cc:2272] T b10d048fcf7d45bb92c4ccea6da3a342 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.315984 12469 tablet_replica.cc:333] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.317203 12469 raft_consensus.cc:2243] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.318737 12469 raft_consensus.cc:2272] T 3f676b06f1c74c848307d5a41962ce5e P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.324563 12469 tablet_replica.cc:333] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
W20260709 12:28:36.324937 29029 consensus_peers.cc:597] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.325937 12469 raft_consensus.cc:2243] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.326941 12469 raft_consensus.cc:2272] T 393b2cf0f8e7449da9cde08f89185150 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.330753 12469 tablet_replica.cc:333] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.332795 12469 raft_consensus.cc:2243] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.334383 12469 raft_consensus.cc:2272] T f8e9159ea91642abb4fcb73bb9f79f93 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.339708 12469 tablet_replica.cc:333] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.340939 12469 raft_consensus.cc:2243] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.341915 12469 raft_consensus.cc:2272] T f3575a1ff5a34e659f8921eb7bab27ab P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.346987 12469 tablet_replica.cc:333] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.348626 12469 raft_consensus.cc:2243] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.349522 12469 raft_consensus.cc:2272] T 7bcbc267390d464ab001c38efea39c00 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.352962 12469 tablet_replica.cc:333] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.354095 12469 raft_consensus.cc:2243] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.354893 12469 raft_consensus.cc:2272] T 897444c2b13e4cb69911b6de6d34b6fe P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.360589 12469 tablet_replica.cc:333] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.362113 12469 raft_consensus.cc:2243] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:28:36.364914 29029 consensus_peers.cc:597] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.365032 12469 raft_consensus.cc:2272] T a5515f4981eb4fc3951f316b5915f12c P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.369011 12469 tablet_replica.cc:333] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.369805 12469 raft_consensus.cc:2243] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.370600 12469 raft_consensus.cc:2272] T 05682701dab0458fba871736c6f6a74a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.373953 12469 tablet_replica.cc:333] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.374761 12469 raft_consensus.cc:2243] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.376497 12469 raft_consensus.cc:2272] T 59b9ca5890804906b936ecc5c501a374 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.379992 12469 tablet_replica.cc:333] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.380831 12469 raft_consensus.cc:2243] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.381599 12469 raft_consensus.cc:2272] T 368b8c8ea73c4ec8a51ff11fb9598522 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.385169 12469 tablet_replica.cc:333] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.386373 12469 raft_consensus.cc:2243] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.387995 12469 raft_consensus.cc:2272] T 0635e538e32f4cd98e440ff04cf0fd55 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.391484 12469 tablet_replica.cc:333] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.392308 12469 raft_consensus.cc:2243] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.393112 12469 raft_consensus.cc:2272] T 869187ef93bc49c0aad674b6b04d6574 P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.398661 12469 tablet_replica.cc:333] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.399592 12469 raft_consensus.cc:2243] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.400463 12469 raft_consensus.cc:2272] T 4a8b2f17137b433296a99c7bc6971f12 P 012f61c51f47429ca57af12350fe42f4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:36.400964 28955 consensus_peers.cc:597] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.405079 12469 tablet_replica.cc:333] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.406322 12469 raft_consensus.cc:2243] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.409198 12469 raft_consensus.cc:2272] T c7514edcd3054478ae21b23d56d58360 P 012f61c51f47429ca57af12350fe42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:36.412148 28955 consensus_peers.cc:597] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.414628 12469 tablet_replica.cc:333] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.415568 12469 raft_consensus.cc:2243] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.417676 12469 raft_consensus.cc:2272] T 1d117af0c3f24c4a8fd44303277d126a P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.421944 12469 tablet_replica.cc:333] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4: stopping tablet replica
I20260709 12:28:36.422886 12469 raft_consensus.cc:2243] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.424132 12469 raft_consensus.cc:2272] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 012f61c51f47429ca57af12350fe42f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.485661 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
W20260709 12:28:36.495205 28955 consensus_peers.cc:597] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45029: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.507480 28955 consensus_peers.cc:597] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45029: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.510792 28955 consensus_peers.cc:597] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45029: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.516376 29029 consensus_peers.cc:597] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45029: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.539381 28955 consensus_peers.cc:597] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45029: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.559595 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
W20260709 12:28:36.604380 29031 consensus_peers.cc:597] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 -> Peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Couldn't send request to peer e59dbf606f6c4bebafd56cbe0e7d4ece. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.613788 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:36.615069 12469 tablet_replica.cc:333] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.616255 12469 raft_consensus.cc:2243] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.617009 12469 raft_consensus.cc:2272] T 910c29c25fbd4ba1a51b3aeef4f79c0d P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.619867 12469 tablet_replica.cc:333] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.620602 12469 raft_consensus.cc:2243] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.621855 12469 raft_consensus.cc:2272] T 3f676b06f1c74c848307d5a41962ce5e P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.624979 12469 tablet_replica.cc:333] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.625828 12469 raft_consensus.cc:2243] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.626631 12469 raft_consensus.cc:2272] T b10d048fcf7d45bb92c4ccea6da3a342 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.630357 12469 tablet_replica.cc:333] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
W20260709 12:28:36.630759 29031 consensus_peers.cc:597] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 -> Peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Couldn't send request to peer e59dbf606f6c4bebafd56cbe0e7d4ece. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.631138 12469 raft_consensus.cc:2243] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.632047 12469 raft_consensus.cc:2272] T 37436ff925f944399a08784cf485f0a3 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.635324 12469 tablet_replica.cc:333] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.636032 12469 raft_consensus.cc:2243] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.636752 12469 raft_consensus.cc:2272] T f3575a1ff5a34e659f8921eb7bab27ab P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.639963 12469 tablet_replica.cc:333] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.640771 12469 raft_consensus.cc:2243] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.641502 12469 raft_consensus.cc:2272] T 59b9ca5890804906b936ecc5c501a374 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.644289 12469 tablet_replica.cc:333] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.645193 12469 raft_consensus.cc:2243] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.647323 12469 raft_consensus.cc:2272] T 7bcbc267390d464ab001c38efea39c00 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:36.648896 29031 consensus_peers.cc:597] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 -> Peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Couldn't send request to peer e59dbf606f6c4bebafd56cbe0e7d4ece. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.650624 12469 tablet_replica.cc:333] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.651461 12469 raft_consensus.cc:2243] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.652909 12469 raft_consensus.cc:2272] T 05682701dab0458fba871736c6f6a74a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.655853 12469 tablet_replica.cc:333] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.656525 12469 raft_consensus.cc:2243] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.657174 12469 raft_consensus.cc:2272] T a5515f4981eb4fc3951f316b5915f12c P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.660492 12469 tablet_replica.cc:333] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.661422 12469 raft_consensus.cc:2243] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.662629 12469 raft_consensus.cc:2272] T 897444c2b13e4cb69911b6de6d34b6fe P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.665714 12469 tablet_replica.cc:333] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.666301 12469 raft_consensus.cc:2243] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.666977 12469 raft_consensus.cc:2272] T 4a8b2f17137b433296a99c7bc6971f12 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.669459 12469 tablet_replica.cc:333] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.670022 12469 raft_consensus.cc:2243] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.670699 12469 raft_consensus.cc:2272] T 0635e538e32f4cd98e440ff04cf0fd55 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.673220 12469 tablet_replica.cc:333] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.673804 12469 raft_consensus.cc:2243] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.674428 12469 raft_consensus.cc:2272] T 869187ef93bc49c0aad674b6b04d6574 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.677606 12469 tablet_replica.cc:333] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.678228 12469 raft_consensus.cc:2243] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.678892 12469 raft_consensus.cc:2272] T 1d117af0c3f24c4a8fd44303277d126a P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.681504 12469 tablet_replica.cc:333] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.682040 12469 raft_consensus.cc:2243] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.682565 12469 raft_consensus.cc:2272] T c7514edcd3054478ae21b23d56d58360 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.685094 12469 tablet_replica.cc:333] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.685652 12469 raft_consensus.cc:2243] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.686174 12469 raft_consensus.cc:2272] T 393b2cf0f8e7449da9cde08f89185150 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.688692 12469 tablet_replica.cc:333] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.689155 12469 raft_consensus.cc:2243] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.690145 12469 raft_consensus.cc:2272] T f8e9159ea91642abb4fcb73bb9f79f93 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.692984 12469 tablet_replica.cc:333] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece: stopping tablet replica
I20260709 12:28:36.693629 12469 raft_consensus.cc:2243] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.695194 12469 raft_consensus.cc:2272] T 368b8c8ea73c4ec8a51ff11fb9598522 P e59dbf606f6c4bebafd56cbe0e7d4ece [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:36.734684 29031 consensus_peers.cc:597] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 -> Peer e59dbf606f6c4bebafd56cbe0e7d4ece (127.12.45.66:46859): Couldn't send request to peer e59dbf606f6c4bebafd56cbe0e7d4ece. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:36.748384 29029 consensus_peers.cc:597] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 -> Peer 012f61c51f47429ca57af12350fe42f4 (127.12.45.65:45029): Couldn't send request to peer 012f61c51f47429ca57af12350fe42f4. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:45029: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:36.760442 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:28:36.818465 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
I20260709 12:28:36.863420 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:36.864430 12469 tablet_replica.cc:333] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.865319 12469 raft_consensus.cc:2243] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.866017 12469 raft_consensus.cc:2272] T 3f676b06f1c74c848307d5a41962ce5e P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.869462 12469 tablet_replica.cc:333] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.870230 12469 raft_consensus.cc:2243] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.871392 12469 raft_consensus.cc:2272] T 910c29c25fbd4ba1a51b3aeef4f79c0d P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.873968 12469 tablet_replica.cc:333] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.874724 12469 raft_consensus.cc:2243] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.875830 12469 raft_consensus.cc:2272] T 37436ff925f944399a08784cf485f0a3 P 09474a5ec3194a23a04d99c316b86f09 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.878778 12469 tablet_replica.cc:333] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.879503 12469 raft_consensus.cc:2243] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.880096 12469 raft_consensus.cc:2272] T f8e9159ea91642abb4fcb73bb9f79f93 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.882751 12469 tablet_replica.cc:333] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.883373 12469 raft_consensus.cc:2243] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.884400 12469 raft_consensus.cc:2272] T 393b2cf0f8e7449da9cde08f89185150 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.886863 12469 tablet_replica.cc:333] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.887549 12469 raft_consensus.cc:2243] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.888103 12469 raft_consensus.cc:2272] T 7bcbc267390d464ab001c38efea39c00 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.890520 12469 tablet_replica.cc:333] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.891127 12469 raft_consensus.cc:2243] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.891741 12469 raft_consensus.cc:2272] T 59b9ca5890804906b936ecc5c501a374 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.894152 12469 tablet_replica.cc:333] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.894785 12469 raft_consensus.cc:2243] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.895424 12469 raft_consensus.cc:2272] T 05682701dab0458fba871736c6f6a74a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.898574 12469 tablet_replica.cc:333] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.899268 12469 raft_consensus.cc:2243] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.899835 12469 raft_consensus.cc:2272] T a5515f4981eb4fc3951f316b5915f12c P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.902452 12469 tablet_replica.cc:333] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.903153 12469 raft_consensus.cc:2243] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.903687 12469 raft_consensus.cc:2272] T 897444c2b13e4cb69911b6de6d34b6fe P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.906160 12469 tablet_replica.cc:333] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.907078 12469 raft_consensus.cc:2243] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.907895 12469 raft_consensus.cc:2272] T c7514edcd3054478ae21b23d56d58360 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.910487 12469 tablet_replica.cc:333] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.911129 12469 raft_consensus.cc:2243] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.911653 12469 raft_consensus.cc:2272] T 1d117af0c3f24c4a8fd44303277d126a P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.914425 12469 tablet_replica.cc:333] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.915110 12469 raft_consensus.cc:2243] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.916183 12469 raft_consensus.cc:2272] T f3575a1ff5a34e659f8921eb7bab27ab P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.918738 12469 tablet_replica.cc:333] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.919337 12469 raft_consensus.cc:2243] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.920010 12469 raft_consensus.cc:2272] T b10d048fcf7d45bb92c4ccea6da3a342 P 09474a5ec3194a23a04d99c316b86f09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.922643 12469 tablet_replica.cc:333] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.923291 12469 raft_consensus.cc:2243] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.923840 12469 raft_consensus.cc:2272] T 368b8c8ea73c4ec8a51ff11fb9598522 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.926717 12469 tablet_replica.cc:333] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.927299 12469 raft_consensus.cc:2243] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:36.927953 12469 raft_consensus.cc:2272] T 0635e538e32f4cd98e440ff04cf0fd55 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.930536 12469 tablet_replica.cc:333] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.931103 12469 raft_consensus.cc:2243] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.932152 12469 raft_consensus.cc:2272] T 869187ef93bc49c0aad674b6b04d6574 P 09474a5ec3194a23a04d99c316b86f09 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.934541 12469 tablet_replica.cc:333] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09: stopping tablet replica
I20260709 12:28:36.935089 12469 raft_consensus.cc:2243] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:36.936081 12469 raft_consensus.cc:2272] T 4a8b2f17137b433296a99c7bc6971f12 P 09474a5ec3194a23a04d99c316b86f09 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:36.986775 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:28:37.037611 12469 master.cc:562] Master@127.12.45.126:46075 shutting down...
I20260709 12:28:37.067961 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:37.068805 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:37.069161 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P 46594ded80ce47f899e03c48072b90eb: stopping tablet replica
I20260709 12:28:37.093755 12469 master.cc:584] Master@127.12.45.126:46075 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21631 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:28:37.164052 12469 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.12.45.126:34061
I20260709 12:28:37.165601 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:37.172770 29399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.172787 29398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.174800 29401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:37.176640 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:37.177775 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:37.177979 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:37.178088 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600117178077 us; error 0 us; skew 500 ppm
I20260709 12:28:37.178668 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:37.181336 12469 webserver.cc:533] Webserver started at http://127.12.45.126:37651/ using document root <none> and password file <none>
I20260709 12:28:37.181818 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:37.181989 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:37.182214 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:37.183887 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/master-0-root/instance:
uuid: "740482ea41d04ec49107ffbefe782377"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.189561 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:37.194135 29406 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.195284 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:37.195597 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/master-0-root
uuid: "740482ea41d04ec49107ffbefe782377"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.195869 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:37.209658 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:37.211278 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:37.263164 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.126:34061
I20260709 12:28:37.263418 29457 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.126:34061 every 8 connection(s)
I20260709 12:28:37.267776 29458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.283850 29458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377: Bootstrap starting.
I20260709 12:28:37.289685 29458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:37.294688 29458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377: No bootstrap required, opened a new log
I20260709 12:28:37.297384 29458 raft_consensus.cc:359] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740482ea41d04ec49107ffbefe782377" member_type: VOTER }
I20260709 12:28:37.298050 29458 raft_consensus.cc:385] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:37.298377 29458 raft_consensus.cc:740] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 740482ea41d04ec49107ffbefe782377, State: Initialized, Role: FOLLOWER
I20260709 12:28:37.299038 29458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740482ea41d04ec49107ffbefe782377" member_type: VOTER }
I20260709 12:28:37.299496 29458 raft_consensus.cc:399] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:37.299700 29458 raft_consensus.cc:493] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:37.300222 29458 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:37.307206 29458 raft_consensus.cc:515] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740482ea41d04ec49107ffbefe782377" member_type: VOTER }
I20260709 12:28:37.307809 29458 leader_election.cc:304] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 740482ea41d04ec49107ffbefe782377; no voters: 
I20260709 12:28:37.309234 29458 leader_election.cc:290] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:37.309960 29461 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:37.311821 29461 raft_consensus.cc:697] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 1 LEADER]: Becoming Leader. State: Replica: 740482ea41d04ec49107ffbefe782377, State: Running, Role: LEADER
I20260709 12:28:37.312767 29458 sys_catalog.cc:565] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:37.312832 29461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740482ea41d04ec49107ffbefe782377" member_type: VOTER }
I20260709 12:28:37.321063 29462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "740482ea41d04ec49107ffbefe782377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740482ea41d04ec49107ffbefe782377" member_type: VOTER } }
I20260709 12:28:37.321200 29463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 740482ea41d04ec49107ffbefe782377. Latest consensus state: current_term: 1 leader_uuid: "740482ea41d04ec49107ffbefe782377" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "740482ea41d04ec49107ffbefe782377" member_type: VOTER } }
I20260709 12:28:37.322022 29463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:37.323961 29462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:37.330657 29471 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:37.336171 29471 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:37.337571 12469 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:37.347347 29471 catalog_manager.cc:1383] Generated new cluster ID: eeb71c8a3b774c349f3a7e60314580cf
I20260709 12:28:37.347808 29471 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:37.364437 29471 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:37.366550 29471 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:37.384279 29471 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377: Generated new TSK 0
I20260709 12:28:37.385020 29471 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:37.405350 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:37.412631 29479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.413795 29480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.416091 29482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:37.416182 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:37.417539 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:37.417789 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:37.417984 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600117417966 us; error 0 us; skew 500 ppm
I20260709 12:28:37.418654 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:37.421416 12469 webserver.cc:533] Webserver started at http://127.12.45.65:38015/ using document root <none> and password file <none>
I20260709 12:28:37.422067 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:37.422327 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:37.422641 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:37.423955 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-0-root/instance:
uuid: "710651b45fd047d9aea0f26c77946f9f"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.429904 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:37.434247 29487 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.435395 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:37.435847 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-0-root
uuid: "710651b45fd047d9aea0f26c77946f9f"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.436311 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:37.457887 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:37.459616 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:37.461499 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:37.464092 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:37.464355 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.464612 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:37.464807 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.518213 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.65:33633
I20260709 12:28:37.518471 29549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.65:33633 every 8 connection(s)
I20260709 12:28:37.524184 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:37.532001 29554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.533509 29555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:37.540187 29550 heartbeater.cc:344] Connected to a master server at 127.12.45.126:34061
I20260709 12:28:37.540769 29550 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:37.541052 12469 server_base.cc:1061] running on GCE node
W20260709 12:28:37.541154 29557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:37.542130 29550 heartbeater.cc:507] Master 127.12.45.126:34061 requested a full tablet report, sending...
I20260709 12:28:37.543537 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:37.543864 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:37.544082 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600117544062 us; error 0 us; skew 500 ppm
I20260709 12:28:37.544889 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:37.545080 29423 ts_manager.cc:194] Registered new tserver with Master: 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633)
I20260709 12:28:37.548316 29423 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48838
I20260709 12:28:37.548522 12469 webserver.cc:533] Webserver started at http://127.12.45.66:40089/ using document root <none> and password file <none>
I20260709 12:28:37.549330 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:37.549594 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:37.549953 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:37.551415 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-1-root/instance:
uuid: "76171565ecf74c9aac0deaac7c291900"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.556710 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:28:37.561100 29562 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.562217 12469 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:28:37.562621 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-1-root
uuid: "76171565ecf74c9aac0deaac7c291900"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.562897 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:37.577445 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:37.578819 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:37.580487 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:37.582973 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:37.583158 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.583352 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:37.583477 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.634649 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.66:43567
I20260709 12:28:37.634774 29624 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.66:43567 every 8 connection(s)
I20260709 12:28:37.640846 12469 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:37.651090 29628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.653339 29629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:37.654217 29631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:37.655930 12469 server_base.cc:1061] running on GCE node
I20260709 12:28:37.656746 29625 heartbeater.cc:344] Connected to a master server at 127.12.45.126:34061
I20260709 12:28:37.657215 29625 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:37.657212 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:37.657846 12469 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:37.658130 12469 hybrid_clock.cc:648] HybridClock initialized: now 1783600117658107 us; error 0 us; skew 500 ppm
I20260709 12:28:37.658661 29625 heartbeater.cc:507] Master 127.12.45.126:34061 requested a full tablet report, sending...
I20260709 12:28:37.659060 12469 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:37.661567 29423 ts_manager.cc:194] Registered new tserver with Master: 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:37.662616 12469 webserver.cc:533] Webserver started at http://127.12.45.67:40151/ using document root <none> and password file <none>
I20260709 12:28:37.663213 12469 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:37.663452 12469 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:37.663447 29423 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48844
I20260709 12:28:37.664021 12469 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:37.666043 12469 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-2-root/instance:
uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.671689 12469 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:28:37.676493 29636 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.677757 12469 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:28:37.678202 12469 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-2-root
uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
format_stamp: "Formatted at 2026-07-09 12:28:37 on dist-test-slave-90bm"
I20260709 12:28:37.678632 12469 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfq_54g/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599574270161-12469-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:37.694559 12469 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:37.696170 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:37.697871 12469 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:37.700459 12469 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:37.700702 12469 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.700976 12469 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:37.701182 12469 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:37.755811 12469 rpc_server.cc:307] RPC server started. Bound to: 127.12.45.67:36695
I20260709 12:28:37.755973 29698 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.12.45.67:36695 every 8 connection(s)
I20260709 12:28:37.773159 29699 heartbeater.cc:344] Connected to a master server at 127.12.45.126:34061
I20260709 12:28:37.773741 29699 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:37.774744 29699 heartbeater.cc:507] Master 127.12.45.126:34061 requested a full tablet report, sending...
I20260709 12:28:37.777454 29423 ts_manager.cc:194] Registered new tserver with Master: 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:37.777702 12469 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018387518s
I20260709 12:28:37.779803 29423 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48854
I20260709 12:28:37.813833 29423 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48866:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:37.818475 29423 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:37.913967 29515 tablet_service.cc:1511] Processing CreateTablet for tablet f0b1aa4fd2654e4d95e99c057a036d1c (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:37.915016 29513 tablet_service.cc:1511] Processing CreateTablet for tablet 3b45143755dc412183cf1dd882f290b5 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:37.915791 29515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0b1aa4fd2654e4d95e99c057a036d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.915570 29514 tablet_service.cc:1511] Processing CreateTablet for tablet badb69aae97a4c3ba2252e42aa7e01e7 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:37.916618 29513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b45143755dc412183cf1dd882f290b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.917331 29514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet badb69aae97a4c3ba2252e42aa7e01e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.917735 29512 tablet_service.cc:1511] Processing CreateTablet for tablet fba3c94d505a4b7b86347de064750711 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:37.919287 29512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba3c94d505a4b7b86347de064750711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.924713 29511 tablet_service.cc:1511] Processing CreateTablet for tablet 50fb89a8625342928a38a5a005eb7383 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:37.926504 29511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50fb89a8625342928a38a5a005eb7383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.932492 29510 tablet_service.cc:1511] Processing CreateTablet for tablet 1f70f573c7ac4e3cb0cf2a3ff2d93cdf (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:37.934878 29510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f70f573c7ac4e3cb0cf2a3ff2d93cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.938684 29509 tablet_service.cc:1511] Processing CreateTablet for tablet 49def8d7de014ea883f046b20d82c270 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:37.940274 29509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49def8d7de014ea883f046b20d82c270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.939925 29590 tablet_service.cc:1511] Processing CreateTablet for tablet f0b1aa4fd2654e4d95e99c057a036d1c (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:37.943141 29589 tablet_service.cc:1511] Processing CreateTablet for tablet badb69aae97a4c3ba2252e42aa7e01e7 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:37.945864 29588 tablet_service.cc:1511] Processing CreateTablet for tablet 3b45143755dc412183cf1dd882f290b5 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:37.948987 29587 tablet_service.cc:1511] Processing CreateTablet for tablet fba3c94d505a4b7b86347de064750711 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:37.951707 29586 tablet_service.cc:1511] Processing CreateTablet for tablet 50fb89a8625342928a38a5a005eb7383 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:37.954375 29585 tablet_service.cc:1511] Processing CreateTablet for tablet 1f70f573c7ac4e3cb0cf2a3ff2d93cdf (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:37.941713 29590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0b1aa4fd2654e4d95e99c057a036d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.959215 29586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50fb89a8625342928a38a5a005eb7383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.962038 29589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet badb69aae97a4c3ba2252e42aa7e01e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.971462 29588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b45143755dc412183cf1dd882f290b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.974702 29585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f70f573c7ac4e3cb0cf2a3ff2d93cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:37.978574 29587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba3c94d505a4b7b86347de064750711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.005151 29584 tablet_service.cc:1511] Processing CreateTablet for tablet 49def8d7de014ea883f046b20d82c270 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:38.007045 29584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49def8d7de014ea883f046b20d82c270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.007330 29664 tablet_service.cc:1511] Processing CreateTablet for tablet f0b1aa4fd2654e4d95e99c057a036d1c (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:38.009877 29664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0b1aa4fd2654e4d95e99c057a036d1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.012164 29663 tablet_service.cc:1511] Processing CreateTablet for tablet badb69aae97a4c3ba2252e42aa7e01e7 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:38.013751 29661 tablet_service.cc:1511] Processing CreateTablet for tablet fba3c94d505a4b7b86347de064750711 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:38.015501 29662 tablet_service.cc:1511] Processing CreateTablet for tablet 3b45143755dc412183cf1dd882f290b5 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:38.025368 29660 tablet_service.cc:1511] Processing CreateTablet for tablet 50fb89a8625342928a38a5a005eb7383 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:38.028879 29659 tablet_service.cc:1511] Processing CreateTablet for tablet 1f70f573c7ac4e3cb0cf2a3ff2d93cdf (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:38.029335 29513 tablet_service.cc:1511] Processing CreateTablet for tablet 57be42294a1c49ee994eaf58ad5d947d (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:38.036321 29662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b45143755dc412183cf1dd882f290b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.036865 29714 tablet_bootstrap.cc:492] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.043110 29513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57be42294a1c49ee994eaf58ad5d947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.051246 29658 tablet_service.cc:1511] Processing CreateTablet for tablet 49def8d7de014ea883f046b20d82c270 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:38.057592 29583 tablet_service.cc:1511] Processing CreateTablet for tablet 57be42294a1c49ee994eaf58ad5d947d (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:38.059432 29583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57be42294a1c49ee994eaf58ad5d947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.071583 29657 tablet_service.cc:1511] Processing CreateTablet for tablet 57be42294a1c49ee994eaf58ad5d947d (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:38.041652 29661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba3c94d505a4b7b86347de064750711. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.078738 29508 tablet_service.cc:1511] Processing CreateTablet for tablet a178b0a2291d45609f26a615cfbc0289 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:38.080653 29508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a178b0a2291d45609f26a615cfbc0289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.095125 29582 tablet_service.cc:1511] Processing CreateTablet for tablet a178b0a2291d45609f26a615cfbc0289 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:38.097479 29582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a178b0a2291d45609f26a615cfbc0289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.097951 29656 tablet_service.cc:1511] Processing CreateTablet for tablet a178b0a2291d45609f26a615cfbc0289 (DEFAULT_TABLE table=test-workload [id=44b43d175f2748f89563347317859d0a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:38.100018 29714 tablet_bootstrap.cc:654] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.100998 29656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a178b0a2291d45609f26a615cfbc0289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.122148 29715 tablet_bootstrap.cc:492] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.124578 29663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet badb69aae97a4c3ba2252e42aa7e01e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.125591 29659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f70f573c7ac4e3cb0cf2a3ff2d93cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.129112 29658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49def8d7de014ea883f046b20d82c270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.130393 29657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57be42294a1c49ee994eaf58ad5d947d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.135124 29660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50fb89a8625342928a38a5a005eb7383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:38.155112 29716 tablet_bootstrap.cc:492] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.165194 29716 tablet_bootstrap.cc:654] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.192778 29714 tablet_bootstrap.cc:492] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.193703 29714 ts_tablet_manager.cc:1403] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.157s	user 0.032s	sys 0.009s
I20260709 12:28:38.199550 29714 raft_consensus.cc:359] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.200340 29714 raft_consensus.cc:385] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.200656 29714 raft_consensus.cc:740] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.207876 29714 consensus_queue.cc:260] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.211627 29715 tablet_bootstrap.cc:654] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.218309 29550 heartbeater.cc:499] Master 127.12.45.126:34061 was elected leader, sending a full tablet report...
I20260709 12:28:38.220032 29714 ts_tablet_manager.cc:1434] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.026s	user 0.016s	sys 0.000s
I20260709 12:28:38.221302 29714 tablet_bootstrap.cc:492] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.230636 29714 tablet_bootstrap.cc:654] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.233175 29716 tablet_bootstrap.cc:492] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.243013 29716 ts_tablet_manager.cc:1403] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.003s
I20260709 12:28:38.247773 29716 raft_consensus.cc:359] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.248658 29716 raft_consensus.cc:385] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.248973 29716 raft_consensus.cc:740] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.249958 29714 tablet_bootstrap.cc:492] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.250025 29716 consensus_queue.cc:260] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.252522 29714 ts_tablet_manager.cc:1403] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.019s
I20260709 12:28:38.254086 29715 tablet_bootstrap.cc:492] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.254879 29715 ts_tablet_manager.cc:1403] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.133s	user 0.020s	sys 0.026s
I20260709 12:28:38.257462 29714 raft_consensus.cc:359] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.258308 29715 raft_consensus.cc:359] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.258798 29714 raft_consensus.cc:385] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.259403 29715 raft_consensus.cc:385] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.259481 29714 raft_consensus.cc:740] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.259843 29715 raft_consensus.cc:740] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.261006 29715 consensus_queue.cc:260] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.260713 29714 consensus_queue.cc:260] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.264955 29714 ts_tablet_manager.cc:1434] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:28:38.266655 29714 tablet_bootstrap.cc:492] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
W20260709 12:28:38.273542 29551 tablet.cc:2406] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:38.277010 29714 tablet_bootstrap.cc:654] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.288345 29699 heartbeater.cc:499] Master 127.12.45.126:34061 was elected leader, sending a full tablet report...
I20260709 12:28:38.288618 29625 heartbeater.cc:499] Master 127.12.45.126:34061 was elected leader, sending a full tablet report...
I20260709 12:28:38.288880 29716 ts_tablet_manager.cc:1434] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.034s
I20260709 12:28:38.290072 29716 tablet_bootstrap.cc:492] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.290881 29714 tablet_bootstrap.cc:492] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.291481 29714 ts_tablet_manager.cc:1403] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 12:28:38.295684 29714 raft_consensus.cc:359] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.296535 29714 raft_consensus.cc:385] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.296841 29714 raft_consensus.cc:740] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.297987 29714 consensus_queue.cc:260] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.299150 29715 ts_tablet_manager.cc:1434] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.044s	user 0.019s	sys 0.013s
I20260709 12:28:38.299312 29716 tablet_bootstrap.cc:654] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.302857 29714 ts_tablet_manager.cc:1434] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:28:38.304195 29714 tablet_bootstrap.cc:492] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.305074 29719 raft_consensus.cc:493] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.305751 29715 tablet_bootstrap.cc:492] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.305721 29719 raft_consensus.cc:515] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.317279 29715 tablet_bootstrap.cc:654] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.319629 29719 leader_election.cc:290] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
W20260709 12:28:38.325438 29470 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:38.339876 29723 raft_consensus.cc:493] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.341048 29723 raft_consensus.cc:515] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.345471 29723 leader_election.cc:290] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.362907 29714 tablet_bootstrap.cc:654] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.378116 29716 tablet_bootstrap.cc:492] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.379254 29716 ts_tablet_manager.cc:1403] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.089s	user 0.033s	sys 0.052s
I20260709 12:28:38.382957 29716 raft_consensus.cc:359] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.383841 29715 tablet_bootstrap.cc:492] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.383997 29716 raft_consensus.cc:385] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.384595 29715 ts_tablet_manager.cc:1403] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.079s	user 0.029s	sys 0.026s
I20260709 12:28:38.384541 29716 raft_consensus.cc:740] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.385624 29716 consensus_queue.cc:260] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.388060 29715 raft_consensus.cc:359] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.388833 29715 raft_consensus.cc:385] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.389137 29715 raft_consensus.cc:740] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.390033 29715 consensus_queue.cc:260] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.391394 29716 ts_tablet_manager.cc:1434] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:28:38.393025 29716 tablet_bootstrap.cc:492] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.393539 29715 ts_tablet_manager.cc:1434] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:28:38.394755 29715 tablet_bootstrap.cc:492] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.401612 29716 tablet_bootstrap.cc:654] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.402597 29715 tablet_bootstrap.cc:654] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.404762 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
W20260709 12:28:38.406852 29491 leader_election.cc:343] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.414424 29714 tablet_bootstrap.cc:492] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.414942 29716 tablet_bootstrap.cc:492] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.415235 29714 ts_tablet_manager.cc:1403] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.111s	user 0.009s	sys 0.006s
I20260709 12:28:38.415613 29716 ts_tablet_manager.cc:1403] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.001s
I20260709 12:28:38.417948 29722 raft_consensus.cc:493] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.418543 29722 raft_consensus.cc:515] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.419086 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.418996 29714 raft_consensus.cc:359] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.419876 29714 raft_consensus.cc:385] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.420177 29714 raft_consensus.cc:740] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.421008 29714 consensus_queue.cc:260] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:38.421571 29566 leader_election.cc:343] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.421666 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.424059 29722 leader_election.cc:290] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.425053 29715 tablet_bootstrap.cc:492] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
W20260709 12:28:38.425338 29490 leader_election.cc:343] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.425674 29715 ts_tablet_manager.cc:1403] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.010s
I20260709 12:28:38.425920 29490 leader_election.cc:304] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:38.427352 29719 raft_consensus.cc:2749] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:38.427248 29716 raft_consensus.cc:359] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.428406 29716 raft_consensus.cc:385] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.428809 29716 raft_consensus.cc:740] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.429745 29716 consensus_queue.cc:260] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.433121 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:38.433890 29674 raft_consensus.cc:2468] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:38.434346 29716 ts_tablet_manager.cc:1434] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:28:38.434211 29714 ts_tablet_manager.cc:1434] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:28:38.433090 29715 raft_consensus.cc:359] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.435418 29715 raft_consensus.cc:385] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.435711 29716 tablet_bootstrap.cc:492] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.435882 29715 raft_consensus.cc:740] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.437239 29715 consensus_queue.cc:260] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.439286 29714 tablet_bootstrap.cc:492] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.440310 29715 ts_tablet_manager.cc:1434] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:28:38.441524 29715 tablet_bootstrap.cc:492] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.443246 29566 leader_election.cc:304] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:38.444074 29716 tablet_bootstrap.cc:654] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.445919 29723 raft_consensus.cc:2804] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.446841 29723 raft_consensus.cc:493] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.447243 29723 raft_consensus.cc:3060] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.450033 29715 tablet_bootstrap.cc:654] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.463056 29716 tablet_bootstrap.cc:492] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.463690 29716 ts_tablet_manager.cc:1403] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.008s
I20260709 12:28:38.464361 29723 raft_consensus.cc:515] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.467130 29723 leader_election.cc:290] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.467322 29716 raft_consensus.cc:359] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.468120 29716 raft_consensus.cc:385] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.468438 29716 raft_consensus.cc:740] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.469372 29716 consensus_queue.cc:260] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.471760 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:38.474411 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:38.475116 29674 raft_consensus.cc:3060] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.476764 29716 ts_tablet_manager.cc:1434] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
W20260709 12:28:38.480515 29566 leader_election.cc:343] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.486828 29714 tablet_bootstrap.cc:654] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.486956 29674 raft_consensus.cc:2468] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:38.478204 29716 tablet_bootstrap.cc:492] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.488783 29566 leader_election.cc:304] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:38.490159 29723 raft_consensus.cc:2804] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.497112 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.498307 29525 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:38.499418 29723 raft_consensus.cc:697] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:38.500696 29723 consensus_queue.cc:237] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.501313 29640 leader_election.cc:304] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:38.502682 29722 raft_consensus.cc:2804] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.503264 29722 raft_consensus.cc:493] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.503585 29716 tablet_bootstrap.cc:654] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.503718 29722 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.517793 29715 tablet_bootstrap.cc:492] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.518486 29715 ts_tablet_manager.cc:1403] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.039s
I20260709 12:28:38.521993 29715 raft_consensus.cc:359] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.522867 29715 raft_consensus.cc:385] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.522702 29722 raft_consensus.cc:515] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.523309 29715 raft_consensus.cc:740] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.525427 29715 consensus_queue.cc:260] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.527173 29716 tablet_bootstrap.cc:492] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.528103 29716 ts_tablet_manager.cc:1403] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.010s
I20260709 12:28:38.528261 29715 ts_tablet_manager.cc:1434] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:28:38.529507 29715 tablet_bootstrap.cc:492] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.530133 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.530961 29600 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:38.533324 29716 raft_consensus.cc:359] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.535025 29716 raft_consensus.cc:385] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.535480 29716 raft_consensus.cc:740] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.537204 29715 tablet_bootstrap.cc:654] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.536872 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:38.536789 29716 consensus_queue.cc:260] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.537758 29525 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.540397 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:38.540560 29714 tablet_bootstrap.cc:492] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.541115 29600 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.541414 29714 ts_tablet_manager.cc:1403] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.012s
I20260709 12:28:38.546733 29714 raft_consensus.cc:359] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.548111 29714 raft_consensus.cc:385] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.548559 29714 raft_consensus.cc:740] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.550037 29525 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:38.550666 29714 consensus_queue.cc:260] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.551461 29640 leader_election.cc:304] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:38.556854 29722 leader_election.cc:290] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.557775 29722 raft_consensus.cc:2804] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.558331 29722 raft_consensus.cc:697] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:38.558516 29714 ts_tablet_manager.cc:1434] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.008s
I20260709 12:28:38.559842 29714 tablet_bootstrap.cc:492] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.559630 29722 consensus_queue.cc:237] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.561036 29715 tablet_bootstrap.cc:492] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.561606 29715 ts_tablet_manager.cc:1403] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.032s	user 0.005s	sys 0.009s
I20260709 12:28:38.565738 29715 raft_consensus.cc:359] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.566710 29715 raft_consensus.cc:385] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.567035 29715 raft_consensus.cc:740] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.568015 29715 consensus_queue.cc:260] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.556910 29716 ts_tablet_manager.cc:1434] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.028s	user 0.012s	sys 0.001s
I20260709 12:28:38.563391 29600 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:38.572296 29716 tablet_bootstrap.cc:492] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.575203 29715 ts_tablet_manager.cc:1434] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 12:28:38.586614 29715 tablet_bootstrap.cc:492] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.581691 29714 tablet_bootstrap.cc:654] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.588840 29716 tablet_bootstrap.cc:654] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.590858 29423 catalog_manager.cc:5697] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:38.600670 29715 tablet_bootstrap.cc:654] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.603497 29744 raft_consensus.cc:493] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.604211 29744 raft_consensus.cc:515] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.607081 29744 leader_election.cc:290] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.610781 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.612017 29714 tablet_bootstrap.cc:492] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.612356 29743 raft_consensus.cc:493] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.612726 29714 ts_tablet_manager.cc:1403] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.053s	user 0.013s	sys 0.008s
I20260709 12:28:38.612903 29743 raft_consensus.cc:515] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.617601 29714 raft_consensus.cc:359] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.618551 29714 raft_consensus.cc:385] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.619215 29714 raft_consensus.cc:740] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.619459 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.619679 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.620404 29714 consensus_queue.cc:260] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.618489 29421 catalog_manager.cc:5697] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:38.621685 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
W20260709 12:28:38.621793 29639 leader_election.cc:343] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.630963 29524 raft_consensus.cc:2468] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
W20260709 12:28:38.631210 29640 leader_election.cc:343] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.634122 29639 leader_election.cc:304] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42; no voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:38.622006 29743 leader_election.cc:290] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.636497 29714 ts_tablet_manager.cc:1434] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.002s
I20260709 12:28:38.641251 29714 tablet_bootstrap.cc:492] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.643723 29716 tablet_bootstrap.cc:492] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.644556 29716 ts_tablet_manager.cc:1403] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.073s	user 0.010s	sys 0.048s
I20260709 12:28:38.646977 29640 leader_election.cc:304] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:38.647176 29743 raft_consensus.cc:2749] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:38.647909 29639 leader_election.cc:343] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.648023 29744 raft_consensus.cc:2804] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.648588 29744 raft_consensus.cc:493] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.649006 29744 raft_consensus.cc:3060] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.650753 29714 tablet_bootstrap.cc:654] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.650472 29716 raft_consensus.cc:359] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.651314 29716 raft_consensus.cc:385] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.651656 29716 raft_consensus.cc:740] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.652516 29716 consensus_queue.cc:260] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.657967 29716 ts_tablet_manager.cc:1434] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:28:38.659158 29716 tablet_bootstrap.cc:492] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.662380 29744 raft_consensus.cc:515] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.674372 29744 leader_election.cc:290] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.674978 29716 tablet_bootstrap.cc:654] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.678788 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:38.682271 29715 tablet_bootstrap.cc:492] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.683315 29715 ts_tablet_manager.cc:1403] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.097s	user 0.027s	sys 0.003s
W20260709 12:28:38.685995 29639 leader_election.cc:343] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.688100 29715 raft_consensus.cc:359] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.688608 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:38.689172 29715 raft_consensus.cc:385] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.689404 29524 raft_consensus.cc:3060] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.689564 29715 raft_consensus.cc:740] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.691046 29715 consensus_queue.cc:260] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.697507 29715 ts_tablet_manager.cc:1434] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:28:38.701514 29716 tablet_bootstrap.cc:492] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.701855 29524 raft_consensus.cc:2468] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:38.702466 29716 ts_tablet_manager.cc:1403] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.000s
I20260709 12:28:38.703680 29640 leader_election.cc:304] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:38.710012 29744 raft_consensus.cc:2804] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.710966 29744 raft_consensus.cc:697] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:38.712255 29744 consensus_queue.cc:237] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.706974 29716 raft_consensus.cc:359] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.714444 29716 raft_consensus.cc:385] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.708215 29715 tablet_bootstrap.cc:492] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.714715 29714 tablet_bootstrap.cc:492] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.715523 29714 ts_tablet_manager.cc:1403] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.075s	user 0.035s	sys 0.015s
I20260709 12:28:38.714797 29716 raft_consensus.cc:740] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.721206 29716 consensus_queue.cc:260] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.719250 29714 raft_consensus.cc:359] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.723567 29715 tablet_bootstrap.cc:654] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.723958 29714 raft_consensus.cc:385] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.724514 29716 ts_tablet_manager.cc:1434] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.001s
I20260709 12:28:38.725756 29716 tablet_bootstrap.cc:492] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.724359 29714 raft_consensus.cc:740] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.727208 29714 consensus_queue.cc:260] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.732137 29714 ts_tablet_manager.cc:1434] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.001s
I20260709 12:28:38.733788 29714 tablet_bootstrap.cc:492] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.734566 29719 raft_consensus.cc:493] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.735345 29719 raft_consensus.cc:515] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.739145 29421 catalog_manager.cc:5697] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:38.743659 29714 tablet_bootstrap.cc:654] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.743451 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49def8d7de014ea883f046b20d82c270" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.744338 29600 raft_consensus.cc:2468] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 0.
I20260709 12:28:38.745841 29719 leader_election.cc:290] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.746430 29490 leader_election.cc:304] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:38.749298 29716 tablet_bootstrap.cc:654] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.752384 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49def8d7de014ea883f046b20d82c270" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:38.753111 29719 raft_consensus.cc:2804] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.753201 29674 raft_consensus.cc:2468] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 0.
I20260709 12:28:38.753641 29719 raft_consensus.cc:493] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.753918 29715 tablet_bootstrap.cc:492] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.754101 29719 raft_consensus.cc:3060] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.754817 29715 ts_tablet_manager.cc:1403] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.013s
I20260709 12:28:38.758417 29744 raft_consensus.cc:493] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.758416 29715 raft_consensus.cc:359] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.759742 29715 raft_consensus.cc:385] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.759300 29744 raft_consensus.cc:515] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.760200 29715 raft_consensus.cc:740] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.761186 29715 consensus_queue.cc:260] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.763759 29715 ts_tablet_manager.cc:1434] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:28:38.765344 29715 tablet_bootstrap.cc:492] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.763684 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.766732 29524 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:38.768117 29640 leader_election.cc:304] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:38.769068 29743 raft_consensus.cc:2804] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.769447 29743 raft_consensus.cc:493] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.769833 29743 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.772123 29719 raft_consensus.cc:515] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.774019 29715 tablet_bootstrap.cc:654] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.775460 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.776226 29600 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:38.774039 29744 leader_election.cc:290] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.785426 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49def8d7de014ea883f046b20d82c270" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:38.786237 29600 raft_consensus.cc:3060] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.787683 29715 tablet_bootstrap.cc:492] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.788335 29715 ts_tablet_manager.cc:1403] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.009s
I20260709 12:28:38.791216 29743 raft_consensus.cc:515] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.792631 29715 raft_consensus.cc:359] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.793463 29715 raft_consensus.cc:385] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.793778 29715 raft_consensus.cc:740] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.794749 29715 consensus_queue.cc:260] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.795723 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:38.796028 29600 raft_consensus.cc:2468] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:38.796662 29524 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.797528 29490 leader_election.cc:304] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:38.801131 29715 ts_tablet_manager.cc:1434] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:28:38.802758 29715 tablet_bootstrap.cc:492] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:38.804847 29714 tablet_bootstrap.cc:492] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.805687 29714 ts_tablet_manager.cc:1403] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.002s
I20260709 12:28:38.805733 29524 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:38.810314 29640 leader_election.cc:304] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:38.810575 29715 tablet_bootstrap.cc:654] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.811525 29723 raft_consensus.cc:493] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.811636 29744 raft_consensus.cc:2804] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.812445 29719 leader_election.cc:290] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.812201 29723 raft_consensus.cc:515] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.813179 29719 raft_consensus.cc:493] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.813683 29719 raft_consensus.cc:515] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.818123 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49def8d7de014ea883f046b20d82c270" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:38.818845 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:38.820257 29673 raft_consensus.cc:2468] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:38.821250 29762 raft_consensus.cc:2804] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.822052 29762 raft_consensus.cc:697] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:38.818917 29674 raft_consensus.cc:3060] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.824517 29719 leader_election.cc:290] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.825340 29739 raft_consensus.cc:493] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.825929 29739 raft_consensus.cc:515] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.823266 29762 consensus_queue.cc:237] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.828588 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:38.829279 29673 raft_consensus.cc:2468] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 0.
I20260709 12:28:38.822245 29566 leader_election.cc:304] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:38.830482 29491 leader_election.cc:304] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:38.831712 29739 leader_election.cc:290] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.812281 29744 raft_consensus.cc:697] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:38.831764 29723 leader_election.cc:290] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.832669 29739 raft_consensus.cc:2804] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.832851 29761 raft_consensus.cc:2804] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.833097 29739 raft_consensus.cc:493] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.832309 29716 tablet_bootstrap.cc:492] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.833586 29739 raft_consensus.cc:3060] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.833310 29744 consensus_queue.cc:237] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.835691 29743 leader_election.cc:290] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:38.833501 29761 raft_consensus.cc:493] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.836654 29761 raft_consensus.cc:3060] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.837304 29715 tablet_bootstrap.cc:492] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:38.837138 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:38.838016 29600 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.838075 29715 ts_tablet_manager.cc:1403] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.012s
I20260709 12:28:38.839972 29716 ts_tablet_manager.cc:1403] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.114s	user 0.017s	sys 0.004s
I20260709 12:28:38.841502 29715 raft_consensus.cc:359] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.842676 29715 raft_consensus.cc:385] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.843010 29715 raft_consensus.cc:740] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.843096 29716 raft_consensus.cc:359] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.841603 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.844035 29715 consensus_queue.cc:260] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.811642 29714 raft_consensus.cc:359] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.847040 29714 raft_consensus.cc:385] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.847365 29715 ts_tablet_manager.cc:1434] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:28:38.843896 29716 raft_consensus.cc:385] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.847930 29716 raft_consensus.cc:740] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.847334 29714 raft_consensus.cc:740] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.849056 29714 consensus_queue.cc:260] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.849591 29716 consensus_queue.cc:260] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.851817 29714 ts_tablet_manager.cc:1434] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.005s
I20260709 12:28:38.853482 29714 tablet_bootstrap.cc:492] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:38.853701 29674 raft_consensus.cc:2468] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:38.854728 29761 raft_consensus.cc:515] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.856070 29524 raft_consensus.cc:2393] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76171565ecf74c9aac0deaac7c291900 in current term 1: Already voted for candidate 710651b45fd047d9aea0f26c77946f9f in this term.
I20260709 12:28:38.857036 29716 ts_tablet_manager.cc:1434] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:28:38.858228 29716 tablet_bootstrap.cc:492] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:38.867172 29600 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:38.861240 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.868659 29761 leader_election.cc:290] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.860522 29714 tablet_bootstrap.cc:654] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.865865 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:38.864391 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:38.868847 29762 raft_consensus.cc:493] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.871698 29762 raft_consensus.cc:515] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.872218 29673 raft_consensus.cc:2468] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:38.874186 29566 leader_election.cc:304] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:38.874776 29762 leader_election.cc:290] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.876047 29764 raft_consensus.cc:2804] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.879901 29764 raft_consensus.cc:493] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.880753 29716 tablet_bootstrap.cc:654] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:38.877290 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:38.882102 29674 raft_consensus.cc:3060] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.883836 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:38.880707 29714 tablet_bootstrap.cc:492] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:38.862908 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:38.884899 29598 raft_consensus.cc:2468] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 0.
I20260709 12:28:38.885203 29524 raft_consensus.cc:2468] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:38.878317 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:38.878914 29421 catalog_manager.cc:5697] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:38.881536 29764 raft_consensus.cc:3060] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.889304 29490 leader_election.cc:304] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:38.890532 29762 raft_consensus.cc:2804] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:38.891029 29762 raft_consensus.cc:493] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.891932 29762 raft_consensus.cc:3060] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.893580 29739 raft_consensus.cc:515] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.898020 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:38.899935 29739 leader_election.cc:290] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.900688 29599 raft_consensus.cc:2393] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 710651b45fd047d9aea0f26c77946f9f in current term 1: Already voted for candidate 76171565ecf74c9aac0deaac7c291900 in this term.
I20260709 12:28:38.900785 29714 ts_tablet_manager.cc:1403] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.005s
I20260709 12:28:38.888422 29423 catalog_manager.cc:5697] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:38.911134 29491 leader_election.cc:343] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.914100 29714 raft_consensus.cc:359] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.915611 29714 raft_consensus.cc:385] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.916059 29714 raft_consensus.cc:740] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.917135 29714 consensus_queue.cc:260] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.918077 29762 raft_consensus.cc:515] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.920599 29762 leader_election.cc:290] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.922125 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:38.923671 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba3c94d505a4b7b86347de064750711" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
W20260709 12:28:38.923964 29491 leader_election.cc:343] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:38.921998 29525 raft_consensus.cc:3060] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.928233 29714 ts_tablet_manager.cc:1434] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.003s
I20260709 12:28:38.935921 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:38.937126 29599 raft_consensus.cc:3060] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.941414 29739 consensus_queue.cc:1048] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:38.942796 29524 raft_consensus.cc:2393] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 76171565ecf74c9aac0deaac7c291900 in current term 1: Already voted for candidate 710651b45fd047d9aea0f26c77946f9f in this term.
I20260709 12:28:38.944988 29566 leader_election.cc:304] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:38.946467 29739 raft_consensus.cc:2749] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:38.952929 29599 raft_consensus.cc:2468] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:38.955822 29490 leader_election.cc:304] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:38.957998 29762 raft_consensus.cc:2804] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.958743 29762 raft_consensus.cc:697] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:38.959941 29762 consensus_queue.cc:237] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.963428 29739 consensus_queue.cc:1048] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:38.968493 29764 raft_consensus.cc:515] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.970965 29764 leader_election.cc:290] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:38.972527 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:38.972872 29672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:38.973356 29525 raft_consensus.cc:3060] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.973421 29716 tablet_bootstrap.cc:492] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:38.974172 29716 ts_tablet_manager.cc:1403] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.116s	user 0.048s	sys 0.040s
I20260709 12:28:38.973476 29672 raft_consensus.cc:3060] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:38.981473 29716 raft_consensus.cc:359] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.986737 29716 raft_consensus.cc:385] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:38.987159 29716 raft_consensus.cc:740] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:38.987372 29672 raft_consensus.cc:2468] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:38.989073 29566 leader_election.cc:304] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:38.988221 29716 consensus_queue.cc:260] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:38.988217 29674 raft_consensus.cc:2468] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:38.991225 29764 raft_consensus.cc:2804] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:38.993515 29764 raft_consensus.cc:697] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:38.995432 29491 leader_election.cc:304] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:38.996533 29716 ts_tablet_manager.cc:1434] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:28:38.994446 29764 consensus_queue.cc:237] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.001154 29723 raft_consensus.cc:493] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.001637 29762 raft_consensus.cc:2804] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:39.001683 29723 raft_consensus.cc:515] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.004673 29723 leader_election.cc:290] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:39.005846 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:39.006565 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:39.006664 29673 raft_consensus.cc:2468] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:39.007400 29523 raft_consensus.cc:2468] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:39.008175 29566 leader_election.cc:304] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:39.009289 29774 raft_consensus.cc:2804] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:39.009754 29774 raft_consensus.cc:493] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.010145 29774 raft_consensus.cc:3060] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:39.001207 29423 catalog_manager.cc:5697] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:39.022874 29774 raft_consensus.cc:515] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.002213 29762 raft_consensus.cc:697] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:39.027348 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:39.028039 29523 raft_consensus.cc:3060] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:39.028115 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:39.028841 29673 raft_consensus.cc:3060] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:39.029824 29762 consensus_queue.cc:237] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.037369 29774 leader_election.cc:290] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:39.044030 29421 catalog_manager.cc:5697] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:39.038412 29525 raft_consensus.cc:2468] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:39.052385 29523 raft_consensus.cc:2468] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:39.053910 29566 leader_election.cc:304] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:39.056773 29744 consensus_queue.cc:1048] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:39.059386 29673 raft_consensus.cc:2468] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:39.061836 29774 raft_consensus.cc:2804] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:39.062718 29774 raft_consensus.cc:697] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:39.067195 29774 consensus_queue.cc:237] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.075459 29419 catalog_manager.cc:5697] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:39.082136 29743 consensus_queue.cc:1048] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:39.093925 29421 catalog_manager.cc:5697] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:39.123714 29599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:43310
I20260709 12:28:39.124586 29599 raft_consensus.cc:606] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Received request to transfer leadership to TS 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:39.131229 29599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:43310
I20260709 12:28:39.131889 29599 raft_consensus.cc:606] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Received request to transfer leadership to TS 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:39.147449 29674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
mode: GRACEFUL
new_leader_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:36796
I20260709 12:28:39.148109 29674 raft_consensus.cc:606] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Received request to transfer leadership to TS 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:39.168812 29525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:44920
I20260709 12:28:39.169425 29525 raft_consensus.cc:606] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:39.176085 29674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:36796
I20260709 12:28:39.176817 29674 raft_consensus.cc:606] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:39.183724 29525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49def8d7de014ea883f046b20d82c270"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:44920
I20260709 12:28:39.184345 29525 raft_consensus.cc:606] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:39.190546 29674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:36796
I20260709 12:28:39.191190 29674 raft_consensus.cc:606] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:39.198124 29599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:43310
I20260709 12:28:39.198799 29599 raft_consensus.cc:606] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Received request to transfer leadership to TS 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:39.220234 29743 consensus_queue.cc:1048] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:39.232630 29599 raft_consensus.cc:3060] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:39.242214 29774 raft_consensus.cc:493] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.242214 29743 consensus_queue.cc:1048] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:39.242936 29774 raft_consensus.cc:515] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.244984 29774 leader_election.cc:290] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:39.245787 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:39.246279 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:39.248210 29566 leader_election.cc:304] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:39.249341 29739 raft_consensus.cc:2749] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:39.251757 29744 raft_consensus.cc:993] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:39.252349 29743 raft_consensus.cc:1081] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:39.254463 29599 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43302
I20260709 12:28:39.255201 29599 raft_consensus.cc:493] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:39.255596 29599 raft_consensus.cc:3060] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:39.261660 29599 raft_consensus.cc:515] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.263922 29599 leader_election.cc:290] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:39.264791 29525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:39.265591 29525 raft_consensus.cc:3060] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:39.265625 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f70f573c7ac4e3cb0cf2a3ff2d93cdf" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:39.266359 29674 raft_consensus.cc:3055] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:39.266690 29674 raft_consensus.cc:740] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:39.267549 29674 consensus_queue.cc:260] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.269510 29674 raft_consensus.cc:3060] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:39.276494 29525 raft_consensus.cc:2468] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:39.278285 29566 leader_election.cc:304] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:39.278545 29762 consensus_queue.cc:1048] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:39.280467 29739 raft_consensus.cc:2804] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:39.281067 29739 raft_consensus.cc:697] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:39.282058 29739 consensus_queue.cc:237] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:39.282385 29674 raft_consensus.cc:2468] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:39.282758 29743 consensus_queue.cc:1048] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.031481 29796 raft_consensus.cc:993] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: : Instructing follower 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:40.032042 29797 raft_consensus.cc:1081] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Signalling peer 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:40.033952 29523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:44916
I20260709 12:28:40.034664 29523 raft_consensus.cc:493] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.035017 29523 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.037560 29797 raft_consensus.cc:493] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.038524 29797 raft_consensus.cc:515] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.041286 29797 leader_election.cc:290] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:40.042313 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:40.043568 29523 raft_consensus.cc:515] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.046856 29523 leader_election.cc:290] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:40.048687 29672 raft_consensus.cc:3060] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:40.048838 29671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:40.049660 29671 raft_consensus.cc:3055] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.050004 29671 raft_consensus.cc:740] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:40.050920 29671 consensus_queue.cc:260] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.052141 29671 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.054025 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:40.055073 29599 raft_consensus.cc:2393] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ec05c84795f4624b9a3e1f88bee2e42 in current term 1: Already voted for candidate 710651b45fd047d9aea0f26c77946f9f in this term.
I20260709 12:28:40.057360 29789 consensus_queue.cc:1048] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:40.064455 29597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:40.065248 29597 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:40.065918 29525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.008s	sys 0.016s
W20260709 12:28:40.066413 29525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.008s	sys 0.018s
I20260709 12:28:40.068712 29671 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:40.079123 29525 raft_consensus.cc:1275] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.081521 29793 consensus_queue.cc:1048] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.082742 29790 consensus_queue.cc:1048] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:40.083002 29639 leader_election.cc:304] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42; no voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:40.084432 29790 consensus_queue.cc:1048] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.082185 29794 consensus_queue.cc:1048] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:28:40.088197 29794 consensus_queue.cc:1048] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.090533 29491 leader_election.cc:304] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:40.091727 29789 consensus_queue.cc:1048] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.092612 29796 raft_consensus.cc:2749] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:40.093753 29789 raft_consensus.cc:2804] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.102708 29789 raft_consensus.cc:697] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:40.094408 29793 consensus_queue.cc:1048] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.096354 29796 consensus_queue.cc:1048] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.097687 29674 raft_consensus.cc:1275] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.103957 29789 consensus_queue.cc:237] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.109367 29801 consensus_queue.cc:1048] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:40.121053 29793 raft_consensus.cc:993] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:40.121855 29793 raft_consensus.cc:1081] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Signalling peer 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:40.144353 29597 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:40.151094 29421 catalog_manager.cc:5697] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:40.167255 29566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33633, user_credentials={real_user=slave}} blocked reactor thread for 81118us
I20260709 12:28:40.169660 29795 consensus_queue.cc:1048] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:40.176717 29423 catalog_manager.cc:5697] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 1 to 2, leader changed from 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67) to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.187546 29522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:28:40.188360 29522 raft_consensus.cc:493] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.189774 29522 raft_consensus.cc:3060] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.239629 29795 consensus_queue.cc:1048] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:40.235970 29522 raft_consensus.cc:515] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.243016 29522 leader_election.cc:290] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:40.255331 29799 raft_consensus.cc:993] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:40.256160 29789 raft_consensus.cc:993] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:40.256333 29799 raft_consensus.cc:1081] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:40.256830 29789 raft_consensus.cc:1081] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:40.282039 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:40.284674 29673 raft_consensus.cc:3060] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.298960 29597 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43302
I20260709 12:28:40.299868 29597 raft_consensus.cc:493] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.300297 29597 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.310673 29795 raft_consensus.cc:993] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:40.323247 29795 raft_consensus.cc:1081] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Signalling peer 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:40.328333 29673 raft_consensus.cc:2468] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:40.332607 29491 leader_election.cc:304] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:40.335250 29801 raft_consensus.cc:2804] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.336026 29801 raft_consensus.cc:697] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:40.340931 29801 consensus_queue.cc:237] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.343765 29597 raft_consensus.cc:515] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.346333 29597 leader_election.cc:290] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
W20260709 12:28:40.338812 29566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.65:33633, user_credentials={real_user=slave}} blocked reactor thread for 66828.7us
I20260709 12:28:40.349521 29600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0b1aa4fd2654e4d95e99c057a036d1c" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:40.350347 29600 raft_consensus.cc:3055] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.350682 29600 raft_consensus.cc:740] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:40.350716 29789 raft_consensus.cc:993] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:40.351387 29789 raft_consensus.cc:1081] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:40.351706 29600 consensus_queue.cc:260] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.353281 29600 raft_consensus.cc:3060] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.349552 29598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49def8d7de014ea883f046b20d82c270"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43300
I20260709 12:28:40.367887 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:40.368906 29522 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.370544 29598 raft_consensus.cc:493] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.371258 29598 raft_consensus.cc:3060] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.379186 29673 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:36784
I20260709 12:28:40.383814 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:40.397537 29674 raft_consensus.cc:3055] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.398036 29674 raft_consensus.cc:740] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:40.401258 29827 raft_consensus.cc:993] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:40.401913 29827 raft_consensus.cc:1081] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Signalling peer 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:40.401060 29674 consensus_queue.cc:260] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.404021 29674 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.413475 29673 raft_consensus.cc:493] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.414160 29673 raft_consensus.cc:3060] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.417855 29597 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43300
I20260709 12:28:40.419001 29597 raft_consensus.cc:493] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.419543 29597 raft_consensus.cc:3060] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.424222 29524 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:28:40.426272 29673 raft_consensus.cc:515] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.429620 29674 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:40.433985 29521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:40.440107 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a178b0a2291d45609f26a615cfbc0289" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:40.440935 29596 raft_consensus.cc:3055] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.441360 29596 raft_consensus.cc:740] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:40.445824 29596 consensus_queue.cc:260] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.447492 29596 raft_consensus.cc:3060] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.447597 29566 leader_election.cc:304] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
W20260709 12:28:40.449471 29794 consensus_queue.cc:1175] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:28:40.449913 29820 raft_consensus.cc:2804] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.450490 29820 raft_consensus.cc:697] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:40.451318 29820 consensus_queue.cc:237] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.454402 29673 leader_election.cc:290] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:40.454689 29597 raft_consensus.cc:515] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.456092 29599 raft_consensus.cc:1240] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Rejecting Update request from peer 710651b45fd047d9aea0f26c77946f9f for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:40.456947 29597 leader_election.cc:290] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:40.445761 29600 raft_consensus.cc:2468] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:40.458287 29522 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:40.458532 29524 raft_consensus.cc:493] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:40.459148 29524 raft_consensus.cc:3060] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.461750 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:40.462568 29522 raft_consensus.cc:3055] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.462946 29522 raft_consensus.cc:740] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:40.463984 29522 consensus_queue.cc:260] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.464851 29789 consensus_queue.cc:1059] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:40.466401 29522 raft_consensus.cc:3060] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.468289 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50fb89a8625342928a38a5a005eb7383" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:40.469386 29674 raft_consensus.cc:3060] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.475231 29522 raft_consensus.cc:2468] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:40.477103 29566 leader_election.cc:304] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:40.471624 29524 raft_consensus.cc:515] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.478721 29820 raft_consensus.cc:2804] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.479240 29820 raft_consensus.cc:697] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:40.480100 29820 consensus_queue.cc:237] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.484357 29674 raft_consensus.cc:2468] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:40.488140 29524 leader_election.cc:290] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:40.488914 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:40.489673 29599 raft_consensus.cc:3055] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.489848 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "badb69aae97a4c3ba2252e42aa7e01e7" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:40.490494 29599 raft_consensus.cc:740] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:40.490722 29674 raft_consensus.cc:3060] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.491498 29599 consensus_queue.cc:260] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.492929 29599 raft_consensus.cc:3060] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.494997 29596 raft_consensus.cc:2468] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 2.
I20260709 12:28:40.497531 29639 leader_election.cc:304] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:40.499452 29799 raft_consensus.cc:2804] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.499965 29799 raft_consensus.cc:697] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:40.500880 29799 consensus_queue.cc:237] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.504752 29599 raft_consensus.cc:2468] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:40.505916 29598 raft_consensus.cc:515] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.506664 29490 leader_election.cc:304] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:40.507899 29834 raft_consensus.cc:2804] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.508859 29834 raft_consensus.cc:697] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:40.509846 29834 consensus_queue.cc:237] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.507674 29420 catalog_manager.cc:5697] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 1 to 2, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.521539 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49def8d7de014ea883f046b20d82c270" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:40.522405 29673 raft_consensus.cc:3060] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.524722 29599 raft_consensus.cc:1275] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.527827 29834 consensus_queue.cc:1048] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.528800 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49def8d7de014ea883f046b20d82c270" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:40.529628 29524 raft_consensus.cc:3055] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:40.529971 29524 raft_consensus.cc:740] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:40.532584 29674 raft_consensus.cc:2468] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:40.531644 29421 catalog_manager.cc:5697] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 1 to 2, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:40.537248 29673 raft_consensus.cc:2468] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:40.539939 29566 leader_election.cc:304] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:40.537487 29598 leader_election.cc:290] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:40.540897 29820 raft_consensus.cc:2804] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:40.541550 29820 raft_consensus.cc:697] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:40.542744 29820 consensus_queue.cc:237] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.549129 29524 consensus_queue.cc:260] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:40.549916 29673 raft_consensus.cc:1275] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.551844 29833 consensus_queue.cc:1048] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:40.552304 29524 raft_consensus.cc:3060] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:40.563095 29524 raft_consensus.cc:2468] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:40.565027 29421 catalog_manager.cc:5697] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 1 to 2, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.582933 29423 catalog_manager.cc:5697] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 710651b45fd047d9aea0f26c77946f9f (127.12.45.65) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.584782 29423 catalog_manager.cc:5697] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 710651b45fd047d9aea0f26c77946f9f (127.12.45.65) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.586364 29423 catalog_manager.cc:5697] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.781623 29598 raft_consensus.cc:1275] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.783366 29833 consensus_queue.cc:1048] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:40.793359 29673 raft_consensus.cc:1275] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.795261 29834 consensus_queue.cc:1048] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.921630 29673 raft_consensus.cc:1275] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.921774 29524 raft_consensus.cc:1275] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.923473 29804 consensus_queue.cc:1048] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.923651 29598 raft_consensus.cc:1275] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.924238 29820 consensus_queue.cc:1048] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.925203 29834 consensus_queue.cc:1048] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.941298 29673 raft_consensus.cc:1275] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.944063 29834 consensus_queue.cc:1048] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.971952 29673 raft_consensus.cc:1275] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.973505 29820 consensus_queue.cc:1048] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:40.982746 29524 raft_consensus.cc:1275] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:40.985193 29804 consensus_queue.cc:1048] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:41.005918 29524 raft_consensus.cc:1275] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:41.007422 29804 consensus_queue.cc:1048] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:41.017450 29673 raft_consensus.cc:1275] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:28:41.018391 29695 debug-util.cc:398] Leaking SignalData structure 0x7b0800088c20 after lost signal to thread 12472
W20260709 12:28:41.019922 29695 debug-util.cc:398] Leaking SignalData structure 0x7b0800282560 after lost signal to thread 29457
I20260709 12:28:41.020759 29804 consensus_queue.cc:1048] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:28:41.021762 29695 debug-util.cc:398] Leaking SignalData structure 0x7b08002e5500 after lost signal to thread 29549
W20260709 12:28:41.023849 29695 debug-util.cc:398] Leaking SignalData structure 0x7b08001b8000 after lost signal to thread 29624
W20260709 12:28:41.025419 29695 debug-util.cc:398] Leaking SignalData structure 0x7b08001442c0 after lost signal to thread 29698
I20260709 12:28:41.066030 29524 raft_consensus.cc:3060] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:41.073019 29524 raft_consensus.cc:1275] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:41.075331 29841 consensus_queue.cc:1048] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:41.088474 29598 raft_consensus.cc:1275] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:41.090521 29842 consensus_queue.cc:1048] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:42.902808 29420 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48874:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:42.906010 29420 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:28:42.979022 29515 tablet_service.cc:1511] Processing CreateTablet for tablet 53bcb6edc99b479a9355cbededeef31d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:42.979921 29585 tablet_service.cc:1511] Processing CreateTablet for tablet 53bcb6edc99b479a9355cbededeef31d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:42.980660 29515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53bcb6edc99b479a9355cbededeef31d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:42.981271 29585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53bcb6edc99b479a9355cbededeef31d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:42.984612 29582 tablet_service.cc:1511] Processing CreateTablet for tablet fed0a24223a74746836bd12e62bd89fd (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:42.986168 29663 tablet_service.cc:1511] Processing CreateTablet for tablet fed0a24223a74746836bd12e62bd89fd (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:42.987739 29663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fed0a24223a74746836bd12e62bd89fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:42.987298 29660 tablet_service.cc:1511] Processing CreateTablet for tablet 53bcb6edc99b479a9355cbededeef31d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:28:42.988875 29660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53bcb6edc99b479a9355cbededeef31d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:42.991238 29508 tablet_service.cc:1511] Processing CreateTablet for tablet fed0a24223a74746836bd12e62bd89fd (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:28:42.992631 29508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fed0a24223a74746836bd12e62bd89fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:42.992898 29510 tablet_service.cc:1511] Processing CreateTablet for tablet f7d34d78784b4957ac23aaeccd5ce19d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:42.994315 29510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d34d78784b4957ac23aaeccd5ce19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.002245 29658 tablet_service.cc:1511] Processing CreateTablet for tablet 931a13e43aa949649d9255a9ea268c6d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:42.991289 29661 tablet_service.cc:1511] Processing CreateTablet for tablet f7d34d78784b4957ac23aaeccd5ce19d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:43.007193 29513 tablet_service.cc:1511] Processing CreateTablet for tablet 15d244e5d6f04d578e5464a7a1fd942e (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:43.003847 29583 tablet_service.cc:1511] Processing CreateTablet for tablet f7d34d78784b4957ac23aaeccd5ce19d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:28:43.005138 29657 tablet_service.cc:1511] Processing CreateTablet for tablet 15d244e5d6f04d578e5464a7a1fd942e (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:43.008795 29513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d244e5d6f04d578e5464a7a1fd942e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.004108 29658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931a13e43aa949649d9255a9ea268c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.011492 29582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fed0a24223a74746836bd12e62bd89fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.013720 29583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d34d78784b4957ac23aaeccd5ce19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.014719 29661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7d34d78784b4957ac23aaeccd5ce19d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.017485 29657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d244e5d6f04d578e5464a7a1fd942e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.025988 29586 tablet_service.cc:1511] Processing CreateTablet for tablet 15d244e5d6f04d578e5464a7a1fd942e (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:28:43.027685 29586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d244e5d6f04d578e5464a7a1fd942e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.030464 29659 tablet_service.cc:1511] Processing CreateTablet for tablet 3c1925f868da45debb0db3995589d54f (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:43.032310 29659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1925f868da45debb0db3995589d54f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.032698 29656 tablet_service.cc:1511] Processing CreateTablet for tablet e66e5dc3d2054e9bafa0168c32a8bf02 (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:43.034096 29656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66e5dc3d2054e9bafa0168c32a8bf02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.040855 29589 tablet_service.cc:1511] Processing CreateTablet for tablet 3c1925f868da45debb0db3995589d54f (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:43.040745 29584 tablet_service.cc:1511] Processing CreateTablet for tablet 931a13e43aa949649d9255a9ea268c6d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:43.043678 29584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931a13e43aa949649d9255a9ea268c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.043910 29512 tablet_service.cc:1511] Processing CreateTablet for tablet 931a13e43aa949649d9255a9ea268c6d (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:28:43.045433 29512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 931a13e43aa949649d9255a9ea268c6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.045539 29589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1925f868da45debb0db3995589d54f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.047531 29588 tablet_service.cc:1511] Processing CreateTablet for tablet e66e5dc3d2054e9bafa0168c32a8bf02 (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:43.049145 29588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66e5dc3d2054e9bafa0168c32a8bf02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.060649 29664 tablet_service.cc:1511] Processing CreateTablet for tablet cf0ce27942104fccbdf7342f7764de73 (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:43.062511 29664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf0ce27942104fccbdf7342f7764de73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.071878 29858 tablet_bootstrap.cc:492] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.081786 29858 tablet_bootstrap.cc:654] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.081341 29590 tablet_service.cc:1511] Processing CreateTablet for tablet cf0ce27942104fccbdf7342f7764de73 (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:43.081549 29587 tablet_service.cc:1511] Processing CreateTablet for tablet bbb0d544ce804da9a5cdc8d1d287837c (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:43.083873 29511 tablet_service.cc:1511] Processing CreateTablet for tablet 3c1925f868da45debb0db3995589d54f (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:28:43.084117 29509 tablet_service.cc:1511] Processing CreateTablet for tablet e66e5dc3d2054e9bafa0168c32a8bf02 (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:28:43.085758 29509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66e5dc3d2054e9bafa0168c32a8bf02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.087978 29587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb0d544ce804da9a5cdc8d1d287837c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.089135 29656 tablet_service.cc:1511] Processing CreateTablet for tablet bbb0d544ce804da9a5cdc8d1d287837c (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:43.090924 29656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb0d544ce804da9a5cdc8d1d287837c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.092051 29507 tablet_service.cc:1511] Processing CreateTablet for tablet bbb0d544ce804da9a5cdc8d1d287837c (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:28:43.093497 29507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb0d544ce804da9a5cdc8d1d287837c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.095088 29511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1925f868da45debb0db3995589d54f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.099663 29514 tablet_service.cc:1511] Processing CreateTablet for tablet cf0ce27942104fccbdf7342f7764de73 (DEFAULT_TABLE table=soft_deleted_table [id=99a014138d86452999a83b3183e7429b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:28:43.101783 29514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf0ce27942104fccbdf7342f7764de73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.090924 29590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf0ce27942104fccbdf7342f7764de73. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:43.189057 29470 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:43.260174 29858 tablet_bootstrap.cc:492] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.260914 29858 ts_tablet_manager.cc:1403] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.189s	user 0.016s	sys 0.035s
I20260709 12:28:43.268134 29858 raft_consensus.cc:359] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.268985 29858 raft_consensus.cc:385] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.269294 29858 raft_consensus.cc:740] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.276407 29858 consensus_queue.cc:260] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.280143 29858 ts_tablet_manager.cc:1434] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:28:43.284315 29860 tablet_bootstrap.cc:492] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.307781 29861 tablet_bootstrap.cc:492] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.311249 29860 tablet_bootstrap.cc:654] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.316977 29861 tablet_bootstrap.cc:654] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.317838 29858 tablet_bootstrap.cc:492] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.331164 29860 tablet_bootstrap.cc:492] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.331897 29860 ts_tablet_manager.cc:1403] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.012s
I20260709 12:28:43.332751 29858 tablet_bootstrap.cc:654] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.335551 29860 raft_consensus.cc:359] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.336405 29860 raft_consensus.cc:385] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.336694 29860 raft_consensus.cc:740] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.337532 29861 tablet_bootstrap.cc:492] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:43.338155 29861 ts_tablet_manager.cc:1403] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.000s
I20260709 12:28:43.337747 29860 consensus_queue.cc:260] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.340579 29860 ts_tablet_manager.cc:1434] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:43.341746 29860 tablet_bootstrap.cc:492] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.341683 29861 raft_consensus.cc:359] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.344553 29861 raft_consensus.cc:385] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.344887 29861 raft_consensus.cc:740] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.345995 29861 consensus_queue.cc:260] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.349205 29861 ts_tablet_manager.cc:1434] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:28:43.350184 29860 tablet_bootstrap.cc:654] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.352654 29861 tablet_bootstrap.cc:492] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.365343 29861 tablet_bootstrap.cc:654] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.368721 29858 tablet_bootstrap.cc:492] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.369580 29858 ts_tablet_manager.cc:1403] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.010s
I20260709 12:28:43.373378 29858 raft_consensus.cc:359] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.374137 29858 raft_consensus.cc:385] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.379935 29858 raft_consensus.cc:740] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.381562 29858 consensus_queue.cc:260] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.384203 29858 ts_tablet_manager.cc:1434] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 12:28:43.388489 29858 tablet_bootstrap.cc:492] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.411227 29858 tablet_bootstrap.cc:654] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.413257 29860 tablet_bootstrap.cc:492] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.414026 29860 ts_tablet_manager.cc:1403] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.036s
I20260709 12:28:43.416492 29861 tablet_bootstrap.cc:492] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:43.417033 29858 tablet_bootstrap.cc:492] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.417320 29861 ts_tablet_manager.cc:1403] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.012s
I20260709 12:28:43.417680 29858 ts_tablet_manager.cc:1403] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.000s
I20260709 12:28:43.418011 29860 raft_consensus.cc:359] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.418882 29860 raft_consensus.cc:385] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.419224 29860 raft_consensus.cc:740] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.420253 29860 consensus_queue.cc:260] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.421293 29858 raft_consensus.cc:359] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.422080 29858 raft_consensus.cc:385] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.422407 29858 raft_consensus.cc:740] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.422868 29860 ts_tablet_manager.cc:1434] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:28:43.423267 29858 consensus_queue.cc:260] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.424919 29860 tablet_bootstrap.cc:492] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.426071 29858 ts_tablet_manager.cc:1434] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:28:43.432564 29858 tablet_bootstrap.cc:492] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.438462 29860 tablet_bootstrap.cc:654] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.448210 29858 tablet_bootstrap.cc:654] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.426864 29861 raft_consensus.cc:359] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.450162 29861 raft_consensus.cc:385] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.450593 29861 raft_consensus.cc:740] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.451689 29860 tablet_bootstrap.cc:492] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.452560 29860 ts_tablet_manager.cc:1403] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.011s
I20260709 12:28:43.454407 29861 consensus_queue.cc:260] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.456842 29860 raft_consensus.cc:359] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.457469 29861 ts_tablet_manager.cc:1434] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.002s
I20260709 12:28:43.457773 29860 raft_consensus.cc:385] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.458185 29860 raft_consensus.cc:740] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.458588 29861 tablet_bootstrap.cc:492] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.459190 29860 consensus_queue.cc:260] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.462167 29860 ts_tablet_manager.cc:1434] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:28:43.463404 29860 tablet_bootstrap.cc:492] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.471418 29860 tablet_bootstrap.cc:654] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.481487 29847 raft_consensus.cc:493] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.482172 29847 raft_consensus.cc:515] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.483946 29860 tablet_bootstrap.cc:492] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.484647 29860 ts_tablet_manager.cc:1403] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.001s
I20260709 12:28:43.484669 29847 leader_election.cc:290] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.485981 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d34d78784b4957ac23aaeccd5ce19d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:43.486756 29598 raft_consensus.cc:2468] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 0.
I20260709 12:28:43.488176 29860 raft_consensus.cc:359] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.489019 29860 raft_consensus.cc:385] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.488924 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d34d78784b4957ac23aaeccd5ce19d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:43.489529 29860 raft_consensus.cc:740] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.490427 29860 consensus_queue.cc:260] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:43.491657 29491 leader_election.cc:343] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.493248 29860 ts_tablet_manager.cc:1434] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:28:43.494570 29860 tablet_bootstrap.cc:492] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.491755 29861 tablet_bootstrap.cc:654] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.500027 29490 leader_election.cc:304] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:43.501089 29847 raft_consensus.cc:2804] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:43.501591 29847 raft_consensus.cc:493] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.502002 29847 raft_consensus.cc:3060] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.502243 29860 tablet_bootstrap.cc:654] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.518211 29847 raft_consensus.cc:515] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.520648 29847 leader_election.cc:290] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.521673 29858 tablet_bootstrap.cc:492] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.522061 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d34d78784b4957ac23aaeccd5ce19d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:43.522395 29858 ts_tablet_manager.cc:1403] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.090s	user 0.010s	sys 0.029s
I20260709 12:28:43.522799 29598 raft_consensus.cc:3060] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.526299 29858 raft_consensus.cc:359] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.527165 29858 raft_consensus.cc:385] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.524794 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7d34d78784b4957ac23aaeccd5ce19d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:43.527648 29858 raft_consensus.cc:740] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.528612 29858 consensus_queue.cc:260] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.529497 29673 raft_consensus.cc:3060] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.531212 29858 ts_tablet_manager.cc:1434] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:43.534780 29598 raft_consensus.cc:2468] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:43.535056 29844 raft_consensus.cc:493] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.534905 29858 tablet_bootstrap.cc:492] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.535985 29844 raft_consensus.cc:515] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.539506 29673 raft_consensus.cc:2468] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:43.540973 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.542236 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
W20260709 12:28:43.542796 29640 leader_election.cc:343] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.543074 29596 raft_consensus.cc:2468] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:43.545068 29639 leader_election.cc:304] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:43.545625 29490 leader_election.cc:304] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:43.546734 29847 raft_consensus.cc:2804] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.547276 29847 raft_consensus.cc:697] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:43.548255 29847 consensus_queue.cc:237] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.549280 29844 leader_election.cc:290] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:43.550062 29844 raft_consensus.cc:2804] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:43.550676 29844 raft_consensus.cc:493] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.551484 29844 raft_consensus.cc:3060] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.553238 29858 tablet_bootstrap.cc:654] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.566116 29858 tablet_bootstrap.cc:492] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.566736 29858 ts_tablet_manager.cc:1403] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.013s
I20260709 12:28:43.569494 29421 catalog_manager.cc:5697] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:43.572162 29846 raft_consensus.cc:493] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.572710 29846 raft_consensus.cc:515] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.575428 29846 leader_election.cc:290] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.576977 29860 tablet_bootstrap.cc:492] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.577574 29861 tablet_bootstrap.cc:492] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:43.577836 29860 ts_tablet_manager.cc:1403] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.003s
I20260709 12:28:43.578125 29861 ts_tablet_manager.cc:1403] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.001s
I20260709 12:28:43.579392 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.570936 29858 raft_consensus.cc:359] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.581359 29858 raft_consensus.cc:385] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:28:43.581627 29566 leader_election.cc:343] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.581735 29858 raft_consensus.cc:740] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.582870 29858 consensus_queue.cc:260] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.583083 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:43.584080 29860 raft_consensus.cc:359] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:43.584996 29566 leader_election.cc:343] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.585207 29860 raft_consensus.cc:385] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.585671 29860 raft_consensus.cc:740] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.586031 29858 ts_tablet_manager.cc:1434] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:28:43.587261 29858 tablet_bootstrap.cc:492] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.586966 29860 consensus_queue.cc:260] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.590942 29844 raft_consensus.cc:515] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.594517 29566 leader_election.cc:304] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:43.596267 29858 tablet_bootstrap.cc:654] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.599932 29846 raft_consensus.cc:2749] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:43.619365 29860 ts_tablet_manager.cc:1434] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.041s	user 0.013s	sys 0.000s
I20260709 12:28:43.621299 29844 leader_election.cc:290] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:43.621388 29861 raft_consensus.cc:359] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.622375 29861 raft_consensus.cc:385] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.622494 29860 tablet_bootstrap.cc:492] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.622732 29861 raft_consensus.cc:740] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.623528 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:43.624207 29598 raft_consensus.cc:3060] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.624521 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:43.624608 29861 consensus_queue.cc:260] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:43.630620 29640 leader_election.cc:343] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.631747 29861 ts_tablet_manager.cc:1434] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.004s
I20260709 12:28:43.633168 29861 tablet_bootstrap.cc:492] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.630777 29844 raft_consensus.cc:493] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.635342 29844 raft_consensus.cc:515] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.638932 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
W20260709 12:28:43.640496 29640 leader_election.cc:343] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.643663 29860 tablet_bootstrap.cc:654] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:43.646435 29566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.12.45.67:36695, user_credentials={real_user=slave}} blocked reactor thread for 61481.3us
I20260709 12:28:43.647763 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
W20260709 12:28:43.649376 29639 leader_election.cc:343] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.649603 29598 raft_consensus.cc:2468] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:43.649931 29639 leader_election.cc:304] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42; no voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:43.650887 29844 leader_election.cc:290] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:43.651660 29873 raft_consensus.cc:2749] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:43.652596 29639 leader_election.cc:304] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:43.659710 29858 tablet_bootstrap.cc:492] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.660634 29858 ts_tablet_manager.cc:1403] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.007s
I20260709 12:28:43.662142 29844 raft_consensus.cc:2804] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.663084 29844 raft_consensus.cc:697] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:43.664127 29844 consensus_queue.cc:237] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.665026 29861 tablet_bootstrap.cc:654] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.667269 29858 raft_consensus.cc:359] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.668358 29858 raft_consensus.cc:385] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.668744 29858 raft_consensus.cc:740] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.670096 29858 consensus_queue.cc:260] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.673945 29858 ts_tablet_manager.cc:1434] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:28:43.675469 29858 tablet_bootstrap.cc:492] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.683421 29860 tablet_bootstrap.cc:492] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.684197 29860 ts_tablet_manager.cc:1403] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.062s	user 0.022s	sys 0.021s
I20260709 12:28:43.684165 29847 raft_consensus.cc:493] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.684976 29847 raft_consensus.cc:515] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.688333 29847 leader_election.cc:290] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.688108 29860 raft_consensus.cc:359] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.689088 29860 raft_consensus.cc:385] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.689117 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d244e5d6f04d578e5464a7a1fd942e" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:43.689606 29860 raft_consensus.cc:740] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.690706 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d244e5d6f04d578e5464a7a1fd942e" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:43.691282 29861 tablet_bootstrap.cc:492] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:43.691468 29598 raft_consensus.cc:2468] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 0.
I20260709 12:28:43.691073 29860 consensus_queue.cc:260] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.691928 29861 ts_tablet_manager.cc:1403] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.008s
I20260709 12:28:43.693864 29490 leader_election.cc:304] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:43.694778 29860 ts_tablet_manager.cc:1434] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 12:28:43.696025 29861 raft_consensus.cc:359] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:43.696424 29491 leader_election.cc:343] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.696628 29860 tablet_bootstrap.cc:492] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.699101 29847 raft_consensus.cc:2804] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:43.699556 29847 raft_consensus.cc:493] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.699904 29847 raft_consensus.cc:3060] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.700708 29858 tablet_bootstrap.cc:654] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.708510 29861 raft_consensus.cc:385] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.708940 29861 raft_consensus.cc:740] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.709700 29861 consensus_queue.cc:260] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.712232 29861 ts_tablet_manager.cc:1434] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:28:43.713446 29861 tablet_bootstrap.cc:492] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.715906 29860 tablet_bootstrap.cc:654] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.719916 29847 raft_consensus.cc:515] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.723268 29847 leader_election.cc:290] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.723798 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d244e5d6f04d578e5464a7a1fd942e" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:43.724020 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d244e5d6f04d578e5464a7a1fd942e" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:43.724538 29598 raft_consensus.cc:3060] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:43.725387 29491 leader_election.cc:343] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.725840 29858 tablet_bootstrap.cc:492] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.726433 29858 ts_tablet_manager.cc:1403] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.005s
I20260709 12:28:43.729802 29858 raft_consensus.cc:359] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.730701 29858 raft_consensus.cc:385] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.730998 29858 raft_consensus.cc:740] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.731674 29858 consensus_queue.cc:260] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.736441 29598 raft_consensus.cc:2468] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 1.
I20260709 12:28:43.737848 29490 leader_election.cc:304] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:43.736534 29858 ts_tablet_manager.cc:1434] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:28:43.741137 29423 catalog_manager.cc:5697] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:43.742416 29858 tablet_bootstrap.cc:492] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.745923 29847 raft_consensus.cc:2804] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.746753 29847 raft_consensus.cc:697] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:43.747804 29847 consensus_queue.cc:237] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.753891 29858 tablet_bootstrap.cc:654] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.762656 29861 tablet_bootstrap.cc:654] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.766953 29844 raft_consensus.cc:493] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.767603 29844 raft_consensus.cc:515] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.772147 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d244e5d6f04d578e5464a7a1fd942e" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:43.773643 29598 raft_consensus.cc:2393] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2ec05c84795f4624b9a3e1f88bee2e42 in current term 1: Already voted for candidate 710651b45fd047d9aea0f26c77946f9f in this term.
I20260709 12:28:43.772199 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d244e5d6f04d578e5464a7a1fd942e" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.775967 29640 leader_election.cc:304] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42; no voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:43.776895 29873 raft_consensus.cc:3060] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.786659 29873 raft_consensus.cc:2749] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:43.772486 29844 leader_election.cc:290] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:43.791577 29858 tablet_bootstrap.cc:492] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.792145 29858 ts_tablet_manager.cc:1403] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.003s
I20260709 12:28:43.797144 29860 tablet_bootstrap.cc:492] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.797163 29861 tablet_bootstrap.cc:492] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:43.797973 29861 ts_tablet_manager.cc:1403] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.085s	user 0.013s	sys 0.000s
I20260709 12:28:43.797771 29423 catalog_manager.cc:5697] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 1 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:43.799065 29858 raft_consensus.cc:359] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.799880 29858 raft_consensus.cc:385] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.800177 29858 raft_consensus.cc:740] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.797861 29860 ts_tablet_manager.cc:1403] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.102s	user 0.005s	sys 0.024s
I20260709 12:28:43.802078 29861 raft_consensus.cc:359] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.802937 29861 raft_consensus.cc:385] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.803232 29861 raft_consensus.cc:740] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.804170 29861 consensus_queue.cc:260] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.804369 29860 raft_consensus.cc:359] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.805152 29860 raft_consensus.cc:385] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.805478 29860 raft_consensus.cc:740] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.806313 29860 consensus_queue.cc:260] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.806861 29861 ts_tablet_manager.cc:1434] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:43.808077 29861 tablet_bootstrap.cc:492] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.809171 29860 ts_tablet_manager.cc:1434] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:28:43.810374 29858 consensus_queue.cc:260] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.813246 29858 ts_tablet_manager.cc:1434] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:28:43.813897 29844 raft_consensus.cc:493] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.814495 29844 raft_consensus.cc:515] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.818995 29877 raft_consensus.cc:493] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.819620 29877 raft_consensus.cc:515] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.820356 29844 leader_election.cc:290] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:43.821795 29858 tablet_bootstrap.cc:492] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: Bootstrap starting.
I20260709 12:28:43.823038 29861 tablet_bootstrap.cc:654] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.826752 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.827605 29523 raft_consensus.cc:2468] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:43.829485 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:43.830298 29674 raft_consensus.cc:2468] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:43.831411 29858 tablet_bootstrap.cc:654] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.836154 29861 tablet_bootstrap.cc:492] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:43.836895 29861 ts_tablet_manager.cc:1403] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.001s
I20260709 12:28:43.841444 29861 raft_consensus.cc:359] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.842621 29861 raft_consensus.cc:385] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.842885 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.843950 29522 raft_consensus.cc:2468] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:43.843007 29861 raft_consensus.cc:740] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.845346 29640 leader_election.cc:304] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:43.822250 29860 tablet_bootstrap.cc:492] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:43.846685 29844 raft_consensus.cc:2804] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:43.847985 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:43.848241 29844 raft_consensus.cc:493] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.848641 29844 raft_consensus.cc:3060] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:43.854969 29639 leader_election.cc:343] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.858891 29860 tablet_bootstrap.cc:654] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.860764 29858 tablet_bootstrap.cc:492] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: No bootstrap required, opened a new log
I20260709 12:28:43.846028 29861 consensus_queue.cc:260] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.865206 29861 ts_tablet_manager.cc:1434] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.003s
I20260709 12:28:43.866330 29861 tablet_bootstrap.cc:492] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:43.868645 29858 ts_tablet_manager.cc:1403] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.014s
I20260709 12:28:43.821868 29877 leader_election.cc:290] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.862596 29844 raft_consensus.cc:515] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.874228 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:43.875021 29522 raft_consensus.cc:3060] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.845973 29566 leader_election.cc:304] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:43.877713 29846 raft_consensus.cc:2804] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:43.878091 29846 raft_consensus.cc:493] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.878487 29846 raft_consensus.cc:3060] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.881362 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
W20260709 12:28:43.883522 29639 leader_election.cc:343] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.887714 29846 raft_consensus.cc:515] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.887903 29858 raft_consensus.cc:359] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.878437 29844 leader_election.cc:290] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:43.889537 29858 raft_consensus.cc:385] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.889865 29858 raft_consensus.cc:740] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.890677 29858 consensus_queue.cc:260] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.893621 29858 ts_tablet_manager.cc:1434] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 12:28:43.903534 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:43.904345 29524 raft_consensus.cc:3060] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.904564 29861 tablet_bootstrap.cc:654] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.906680 29877 raft_consensus.cc:493] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.907335 29877 raft_consensus.cc:515] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.908447 29846 leader_election.cc:290] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.912902 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0ce27942104fccbdf7342f7764de73" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.913712 29523 raft_consensus.cc:2468] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:43.916287 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0ce27942104fccbdf7342f7764de73" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:43.917238 29673 raft_consensus.cc:2468] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:43.919092 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:43.920013 29674 raft_consensus.cc:3060] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.909651 29877 leader_election.cc:290] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.922715 29877 raft_consensus.cc:493] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.923357 29877 raft_consensus.cc:515] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.925465 29524 raft_consensus.cc:2468] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:43.926167 29877 leader_election.cc:290] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:43.938275 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:43.939145 29673 raft_consensus.cc:2468] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:43.927146 29522 raft_consensus.cc:2468] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:43.934505 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:43.935163 29566 leader_election.cc:304] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:43.942540 29566 leader_election.cc:304] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:43.942642 29640 leader_election.cc:304] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 76171565ecf74c9aac0deaac7c291900
W20260709 12:28:43.943840 29566 leader_election.cc:343] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:43.944149 29844 raft_consensus.cc:2804] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.944765 29844 raft_consensus.cc:697] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:43.945996 29846 raft_consensus.cc:2804] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:43.946892 29846 raft_consensus.cc:493] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.947453 29846 raft_consensus.cc:3060] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.945731 29844 consensus_queue.cc:237] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.949743 29877 raft_consensus.cc:2804] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.976416 29860 tablet_bootstrap.cc:492] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:43.977185 29860 ts_tablet_manager.cc:1403] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.167s	user 0.012s	sys 0.013s
I20260709 12:28:43.980844 29860 raft_consensus.cc:359] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.981649 29860 raft_consensus.cc:385] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.981926 29860 raft_consensus.cc:740] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.984915 29847 consensus_queue.cc:1048] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:43.991854 29860 consensus_queue.cc:260] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:43.999307 29860 ts_tablet_manager.cc:1434] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.001s
I20260709 12:28:44.000927 29877 raft_consensus.cc:697] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:44.035321 29860 tablet_bootstrap.cc:492] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: Bootstrap starting.
I20260709 12:28:44.035413 29877 consensus_queue.cc:237] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.017382 29674 raft_consensus.cc:2468] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
W20260709 12:28:44.026057 29566 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:36695, user_credentials={real_user=slave}} blocked reactor thread for 80861.8us
I20260709 12:28:44.039639 29566 leader_election.cc:304] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:44.043912 29860 tablet_bootstrap.cc:654] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.043466 29423 catalog_manager.cc:5697] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:44.045244 29846 raft_consensus.cc:515] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.049211 29894 raft_consensus.cc:2804] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.049911 29894 raft_consensus.cc:493] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.050379 29894 raft_consensus.cc:3060] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.052495 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0ce27942104fccbdf7342f7764de73" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:44.053310 29523 raft_consensus.cc:3060] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.058214 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf0ce27942104fccbdf7342f7764de73" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:44.059336 29674 raft_consensus.cc:3060] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.060688 29861 tablet_bootstrap.cc:492] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:44.061376 29861 ts_tablet_manager.cc:1403] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.195s	user 0.048s	sys 0.086s
I20260709 12:28:44.064811 29861 raft_consensus.cc:359] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.065599 29861 raft_consensus.cc:385] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.065893 29861 raft_consensus.cc:740] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.066802 29861 consensus_queue.cc:260] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.069430 29861 ts_tablet_manager.cc:1434] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:44.072194 29861 tablet_bootstrap.cc:492] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:44.089391 29523 raft_consensus.cc:2468] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:44.093940 29833 consensus_queue.cc:1048] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.098003 29896 raft_consensus.cc:493] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.098842 29896 raft_consensus.cc:515] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.099550 29861 tablet_bootstrap.cc:654] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.099931 29566 leader_election.cc:304] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.101363 29896 leader_election.cc:290] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.101828 29895 raft_consensus.cc:2804] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.104700 29895 raft_consensus.cc:697] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:44.104554 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:44.105873 29895 consensus_queue.cc:237] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.109120 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:44.112838 29419 catalog_manager.cc:5697] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.110112 29522 raft_consensus.cc:2393] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76171565ecf74c9aac0deaac7c291900 in current term 1: Already voted for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in this term.
I20260709 12:28:44.115374 29833 raft_consensus.cc:493] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.116466 29833 raft_consensus.cc:515] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.048897 29846 leader_election.cc:290] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.128345 29844 consensus_queue.cc:1048] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.135654 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:44.122087 29566 leader_election.cc:304] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76171565ecf74c9aac0deaac7c291900; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:44.140748 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:44.140831 29877 raft_consensus.cc:3060] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.123623 29833 leader_election.cc:290] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.148068 29490 leader_election.cc:304] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:44.149469 29833 raft_consensus.cc:2749] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.153419 29674 raft_consensus.cc:2468] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:44.155346 29894 raft_consensus.cc:515] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.158288 29894 leader_election.cc:290] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.161011 29833 consensus_queue.cc:1048] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:44.162784 29860 tablet_bootstrap.cc:492] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: No bootstrap required, opened a new log
I20260709 12:28:44.163439 29860 ts_tablet_manager.cc:1403] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: Time spent bootstrapping tablet: real 0.128s	user 0.015s	sys 0.028s
I20260709 12:28:44.164073 29877 raft_consensus.cc:2749] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.165560 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:44.157738 29419 catalog_manager.cc:5697] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.166085 29674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:44.166802 29674 raft_consensus.cc:3060] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.168089 29860 raft_consensus.cc:359] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.168787 29860 raft_consensus.cc:385] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.171687 29522 raft_consensus.cc:3060] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.175644 29674 raft_consensus.cc:2468] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:44.177402 29566 leader_election.cc:304] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.178275 29894 raft_consensus.cc:2804] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.178787 29894 raft_consensus.cc:697] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:44.179680 29894 consensus_queue.cc:237] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:44.180327 29566 leader_election.cc:343] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:44.183939 29861 tablet_bootstrap.cc:492] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:44.184674 29861 ts_tablet_manager.cc:1403] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.113s	user 0.011s	sys 0.006s
I20260709 12:28:44.187974 29861 raft_consensus.cc:359] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.188709 29861 raft_consensus.cc:385] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.188994 29861 raft_consensus.cc:740] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.189316 29873 consensus_queue.cc:1048] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.169056 29860 raft_consensus.cc:740] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.190732 29860 consensus_queue.cc:260] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
W20260709 12:28:44.191360 29470 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:44.193739 29860 ts_tablet_manager.cc:1434] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:28:44.195103 29861 consensus_queue.cc:260] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.198567 29861 ts_tablet_manager.cc:1434] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:28:44.199769 29861 tablet_bootstrap.cc:492] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: Bootstrap starting.
I20260709 12:28:44.206907 29861 tablet_bootstrap.cc:654] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:44.220353 29861 tablet_bootstrap.cc:492] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: No bootstrap required, opened a new log
I20260709 12:28:44.220988 29861 ts_tablet_manager.cc:1403] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 12:28:44.221549 29421 catalog_manager.cc:5697] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.224340 29861 raft_consensus.cc:359] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.225152 29861 raft_consensus.cc:385] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:44.225448 29861 raft_consensus.cc:740] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Initialized, Role: FOLLOWER
I20260709 12:28:44.226457 29861 consensus_queue.cc:260] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.228797 29861 ts_tablet_manager.cc:1434] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:28:44.231532 29833 consensus_queue.cc:1048] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.240995 29833 raft_consensus.cc:493] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.241598 29833 raft_consensus.cc:515] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.244797 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:44.250699 29833 leader_election.cc:290] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.253244 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:44.254200 29673 raft_consensus.cc:2393] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 710651b45fd047d9aea0f26c77946f9f in current term 1: Already voted for candidate 76171565ecf74c9aac0deaac7c291900 in this term.
I20260709 12:28:44.255321 29491 leader_election.cc:304] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f; no voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:44.256983 29847 raft_consensus.cc:3060] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.265503 29847 raft_consensus.cc:2749] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:44.272943 29873 raft_consensus.cc:493] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.273600 29873 raft_consensus.cc:515] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.275604 29873 leader_election.cc:290] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:44.276805 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:44.277714 29522 raft_consensus.cc:2468] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:44.278100 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900" is_pre_election: true
I20260709 12:28:44.278851 29598 raft_consensus.cc:2468] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 0.
I20260709 12:28:44.280134 29639 leader_election.cc:304] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.281629 29873 raft_consensus.cc:2804] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.281987 29873 raft_consensus.cc:493] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.282315 29873 raft_consensus.cc:3060] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.288211 29873 raft_consensus.cc:515] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.290568 29873 leader_election.cc:290] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:44.291330 29598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:44.291332 29522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:44.292219 29598 raft_consensus.cc:3060] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.292312 29522 raft_consensus.cc:3060] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.301165 29598 raft_consensus.cc:2468] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:44.302776 29639 leader_election.cc:304] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.303689 29873 raft_consensus.cc:2804] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.304186 29522 raft_consensus.cc:2468] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 1.
I20260709 12:28:44.304257 29873 raft_consensus.cc:697] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:44.306421 29873 consensus_queue.cc:237] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.317310 29420 catalog_manager.cc:5697] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 1 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:44.408538 29844 consensus_queue.cc:1048] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.411377 29844 consensus_queue.cc:1048] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:44.429068 29895 raft_consensus.cc:493] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.429739 29895 raft_consensus.cc:515] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.431972 29895 leader_election.cc:290] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.436240 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" is_pre_election: true
I20260709 12:28:44.437057 29673 raft_consensus.cc:2468] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:44.438476 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f" is_pre_election: true
I20260709 12:28:44.439289 29523 raft_consensus.cc:2468] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 0.
I20260709 12:28:44.440027 29877 consensus_queue.cc:1048] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:44.441339 29566 leader_election.cc:304] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.442235 29877 raft_consensus.cc:2804] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:44.442662 29877 raft_consensus.cc:493] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.442997 29877 raft_consensus.cc:3060] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.476399 29895 consensus_queue.cc:1048] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.477452 29877 raft_consensus.cc:515] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.480002 29877 leader_election.cc:290] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.487246 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:44.488080 29523 raft_consensus.cc:3060] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.498200 29523 raft_consensus.cc:2468] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:44.499788 29566 leader_election.cc:304] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.501091 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:44.501927 29673 raft_consensus.cc:3060] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:44.501341 29877 raft_consensus.cc:2804] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:44.504060 29877 raft_consensus.cc:697] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:44.505059 29877 consensus_queue.cc:237] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.514129 29673 raft_consensus.cc:2468] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 1.
I20260709 12:28:44.520701 29420 catalog_manager.cc:5697] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 1 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.531915 29877 consensus_queue.cc:1048] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:44.545877 29877 consensus_queue.cc:1048] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.577955 29877 consensus_queue.cc:1048] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:44.590214 29846 consensus_queue.cc:1048] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.611511 29598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:43310
I20260709 12:28:44.612259 29598 raft_consensus.cc:606] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Received request to transfer leadership to TS 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:44.620149 29598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:43310
I20260709 12:28:44.620841 29598 raft_consensus.cc:606] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Received request to transfer leadership to TS 710651b45fd047d9aea0f26c77946f9f
I20260709 12:28:44.633818 29673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:36796
I20260709 12:28:44.634449 29673 raft_consensus.cc:606] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:44.640427 29673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c1925f868da45debb0db3995589d54f"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:36796
I20260709 12:28:44.640990 29673 raft_consensus.cc:606] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:44.646715 29673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
mode: GRACEFUL
new_leader_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:36796
I20260709 12:28:44.647248 29673 raft_consensus.cc:606] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Received request to transfer leadership to TS 76171565ecf74c9aac0deaac7c291900
I20260709 12:28:44.656442 29598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:43310
I20260709 12:28:44.657167 29598 raft_consensus.cc:606] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Received request to transfer leadership to TS 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:44.660218 29873 raft_consensus.cc:993] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:44.660737 29844 raft_consensus.cc:1081] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:44.662423 29596 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43302
I20260709 12:28:44.662990 29596 raft_consensus.cc:493] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.663316 29596 raft_consensus.cc:3060] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.671424 29596 raft_consensus.cc:515] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.673593 29596 leader_election.cc:290] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.674666 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:44.675101 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e66e5dc3d2054e9bafa0168c32a8bf02" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:44.675426 29523 raft_consensus.cc:3060] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.675823 29673 raft_consensus.cc:3055] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.676165 29673 raft_consensus.cc:740] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:44.677047 29673 consensus_queue.cc:260] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.678292 29673 raft_consensus.cc:3060] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.685985 29523 raft_consensus.cc:2468] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:44.687345 29566 leader_election.cc:304] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.688226 29846 raft_consensus.cc:2804] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.688666 29673 raft_consensus.cc:2468] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:44.688807 29846 raft_consensus.cc:697] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:44.690124 29846 consensus_queue.cc:237] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.700736 29420 catalog_manager.cc:5697] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.777070 29844 consensus_queue.cc:1048] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:44.789516 29844 consensus_queue.cc:1048] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:44.806941 29844 raft_consensus.cc:993] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:44.807606 29844 raft_consensus.cc:1081] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:44.809940 29598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c1925f868da45debb0db3995589d54f"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43302
I20260709 12:28:44.810688 29598 raft_consensus.cc:493] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.811241 29598 raft_consensus.cc:3060] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.817662 29598 raft_consensus.cc:515] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.821066 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:44.821224 29598 leader_election.cc:290] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:44.821784 29523 raft_consensus.cc:3060] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.824647 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c1925f868da45debb0db3995589d54f" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:44.825430 29673 raft_consensus.cc:3055] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:44.825773 29673 raft_consensus.cc:740] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:44.826712 29673 consensus_queue.cc:260] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.828234 29673 raft_consensus.cc:3060] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:44.833959 29523 raft_consensus.cc:2468] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:44.836102 29566 leader_election.cc:304] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:44.837993 29673 raft_consensus.cc:2468] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:44.839927 29895 raft_consensus.cc:2804] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:44.840545 29895 raft_consensus.cc:697] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:44.841457 29895 consensus_queue.cc:237] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:44.850732 29420 catalog_manager.cc:5697] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:44.994037 29877 raft_consensus.cc:993] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:44.994689 29895 raft_consensus.cc:1081] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Signalling peer 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:44.996922 29522 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:28:44.997627 29522 raft_consensus.cc:493] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:44.998042 29522 raft_consensus.cc:3060] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.005452 29926 raft_consensus.cc:993] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: : Instructing follower 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:45.006084 29873 raft_consensus.cc:1081] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Signalling peer 76171565ecf74c9aac0deaac7c291900 to start an election
I20260709 12:28:45.011118 29598 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
 from {username='slave'} at 127.0.0.1:43302
I20260709 12:28:45.011647 29598 raft_consensus.cc:493] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.011943 29598 raft_consensus.cc:3060] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.012766 29522 raft_consensus.cc:515] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.017077 29522 leader_election.cc:290] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:45.021024 29598 raft_consensus.cc:515] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.024044 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:45.025102 29596 raft_consensus.cc:3055] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.025455 29596 raft_consensus.cc:740] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:45.026448 29596 consensus_queue.cc:260] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.027668 29596 raft_consensus.cc:3060] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.026612 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53bcb6edc99b479a9355cbededeef31d" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:45.029070 29673 raft_consensus.cc:3060] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.030742 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:45.031513 29523 raft_consensus.cc:3060] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.036798 29596 raft_consensus.cc:2468] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:45.038203 29673 raft_consensus.cc:2468] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:45.038930 29490 leader_election.cc:304] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:45.039997 29847 raft_consensus.cc:2804] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.040516 29847 raft_consensus.cc:697] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:45.040004 29598 leader_election.cc:290] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:45.041738 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "931a13e43aa949649d9255a9ea268c6d" candidate_uuid: "76171565ecf74c9aac0deaac7c291900" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:45.041447 29847 consensus_queue.cc:237] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.042250 29523 raft_consensus.cc:2468] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:45.042446 29673 raft_consensus.cc:3055] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.042950 29673 raft_consensus.cc:740] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:45.043773 29566 leader_election.cc:304] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:45.044831 29895 raft_consensus.cc:2804] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.043942 29673 consensus_queue.cc:260] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.045513 29895 raft_consensus.cc:697] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Leader. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:45.046626 29673 raft_consensus.cc:3060] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.046659 29895 consensus_queue.cc:237] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.055846 29877 raft_consensus.cc:993] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:45.056058 29673 raft_consensus.cc:2468] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76171565ecf74c9aac0deaac7c291900 in term 2.
I20260709 12:28:45.056380 29846 raft_consensus.cc:1081] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Signalling peer 710651b45fd047d9aea0f26c77946f9f to start an election
I20260709 12:28:45.058545 29523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
 from {username='slave'} at 127.0.0.1:44912
I20260709 12:28:45.059475 29523 raft_consensus.cc:493] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.059959 29523 raft_consensus.cc:3060] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.059679 29420 catalog_manager.cc:5697] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 1 to 2, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.066704 29421 catalog_manager.cc:5697] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 reported cstate change: term changed from 1 to 2, leader changed from 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67) to 76171565ecf74c9aac0deaac7c291900 (127.12.45.66). New cstate: current_term: 2 leader_uuid: "76171565ecf74c9aac0deaac7c291900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.069375 29523 raft_consensus.cc:515] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.079931 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:45.080694 29596 raft_consensus.cc:3055] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.081068 29596 raft_consensus.cc:740] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:45.082152 29596 consensus_queue.cc:260] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.083454 29523 leader_election.cc:290] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Requested vote from peers 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567), 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67:36695)
I20260709 12:28:45.084131 29596 raft_consensus.cc:3060] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.085669 29673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fed0a24223a74746836bd12e62bd89fd" candidate_uuid: "710651b45fd047d9aea0f26c77946f9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
I20260709 12:28:45.086369 29673 raft_consensus.cc:3060] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.093004 29596 raft_consensus.cc:2468] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:45.094470 29490 leader_election.cc:304] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 710651b45fd047d9aea0f26c77946f9f, 76171565ecf74c9aac0deaac7c291900; no voters: 
I20260709 12:28:45.095448 29847 raft_consensus.cc:2804] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.095958 29847 raft_consensus.cc:697] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Becoming Leader. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:45.096724 29847 consensus_queue.cc:237] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.097154 29673 raft_consensus.cc:2468] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 710651b45fd047d9aea0f26c77946f9f in term 2.
I20260709 12:28:45.098803 29895 consensus_queue.cc:1048] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:45.107642 29522 raft_consensus.cc:1275] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.109387 29846 consensus_queue.cc:1048] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.109737 29895 consensus_queue.cc:1048] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:45.111586 29420 catalog_manager.cc:5697] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f reported cstate change: term changed from 1 to 2, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 710651b45fd047d9aea0f26c77946f9f (127.12.45.65). New cstate: current_term: 2 leader_uuid: "710651b45fd047d9aea0f26c77946f9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:45.145788 29877 raft_consensus.cc:993] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:45.146679 29877 raft_consensus.cc:1081] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Signalling peer 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:45.147601 29673 raft_consensus.cc:1275] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.148912 29671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:36784
I20260709 12:28:45.149749 29671 raft_consensus.cc:493] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:45.150130 29671 raft_consensus.cc:3060] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.154968 29895 consensus_queue.cc:1048] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:45.170392 29671 raft_consensus.cc:515] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.175935 29671 leader_election.cc:290] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 2 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:45.176653 29673 raft_consensus.cc:1240] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Rejecting Update request from peer 76171565ecf74c9aac0deaac7c291900 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:45.177106 29523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:45.177917 29523 raft_consensus.cc:3060] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.181480 29895 consensus_queue.cc:1059] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:45.177155 29596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb0d544ce804da9a5cdc8d1d287837c" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:45.182447 29895 raft_consensus.cc:3055] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:45.183084 29895 raft_consensus.cc:740] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:45.184067 29895 consensus_queue.cc:260] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.185439 29895 raft_consensus.cc:3060] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:45.187283 29523 raft_consensus.cc:2468] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 2.
I20260709 12:28:45.188786 29640 leader_election.cc:304] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:45.189765 29926 raft_consensus.cc:2804] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:45.190529 29926 raft_consensus.cc:697] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:45.191715 29926 consensus_queue.cc:237] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:45.205997 29596 raft_consensus.cc:2468] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 2.
I20260709 12:28:45.210325 29423 catalog_manager.cc:5697] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 1 to 2, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 2 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:45.427233 29673 raft_consensus.cc:1275] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.428653 29895 consensus_queue.cc:1048] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.441311 29523 raft_consensus.cc:1275] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.444639 29894 consensus_queue.cc:1048] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.489449 29596 raft_consensus.cc:1275] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.491308 29847 consensus_queue.cc:1048] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.497200 29598 raft_consensus.cc:1275] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.499516 29847 consensus_queue.cc:1048] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:45.517532 29671 raft_consensus.cc:1275] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.517532 29673 raft_consensus.cc:1275] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 710651b45fd047d9aea0f26c77946f9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.532490 29833 consensus_queue.cc:1048] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:45.540972 29931 consensus_queue.cc:1048] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:45.595826 29671 raft_consensus.cc:1275] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.597390 29846 consensus_queue.cc:1048] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:45.609028 29523 raft_consensus.cc:1275] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 76171565ecf74c9aac0deaac7c291900: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.610899 29846 consensus_queue.cc:1048] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:45.808579 29522 raft_consensus.cc:1275] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.810734 29926 consensus_queue.cc:1048] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:45.825949 29596 raft_consensus.cc:1275] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:45.827894 29873 consensus_queue.cc:1048] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:47.661331 29420 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48874:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:28:47.688745 29420 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48874:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:28:47.756404 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:48.096171 29491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:36695, user_credentials={real_user=slave}} blocked reactor thread for 63086.3us
W20260709 12:28:48.101429 29490 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.66:43567, user_credentials={real_user=slave}} blocked reactor thread for 45685.6us
W20260709 12:28:48.139459 29983 log.cc:927] Time spent T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
W20260709 12:28:48.170435 29984 log.cc:927] Time spent T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:28:48.163705 30004 tablet.cc:1724] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:48.185963 29988 log.cc:927] Time spent T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: Append to log took a long time: real 0.093s	user 0.003s	sys 0.000s
I20260709 12:28:48.155489 30002 tablet.cc:1724] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:28:48.173336 29985 log.cc:927] Time spent T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
W20260709 12:28:48.217106 29491 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.12.45.67:36695, user_credentials={real_user=slave}} blocked reactor thread for 113777us
I20260709 12:28:48.261488 29524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5"
dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
mode: GRACEFUL
new_leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:55866
I20260709 12:28:48.262557 29524 raft_consensus.cc:606] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Received request to transfer leadership to TS 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:48.280926 30008 tablet.cc:1724] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.284984 30002 tablet.cc:1724] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.303485 30001 tablet.cc:1724] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.260526 30009 tablet.cc:1724] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.314055 30015 tablet.cc:1724] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.318861 30007 tablet.cc:1724] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.331800 30010 tablet.cc:1724] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.335492 30011 tablet.cc:1724] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.347308 29420 catalog_manager.cc:5789] TS 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633) does not have the latest schema for tablet 15d244e5d6f04d578e5464a7a1fd942e (table soft_deleted_table [id=99a014138d86452999a83b3183e7429b]). Expected version 1 got 0
I20260709 12:28:48.356408 30009 tablet.cc:1724] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.375713 30002 tablet.cc:1724] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.325749 30013 tablet.cc:1724] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.397529 30017 tablet.cc:1724] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.398154 30012 tablet.cc:1724] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.397647 30010 tablet.cc:1724] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.424038 30008 tablet.cc:1724] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.431420 30004 tablet.cc:1724] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.416796 30007 tablet.cc:1724] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.447846 29599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d"
dest_uuid: "76171565ecf74c9aac0deaac7c291900"
mode: GRACEFUL
new_leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:53918
I20260709 12:28:48.448750 29599 raft_consensus.cc:606] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Received request to transfer leadership to TS 2ec05c84795f4624b9a3e1f88bee2e42
I20260709 12:28:48.450862 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:28:48.452808 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:48.454157 30013 tablet.cc:1724] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.454612 30001 tablet.cc:1724] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.452726 30015 tablet.cc:1724] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.461860 30009 tablet.cc:1724] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.480244 30008 tablet.cc:1724] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.482601 30011 tablet.cc:1724] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.498373 29420 catalog_manager.cc:5789] TS 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567) does not have the latest schema for tablet cf0ce27942104fccbdf7342f7764de73 (table soft_deleted_table [id=99a014138d86452999a83b3183e7429b]). Expected version 1 got 0
I20260709 12:28:48.499694 29420 catalog_manager.cc:5789] TS 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567) does not have the latest schema for tablet 931a13e43aa949649d9255a9ea268c6d (table soft_deleted_table [id=99a014138d86452999a83b3183e7429b]). Expected version 1 got 0
I20260709 12:28:48.501573 30015 tablet.cc:1724] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.519065 30012 tablet.cc:1724] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:28:48.551733 29421 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377: soft_deleted_table [id=99a014138d86452999a83b3183e7429b] alter complete (version 1)
I20260709 12:28:48.627454 30005 raft_consensus.cc:993] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: : Instructing follower 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:48.628151 29953 raft_consensus.cc:1081] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Signalling peer 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:48.629969 29670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:36776
I20260709 12:28:48.630618 29670 raft_consensus.cc:493] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:48.630997 29670 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:48.638159 29670 raft_consensus.cc:515] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:48.640551 29670 leader_election.cc:290] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 3 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:48.641788 29521 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:48.642726 29521 raft_consensus.cc:3055] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:48.643075 29521 raft_consensus.cc:740] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 710651b45fd047d9aea0f26c77946f9f, State: Running, Role: LEADER
I20260709 12:28:48.643980 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b45143755dc412183cf1dd882f290b5" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:48.644207 29521 consensus_queue.cc:260] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:48.645016 29599 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:48.645933 29521 raft_consensus.cc:3060] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:48.655061 29521 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 3.
I20260709 12:28:48.655061 29599 raft_consensus.cc:2468] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 3.
I20260709 12:28:48.656744 29640 leader_election.cc:304] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:48.657928 29873 raft_consensus.cc:2804] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:48.658488 29873 raft_consensus.cc:697] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:48.659600 29873 consensus_queue.cc:237] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:48.670769 29420 catalog_manager.cc:5697] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 2 to 3, leader changed from 710651b45fd047d9aea0f26c77946f9f (127.12.45.65) to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.942772 29969 raft_consensus.cc:993] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: : Instructing follower 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:48.943384 29977 raft_consensus.cc:1081] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Signalling peer 2ec05c84795f4624b9a3e1f88bee2e42 to start an election
I20260709 12:28:48.945034 29669 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d"
dest_uuid: "2ec05c84795f4624b9a3e1f88bee2e42"
 from {username='slave'} at 127.0.0.1:36784
I20260709 12:28:48.945639 29669 raft_consensus.cc:493] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:48.945990 29669 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:48.952864 29669 raft_consensus.cc:515] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:48.955017 29669 leader_election.cc:290] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 3 election: Requested vote from peers 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633), 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567)
I20260709 12:28:48.956380 29524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "710651b45fd047d9aea0f26c77946f9f"
I20260709 12:28:48.956653 29599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57be42294a1c49ee994eaf58ad5d947d" candidate_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "76171565ecf74c9aac0deaac7c291900"
I20260709 12:28:48.957194 29524 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:48.957337 29599 raft_consensus.cc:3055] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:48.957798 29599 raft_consensus.cc:740] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 76171565ecf74c9aac0deaac7c291900, State: Running, Role: LEADER
I20260709 12:28:48.959012 29599 consensus_queue.cc:260] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:48.960340 29599 raft_consensus.cc:3060] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:48.965371 29524 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 3.
I20260709 12:28:48.966435 29599 raft_consensus.cc:2468] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ec05c84795f4624b9a3e1f88bee2e42 in term 3.
I20260709 12:28:48.966768 29640 leader_election.cc:304] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ec05c84795f4624b9a3e1f88bee2e42, 710651b45fd047d9aea0f26c77946f9f; no voters: 
I20260709 12:28:48.967775 29873 raft_consensus.cc:2804] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:48.968313 29873 raft_consensus.cc:697] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 LEADER]: Becoming Leader. State: Replica: 2ec05c84795f4624b9a3e1f88bee2e42, State: Running, Role: LEADER
I20260709 12:28:48.969381 29873 consensus_queue.cc:237] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } }
I20260709 12:28:48.979210 29420 catalog_manager.cc:5697] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 reported cstate change: term changed from 2 to 3, leader changed from 76171565ecf74c9aac0deaac7c291900 (127.12.45.66) to 2ec05c84795f4624b9a3e1f88bee2e42 (127.12.45.67). New cstate: current_term: 3 leader_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ec05c84795f4624b9a3e1f88bee2e42" member_type: VOTER last_known_addr { host: "127.12.45.67" port: 36695 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:49.120702 29524 raft_consensus.cc:1275] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:49.122088 29873 consensus_queue.cc:1048] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:49.134168 29600 raft_consensus.cc:1275] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:49.137349 29873 consensus_queue.cc:1048] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:49.454456 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:49.460788 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:49.461405 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:49.475638 29524 raft_consensus.cc:1275] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:49.477100 30026 consensus_queue.cc:1048] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "710651b45fd047d9aea0f26c77946f9f" member_type: VOTER last_known_addr { host: "127.12.45.65" port: 33633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:49.488562 29599 raft_consensus.cc:1275] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Refusing update from remote peer 2ec05c84795f4624b9a3e1f88bee2e42: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:28:49.491837 30026 consensus_queue.cc:1048] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76171565ecf74c9aac0deaac7c291900" member_type: VOTER last_known_addr { host: "127.12.45.66" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:50.462749 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:50.469291 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:50.469872 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:51.470933 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:51.479132 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:51.479760 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:52.481055 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:52.487143 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:52.487834 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:53.489219 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:53.494918 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:53.495466 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:54.496548 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:54.503383 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:54.503932 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:55.505023 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:55.510499 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:55.511103 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:56.513019 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:56.524681 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:56.525195 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:57.526540 12469 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:57.531841 12469 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:28:57.532424 12469 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:58.604290 12469 tablet_server.cc:179] TabletServer@127.12.45.65:0 shutting down...
W20260709 12:28:58.638077 29566 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 19 similar messages]
W20260709 12:28:58.644423 29566 consensus_peers.cc:597] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.681607 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:58.682777 12469 tablet_replica.cc:333] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.684054 12469 raft_consensus.cc:2243] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.685164 12469 raft_consensus.cc:2272] T 3c1925f868da45debb0db3995589d54f P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.688845 12469 tablet_replica.cc:333] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.689916 12469 raft_consensus.cc:2243] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.690753 12469 raft_consensus.cc:2272] T cf0ce27942104fccbdf7342f7764de73 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.693972 12469 tablet_replica.cc:333] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.694778 12469 raft_consensus.cc:2243] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.696216 12469 raft_consensus.cc:2272] T fed0a24223a74746836bd12e62bd89fd P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.700271 12469 tablet_replica.cc:333] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.701088 12469 raft_consensus.cc:2243] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.702193 12469 raft_consensus.cc:2272] T 15d244e5d6f04d578e5464a7a1fd942e P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.705376 12469 tablet_replica.cc:333] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.706004 12469 raft_consensus.cc:2243] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.706699 12469 raft_consensus.cc:2272] T 49def8d7de014ea883f046b20d82c270 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.709599 12469 tablet_replica.cc:333] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.710700 12469 raft_consensus.cc:2243] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.712129 12469 raft_consensus.cc:2272] T f7d34d78784b4957ac23aaeccd5ce19d P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.715114 12469 tablet_replica.cc:333] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.715786 12469 raft_consensus.cc:2243] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.716387 12469 raft_consensus.cc:2272] T bbb0d544ce804da9a5cdc8d1d287837c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.719369 12469 tablet_replica.cc:333] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.720038 12469 raft_consensus.cc:2243] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.721431 12469 raft_consensus.cc:2272] T f0b1aa4fd2654e4d95e99c057a036d1c P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.725106 12469 tablet_replica.cc:333] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.725935 12469 raft_consensus.cc:2243] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.727003 12469 raft_consensus.cc:2272] T 53bcb6edc99b479a9355cbededeef31d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.730067 12469 tablet_replica.cc:333] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.730731 12469 raft_consensus.cc:2243] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.731323 12469 raft_consensus.cc:2272] T 3b45143755dc412183cf1dd882f290b5 P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.734455 12469 tablet_replica.cc:333] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.735541 12469 raft_consensus.cc:2243] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.737102 12469 raft_consensus.cc:2272] T fba3c94d505a4b7b86347de064750711 P 710651b45fd047d9aea0f26c77946f9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.741573 12469 tablet_replica.cc:333] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.742482 12469 raft_consensus.cc:2243] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.743580 12469 raft_consensus.cc:2272] T badb69aae97a4c3ba2252e42aa7e01e7 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.747059 12469 tablet_replica.cc:333] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.747689 12469 raft_consensus.cc:2243] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.748302 12469 raft_consensus.cc:2272] T 57be42294a1c49ee994eaf58ad5d947d P 710651b45fd047d9aea0f26c77946f9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.751238 12469 tablet_replica.cc:333] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.751804 12469 raft_consensus.cc:2243] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.752362 12469 raft_consensus.cc:2272] T 931a13e43aa949649d9255a9ea268c6d P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.756819 12469 tablet_replica.cc:333] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.757683 12469 raft_consensus.cc:2243] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:58.758399 29566 consensus_peers.cc:597] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.758536 12469 raft_consensus.cc:2272] T 50fb89a8625342928a38a5a005eb7383 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:58.760630 29566 consensus_peers.cc:597] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.762633 12469 tablet_replica.cc:333] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.763445 12469 raft_consensus.cc:2243] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.764091 12469 raft_consensus.cc:2272] T e66e5dc3d2054e9bafa0168c32a8bf02 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.767144 12469 tablet_replica.cc:333] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.767783 12469 raft_consensus.cc:2243] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.768325 12469 raft_consensus.cc:2272] T a178b0a2291d45609f26a615cfbc0289 P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.771670 12469 tablet_replica.cc:333] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f: stopping tablet replica
I20260709 12:28:58.772274 12469 raft_consensus.cc:2243] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.772832 12469 raft_consensus.cc:2272] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 710651b45fd047d9aea0f26c77946f9f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:58.825323 29640 consensus_peers.cc:597] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:58.828441 29566 consensus_peers.cc:597] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.831094 12469 tablet_server.cc:196] TabletServer@127.12.45.65:0 shutdown complete.
W20260709 12:28:58.840751 29640 consensus_peers.cc:597] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.890415 12469 tablet_server.cc:179] TabletServer@127.12.45.66:0 shutting down...
W20260709 12:28:58.915652 29566 consensus_peers.cc:597] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.935300 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:58.936304 12469 tablet_replica.cc:333] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.937810 12469 raft_consensus.cc:2243] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.938727 12469 raft_consensus.cc:2272] T 15d244e5d6f04d578e5464a7a1fd942e P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.941777 12469 tablet_replica.cc:333] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.942606 12469 raft_consensus.cc:2243] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.943725 12469 raft_consensus.cc:2272] T 3c1925f868da45debb0db3995589d54f P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.946498 12469 tablet_replica.cc:333] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.947158 12469 raft_consensus.cc:2243] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.947753 12469 raft_consensus.cc:2272] T fed0a24223a74746836bd12e62bd89fd P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.950356 12469 tablet_replica.cc:333] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.951197 12469 raft_consensus.cc:2243] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.952207 12469 raft_consensus.cc:2272] T cf0ce27942104fccbdf7342f7764de73 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.954870 12469 tablet_replica.cc:333] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.955576 12469 raft_consensus.cc:2243] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.956596 12469 raft_consensus.cc:2272] T 49def8d7de014ea883f046b20d82c270 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.959352 12469 tablet_replica.cc:333] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.960003 12469 raft_consensus.cc:2243] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.960652 12469 raft_consensus.cc:2272] T f7d34d78784b4957ac23aaeccd5ce19d P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.963217 12469 tablet_replica.cc:333] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.963836 12469 raft_consensus.cc:2243] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.964377 12469 raft_consensus.cc:2272] T badb69aae97a4c3ba2252e42aa7e01e7 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.966902 12469 tablet_replica.cc:333] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.967533 12469 raft_consensus.cc:2243] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.968073 12469 raft_consensus.cc:2272] T 57be42294a1c49ee994eaf58ad5d947d P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.970824 12469 tablet_replica.cc:333] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.971455 12469 raft_consensus.cc:2243] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.972432 12469 raft_consensus.cc:2272] T 931a13e43aa949649d9255a9ea268c6d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.975075 12469 tablet_replica.cc:333] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.975684 12469 raft_consensus.cc:2243] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.976758 12469 raft_consensus.cc:2272] T 50fb89a8625342928a38a5a005eb7383 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.979453 12469 tablet_replica.cc:333] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.980054 12469 raft_consensus.cc:2243] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.980618 12469 raft_consensus.cc:2272] T a178b0a2291d45609f26a615cfbc0289 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.983095 12469 tablet_replica.cc:333] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.983700 12469 raft_consensus.cc:2243] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.984766 12469 raft_consensus.cc:2272] T e66e5dc3d2054e9bafa0168c32a8bf02 P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.987471 12469 tablet_replica.cc:333] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.988065 12469 raft_consensus.cc:2243] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:58.989212 12469 raft_consensus.cc:2272] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.992161 12469 tablet_replica.cc:333] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
W20260709 12:28:58.992379 29639 consensus_peers.cc:597] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 -> Peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Couldn't send request to peer 76171565ecf74c9aac0deaac7c291900. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:58.992853 12469 raft_consensus.cc:2243] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.993513 12469 raft_consensus.cc:2272] T 53bcb6edc99b479a9355cbededeef31d P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:58.996542 12469 tablet_replica.cc:333] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:58.997190 12469 raft_consensus.cc:2243] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:58.997742 12469 raft_consensus.cc:2272] T 3b45143755dc412183cf1dd882f290b5 P 76171565ecf74c9aac0deaac7c291900 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.000644 12469 tablet_replica.cc:333] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:59.001297 12469 raft_consensus.cc:2243] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.001834 12469 raft_consensus.cc:2272] T fba3c94d505a4b7b86347de064750711 P 76171565ecf74c9aac0deaac7c291900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.004657 12469 tablet_replica.cc:333] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:59.005548 12469 raft_consensus.cc:2243] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.006458 12469 raft_consensus.cc:2272] T bbb0d544ce804da9a5cdc8d1d287837c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.009073 12469 tablet_replica.cc:333] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900: stopping tablet replica
I20260709 12:28:59.009644 12469 raft_consensus.cc:2243] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.010149 12469 raft_consensus.cc:2272] T f0b1aa4fd2654e4d95e99c057a036d1c P 76171565ecf74c9aac0deaac7c291900 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.024945 29639 consensus_peers.cc:597] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 -> Peer 76171565ecf74c9aac0deaac7c291900 (127.12.45.66:43567): Couldn't send request to peer 76171565ecf74c9aac0deaac7c291900. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.066552 12469 tablet_server.cc:196] TabletServer@127.12.45.66:0 shutdown complete.
I20260709 12:28:59.120639 12469 tablet_server.cc:179] TabletServer@127.12.45.67:0 shutting down...
W20260709 12:28:59.162727 29640 consensus_peers.cc:597] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.166630 12469 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:59.167502 12469 tablet_replica.cc:333] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.168347 12469 raft_consensus.cc:2243] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.168944 12469 raft_consensus.cc:2272] T 3c1925f868da45debb0db3995589d54f P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.171590 12469 tablet_replica.cc:333] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.172152 12469 raft_consensus.cc:2243] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.172629 12469 raft_consensus.cc:2272] T 15d244e5d6f04d578e5464a7a1fd942e P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.175668 12469 tablet_replica.cc:333] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.176429 12469 raft_consensus.cc:2243] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.177175 12469 raft_consensus.cc:2272] T cf0ce27942104fccbdf7342f7764de73 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.180078 12469 tablet_replica.cc:333] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.180696 12469 raft_consensus.cc:2243] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.181227 12469 raft_consensus.cc:2272] T fed0a24223a74746836bd12e62bd89fd P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.184051 12469 tablet_replica.cc:333] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.184778 12469 raft_consensus.cc:2243] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.185578 12469 raft_consensus.cc:2272] T 50fb89a8625342928a38a5a005eb7383 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:59.188247 29640 consensus_peers.cc:597] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 -> Peer 710651b45fd047d9aea0f26c77946f9f (127.12.45.65:33633): Couldn't send request to peer 710651b45fd047d9aea0f26c77946f9f. Status: Network error: Client connection negotiation failed: client connection to 127.12.45.65:33633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.189173 12469 tablet_replica.cc:333] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.189854 12469 raft_consensus.cc:2243] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.190472 12469 raft_consensus.cc:2272] T 931a13e43aa949649d9255a9ea268c6d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.193233 12469 tablet_replica.cc:333] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.194232 12469 raft_consensus.cc:2243] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.195425 12469 raft_consensus.cc:2272] T 57be42294a1c49ee994eaf58ad5d947d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.198177 12469 tablet_replica.cc:333] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.198983 12469 raft_consensus.cc:2243] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.199528 12469 raft_consensus.cc:2272] T badb69aae97a4c3ba2252e42aa7e01e7 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.202059 12469 tablet_replica.cc:333] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.202725 12469 raft_consensus.cc:2243] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.203248 12469 raft_consensus.cc:2272] T 53bcb6edc99b479a9355cbededeef31d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.205647 12469 tablet_replica.cc:333] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.206229 12469 raft_consensus.cc:2243] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.207333 12469 raft_consensus.cc:2272] T 3b45143755dc412183cf1dd882f290b5 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.209817 12469 tablet_replica.cc:333] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.210529 12469 raft_consensus.cc:2243] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.211031 12469 raft_consensus.cc:2272] T fba3c94d505a4b7b86347de064750711 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.213374 12469 tablet_replica.cc:333] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.213925 12469 raft_consensus.cc:2243] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.215173 12469 raft_consensus.cc:2272] T a178b0a2291d45609f26a615cfbc0289 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.217937 12469 tablet_replica.cc:333] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.218626 12469 raft_consensus.cc:2243] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.219334 12469 raft_consensus.cc:2272] T e66e5dc3d2054e9bafa0168c32a8bf02 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.222628 12469 tablet_replica.cc:333] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.223351 12469 raft_consensus.cc:2243] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.224220 12469 raft_consensus.cc:2272] T 1f70f573c7ac4e3cb0cf2a3ff2d93cdf P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.227999 12469 tablet_replica.cc:333] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.228726 12469 raft_consensus.cc:2243] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.229414 12469 raft_consensus.cc:2272] T f7d34d78784b4957ac23aaeccd5ce19d P 2ec05c84795f4624b9a3e1f88bee2e42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.232561 12469 tablet_replica.cc:333] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.233224 12469 raft_consensus.cc:2243] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.233775 12469 raft_consensus.cc:2272] T 49def8d7de014ea883f046b20d82c270 P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.236558 12469 tablet_replica.cc:333] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.237267 12469 raft_consensus.cc:2243] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.238384 12469 raft_consensus.cc:2272] T bbb0d544ce804da9a5cdc8d1d287837c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.241102 12469 tablet_replica.cc:333] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42: stopping tablet replica
I20260709 12:28:59.241680 12469 raft_consensus.cc:2243] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:59.242220 12469 raft_consensus.cc:2272] T f0b1aa4fd2654e4d95e99c057a036d1c P 2ec05c84795f4624b9a3e1f88bee2e42 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.325367 12469 tablet_server.cc:196] TabletServer@127.12.45.67:0 shutdown complete.
I20260709 12:28:59.374011 12469 master.cc:562] Master@127.12.45.126:34061 shutting down...
I20260709 12:28:59.405786 12469 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:59.406702 12469 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:59.407087 12469 tablet_replica.cc:333] T 00000000000000000000000000000000 P 740482ea41d04ec49107ffbefe782377: stopping tablet replica
I20260709 12:28:59.431106 12469 master.cc:584] Master@127.12.45.126:34061 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (22331 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (43962 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (565030 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:28:59.497959 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2241 messages since previous log ~33 seconds ago
I20260709 12:28:59.498250 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2241 messages since previous log ~33 seconds ago
I20260709 12:28:59.498512 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2237 messages since previous log ~33 seconds ago
I20260709 12:28:59.498690 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:28:59.498853 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:28:59.499008 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:4755: suppressed but not reported on 1 messages since previous log ~98 seconds ago
I20260709 12:28:59.499161 12469 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 1 messages since previous log ~279 seconds ago
I20260709 12:28:59.499327 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 1 messages since previous log ~279 seconds ago
I20260709 12:28:59.499469 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/consensus/raft_consensus.cc:670: suppressed but not reported on 2 messages since previous log ~374 seconds ago
I20260709 12:28:59.499626 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 3 messages since previous log ~148 seconds ago
I20260709 12:28:59.499843 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 12:28:59.500113 12469 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~21 seconds ago
